I have just released merle 0.2. It has some new memcached commands as well as a cleaner/easier API. This also includes some contributions from Nick Gerakines. Also note that I have removed support for direct calls to memcached, we decided it wasn’t really needed. If this is something you would like to see added back into the code please let me know. Also if you run into any bugs or want to see improvements don’t hesitate to comment here or on github.
You can clone the code from github or download the tarball. Enjoy!
This is just a thought of mine – I’m thinking about the same problem for another client library that I’m writing. If you write a memcached client as a gen_server, then all the memcached client requests are inherently being processed one after another by 1 gen_server process in a serial fashion. Isn’t this a bottleneck considering memcached was specifically built so that it can handle lots of connections efficiently (via libevent)?
I believe you are correct. My understanding is that the process’ mailbox has to be scanned and each item in the mailbox is processed in the order they were received. One option to continue to use gen_server and allow for multiple processes is to adjust start_link to look like:
gen_server:start_link(?MODULE, [Host, Port], []).
rather than:
gen_server:start_link({local, ?SERVER}, ?MODULE, [Host, Port], []).
This allows for starting of more than one process:
1> {ok, P} = merle:connect().
{ok,<0.33.0>}
2> {ok, P1} = merle:connect().
{ok,<0.37.0>}
The downside is that you have to know the pid anytime you make a call. An example for merle would be:
version(Pid) ->
gen_server:call(Pid, {version}).
5> merle:version(P).
["VERSION 1.2.6"]
6> merle:version(P1).
["VERSION 1.2.6"]
Rather than:
version() ->
gen_server:call(?SERVER, {version}).
The other alternative is to run multiple VM’s so ?SERVER doesn’t collide, since it can be registered locally.
As far as being a bottleneck I’m not sure. I haven’t done enough testing to see if there is an issue doing it this way. But it sounds like a test I should do.
I have seen plenty of other gen_servers setup this way so I am also banking on that. Hope this helps.
Thanks for the reply. After writing my comment above, I stumbled upon the erlang-mysql-driver code by Yariv Sadin. They take an interesting approach to this problem. They’ve a non-gen_server module called mysql_connection which basically owns 1 socket and speaks the MySQL protocol on that socket. This module can be used as-is if you desire so.
Then they provide a gen_server module which offers “connection pooling”. You can ask it to spawn ‘n’ connections and put them all in one connection pool identified by a PoolId. Whenever you make an API call, you’ve to specify the PoolId and it’ll automatically redirect the query to an unused connection from the pool.
http://code.google.com/p/erlang-mysql-driver/
Thanks, I will have to check it out.
Hi,
I think Erlang’s pool module can help here [http://erlang.org/doc/man/pool.html]. The pool module supports load balancing messages to different nodes, but it can also support the same with different processes in the same node.
See this link (http://weblog.hypotheticalabs.com/?p=224)
Ahmed