Comments on: CouchDB Load Balancing and Replication using HAProxy.

By: Ed Ruder

Ed Ruder — Wed, 26 Aug 2009 21:26:33 +0000

Thought I’d add a bit more detail that clarifies a couple of gotchas I ran into while bringing up a second CouchDB instance on the same machine.

The first problem I had was in specifying the separate .ini file. I had assumed that -c SOME_PATH/couchdb2.ini would work just like local.ini did for the first instance–i.e., it was an override on top of default.ini. Not so. -c (lowercase C) doesn’t build on the “system default” configuration file chain. -C (uppercase C) does. You can explicitly include default.ini and your instance’s .ini file using -c or include just your instance’s .ini file using -C (or use -c if your instance’s .ini file doesn’t assume it’s building on top of default.ini).

The second has to do with directory and file ownership–if you are running couchdb under the couchdb as I was (and as the default startup script does), the directory containing your database and view needs to be read/write-able by the couchdb user. The log file also has to be writable by the couchdb user. An easy way to make sure this is the case is to chown the directories and log file to be owned by the couchdb user.

Thanks for the article!

By: joe

joe — Wed, 15 Jul 2009 19:43:14 +0000

It works pretty well and is in production. Regarding replication I believe there will be configurable permanent replicators coming to couch soon.

By: Curtis

Curtis — Tue, 14 Jul 2009 18:02:58 +0000

Great Post. How has the HAProxy setup been working for you? I do worry that the replication step involves too many steps.

By: links for 2009-01-27 « Bloggitation

links for 2009-01-27 « Bloggitation — Wed, 28 Jan 2009 07:04:00 +0000

[...] CouchDB Load Balancing and Replication using HAProxy. (tags: sysadmin cluster database couchdb) [...]

By: Charlie

Charlie — Tue, 27 Jan 2009 20:43:45 +0000

Nice! I like how easy it is to get the master/slave behavior by simply configuring a proxy server.

I’m wondering about better ways to do this replication though. It looks like we have the following sequence of steps:

1. CouchDB instance A completes an update operation
2. Instance A runs some kind of user-supplied script?
3. Script sends instance A a POST request giving a command to replicate with CouchDB instance B
4. Instance A connects to instance B to get a list of documents on instance B.
5. Instance A determines which documents it has that instance B doesn’t have, and sends those documents to instance B using a batch request.
6. Go back to step 3, replacing B with C
7. …

That seems kind of inefficient. I’m wondering if we could do something more like:

1. CouchDB instance A completes an update operation
2. Instance A runs some kind of user-supplied script?
3. Script sends the same update to instance B
4. Script sends the same update to instance C
5. …

or, as a variation on that theme:

3. Script publishes the update to a messaging server. In AMQP terminology, publish the update to and exchange. In JMS terminology, publish it to a topic.

4. ‘Slave’ databases have a live connection over which they are listening for these updates.