Joe’s Blog!

Headed to Boston next week, planning to meet-up next Tuesday (7/28) 7pm at Cambridge Brewing Co. Drop by for a beer, food and maybe a little Erlang.

Inspired by Holger Just’s haproxy configuration tool I decided to write one my own that worked better for my setup and haproxy_join was born. It’s a simple Ruby script that allows you to break up a monolithic haproxy configuration file in to pieces.

haproxy_join expects files and directories to be in the following scheme:

HAPROXY_PATH/conf/global.cfg (file)
HAPROXY_PATH/conf/defaults.cfg (file)
HAPROXY_PATH/conf/frontend.cfg (file)
HAPROXY_PATH/conf/frontend.d (dir of frontend configs)
HAPROXY_PATH/conf/backend.d (dir backend configs)


The HAPROXY_PATH and resulting configuration file are specified when running the haproxy_join command. On most systems it would look like the following, all you need to do is break up your current config and put it in the above structure.

haproxy_join haproxy.cfg /etc/haproxy/


haproxy_join will also attempt to backup your configuration file before writing a new one.

This works great in tandem with a tool like Chef, allowing you to have Chef manage each small configuration file with a template and haproxy_join to concatenate them together each time they are changed. You can achieve this by using a Chef recipe based on the default opscode haproxy recipe and a slightly modified haproxy init script based on Holger Just’s haproxy init script. The recipe will notify haproxy Chef to restart haproxy if a configuration has changed and the init script will run haproxy_join before it restarts haproxy. I have posted an example of the cookbook recipe and the init script. Neither of these have been heavily used/tested so try them out before you put them into production.

Hope this helps anyone with large haproxy configurations. Let me know if you have any questions in the comments.

Working at Cloudant I use CouchDB on a daily basis. This evening for fun I decided to write some Ruby to take update notifications and push them into RabbitMQ. There are other examples of using the update notifications and Ruby in Couch such as the view updater out on the Couch wiki. It turned out super simple. There are a few AMQP libraries for Ruby, in this example I am going to use carrot.  It’s based on the  amqp library without all the eventmachine stuff. So here it goes:

couch_amqp.rb :

#!/usr/bin/ruby

require ‘rubygems’
require ‘carrot’

def main
queue = “couchdb”
run = true
couchq = Carrot.queue(:queue => queue)

while run do

notifications = gets

if notifications == nil
run = false
else
couchq.publish(notifications)
end

end
end

main

As you can tell we connect to a queue called “couchdb” on by default this is on localhost. Next we have a loop that continually runs and grabs updates from stdin. I then publish each notification to the queue and that’s that. To get the messages out of the queue I used irb and carrot.

[user@host ~]$ irb
irb(main):001:0> require ‘rubygems’
=> true
irb(main):002:0> require ‘carrot’
=> true
irb(main):003:0> couchq = Carrot.queue(:queue => “couchdb”)
=> #<Carrot::AMQP::Queue:0×7f8d2284b640 <snip>
irb(main):004:0> couchq.pop
=> “{\”type\”:\”updated\”,\”db\”:\”test1\”}\n”

So yeah, pretty simple stuff. Go ahead relax!

[EDIT 06/05/2009 2326 PST : Don't forget to add the entry to your local.ini]

[update_notification]

couch_amqp=/PATH/TO/couch_amqp.rb

I recently switched to using Claws Mail from Mozilla Thunderbird. This was mostly to try something new and because of the seeming stagnation of the Thunderbird project. So far so good, here are a couple things that I think are cool.

Lots of plugins and themes. Claws Mail has for all sorts of tasks and features. The ones that I think are key are GTKHTML, Notification (libnotify/notify-osd) and SpamAssassin. SpamAssassin in interesting because it actually uses a full SpamAssassin installation like you would see on a mail server. The plugin connects to the spamd daemon running on your system and you can teach it what is spam and ham. If you are running Ubuntu all/most of the plugins are available in the normal repositories.

Also very cool is its auto-generated mail filters. Basically you can right click any email and create a filter based on the headers. This worked great for all the mailing lists and ticketing systems I use.

Claws also has a network log which worked great for diagnosing issues with an IMAP or SMTP session.

There are a few quirks, the first that I noticed was that when Claws is checking the mail servers for new mail it locks out some of the menu items. This is annoying if an automatic check happens when you are trying to change a config item. Another annoyance is when using the up and down keys to go through messages you either have to click on the message or hit enter to view it. It would be nice if they loaded as soon as you arrowed to it. Lastly, the interface seems a little less polished than Thunderbird but that may be just because I am more accustomed to TBird after years of use.

If you are interested in trying Claws out and use Ubuntu I recommend adding the Claws PPA to your apt sources to get the latest version and claws-mail-extra-plugins.

I recently switched an application I am working on from nginx and mongrel to nginx and passenger. The setup is easy as can be but I noticed an issue on one of my long running operations. I have a controller that spawns some threads, performs operations and returns back a result to the page. Sometimes this takes a minute and using mongrel you would just increase the proxy_read_timeout in nginx. With passenger it times out after 60 seconds. There is not a configuration parameter for adjusting this. I found that adjusting upstream.read_timeout and upstream.send_timeout in /usr/lib/ruby/gems/1.8/gems/passenger-2.2.1/ext/nginx/Configuration.c (below) to a higher value and reinstalling the passenger module solves the problem.

/usr/lib/ruby/gems/1.8/gems/passenger-2.2.1/ext/nginx/Configuration.c

::snip::
ngx_conf_merge_msec_value(conf->upstream.send_timeout,
prev->upstream.send_timeout, 60000);

ngx_conf_merge_msec_value(conf->upstream.read_timeout,
prev->upstream.read_timeout, 60000);
::snip::

Thanks to the passenger team and the quick response on the mailing list.