Joe’s Blog!

Headed to Boston next week, planning to meet-up next Tuesday (7/28) 7pm at Cambridge Brewing Co. Drop by for a beer, food and maybe a little Erlang.

Working at Cloudant I use CouchDB on a daily basis. This evening for fun I decided to write some Ruby to take update notifications and push them into RabbitMQ. There are other examples of using the update notifications and Ruby in Couch such as the view updater out on the Couch wiki. It turned out super simple. There are a few AMQP libraries for Ruby, in this example I am going to use carrot.  It’s based on the  amqp library without all the eventmachine stuff. So here it goes:

couch_amqp.rb :

#!/usr/bin/ruby

require ‘rubygems’
require ‘carrot’

def main
queue = “couchdb”
run = true
couchq = Carrot.queue(:queue => queue)

while run do

notifications = gets

if notifications == nil
run = false
else
couchq.publish(notifications)
end

end
end

main

As you can tell we connect to a queue called “couchdb” on by default this is on localhost. Next we have a loop that continually runs and grabs updates from stdin. I then publish each notification to the queue and that’s that. To get the messages out of the queue I used irb and carrot.

[user@host ~]$ irb
irb(main):001:0> require ‘rubygems’
=> true
irb(main):002:0> require ‘carrot’
=> true
irb(main):003:0> couchq = Carrot.queue(:queue => “couchdb”)
=> #<Carrot::AMQP::Queue:0×7f8d2284b640 <snip>
irb(main):004:0> couchq.pop
=> “{\”type\”:\”updated\”,\”db\”:\”test1\”}\n”

So yeah, pretty simple stuff. Go ahead relax!

[EDIT 06/05/2009 2326 PST : Don't forget to add the entry to your local.ini]

[update_notification]

couch_amqp=/PATH/TO/couch_amqp.rb

I recently switched an application I am working on from nginx and mongrel to nginx and passenger. The setup is easy as can be but I noticed an issue on one of my long running operations. I have a controller that spawns some threads, performs operations and returns back a result to the page. Sometimes this takes a minute and using mongrel you would just increase the proxy_read_timeout in nginx. With passenger it times out after 60 seconds. There is not a configuration parameter for adjusting this. I found that adjusting upstream.read_timeout and upstream.send_timeout in /usr/lib/ruby/gems/1.8/gems/passenger-2.2.1/ext/nginx/Configuration.c (below) to a higher value and reinstalling the passenger module solves the problem.

/usr/lib/ruby/gems/1.8/gems/passenger-2.2.1/ext/nginx/Configuration.c

::snip::
ngx_conf_merge_msec_value(conf->upstream.send_timeout,
prev->upstream.send_timeout, 60000);

ngx_conf_merge_msec_value(conf->upstream.read_timeout,
prev->upstream.read_timeout, 60000);
::snip::

Thanks to the passenger team and the quick response on the mailing list.

I decided to play around with RabbitMQ and chef (Opscode recently released new version) this afternoon. My main objective was to see if I could replace stompserver with something a bit more robust. In this case RabbitMQ and it’s STOMP plugin, the following is a basic outline of what I did. Please note that this was done on a Ubuntu 8.10 machine, if you are using another non-apt distro you will likely need to adjust the package installation commands and etc.

The first thing that needs to be done, if you do not already have a working chef/chef-server/ohai installation, is to get one up and running. If you already have a working install you should be able to skip down to the Erlang install portion and start from there.

Installing all this is pretty basic, check out the instructions. You should end up with a working installation with chef-server using stompserver (which we will be replacing) and couchdb. Here’s what I did:

sudo apt-get install ruby ruby1.8-dev rubygems libopenssl-ruby1.8 build-essential wget
wget http://rubyforge.org/frs/download.php/45904/rubygems-update-1.3.1.gem
sudo gem install rubygems-update-1.3.1.gem
sudo /var/lib/gems/1.8/bin/update_rubygems
sudo rm /usr/bin/gem
sudo ln -s /usr/bin/gem1.8 /usr/bin/gem
sudo gem sources -a http://gems.opscode.com
sudo gem install chef ohai chef-server
apt-get install couchdb

I then added in my configuration for chef-server:

vi /etc/chef/server.rb

#
## Chef Server Config File
##
#
log_level :info
log_location STDOUT
ssl_verify_mode :verify_none
registration_url “http://localhost:4000″
openid_url “http://localhost:4000″
template_url “http://localhost:4000″
remotefile_url “http://localhost:4000″
search_url “http://localhost:4000″
cookbook_path [ "/var/localhost/site-cookbooks", "/var/localhost/cookbooks" ]
merb_log_path “/var/log/localhost-server-merb.log”
queue_user “guest”
queue_password “guest”

Chef::Log::Formatter.show_time = false

Since I am using a single machine for my chef server I started two chef-server instances, one with the defaults chef-server and another chef-server -p 4001. Two instances are needed for OpenID to work properly.

The next objective is to install Erlang.

sudo apt-get install erlang

Now for the RabbitMQ and STOMP plugin install. Per Update 2 (below) using the latest plugin and version 1.5.1 of RabbitMQ doesn’t work. The latest revisions from the repository for both is needed.

apt-get install mercurial
cd /opt/
hg clone http://hg.rabbitmq.com/rabbitmq-codegen
hg clone http://hg.rabbitmq.com/rabbitmq-server
hg clone http://hg.rabbitmq.com/rabbitmq-stomp
apt-get install python-simplejson
make -C rabbitmq-server
make -C rabbitmq-stomp

I used the init script for rabbitmq-server found in debian package for 1.5.1 with a slight change, pointing to my installation in /opt. I put a copy of it here.

DAEMON=/opt/rabbitmq-server/scripts/rabbitmq-multi

You will want to edit the config for rabbitmq to start up with STOMP activated and then restart the service.

vi /etc/default/rabbitmq

SERVER_START_ARGS=’
-pa /opt/rabbitmq-stomp/ebin
-rabbit
stomp_listeners [{"0.0.0.0",61613}]
extra_startup_steps [{"STOMP-listeners",rabbit_stomp,kickstart,[]}]’

Finally, start everything up, minus stompserver.

/etc/init.d/couchdb start
/etc/init.d/rabbitmq-server restart
chef-server
chef-server -p 4001
chef-indexer

After all that you should have a working chef-server using RabbitMQ/STOMP rather than stompserver. The rest is up to you.

Update 2:
It looks like the issue (below) was caused by using incompatible versions of STOMP plugin and RabbitMQ. Using the latest server and plugin from their repository fixes the issue. I have adjusted the above instructions to state this change.

Update:
It seems that I am having issues with the chef-indexer connecting properly to rabbitmq-stomp. From what I can tell this seems to be an issue with the RabbitMQ STOMP plugin since the stomp ruby client seems to work fine against stompserver. I suppose the RabbitMQ folks do label it “experimental”. Perhaps in a future revision it will be corrected. Anywho, here are some examples of what I am seeing.

root@ubuntu-810:~# chef-indexer
/usr/lib/ruby/gems/1.8/gems/stomp-1.0.6/lib/stomp.rb:233:in `write’: Broken pipe (Errno::EPIPE)
from /usr/lib/ruby/gems/1.8/gems/stomp-1.0.6/lib/stomp.rb:233:in `puts’
from /usr/lib/ruby/gems/1.8/gems/stomp-1.0.6/lib/stomp.rb:233:in `_transmit’
from /usr/lib/ruby/gems/1.8/gems/stomp-1.0.6/lib/stomp.rb:232:in `synchronize’
from /usr/lib/ruby/gems/1.8/gems/stomp-1.0.6/lib/stomp.rb:232:in `_transmit’
from /usr/lib/ruby/gems/1.8/gems/stomp-1.0.6/lib/stomp.rb:220:in `transmit’
from /usr/lib/ruby/gems/1.8/gems/stomp-1.0.6/lib/stomp.rb:117:in `subscribe’
from /usr/lib/ruby/gems/1.8/gems/chef-0.5.2/lib/chef/queue.rb:79:in `subscribe’
from /usr/lib/ruby/gems/1.8/gems/chef-server-0.5.2/bin/chef-indexer:80
from /usr/bin/chef-indexer:19:in `load’
from /usr/bin/chef-indexer:19

root@ubuntu-810:~# irb
irb(main):001:0> require ‘rubygems’
=> true
irb(main):002:0> require ’stomp’
=> true
irb(main):003:0> conn = Stomp::Connection.open(‘guest’, ‘guest’, ‘localhost’)
=> #, @login=”guest”, @reconnectDelay=5, @transmit_semaphore=#, @failure=nil, @port=61613, @connect=#>,n <<"/">>]},n {rabbit_stomp,do_login,5},n {rabbit_stomp,process_received_bytes,2},n {rabbit_stomp,init,1},n {proc_lib,init_p,5}]}n”, @headers={“content-type”=>”text/plain”, “content-length”=>”290″, “message”=>”Processing error”}>, @reliable=false, @clientId=nil, @subscriptions={}, @host=”localhost”, @socket_semaphore=#, @socket=#, @passcode=”guest”, @closed=false>

root@ubuntu-810:~# tail /var/log/rabbitmq/rabbit.log
=INFO REPORT==== 3-Feb-2009::02:11:42 ===
starting STOMP connection <0.3746.0> from 127.0.0.1:59035

=ERROR REPORT==== 3-Feb-2009::02:11:42 ===
STOMP error frame sent:
Message: “Processing error”
Detail: “{undef,[{rabbit_channel,start_link,n [rabbit_stomp,<0.3746.0>,<0.3746.0>,<<"guest">>,n <<"/">>]},n {rabbit_stomp,do_login,5},n {rabbit_stomp,process_received_bytes,2},n {rabbit_stomp,init,1},n {proc_lib,init_p,5}]}n”

=INFO REPORT==== 3-Feb-2009::02:11:42 ===
ending STOMP connection <0.3746.0> from 127.0.0.1:59035

Here are a few interesting articles I have seen this week.

• Burstsort: Fastest known algorithm to sort large set of strings
• Erlang Programming for Multicore
• LShift : My thoughts on real time full-text search
• Forcing a Process to Garbage Collect in Erlang
• Sysenter Based System Call Mechanism in Linux 2.6
• Writing Linux programs in raw binary
• Calculating the Pearson correlation coefficient using R and Ruby

Have a great weekend!