As some of you might have heard I am getting married and afterwards moving to Seattle (was San Diego). As such I need to get rid of a few things.
Here’s what I got:
Photos of everything can be found here.
Looking for ways to have Erlang and other languages inter-operate I found Erlectricity. It is a ruby library (gem install erlectricity) that assists in message passing between Erlang and Ruby. Electricity basically allows Ruby and Erlang to talk using an Erlang port to a Ruby process.
I checked out a couple of the examples contained in the source and decided to expand upon the echo program. My version takes a tuple, the first item being what job to do and the second is a URL. Erlang determines what to send along to Ruby, sends it and Ruby runs either job and returns the result. The two options are scrape and url, the first will scrape the URL provided and return the HTML the latter will simply return the URL you provided to it.
The Erlang side:
-module(echo).
-export([job/1]).job(Info) ->
Cmd = “ruby webscrape.rb”,
Port = open_port({spawn, Cmd}, [{packet, 4}, use_stdio, exit_status, binary]),case Info of
{scrape, Site} ->
Payload = term_to_binary({scrape, Site}),
port_command(Port, Payload);
{url, Site} ->
Payload = term_to_binary({url, Site}),
port_command(Port, Payload);
Error ->
Error
end,receive
{Port, {data, Data}} ->
{result, Text} = binary_to_term(Data),
io:format(“~p~n”, [Text])
end.
The Ruby side:
require ‘net/http’
require ‘rubygems’
require ‘erlectricity’
require ’stringio’receive do |f|
f.when(:scrape, String) do |text|
scrape = Net::HTTP.get_response(URI.parse(text))
f.send!(:result, scrape.body)
f.receive_loop
end
f.when(:url, String) do |text|
f.send!(:result, “You gave #{text}”)
f.receive_loop
end
end
Here’s an example of its usage:
10> echo:job({url, <<"http://www.joetify.com">>}).
<<"You gave http://www.joetify.com">>
ok
11> echo:job({scrape, <<"http://www.joetify.com">>}).
<<"html>ncenter>nimg src=”./joetify_logo.png”>ncenter>nhtml>n”>>
ok
I modified the HTML code so it shows up properly above.
Pretty cool stuff. If you find this interesting you should certainly check out fuzed, a Yaws based web server that allows you to run Ruby on Rails apps behind it. I have played with it a bit and the clustering of rails nodes is pretty sweet.
Don’t use the latest Ibex Alpha 6 if you run an Intel Gigabit ethernet card, there is a bug (here too) currently that will screw with the firmware render it inoperable by making the checksum fail. This applies to the e1000e driver but can cause issues if you have used e1000 in the past. The image download page says:
Due to an unresolved bug in the Linux kernel included in these images, they should not be used on Intel ethernet hardware supported by the e1000e driver (Intel GigE). Doing so may render your network hardware permanently inoperable.
Older Intel ethernet hardware which uses the e1000 driver is not affected by this; however, some hardware which used the e1000 driver in previous Ubuntu releases, such as hardware that uses a PCI Express bus, has been moved from e1000 to e1000e in the latest kernel releases. If in doubt, do not use these images, and subscribe to https://bugs.launchpad.net/ubuntu/+source/linux/+bug/263555 to be notified when the bug is fixed.
Yikes! Hope they have it fixed by the 10th.
Update: More info can be found in a discussion on the kernel mailing list.
Update #2: Looks like the main bug report is here. Looks like they are getting close to a resolution.
Update #3: Seems that a fix as been released and the final release of Ibex (8.10) will be out Oct 30th.
This past weekend I decided to work on another erlang project. Similar to the last one it revolves around parsing XML. This time I am using xmerl to parse Amazon wishlist data. I used a number of examples (here, here and here) and a mailing list post to figure this one out and I must say that it is much more complicated that the first. The program accepts an email address and then gives you a list of items on all of your wishlists. So here it is.
First, I define the URLs we need to grab the wishlists from Amazon web services.
-module(amazon_wishlist_xmerl).
-export([get_wishlist/1]).
-include_lib(“xmerl/include/xmerl.hrl”).-define(LIST_ID_URL,
“http://webservices.amazon.com/onca/xml?”
“Service=AWSECommerceService&Operation=ListSearch”
“&SubscriptionId=YOURAWSIDHERE&”
“&ListType=WishList”
“&Email=”).-define(LIST_URL,
“http://webservices.amazon.com/onca/xml?”
“Service=AWSECommerceService&Operation=ListLookup”
“&SubscriptionId=YOURAWSIDHERE”
“&ListType=WishList”
“&ResponseGroup=ListItems”
“&ListId=”).
I then use an anonymous function to get the value that I need from the XML input using a list comprehension.
-define(Val(X),
(fun() ->
[ V || #xmlElement{ content = [#xmlText{value = V}|_]} <- X]
end)()).
I then have two functions which first build the URL, then use http:request to get the XML, next it checks to see if the HTTP status code is good (200 rather than 404 or etc). Each function then scans the XML for the appropriate value using the above defined anonymous function. build_list_id returns the wishlist ID’s to build_wishlist per the list map in get_wishlist. build_wishlist then prints each item title using a list comprehension.
build_list_id(Email_addr) ->
List_id_url = ?LIST_ID_URL ++ Email_addr,
{ ok, { _Status, _Headers, Body }} = http:request(List_id_url),
check_status(_Status),
{ Xml, _Rest } = xmerl_scan:string(Body),
?Val(xmerl_xpath:string(“//ListId”, Xml)).build_wishlist(List_id) ->
List_url = ?LIST_URL ++ List_id,
{ ok, { _Status, _Headers, Body }} = http:request(List_url),
check_status(_Status),
{ Xml, _Rest } = xmerl_scan:string(Body),
Titles = ?Val(xmerl_xpath:string(“//Title”, Xml)),
[ io:format("~p~n", [T]) || T <- Titles ].check_status(_Status) ->
case _Status of
{“HTTP/1.1″,200,”OK”} ->
ok;
_ ->
io:format(“Error! Bad Status Code. ~p ~n”, [_Status]),
exit(not_200_status_code)
end.get_wishlist(Email_addr) ->
lists:map( fun build_wishlist/1, build_list_id(Email_addr) ).
Here’s what it looks like when I run it against my wishlists.
[zeusfaber@der-dieb amazon_wishlist_xmerl]$ erl
Erlang (BEAM) emulator version 5.6.4 [source] [64-bit] [smp:2] [async-threads:0] [hipe] [kernel-poll:false]
Eshell V5.6.4 (abort with ^G)
1> inets:start().
ok
2> c(amazon_wishlist_xmerl).
{ok,amazon_wishlist_xmerl}
3> amazon_wishlist_xmerl:get_wishlist(“MYEMAILADDRESS”).
“Erlang Programming”
“Notes on the Underground, New Edition: An Essay on Technology, Society, and the Imagination”
“Programming Embedded Systems: With C and GNU Development Tools, 2nd Edition”
“The Supply and Demand Paradox: A Treatise on Economics”
“The Post-American World”
“Building Clustered Linux Systems (HP Professional Series)”
“High Performance Computing (RISC Architectures, Optimization ”
“Metal Gear Solid Portable Ops”
“House of Bush, House of Saud: The Secret Relationship Between the World’s Two Most Powerful Dynasties”
“The long, lonely leap,”
“Playstation 3 80GB”
“Clumsy Crab (Tiger Tales)”
“Stuff: The Secret Lives of Everyday Things (New Report, No 4)”
[[ok,ok,ok,ok,ok,ok,ok,ok,ok,ok],[ok,ok],[ok],[]]
Seems to work well. For my next project I think I will attempt to replicate the functionality of this program using erlsom’s SAX parser rather than xmerl. I’ll post it if/when I get it done.