I have been watching Kevin Smith’s Erlang videos lately and have come up with a couple tips to supplement a couple topics I have seen. First, using the Erlang ping function with a machine that is behind a firewall. I have found that there are ports that need to be opened to make sure your ping works properly.
Remote Machine:
[joe@catalpa02 ~]$ erl -sname coppi -setcookie pingtest
Erlang (BEAM) emulator version 5.5.2 [source] [async-threads:0] [hipe] [kernel-poll:false]
Eshell V5.5.2 (abort with ^G)
(coppi@catalpa02)1>
Local Machine:
[zeusfaber@der-dieb ~]$ erl -sname merckx -setcookie pingtest
Erlang (BEAM) emulator version 5.6.3 [source] [64-bit] [smp:2] [async-threads:0] [hipe] [kernel-poll:false]
Eshell V5.6.3 (abort with ^G)
(merckx@der-dieb)1> net_adm:ping(coppi@catalpa02).
pang
(merckx@der-dieb)2> net_adm:ping(coppi@catalpa02).
pang
As you can see above when I initially attempted to ping my remote machine (with shorewall, iptables based firewall installed) I received a ‘pang’ which means it didn’t work. I also checked the logs on catalpa02 and saw the following messages:
10.1.1.195 DST=10.1.1.200 LEN=60 TOS=0×00 PREC=0×00 TTL=64 ID=23194 DF PROTO=TCP SPT=43037 DPT=4369 WINDOW=5840 RES=0×00 SYN URGP=0
Jul 29 23:07:27 catalpa02 kernel: Shorewall:net2all:DROP:IN=eth0 OUT= MAC=00:0b:db:a8:b5:06:00:1e:4c:a2:9d:36:08:00 SRC=10.1.1.195 DST=10.1.1.200 LEN=60 TOS=0×00 PREC=0×00 TTL=64 ID=23195 DF PROTO=TCP SPT=43037 DPT=4369 WINDOW=5840 RES=0×00 SYN URGP=0
So, it looks like port 4369 needs to be open since the firewall is dropping them. I opened it and attempted a ping again and received:
Jul 29 23:13:47 catalpa02 kernel: Shorewall:net2all:DROP:IN=eth0 OUT= MAC=00:0b:db:a8:b5:06:00:1e:4c:a2:9d:36:08:00 SRC=10.1.1.195 DST=10.1.1.200 LEN=60 TOS=0×00 PREC=0×00 TTL=64 ID=38806 DF PROTO=TCP SPT=43200 DPT=57783 WINDOW=5840 RES=0×00 SYN URGP=0
Jul 29 23:13:50 catalpa02 kernel: Shorewall:net2all:DROP:IN=eth0 OUT= MAC=00:0b:db:a8:b5:06:00:1e:4c:a2:9d:36:08:00 SRC=10.1.1.195 DST=10.1.1.200 LEN=60 TOS=0×00 PREC=0×00 TTL=64 ID=38807 DF PROTO=TCP SPT=43200 DPT=57783 WINDOW=5840 RES=0×00 SYN URGP=0
It looks like 57783 is also being blocked. After opening it everything seemed to work:
(merckx@der-dieb)4> net_adm:ping(coppi@catalpa02).
pong
Unfortunately the second port that we needed to open changes for each new Erlang session (the next time I attempted a ping it used port 54328). So it may be more advantageous to run your Erlang nodes across a private network linking your machines rather than across your public interface. This generally a good practice anyway since it saves you from poking to many holes in your firewall, sending unencrypted data across a public network link and not to mention bandwidth costs.
Tip number two is sort of related, it is when pinging hostnames with hyphens in them. Erlang treats node names as an atom so if you attempt to use a hyphen without any precautions you will likely get an error like the following.
[joe@catalpa02 ~]$ erl -sname coppi -setcookie pingtest1
Erlang (BEAM) emulator version 5.5.2 [source] [async-threads:0] [hipe] [kernel-poll:false]
Eshell V5.5.2 (abort with ^G)
(coppi@catalpa02)1> net_adm:ping(merckx@der-dieb).
** exited: {badarith,[{erl_eval,eval_op,3},
{erl_eval,expr_list,6},
{erl_eval,expr,5},
{shell,exprs,6},
{shell,eval_loop,3}]} **
=ERROR REPORT==== 29-Jul-2008::23:24:15 ===
Error in process <0.35.0> on node ‘coppi@catalpa02′ with exit value: {badarith,[{erl_eval,eval_op,3},{erl_eval,expr_list,6},{erl_eval,expr,5},{shell,exprs,6},{shell,eval_loop,3}]}
Simply by adding single quotes around the node name we can get around this error.
(coppi@catalpa02)2> net_adm:ping(’merckx@der-dieb’).
pong
I have gotten a few comments and etc regarding whether or not I was able to get Hadoop to talk to Ganglia. Sadly I wasn’t able to get this to work properly either but I did contact the Hadoop mailing list (this thread) and got the following information. There is actually a bug. The link includes a patch but note that the trunk has changed and the patch currently only works on Hadoop version 0.16.0. I have not had a chance to test everything out yet but it is at least a step in the right direction for those of you who are curious. Hope this helps.
As some of you may have heard there is a new DNS cache poisoning ’sploit in the wild (CVE). Currently there are two exploits available, one for a domain and the other for a hostname. Check out the info on each here and here. The basic idea is described in the files.
This exploit targets a fairly ubiquitous flaw in DNS implementations
which allow the insertion of malicious DNS records into the cache of the
target nameserver. This exploit caches a single malicious host entry
into the target nameserver. By causing the target nameserver to query
for random hostnames at the target domain, the attacker can spoof a
response to the target server including an answer for the query, an
authority server record, and an additional record for that server,
causing target nameserver to insert the additional record into the
cache.
Metasploit (the latest from their SVN) includes each exploit. This makes it easy to test and attempt the exploit against your servers. Luckily my name servers are running TinyDNS which does not have this flaw. For example:
msf auxiliary(bailiwicked_host) > check
[*] Using the Metasploit service to verify exploitability…
[*] UNKNOWN: This server did not reply to our vulnerability check requests
You will see some requests in your DNS logs that look like this:
2008-07-25 14:26:45.896813500 3ff6167e:373a:0009 - 0010 spoofprobe-check-1-11391593674.red.metasploit.com
2008-07-25 14:26:55.899392500 3ff6167e:373a:000a - 0010 spoofprobe-check-2-1139140392.red.metasploit.com
2008-07-25 14:27:05.903259500 3ff6167e:373a:000b - 0010 spoofprobe-check-3-11391648643.red.metasploit.com
So yeah, if you aren’t running TinyDNS patch your name servers!
More info at hackaday and Dan Kaminsky’s blog.
I have been doing some work with Erlang lately and wanted to do figure out how to do XML parsing. After a bit of looking I found Erlsom, which is a XML parsing library for Erlang. It has a few modes including a SAX parser and a “simple sort of DOM parser”. I have had experience using Java, xerces and jdom so this sounded good to me. So I created an XML file containing music data, stuff like artist, album, song title and etc. So I decided to screw around with some of the examples found in their docs.
First, I had to install Erlsom. I just downloaded the tarball and extracted it and did the normal configure, make, make install. One issue I noticed is a error with non-visual (”^M”) characters in the config* files. I just used dos2unix to remove the bad characters and the configure script then worked fine.
Then I threw together my XML file, I had some old ID3 tag parsing code from a project of years ago that I used to create it. It basically looked like this. Then I just started up the Erlang console, loaded and parsed the XML file.
[zeusfaber@der-dieb ~]$ erl
Erlang (BEAM) emulator version 5.6.3 [source] [64-bit] [smp:2] [async-threads:0] [hipe] [kernel-poll:false]
Eshell V5.6.3 (abort with ^G)
1> {ok, Xml} = file:read_file(”music-library.xml”).
{ok,<<”\r\n
Arms and SleepersLimited Edition EP”…>>}
2> erlsom:parse_sax(Xml, [], fun(Event, Acc) -> io:format(”~p~n”, [Event]), Acc end).
startDocument
{processingInstruction,”xml”,
” version=\”1.0\” encoding=\”UTF-8\”"}
{startElement,[],”Library”,[],[]}
{startElement,[],”ArtistName”,[],[]}
{characters,”Arms and Sleepers”}
{startElement,[],”AlbumTitle”,[],[]}
{characters,”Limited Edition EP”}
{startElement,[],”SongTitle”,[],
[{attribute,"SongDate",[],[],”Unknown”},
{attribute,”SongGenre”,[],[],”Unknown”}]}
{characters,”We’re all in Paris Now (pt. 1)”}
–SNIP–
{startElement,[],”ArtistName”,[],[]}
{characters,”Wolf Parade”}
{startElement,[],”AlbumTitle”,[],[]}
{characters,”At Mount Zoomer”}
{startElement,[],”SongTitle”,[],
[{attribute,"SongDate",[],[],”2008″},
{attribute,”SongGenre”,[],[],”Unknown”}]}
{characters,”Kissing the Beehive”}
{endElement,[],”SongTitle”,[]}
{endElement,[],”AlbumTitle”,[]}
{endElement,[],”ArtistName”,[]}
{endElement,[],”Library”,[]}
endDocument
{ok,[],”\r\n”}
Once it loads the file you can do operations on the data. I did a few counts of artists and songs. The first counts how many times “Wolf Parade” shows up in the ‘characters’ field (ie {characters,”Wolf Parade”}).
3> CountWolfParade = fun(Event, Acc) -> case Event of {characters, “Wolf Parade”} -> Acc + 1; _ -> Acc end end.
#Fun
4> erlsom:parse_sax(Xml, 0, CountWolfParade).
{ok,9,”\r\n”}
So I have nine entires of “Wolf Parade”. Next, I ran it on the other artist in the XML, Arms and Sleepers.
5> CountArmsAndSleepers = fun(Event, Acc) -> case Event of {characters, “Arms and Sleepers”} -> Acc + 1; _ -> Acc end end.
#Fun
6> erlsom:parse_sax(Xml, 0, CountArmsAndSleepers).
{ok,6,”\r\n”}
This time I have six. In both cases the counts matched what was in the XML file. Next, I decided to count not a characters field but based on one of the element names, specifically “SongTitle”. The count should give me the total number of songs.
7> CountTotalSongs = fun(Event, Acc) -> case Event of {startElement, _, “SongTitle”, _, _} -> Acc + 1; _ -> Acc end end.
#Fun
8> erlsom:parse_sax(Xml, 0, CountTotalSongs).
{ok,15,”\r\n”}
Don’t forget that things need to match the ‘{startElement,[],”SongTitle”,[],[]}’ directive so your patten needs to look something like ‘{startElement, _, “SongTitle”, _, _}’ so it takes into account the empty brackets (’[]‘).
The pattern matching in Erlang and Erlsom makes parsing pretty easy although I have heard that using Erlang alone to parse XML is troublesome.
OOPS! How did that Micera get in my musette?
