Joe’s Blog!

I was finally able to get a clean build of gexec with the ‘–with-ganglia’ option. Here’s what I did:
I downloaded the tarball available at http://therealms.org/oss/ganglia/gexec-0.3.8.tar.gz (thanks to Bernard on the Ganglia mailing list). Then run:
rpmbuild -tb gexec-0.3.8.tar.gz
This created a RPM and SRPM, the RPM can be deleted and I installed the SRPM. Should be located [...]

[Read more →]

So I began building a new head cluster node in a KVM, just as a test run and to refine my methodology. I decided to drop Unicluster due to an unresolved issue, this time around I decided to install everything myself. … Java, check … Hadoop, check … Pig, check … Grid Engine, check … [...]

[Read more →]

Over at GridEngine.info they found a link on DanT’s Sun blog that has a sweet tutorial on setting up Hadoop using SGE’s parallel environments with loose integration.
Here we are relying on master node to start othe daemons ( [rs]sh the machine and start daemons) and distribute jobs , and we donot have control on the [...]

[Read more →]

Pig is a query language for use with Hadoop. It allows users to query hadoop data similar to a SQL database. Formally, according to their website:
Pig is a platform for analyzing large data sets that consists of a high-level language for expressing data analysis programs, coupled with infrastructure for evaluating these programs. The salient property [...]

[Read more →]

Over at GridGuru’s they have a interesting article regarding Map Reduce its applications. The Map Reduce crowd has been growing of late and is out spoken about what a great tool it is. Without a doubt it is, but something I learned a long time ago is that for each job there is a correct [...]

[Read more →]