Install Pseudo-distributed HAMA with Hadoop on Mac OS X

Here is a note on installing Pseudo-distributed HAMA on Hadoop. I use hadoop 0.20.2 (hadoop-0.20.2.tar.gz) and hama (hama-0.3.0-incubating.tar.gz). The current stable release of HAMA works with Hadoop 0.20.0. 

Prerequisites :

You need to have Pseudo-distributed Hadoop installed and working first. In my case, I have install Hadoop on /usr/local/hadoop20 and also you need to put Hadoop on PATH. 

export HADOOP_HOME=/usr/local/hadoop20 export PATH=$PATH:$HADOOP_HOME/bin

Steps to Install HBase

1. Download hama-0.3.0-incubating.tar.gz and place it at /usr/local and untar it at /usr/local

$cd /usr/local $tar xvf hama-0.3.0-incubating.tar.gz $ln -s hama-0.3.0-incubating hama

2. Config network settings: 

Run the command to find the network setting:

lei:lei$ ifconfig lo0: flags=8049<UP,LOOPBACK,RUNNING,MULTICAST> mtu 16384 inet6 ::1 prefixlen 128 inet6 fe80::1%lo0 prefixlen 64 scopeid 0x1 inet 127.0.0.1 netmask 0xff000000 gif0: flags=8010<POINTOPOINT,MULTICAST> mtu 1280 stf0: flags=0<> mtu 1280 en0: flags=8863<UP,BROADCAST,SMART,RUNNING,SIMPLEX,MULTICAST> mtu 1500 ether 00:25:4c:e3:b9:7d media: autoselect status: inactive en1: flags=8863<UP,BROADCAST,SMART,RUNNING,SIMPLEX,MULTICAST> mtu 1500 ether 00:25:00:4d:74:9f inet6 fe80::225:ff:fe4c:749e%en1 prefixlen 64 scopeid 0x5 inet 192.168.5.62 netmask 0xffffff00 broadcast 192.168.5.255 media: autoselect status: active fw0: flags=8863<UP,BROADCAST,SMART,RUNNING,SIMPLEX,MULTICAST> mtu 4078 lladdr 00:25:4c:gg:gf:e3:b9:7d media: autoselect <full-duplex> status: inactive

Here is content of my /etc/hosts

$ cat /etc/hosts 127.0.0.1 lei.hadoop.local hbase localhost lei 192.168.5.62 home.lei.local

3 Config HAMA

Add the following environment variable. 

$HAMA_HOME=/usr/local/hama $ cd /usr/local/hama/conf $ vi hama-env.sh

add the following line:

$HAMA_HOME=/usr/local/hama export JAVA_HOME=/Library/Java/Home $vi hama-site.xml

add the following settings in hama-site.xml:

<configuration> <property> <name>bsp.master.address</name> <value>lei.hadoop.local:40000</value> <description></description> </property> <property> <name>fs.default.name</name> <value>hdfs://lei.hadoop.local:9000/hama</value> <description></description> </property> <property> <name>hama.zookeeper.quorum</name> <value>home.lei.local</value> <description></description> </property> </configuration>

Here is the content of gromservers:

lei$ cat groomservers lei.hadoop.local home.lei.local

Start and Run Hadoop and HAMA:

1. start Hadoop: 

lei$ start-all.sh $ hadoop fs -ls

2. start Hbase:

$ $HAMA_HOME/bin/start-bspd.sh $ jps 26885 BSPMasterRunner 293 26969 GroomServerRunner 26653 SecondaryNameNode 26591 DataNode 26853 ZooKeeperRunner 27008 Jps 26527 NameNode

3. Verify HAMA install:



$ $HAMA_HOME/bin/hama jar $HAMA_HOME/hama-examples-0.3.0-incubating.jar pi 12/02/15 00:07:01 INFO bsp.BSPJobClient: Running job: job_201202150004_0001 12/02/15 00:07:04 INFO bsp.BSPJobClient: Current supersteps number: 0 12/02/15 00:07:16 INFO bsp.BSPJobClient: Current supersteps number: 1 12/02/15 00:07:16 INFO bsp.BSPJobClient: The total number of supersteps: 1 Estimated value of PI is 3.1152 Job Finished in 16.748 seconds

4. stop HAMA and Hadoop: 

$ $HAMA_HOME/bin/stop-bspd.sh stopping bspmaster lei.hadoop.local: no groom to stop home.lei.local: no groom to stop home.lei.local: stopping zookeeper $ stop-all.sh no jobtracker to stop lei.hadoop.local: no tasktracker to stop stopping namenode lei.hadoop.local: stopping datanode lei.hadoop.local: stopping secondarynamenode

You can find useful trouble shooting link here: 

http://wiki.apache.org/hadoop/HowToSetupYourDevelopmentEnvironment

Enjoy.