Tuesday, February 14, 2012

Build and Run Hadoop Java program with Ant and Maven


Recently I got a chance to experiment with ant/maven to build and run Hadoop Java program. I don't have time to read the manuals, and here is what I did on Ant and Maven scripts to build and run a modified WordCount program. 


I have hadoop-0.20.205.0 installed at /usr/local/hadoop20 and environment variable is set accordingly. The build target is to build jar file LeiBigTop-1.1.jar.


Here is the build.xml:




Assuming you have Hadoop running correctly. You can run the following to build and run the com.lei.bigtop.hadoop.wordcount.CountWordsV2. 


lei:LeiBigTop lei$ ant -version
Apache Ant(TM) version 1.8.2 compiled on December 20 2010


lei:LeiBigTop lei$ ant


lei:LeiBigTop lei$ ant runWordCounts 


Here is the pom.xml.  






lei:LeiBigTop lei$ mvn -version
Apache Maven 3.0.3 (r1075438; 2011-02-28 09:31:09-0800)
Maven home: /usr/local/apache-maven-3.0.3
Java version: 1.6.0_29, vendor: Apple Inc.
Java home: /System/Library/Java/JavaVirtualMachines/1.6.0.jdk/Contents/Home
Default locale: en_US, platform encoding: MacRoman
OS name: "mac os x", version: "10.6.8", arch: "x86_64", family: "mac"

lei:LeiBigTop lei$ mvn clean compile package verify




Enjoy.  

2 comments:

  1. From my view, the only difference between learning at hadoop online training and learning from the informative blogs like this is that, here in these blogs we can connect to more examples along with the videos which helps us to understand the subject to the point.

    ReplyDelete
  2. Hi, Great.. Tutorial is just awesome..It is really helpful for a newbie like me.. I am a regular follower of your blog. Really very informative post you shared here. Kindly keep blogging. If anyone wants to become a Java developer learn from Java Training in Chennai. or learn thru Java EE Online Training from India . Nowadays Java has tons of job opportunities on various vertical industry.

    ReplyDelete