Running Hadoop code from the Definitive Guide book

bigpanda133

I am trying to compile and run the code of Hadoop Definitive Guide book, but I am stuck with some errors. Can anyone help me out with this?

earl_hudson

Assuming you are using Linux, you can compile Hadoop code from the Definitive Guide book as follows:

Download the Hadoop source code from the Apache Hadoop releases page.
Unpack the source code archive.
Build Hadoop using Apache Maven. You can do this by running the following command from the root of the Hadoop source tree:

$ mvn clean install -DskipTests=true

Once Hadoop has been built successfully, you can run it by starting the various Hadoop daemons. The following commands will start a single node Hadoop cluster, consisting of a Namenode, a Datanode, and a local JobTracker:

$ hadoop-daemon.sh start namenode
$ hadoop-daemon.sh start datanode
$ hadoop-daemon.sh start jobtracker

You can then run Hadoop jobs on your cluster. For example, to run the example WordCount job from the Hadoop Definitive Guide, you can use the following command:

$ hadoop jar hadoop-*-examples.jar org.apache.hadoop.examples.WordCount /path/to/input /path/to/output

goldenmouse

The code for the Hadoop Definitive Guide can be found on the book's website. To compile the code, you will need a Java development environment such as Eclipse or IntelliJ IDEA. For more information on how to set up a Java development environment, please see the following link:

https://www3.ntu.edu.sg/home/ehchua/programming/howto/EclipseJava_HowTo.html

Once you have your development environment set up, you can compile the code by running the following command:

javac -cp hadoop-core-1.2.1.jar *.java

This will compile all of the Java code in the current directory. To run the code, you will need to have a Hadoop cluster up and running. For more information on how to set up a Hadoop cluster, please see the following link:

https://hadoop.apache.org/docs/r1.2.1/cluster_setup.html

Once your Hadoop cluster is up and running, you can run the code by submitting it to the Hadoop cluster. For more information on how to submit code to a Hadoop cluster, please see the following link:

https://hadoop.apache.org/docs/r1.2.1/mapred_tutorial.html#Job+Submission