Get started with Hadoop in pdf

The purpose of this tutorial is to get you started with Hadoop and to get you acquainted with the code and homework submission system. This tutorial is to be completed individually. Here you will learn how to write, compile, debug and execute a simple Hadoop program.

First part of the assignment serves as a tutorial and the second part asks you to write your own Hadoop program.

Section 1 describes the virtual machine environment. Instead of the virtual machine, you are welcome to setup your own pseudo-distributed or fully distributed cluster if you prefer. Any version of Hadoop that is at least 1.0 will suffice. (For an easy way to set up a cluster, try Cloudera Manager: http://archive.cloudera.com/cm5/installer/latest/ cloudera-manager-installer.bin.) If you choose to setup your own cluster, you are responsible for making sure the cluster is working properly. The TAs will be unable to help you debug configuration issues in your own cluster.

Section 2 explains how to use the Eclipse environment in the virtual machine, including how to create a project, how to run jobs, and how to debug jobs. Section 2.5 gives an end-to-end example of creating a project, adding code, building, running, and debugging it. Section 3 is the actual homework assignment.