Avkash Chauhan's Blog

Windows Azure, Windows 8, Cloud Computing, Big Data and Hadoop: All together at one place.. One problem, One solution at One time...

Listing current running Hadoop Jobs and Killing running Jobs

Listing current running Hadoop Jobs and Killing running Jobs

Rate This
  • Comments 1

When you have jobs running in Hadoop, you can use the map/reduce web view to list the current running jobs however what if you would need to kill any current running job because the submitted jobs started malfunctioning or in worst case scenario, the job is stuck in infinite loops. I have seen several scenarios when a submitted job got stuck in problematic states due to code defect in map/reduce job or the Hadoop cluster itself. In any of such situation, you would need to manually kill the job which is already started.

 

To kill a currently running Hadoop job first you need Job ID and then Kill the job using the as following commands:

  • Hadoop job -list
  • Hadoop job –kill <JobID>

To list current running job in Hadoop Command shell please use below command:

 

On Linux:      $ bin/hadoop job –list
On Windows:    HADOOP_HOME = C:\Apps\Dist\ 
               HADOOP_HOME\bin\Hadoop job list

 

Above command will return job details as below:

 

[Linux]

1 jobs currently running
JobId                  State          StartTime       UserName
job_201203293423_0001   1             1334506474312   avkash
 
[Windows]
c:\apps\dist>hadoop job -list
1 jobs currently running
JobId                  State   StartTime       UserName        Priority        SchedulingInfo
job_201204011859_0002   1       1333307249654   avkash       NORMAL          NA
 
 

Once you have the JobID you can use the following command to kill the job:

 
On Linux:      $ bin/hadoop job -kill jobid
On Windows:    HADOOP_HOME = C:\Apps\Dist\ 
               HADOOP_HOME\bin\Hadoop job –kill <Job_ID>
 
[Windows]
c:\apps\dist>hadoop job -kill job_201204011859_0002
Killed job job_201204011859_0002

 

 

Leave a Comment
  • Please add 3 and 7 and type the answer here:
  • Post
  • I have tried this and this works. But sometimes when I kill jobs that are running for longer times than anticipated, It shows message that Killed Job_ID but the job ID is still shown in hadoop job list. It occasionally happens, most of the time I am able to kill jobs successfully. I am not able to identify the reason for this. Can you help?

Page 1 of 1 (1 items)