Avkash Chauhan's Blog

Windows Azure, Windows 8, Cloud Computing, Big Data and Hadoop: All together at one place.. One problem, One solution at One time...

Programmatically retrieving Task ID and Unique Reducer ID in MapReduce

Programmatically retrieving Task ID and Unique Reducer ID in MapReduce

  • Comments 1

For each Mapper and Reducer you can get Task attempt id and Task ID both. This can be done when you set up your map using the Context object. You may also know that the when setting a Reducer an unique reduce ID is used inside reducer class setup method. You can get this ID as well.

 

There are multiple ways you can get this info:

1. Using JobConf Class.

  • JobConf.get("mapred.task.id") will provide most of the info related with Map and Reduce task along with attempt id.

 

2. You can use Context Class and use as below:

  • To get task attempt ID - context.getTaskAttemptID()
  • Reducer Task ID - Context.getTaskAttemptID().getTaskID()
  • Reducer Number - Context.getTaskAttemptID().getTaskID().getId()

Keyword: Hadoop, Map/Reduce, Jobs Performance, Mapper, Reducer

Leave a Comment
  • Please add 2 and 6 and type the answer here:
  • Post
  • Please can you post an example.

    I tried  and it did not work for me.

Page 1 of 1 (1 items)