MapReduce Job Execution Process – Job scheduling

Hi Hadoopers,

We shall talk about 3rd circle today, as we talk about Job submission and Job initialilzation already.

Scheduling the jobs is an interesting concept. I’m really excited to see the communication between Scheduler, Job tracker and  Task tracker.

1. The task tracker keeps on sending heartbeats to Job Tracker about the status of the job. So, it says to Job Tracker that job is completed and it wants more jobs.
2. Job Tracker updates the task status and make a note of Task Tracker’s message.
3. Job Tracker goes to Scheduler asking for tasks.
4. Scheduler updates the tasks scheduler record. Based on job scheduling policy, either it makes the job client to wait or process the job. It is based on execution policy, priority etc.
5. Job tracker gets the task.
6. It submits the task to the task tracker.