当同时在集群中运行多个作业时,默认情况下,Hadoop将提交的作业放入一个FIFO,一个作业结束后,Hadoop就启动下一个作业。
当一个运行时间长但是优先级较低的作业先于运行时间短而优先级较高的作业提交时,优先级高的作业会长时间排队等待。 为了解决这个问题,Hadoop定义了5种不同的作业优先级,分别是:VERY_HIGH,HIGH,NORMAL,LOW,VERY_LOW,作业的默认优先级是NORMAL,可以通过$hadoop job -set-priority进行修改。例子:1.在集群中启动1个运行时间较长的作业caiyong@caiyong:/opt/hadoop$ bin/hadoop jar hadoop-examples-1.2.1.jar pi 2000 20002.查看作业列表caiyong@caiyong:/opt/hadoop$ bin/hadoop job -list1 jobs currently runningJobId State StartTime UserName Priority SchedulingInfojob_201503171201_0003 1 1426565671593 caiyong NORMAL NA3.查看作业的运行状态caiyong@caiyong:/opt/hadoop$ bin/hadoop job -status job_201503171201_0003Job: job_201503171201_0003file: hdfs://127.0.0.1:8020/home/caiyong/tmp/mapred/staging/caiyong/.staging/job_201503171201_0003/job.xmltracking URL:http://localhost:50030/jobdetails.jsp?jobid=job_201503171201_0003map() completion: 0.012500001reduce() completion: 0.0Counters: 19 Job Counters SLOTS_MILLIS_MAPS=117080 Launched map tasks=26 Data-local map tasks=26 File Input Format Counters Bytes Read=2832 FileSystemCounters HDFS_BYTES_READ=5870 FILE_BYTES_WRITTEN=1316654 Map-Reduce Framework Map output materializedbytes=672 Map input records=24 Spilled Records=48 Map output bytes=432 Total committed heap usage(bytes)=3815768064 CPU time spent (ms)=9530 Map input bytes=576 SPLIT_RAW_BYTES=3038 Combine input records=0 Combine output records=0 Physical memory (bytes)snapshot=4156928000 Virtual memory (bytes) snapshot=9500446720 Map output records=484.把作业的优先级提高为VERY_HIGHcaiyong@caiyong:/opt/hadoop$ bin/hadoop job -set-priority job_201503171201_0003 VERY_HIGHChanged job priority.5.查看更改后的作业优先级caiyong@caiyong:/opt/hadoop$ bin/hadoop job -list1 jobs currently runningJobId State StartTime UserName Priority SchedulingInfojob_201503171201_0003 1 1426565671593 caiyong VERY_HIGH NA6.强制结束正在运行的作业caiyong@caiyong:/opt/hadoop$ bin/hadoop job -kill job_201503171201_0003Killed job job_201503171201_0003