Commit Graph

296 Commits

Author SHA1 Message Date
Arun Suresh 7af9b8ad1e Fixing Job History Server Configuration parsing. (Jason Lowe via asuresh) 2017-11-09 15:10:21 -08:00
Arun Suresh 71a55bebfb Preparing for 2.10.0 development 2017-10-31 12:51:14 -07:00
Jason Lowe 50dfef7275 MAPREDUCE-6927. MR job should only set tracking url if history was successfully written. Contributed by Eric Badger
(cherry picked from commit 735fce5bec)

Conflicts:
	hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-app/src/main/java/org/apache/hadoop/mapreduce/v2/app/MRAppMaster.java
2017-08-08 14:55:55 -05:00
Junping Du cae33098e6 Revert "MAPREDUCE-5875. Make Counter limits consistent across JobClient, MRAppMaster, and YarnChild. (Gera Shegalov via kasha)"
This reverts commit e8a31f2e1c.

(cherry picked from commit fbb7d6bcbb)
2017-07-31 14:15:06 -07:00
Masatake Iwasaki 2f4cf07fb1 MAPREDUCE-6644. Use doxia macro to generate in-page TOC of MapReduce site documentation. (iwasakims)
(cherry picked from commit 3619ae32be)
2017-02-02 01:23:02 +09:00
Akira Ajisaka 3e2d26a97f HADOOP-12733. Remove references to obsolete io.seqfile configuration variables. Contributed by Ray Chiang.
(cherry picked from commit 01d31fe938)
2017-01-04 14:15:00 +09:00
Karthik Kambatla 0aaaeea268 MAPREDUCE-6817. The format of job start time in JHS is different from those of submit and finish time. (Haibo Chen via kasha)
(cherry picked from commit a793cec2c9)
2016-12-07 13:37:12 -08:00
Robert Kanter 99b046f8a9 MAPREDUCE-6787. Allow job_conf.xml to be downloadable on the job overview page in JHS (haibochen via rkanter)
(cherry picked from commit c87b3a448a)
2016-12-01 17:30:02 -08:00
Mingliang Liu 4e0fcff8ab HADOOP-13427. Eliminate needless uses of FileSystem#{exists(), isFile(), isDirectory()}. Contributed by Steve Loughran and Mingliang Liu
(cherry picked from commit 5af572b644)
2016-11-15 11:36:08 -08:00
Jason Lowe 5ffb26c2a9 MAPREDUCE-6797. Job history server scans can become blocked on a single, slow entry. Contributed by Prabhu Joseph
(cherry picked from commit 99c2bbd337)
2016-11-14 20:30:36 +00:00
Varun Saxena a942d968c9 MAPREDUCE-6782. JHS task page search based on each individual column not working (Ajith S via Varun Saxena)
(cherry picked from commit 026b39ad9d)
2016-11-08 15:02:46 +05:30
Varun Saxena 65fa9827fc YARN-5757. RM REST API documentation is not up to date (Miklos Szegedi via Varun Saxena) 2016-10-27 17:24:24 +05:30
Varun Saxena 14ce42a8ed MAPREDUCE-6798. Fix intermittent failure of TestJobHistoryParsing.testJobHistoryMethods (Haibo Chen via Varun Saxena) 2016-10-27 15:41:27 +05:30
Robert Kanter 85724268c2 MAPREDUCE-6718. add progress log to JHS during startup (haibochen via rkanter)
(cherry picked from commit 0d6778d800)
2016-09-28 16:04:48 -07:00
Varun Saxena 705307eda3 MAPREDUCE-6750. Fix TestHSAdminServer#testRefreshSuperUserGroups (Kihwal Lee via Varun Saxena) 2016-08-09 23:43:27 +05:30
Akira Ajisaka f4d94f2246 MAPREDUCE-6738. TestJobListCache.testAddExisting failed intermittently in slow VM testbed. Contributed by Junping Du.
(cherry picked from commit ecff7d0359)
2016-07-21 11:38:02 -07:00
Robert Kanter 233ffaf3f7 MAPREDUCE-6652. Add configuration property to prevent JHS from loading jobs with a task count greater than X (haibochen via rkanter) 2016-07-15 13:37:04 -07:00
Junping Du 51a3131a87 MAPREDUCE-6733. MapReduce JerseyTest tests failing with java.net.BindException: Address already in use. Contributed by Vinod Kumar Vavilapalli.
(cherry picked from commit 1d053836b1)

Conflicts:

	hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-app/src/test/java/org/apache/hadoop/mapreduce/v2/app/webapp/TestAMWebServices.java
	hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-app/src/test/java/org/apache/hadoop/mapreduce/v2/app/webapp/TestAMWebServicesAttempt.java
	hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-app/src/test/java/org/apache/hadoop/mapreduce/v2/app/webapp/TestAMWebServicesAttempts.java
	hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-app/src/test/java/org/apache/hadoop/mapreduce/v2/app/webapp/TestAMWebServicesJobConf.java
	hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-app/src/test/java/org/apache/hadoop/mapreduce/v2/app/webapp/TestAMWebServicesJobs.java
	hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-app/src/test/java/org/apache/hadoop/mapreduce/v2/app/webapp/TestAMWebServicesTasks.java
	hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-hs/src/test/java/org/apache/hadoop/mapreduce/v2/hs/webapp/TestHsWebServices.java
	hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-hs/src/test/java/org/apache/hadoop/mapreduce/v2/hs/webapp/TestHsWebServicesAttempts.java
	hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-hs/src/test/java/org/apache/hadoop/mapreduce/v2/hs/webapp/TestHsWebServicesJobConf.java
	hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-hs/src/test/java/org/apache/hadoop/mapreduce/v2/hs/webapp/TestHsWebServicesJobs.java
	hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-hs/src/test/java/org/apache/hadoop/mapreduce/v2/hs/webapp/TestHsWebServicesJobsQuery.java
	hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-hs/src/test/java/org/apache/hadoop/mapreduce/v2/hs/webapp/TestHsWebServicesTasks.java
2016-07-15 11:36:04 -07:00
Wangda Tan 863bfa4d6a YARN-1942. Deprecate toString/fromString methods from ConverterUtils and move them to records classes like ContainerId/ApplicationId, etc. (wangda) 2016-06-14 15:12:00 -07:00
Junping Du 2703ec6871 YARN-5076. YARN web interfaces lack XFS protection. Contributed by Jonathan Maron. 2016-05-19 14:10:15 -07:00
Junping Du 7e36b9159e MAPREDUCE-6657. Job history server can fail on startup when NameNode is in start phase. Contributed by Haibo Chen.
(cherry picked from commit f6ef876fe1)
2016-05-17 14:42:11 -07:00
Jason Lowe 9c47862d87 MAPREDUCE-6698. Increase timeout on TestUnnecessaryBlockingOnHistoryFileInfo.testTwoThreadsQueryingDifferentJobOfSameUser. Contributed by Haibo Chen
(cherry picked from commit 34fddd1e91)
2016-05-17 15:04:02 +00:00
Jason Lowe 51e06c085f MAPREDUCE-6701. application master log can not be available when clicking jobhistory's am logs link. Contributed by Haibo Chen
(cherry picked from commit 12fa4ec141)
2016-05-17 14:56:23 +00:00
Jason Lowe d3dfed3fd0 MAPREDUCE-6684. High contention on scanning of user directory under immediate_done in Job History Server. Contributed by Haibo Chen
(cherry picked from commit 5ffb54694b)
2016-05-10 16:06:26 +00:00
Jian He 70a31458e0 MAPREDUCE-6680. JHS UserLogDir scan algorithm sometime could skip directory with update in CloudFS (Azure FileSystem, S3, etc. Contributed by Junping Du
(cherry picked from commit 1e48eefe58)
2016-04-20 19:02:58 -07:00
Eric Payne edb096eab7 MAPREDUCE-6649. getFailureInfo not returning any failure info. Contributed by Eric Badger 2016-04-16 22:22:42 +00:00
Junping Du 5574f82799 MAPREDUCE-6670. TestJobListCache#testEviction sometimes fails on Windows with timeout. Contributed by Gergely Novák.
(cherry picked from commit de96d7c88a)
2016-04-06 08:35:51 -07:00
Rohith Sharma K S f22ce96b50 MAPREDUCE-6662. Clear ASF Warnings on test data files. Contributed by Vinayakumar B
(cherry picked from commit 0ef8bbfd87)
2016-03-29 10:35:04 +08:00
Varun Vasudev e9a0ffc7f1 YARN-4737. Add CSRF filter support in YARN. Contributed by Jonathan Maron.
(cherry picked from commit 43416187c07afb35e3267f94d0a41d8d3cfb5735)
2016-03-07 15:23:36 +05:30
Robert Kanter 48cb4b9ba7 MAPREDUCE-6622. Add capability to set JHS job cache to a task-based limit (rchiang via rkanter)
(cherry picked from commit 0f72da7e28)
2016-02-26 17:57:45 -08:00
Robert Kanter ec48848711 YARN-4701. When task logs are not available, port 8041 is referenced instead of port 8042 (haibochen via rkanter)
(cherry picked from commit c4d4df8de0)
2016-02-25 12:48:29 -08:00
Robert Kanter d9f01f28a0 MAPREDUCE-6640. mapred job -history command should be able to take Job ID (rkanter)
(cherry picked from commit 4343a4cf77)
2016-02-23 17:38:25 -08:00
Varun Vasudev 212c519ad3 MAPREDUCE-6634. Log uncaught exceptions/errors in various thread pools in mapreduce. Contributed by Sidharta Seethana.
(cherry picked from commit c50d1e54000c76880a041ce5959a2eb23c86bd35)
2016-02-18 14:19:01 +05:30
Robert Kanter 358c49d830 MAPREDUCE-6620. Jobs that did not start are shown as starting in 1969 in the JHS web UI (haibochen via rkanter)
(cherry picked from commit ccbba4a4de)
2016-02-02 15:21:51 -08:00
Akira Ajisaka 7f215ffdd5 MAPREDUCE-6605. Fix typos mapreduce.map.skip.proc.count.autoincr and mapreduce.reduce.skip.proc.count.autoincr in mapred-default.xml. Contributed by Kai Sasaki. 2016-01-22 18:37:17 +09:00
Karthik Kambatla ef59521094 YARN-4526. Make SystemClock singleton so AppSchedulingInfo could use it. (kasha)
(cherry picked from commit d40859fab1)

Conflicts:
	hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-app/src/test/java/org/apache/hadoop/mapred/TestTaskAttemptListenerImpl.java
2016-01-18 10:59:34 +01:00
Rohith Sharma K S 2d0348c907 MAPREDUCE-6419. JobHistoryServer doesn't sort properly based on Job ID when Job id's exceed 9999. (Mohammad Shahid Khan via rohithsharmaks)
(cherry picked from commit a308e86621)
2015-12-24 12:00:37 +05:30
Zhihai Xu da7bde60c0 MAPREDUCE-6436. JobHistory cache issue. Contributed by Kai Sasaki
(cherry picked from commit 5b7078d069)
2015-12-15 01:03:10 -08:00
Steve Loughran af14458de7 HADOOP-12321. Make JvmPauseMonitor an AbstractService. (Sunil G via Stevel) [includes HDFS-8947 MAPREDUCE-6462 and YARN-4072] 2015-12-06 17:42:56 +00:00
Vinod Kumar Vavilapalli b20e77eba3 Preparing for 2.9.0 development: mvn versions:set -DnewVersion=2.9.0 2015-11-25 17:47:22 -08:00
Akira Ajisaka 8de7911eb3 MAPREDUCE-6499. Add elapsed time for retired job in JobHistoryServer WebUI. Contributed by Lin Yiqun.
(cherry picked from commit a7f776ffdf)
2015-11-17 15:18:32 +09:00
Jason Lowe a7d8895b60 MAPREDUCE-6528. Memory leak for HistoryFileManager.getJobSummary(). Contributed by Junping Du
(cherry picked from commit 6344b6a769)
2015-10-30 15:32:48 +00:00
Jason Lowe 145058003a MAPREDUCE-6515. Update Application priority in AM side from AM-RM heartbeat. Contributed by Sunil G
(cherry picked from commit cf953b6258)
2015-10-29 18:05:56 +00:00
Haohui Mai ee04add696 HDFS-9167. Update pom.xml in other modules to depend on hdfs-client instead of hdfs. Contributed by Mingliang Liu. 2015-10-13 15:24:29 -07:00
Junping Du c8469357ba MAPREDUCE-6443. Add JvmPauseMonitor to JobHistoryServer. Contributed by Robert Kanter. 2015-08-06 07:00:42 -07:00
Jason Lowe 508da4d461 MAPREDUCE-6394. Speed up Task processing loop in HsTasksBlock#render(). Contributed by Ray Chiang
(cherry picked from commit d0e0ba8010)
2015-07-31 18:19:52 +00:00
Vinod Kumar Vavilapalli 8c9e2cf10a MAPREDUCE-6410. Fixed MapReduce JobHistory server to use the right (login) UGI to refresh log and cleaner settings. Contributed by Varun Saxena.
(cherry picked from commit d481684c7c)
2015-06-23 11:41:05 -07:00
Gera Shegalov ee63c3e831 MAPREDUCE-6305. AM/Task log page should be able to link back to the job. (Siqi Li via gera)
(cherry picked from commit 055cd5a9a1)
2015-06-20 12:40:59 -07:00
Gera Shegalov 34e744ce1a MAPREDUCE-6316. Task Attempt List entries should link to the task overview. (Siqi Li via gera)
(cherry picked from commit 63d40d5286)
2015-06-20 11:52:11 -07:00
Devaraj K 7f075f2b75 MAPREDUCE-6377. JHS sorting on state column not working in webUi.
Contributed by zhihai xu.

(cherry picked from commit 790a861766)

Conflicts:
	hadoop-mapreduce-project/CHANGES.txt
2015-06-05 16:03:02 +05:30