Apache/hadoop
1
0
Fork 0
Code Issues Pull Requests Packages Projects Releases Wiki Activity
11,841 Commits 326 Branches 371 Tags
Commit Graph

281 Commits

Author SHA1 Message Date
Akira Ajisaka
f4d94f2246 MAPREDUCE-6738. TestJobListCache.testAddExisting failed intermittently in slow VM testbed. Contributed by Junping Du. 
(cherry picked from commit ecff7d03595b45d7a5d6aed73b9931fef85a56ee)
 2016-07-21 11:38:02 -07:00
Robert Kanter
233ffaf3f7 MAPREDUCE-6652. Add configuration property to prevent JHS from loading jobs with a task count greater than X (haibochen via rkanter) 2016-07-15 13:37:04 -07:00
Junping Du
51a3131a87 MAPREDUCE-6733. MapReduce JerseyTest tests failing with java.net.BindException: Address already in use. Contributed by Vinod Kumar Vavilapalli. 
(cherry picked from commit 1d053836b118de2ebda254ed2182db68a45baaa1)

Conflicts:

	hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-app/src/test/java/org/apache/hadoop/mapreduce/v2/app/webapp/TestAMWebServices.java
	hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-app/src/test/java/org/apache/hadoop/mapreduce/v2/app/webapp/TestAMWebServicesAttempt.java
	hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-app/src/test/java/org/apache/hadoop/mapreduce/v2/app/webapp/TestAMWebServicesAttempts.java
	hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-app/src/test/java/org/apache/hadoop/mapreduce/v2/app/webapp/TestAMWebServicesJobConf.java
	hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-app/src/test/java/org/apache/hadoop/mapreduce/v2/app/webapp/TestAMWebServicesJobs.java
	hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-app/src/test/java/org/apache/hadoop/mapreduce/v2/app/webapp/TestAMWebServicesTasks.java
	hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-hs/src/test/java/org/apache/hadoop/mapreduce/v2/hs/webapp/TestHsWebServices.java
	hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-hs/src/test/java/org/apache/hadoop/mapreduce/v2/hs/webapp/TestHsWebServicesAttempts.java
	hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-hs/src/test/java/org/apache/hadoop/mapreduce/v2/hs/webapp/TestHsWebServicesJobConf.java
	hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-hs/src/test/java/org/apache/hadoop/mapreduce/v2/hs/webapp/TestHsWebServicesJobs.java
	hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-hs/src/test/java/org/apache/hadoop/mapreduce/v2/hs/webapp/TestHsWebServicesJobsQuery.java
	hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-hs/src/test/java/org/apache/hadoop/mapreduce/v2/hs/webapp/TestHsWebServicesTasks.java
 2016-07-15 11:36:04 -07:00
Wangda Tan
863bfa4d6a YARN-1942. Deprecate toString/fromString methods from ConverterUtils and move them to records classes like ContainerId/ApplicationId, etc. (wangda) 2016-06-14 15:12:00 -07:00
Junping Du
2703ec6871 YARN-5076. YARN web interfaces lack XFS protection. Contributed by Jonathan Maron. 2016-05-19 14:10:15 -07:00
Junping Du
7e36b9159e MAPREDUCE-6657. Job history server can fail on startup when NameNode is in start phase. Contributed by Haibo Chen. 
(cherry picked from commit f6ef876fe158a5334cad7075f1966573a1c4dec9)
 2016-05-17 14:42:11 -07:00
Jason Lowe
9c47862d87 MAPREDUCE-6698. Increase timeout on TestUnnecessaryBlockingOnHistoryFileInfo.testTwoThreadsQueryingDifferentJobOfSameUser. Contributed by Haibo Chen 
(cherry picked from commit 34fddd1e912dcd9f898e500b8df11295ee49c0d8)
 2016-05-17 15:04:02 +00:00
Jason Lowe
51e06c085f MAPREDUCE-6701. application master log can not be available when clicking jobhistory's am logs link. Contributed by Haibo Chen 
(cherry picked from commit 12fa4ec141f8d0cb105b6f60c3739a4b26890fd5)
 2016-05-17 14:56:23 +00:00
Jason Lowe
d3dfed3fd0 MAPREDUCE-6684. High contention on scanning of user directory under immediate_done in Job History Server. Contributed by Haibo Chen 
(cherry picked from commit 5ffb54694b52657f3b7de4560474ab740734e1b2)
 2016-05-10 16:06:26 +00:00
Jian He
70a31458e0 MAPREDUCE-6680. JHS UserLogDir scan algorithm sometime could skip directory with update in CloudFS (Azure FileSystem, S3, etc. Contributed by Junping Du 
(cherry picked from commit 1e48eefe5800975ea0c4295c9911ae3f572ed37d)
 2016-04-20 19:02:58 -07:00
Eric Payne
edb096eab7 MAPREDUCE-6649. getFailureInfo not returning any failure info. Contributed by Eric Badger 2016-04-16 22:22:42 +00:00
Junping Du
5574f82799 MAPREDUCE-6670. TestJobListCache#testEviction sometimes fails on Windows with timeout. Contributed by Gergely Novák. 
(cherry picked from commit de96d7c88a42cd54bd88ce2de63122998e967efa)
 2016-04-06 08:35:51 -07:00
Rohith Sharma K S
f22ce96b50 MAPREDUCE-6662. Clear ASF Warnings on test data files. Contributed by Vinayakumar B 
(cherry picked from commit 0ef8bbfd8791899c7cfed3dd9c1670182fd87575)
 2016-03-29 10:35:04 +08:00
Varun Vasudev
e9a0ffc7f1 YARN-4737. Add CSRF filter support in YARN. Contributed by Jonathan Maron. 
(cherry picked from commit 43416187c07afb35e3267f94d0a41d8d3cfb5735)
 2016-03-07 15:23:36 +05:30
Robert Kanter
48cb4b9ba7 MAPREDUCE-6622. Add capability to set JHS job cache to a task-based limit (rchiang via rkanter) 
(cherry picked from commit 0f72da7e281376f4fcbfbf3fb33f5d7fedcdb1aa)
 2016-02-26 17:57:45 -08:00
Robert Kanter
ec48848711 YARN-4701. When task logs are not available, port 8041 is referenced instead of port 8042 (haibochen via rkanter) 
(cherry picked from commit c4d4df8de09ee0c89ea8176bd8149900becd3c0c)
 2016-02-25 12:48:29 -08:00
Robert Kanter
d9f01f28a0 MAPREDUCE-6640. mapred job -history command should be able to take Job ID (rkanter) 
(cherry picked from commit 4343a4cf7790b47dc7ecd18f8634e6b9c805e775)
 2016-02-23 17:38:25 -08:00
Varun Vasudev
212c519ad3 MAPREDUCE-6634. Log uncaught exceptions/errors in various thread pools in mapreduce. Contributed by Sidharta Seethana. 
(cherry picked from commit c50d1e54000c76880a041ce5959a2eb23c86bd35)
 2016-02-18 14:19:01 +05:30
Robert Kanter
358c49d830 MAPREDUCE-6620. Jobs that did not start are shown as starting in 1969 in the JHS web UI (haibochen via rkanter) 
(cherry picked from commit ccbba4a4deb8ade54a04137c993526e461bcb46e)
 2016-02-02 15:21:51 -08:00
Akira Ajisaka
7f215ffdd5 MAPREDUCE-6605. Fix typos mapreduce.map.skip.proc.count.autoincr and mapreduce.reduce.skip.proc.count.autoincr in mapred-default.xml. Contributed by Kai Sasaki. 2016-01-22 18:37:17 +09:00
Karthik Kambatla
ef59521094 YARN-4526. Make SystemClock singleton so AppSchedulingInfo could use it. (kasha) 
(cherry picked from commit d40859fab1ad977636457a6cc96b6a4f9b903afc)

Conflicts:
	hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-app/src/test/java/org/apache/hadoop/mapred/TestTaskAttemptListenerImpl.java
 2016-01-18 10:59:34 +01:00
Rohith Sharma K S
2d0348c907 MAPREDUCE-6419. JobHistoryServer doesn't sort properly based on Job ID when Job id's exceed 9999. (Mohammad Shahid Khan via rohithsharmaks) 
(cherry picked from commit a308e86621b1a7692fe1004ed14c919fdaba03cd)
 2015-12-24 12:00:37 +05:30
Zhihai Xu
da7bde60c0 MAPREDUCE-6436. JobHistory cache issue. Contributed by Kai Sasaki 
(cherry picked from commit 5b7078d06921893200163a3d29c8901c3c0107cb)
 2015-12-15 01:03:10 -08:00
Steve Loughran
af14458de7 HADOOP-12321. Make JvmPauseMonitor an AbstractService. (Sunil G via Stevel) [includes HDFS-8947 MAPREDUCE-6462 and YARN-4072] 2015-12-06 17:42:56 +00:00
Vinod Kumar Vavilapalli
b20e77eba3 Preparing for 2.9.0 development: mvn versions:set -DnewVersion=2.9.0 2015-11-25 17:47:22 -08:00
Akira Ajisaka
8de7911eb3 MAPREDUCE-6499. Add elapsed time for retired job in JobHistoryServer WebUI. Contributed by Lin Yiqun. 
(cherry picked from commit a7f776ffdfc9f96d3091d99a2a87157b290cbfe0)
 2015-11-17 15:18:32 +09:00
Jason Lowe
a7d8895b60 MAPREDUCE-6528. Memory leak for HistoryFileManager.getJobSummary(). Contributed by Junping Du 
(cherry picked from commit 6344b6a7694c70f296392b6462dba452ff762109)
 2015-10-30 15:32:48 +00:00
Jason Lowe
145058003a MAPREDUCE-6515. Update Application priority in AM side from AM-RM heartbeat. Contributed by Sunil G 
(cherry picked from commit cf953b6258b036fc482456b4591cfb98054f48f2)
 2015-10-29 18:05:56 +00:00
Haohui Mai
ee04add696 HDFS-9167. Update pom.xml in other modules to depend on hdfs-client instead of hdfs. Contributed by Mingliang Liu. 2015-10-13 15:24:29 -07:00
Junping Du
c8469357ba MAPREDUCE-6443. Add JvmPauseMonitor to JobHistoryServer. Contributed by Robert Kanter. 2015-08-06 07:00:42 -07:00
Jason Lowe
508da4d461 MAPREDUCE-6394. Speed up Task processing loop in HsTasksBlock#render(). Contributed by Ray Chiang 
(cherry picked from commit d0e0ba8010b72f58ddede5303f2b88404263d4bf)
 2015-07-31 18:19:52 +00:00
Vinod Kumar Vavilapalli
8c9e2cf10a MAPREDUCE-6410. Fixed MapReduce JobHistory server to use the right (login) UGI to refresh log and cleaner settings. Contributed by Varun Saxena. 
(cherry picked from commit d481684c7c9293a94f54ef622a92753531c6acc7)
 2015-06-23 11:41:05 -07:00
Gera Shegalov
ee63c3e831 MAPREDUCE-6305. AM/Task log page should be able to link back to the job. (Siqi Li via gera) 
(cherry picked from commit 055cd5a9a11bd3e544a8a67c6c44b53efcb3a9ed)
 2015-06-20 12:40:59 -07:00
Gera Shegalov
34e744ce1a MAPREDUCE-6316. Task Attempt List entries should link to the task overview. (Siqi Li via gera) 
(cherry picked from commit 63d40d528654707b3f56619a4eb20e21cdeadc7e)
 2015-06-20 11:52:11 -07:00
Devaraj K
7f075f2b75 MAPREDUCE-6377. JHS sorting on state column not working in webUi. 
Contributed by zhihai xu.

(cherry picked from commit 790a861766dcd60212d83f444f2f96d3acf20a94)

Conflicts:
	hadoop-mapreduce-project/CHANGES.txt
 2015-06-05 16:03:02 +05:30
Gera Shegalov
bae1ec5f29 MAPREDUCE-6382. Don't escape HTML links in Diagnostics in JHS job overview. (Siqi Li via gera) 
(cherry picked from commit e8302071ff16247a3d0cfc2dcca02da1bbf4db18)
 2015-06-03 23:52:16 -07:00
Devaraj K
b9865cbf3d MAPREDUCE-6273. HistoryFileManager should check whether summaryFile exists 
to avoid FileNotFoundException causing HistoryFileInfo into MOVE_FAILED
state. Contributed by zhihai xu.

(cherry picked from commit 1ea90144d58443a7431ff33ba49ee19278ebe42b)
 2015-05-15 12:34:08 +05:30
Jason Lowe
53e9973afc MAPREDUCE-5465. Tasks are often killed before they exit on their own. Contributed by Ming Ma 2015-05-11 22:40:05 +00:00
Junping Du
a1a7deebf8 YARN-3587. Fix the javadoc of DelegationTokenSecretManager in yarn, etc. projects. Contributed by Gabor Liptak. 
(cherry picked from commit 7e543c27fa2881aa65967be384a6203bd5b2304f)
 2015-05-11 05:28:25 -07:00
Devaraj K
ade5165ecf MAPREDUCE-6342. Make POM project names consistent. Contributed by Rohith. 2015-05-08 15:20:35 +05:30
Akira Ajisaka
f7c011cbaa MAPREDUCE-6356. Misspelling of threshold in log4j.properties for tests. Contributed by Brahma Reddy Battula. 
(cherry picked from commit d335071398fdb14153c854b740814b4fd385b658)
 2015-05-07 19:41:44 +09:00
Devaraj K
84ae26573c MAPREDUCE-6252. JobHistoryServer should not fail when encountering a 
missing directory. Contributed by Craig Welch.

(cherry picked from commit 5e67c4d384193b38a85655c8f93193596821faa5)
 2015-04-27 15:02:44 +05:30
Tsuyoshi Ozawa
542d53a268 MAPREDUCE-6330. Fix typo in some Task Attempt API's URL. Contributed by Ryu Kobayashi. 
(cherry picked from commit e33594dd87f83953b59b7416c82992de760eb4cd)
 2015-04-23 03:22:02 +09:00
Gera Shegalov
6f06e3b7ea MAPREDUCE-6297. Task Id of the failed task in diagnostics should link to the task page. (Siqi Li via gera) 
(cherry picked from commit 89ded89e86e5d9a634d92a5d8a7c889744d97f94)
 2015-04-21 14:21:57 -07:00
Akira Ajisaka
51c458a5c4 MAPREDUCE-6300. Task list sort by task id broken. Contributed by Siqi Li. 
(cherry picked from commit 1fa8075794894b3e0053d251788277fb88e85533)
 2015-04-16 10:06:03 -05:00
Akira Ajisaka
4801b0b478 Revert "MAPREDUCE-6300. Task list sort by task id broken. Contributed by Siqi Li." 
This reverts commit 3de0bf9a35094f3c2ac216992d861729251b6a3d.
 2015-04-11 12:37:44 +09:00
Akira Ajisaka
3de0bf9a35 MAPREDUCE-6300. Task list sort by task id broken. Contributed by Siqi Li. 
(cherry picked from commit 4cec9975c0e2c2fcca090888dc2c5e149c5be1e9)
 2015-04-11 01:31:44 +09:00
Akira Ajisaka
f603b9ccca MAPREDUCE-6307. Remove property mapreduce.tasktracker.taskmemorymanager.monitoringinterval. Contributed by J.Andreina. 
(cherry picked from commit 83979e61a49daa67a05eed1ba64c24d31a41610a)

Conflicts:
	hadoop-mapreduce-project/CHANGES.txt
 2015-04-10 20:23:46 +09:00
Robert Kanter
0fdd5c23c3 MAPREDUCE-6266. Job#getTrackingURL should consistently return a proper URL (rchiang via rkanter) 
(cherry picked from commit c4986b2d00d327f18d0c7e2f8805b69a4e07a19b)
 2015-04-09 13:49:03 -07:00
Vinod Kumar Vavilapalli
defae1d399 Revert "MAPREDUCE-6288. Changed permissions on JobHistory server's done directory so that user's client can load the conf files directly. Contributed by Robert Kanter." 
This reverts commit 5358b83167777a7108b32c9900fb0d01ca0fe961.

(cherry picked from commit cc0a01c50308fe599b4bdd5a80068d6244062e4f)
 2015-03-30 14:52:36 -07:00