Apache/hadoop
1
0
Fork 0
Code Issues Pull Requests Packages Projects Releases Wiki Activity
11,977 Commits 326 Branches 371 Tags
Commit Graph

273 Commits

Author SHA1 Message Date
Junping Du
7706a63fb4 Preparing for 2.8.1 development after cutting off 2.8.0 branch. 2017-01-10 18:03:51 -08:00
Rohith Sharma K S
0c1615c368 MAPREDUCE-6662. Clear ASF Warnings on test data files. Contributed by Vinayakumar B 
(cherry picked from commit 0ef8bbfd8791899c7cfed3dd9c1670182fd87575)
(cherry picked from commit f22ce96b50f8148e95a453ff41f84b99a9197bf6)
 2017-01-05 18:03:48 +09:00
Akira Ajisaka
58a1753373 HADOOP-12733. Remove references to obsolete io.seqfile configuration variables. Contributed by Ray Chiang. 
(cherry picked from commit 01d31fe9389ccdc153d7f4bf6574bf8e509867c1)
(cherry picked from commit 3e2d26a97f83461d491180bbc6aa0ba926c0681b)
 2017-01-04 14:20:04 +09:00
Junping Du
5f36b62265 Revert "MAPREDUCE-5875. Make Counter limits consistent across JobClient, MRAppMaster, and YarnChild. (Gera Shegalov via kasha)" 
This reverts commit 7bfd9e068d205f752c05d51778a4a4702329d31e.

Conflicts:
	hadoop-mapreduce-project/CHANGES.txt
	hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-core/src/main/java/org/apache/hadoop/mapreduce/JobSubmitter.java
	hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-core/src/main/java/org/apache/hadoop/mapreduce/jobhistory/HistoryViewer.java
	hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-hs/src/main/java/org/apache/hadoop/mapreduce/v2/hs/CompletedJob.java
 2016-12-28 17:37:20 -08:00
Jason Lowe
5b1b825ac8 MAPREDUCE-6797. Job history server scans can become blocked on a single, slow entry. Contributed by Prabhu Joseph 
(cherry picked from commit 99c2bbd337942e4bc7b246a88dff53f98e530651)
 2016-11-14 20:31:38 +00:00
Varun Saxena
e714d82196 MAPREDUCE-6750. Fix TestHSAdminServer#testRefreshSuperUserGroups (Kihwal Lee via Varun Saxena) 2016-08-22 11:05:01 -05:00
Akira Ajisaka
3dcbd2df56 MAPREDUCE-6738. TestJobListCache.testAddExisting failed intermittently in slow VM testbed. Contributed by Junping Du. 
(cherry picked from commit ecff7d03595b45d7a5d6aed73b9931fef85a56ee)
(cherry picked from commit f4d94f2246da3a015a105cb7f55ec1ebc451e0e7)
 2016-07-21 11:38:26 -07:00
Wangda Tan
d838c6443d YARN-1942. Deprecate toString/fromString methods from ConverterUtils and move them to records classes like ContainerId/ApplicationId, etc. (wangda) 2016-06-14 15:21:41 -07:00
Jason Lowe
c983115e4c MAPREDUCE-6698. Increase timeout on TestUnnecessaryBlockingOnHistoryFileInfo.testTwoThreadsQueryingDifferentJobOfSameUser. Contributed by Haibo Chen 
(cherry picked from commit 34fddd1e912dcd9f898e500b8df11295ee49c0d8)
 2016-05-17 15:04:33 +00:00
Jason Lowe
da48923d3a MAPREDUCE-6684. High contention on scanning of user directory under immediate_done in Job History Server. Contributed by Haibo Chen 
(cherry picked from commit 5ffb54694b52657f3b7de4560474ab740734e1b2)

Conflicts:

	hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-hs/src/main/java/org/apache/hadoop/mapreduce/v2/hs/HistoryFileManager.java
 2016-05-10 16:16:32 +00:00
Jian He
0b008dc21c MAPREDUCE-6680. JHS UserLogDir scan algorithm sometime could skip directory with update in CloudFS (Azure FileSystem, S3, etc. Contributed by Junping Du 
(cherry picked from commit 1e48eefe5800975ea0c4295c9911ae3f572ed37d)
 2016-04-20 19:03:19 -07:00
Eric Payne
1e33c05a7e MAPREDUCE-6649. getFailureInfo not returning any failure info. Contributed by Eric Badger 2016-04-16 22:37:00 +00:00
Junping Du
80c151b889 MAPREDUCE-6670. TestJobListCache#testEviction sometimes fails on Windows with timeout. Contributed by Gergely Novák. 
(cherry picked from commit de96d7c88a42cd54bd88ce2de63122998e967efa)
(cherry picked from commit 5574f827991cfc3d14f71184296866d387ed174f)
 2016-04-13 08:58:24 -07:00
Robert Kanter
3d8ae9c577 MAPREDUCE-6622. Add capability to set JHS job cache to a task-based limit (rchiang via rkanter) 
(cherry picked from commit 0f72da7e281376f4fcbfbf3fb33f5d7fedcdb1aa)

Conflicts:
	hadoop-mapreduce-project/CHANGES.txt
 2016-03-06 19:26:12 -08:00
Akira Ajisaka
b85b8d90d1 MAPREDUCE-6605. Fix typos mapreduce.map.skip.proc.count.autoincr and mapreduce.reduce.skip.proc.count.autoincr in mapred-default.xml. Contributed by Kai Sasaki. 
(cherry picked from commit 7f215ffdd5219fc585ec484073b317a940895a60)
 2016-01-22 18:38:08 +09:00
Rohith Sharma K S
550840f402 MAPREDUCE-6419. JobHistoryServer doesn't sort properly based on Job ID when Job id's exceed 9999. (Mohammad Shahid Khan via rohithsharmaks) 
(cherry picked from commit a308e86621b1a7692fe1004ed14c919fdaba03cd)
 2015-12-24 12:02:53 +05:30
Zhihai Xu
c4febc7d02 MAPREDUCE-6436. JobHistory cache issue. Contributed by Kai Sasaki 
(cherry picked from commit 5b7078d06921893200163a3d29c8901c3c0107cb)
 2015-12-15 22:42:19 -08:00
Akira Ajisaka
8de7911eb3 MAPREDUCE-6499. Add elapsed time for retired job in JobHistoryServer WebUI. Contributed by Lin Yiqun. 
(cherry picked from commit a7f776ffdfc9f96d3091d99a2a87157b290cbfe0)
 2015-11-17 15:18:32 +09:00
Jason Lowe
a7d8895b60 MAPREDUCE-6528. Memory leak for HistoryFileManager.getJobSummary(). Contributed by Junping Du 
(cherry picked from commit 6344b6a7694c70f296392b6462dba452ff762109)
 2015-10-30 15:32:48 +00:00
Jason Lowe
145058003a MAPREDUCE-6515. Update Application priority in AM side from AM-RM heartbeat. Contributed by Sunil G 
(cherry picked from commit cf953b6258b036fc482456b4591cfb98054f48f2)
 2015-10-29 18:05:56 +00:00
Haohui Mai
ee04add696 HDFS-9167. Update pom.xml in other modules to depend on hdfs-client instead of hdfs. Contributed by Mingliang Liu. 2015-10-13 15:24:29 -07:00
Junping Du
c8469357ba MAPREDUCE-6443. Add JvmPauseMonitor to JobHistoryServer. Contributed by Robert Kanter. 2015-08-06 07:00:42 -07:00
Jason Lowe
508da4d461 MAPREDUCE-6394. Speed up Task processing loop in HsTasksBlock#render(). Contributed by Ray Chiang 
(cherry picked from commit d0e0ba8010b72f58ddede5303f2b88404263d4bf)
 2015-07-31 18:19:52 +00:00
Vinod Kumar Vavilapalli
8c9e2cf10a MAPREDUCE-6410. Fixed MapReduce JobHistory server to use the right (login) UGI to refresh log and cleaner settings. Contributed by Varun Saxena. 
(cherry picked from commit d481684c7c9293a94f54ef622a92753531c6acc7)
 2015-06-23 11:41:05 -07:00
Gera Shegalov
ee63c3e831 MAPREDUCE-6305. AM/Task log page should be able to link back to the job. (Siqi Li via gera) 
(cherry picked from commit 055cd5a9a11bd3e544a8a67c6c44b53efcb3a9ed)
 2015-06-20 12:40:59 -07:00
Gera Shegalov
34e744ce1a MAPREDUCE-6316. Task Attempt List entries should link to the task overview. (Siqi Li via gera) 
(cherry picked from commit 63d40d528654707b3f56619a4eb20e21cdeadc7e)
 2015-06-20 11:52:11 -07:00
Devaraj K
7f075f2b75 MAPREDUCE-6377. JHS sorting on state column not working in webUi. 
Contributed by zhihai xu.

(cherry picked from commit 790a861766dcd60212d83f444f2f96d3acf20a94)

Conflicts:
	hadoop-mapreduce-project/CHANGES.txt
 2015-06-05 16:03:02 +05:30
Gera Shegalov
bae1ec5f29 MAPREDUCE-6382. Don't escape HTML links in Diagnostics in JHS job overview. (Siqi Li via gera) 
(cherry picked from commit e8302071ff16247a3d0cfc2dcca02da1bbf4db18)
 2015-06-03 23:52:16 -07:00
Devaraj K
b9865cbf3d MAPREDUCE-6273. HistoryFileManager should check whether summaryFile exists 
to avoid FileNotFoundException causing HistoryFileInfo into MOVE_FAILED
state. Contributed by zhihai xu.

(cherry picked from commit 1ea90144d58443a7431ff33ba49ee19278ebe42b)
 2015-05-15 12:34:08 +05:30
Jason Lowe
53e9973afc MAPREDUCE-5465. Tasks are often killed before they exit on their own. Contributed by Ming Ma 2015-05-11 22:40:05 +00:00
Junping Du
a1a7deebf8 YARN-3587. Fix the javadoc of DelegationTokenSecretManager in yarn, etc. projects. Contributed by Gabor Liptak. 
(cherry picked from commit 7e543c27fa2881aa65967be384a6203bd5b2304f)
 2015-05-11 05:28:25 -07:00
Devaraj K
ade5165ecf MAPREDUCE-6342. Make POM project names consistent. Contributed by Rohith. 2015-05-08 15:20:35 +05:30
Akira Ajisaka
f7c011cbaa MAPREDUCE-6356. Misspelling of threshold in log4j.properties for tests. Contributed by Brahma Reddy Battula. 
(cherry picked from commit d335071398fdb14153c854b740814b4fd385b658)
 2015-05-07 19:41:44 +09:00
Devaraj K
84ae26573c MAPREDUCE-6252. JobHistoryServer should not fail when encountering a 
missing directory. Contributed by Craig Welch.

(cherry picked from commit 5e67c4d384193b38a85655c8f93193596821faa5)
 2015-04-27 15:02:44 +05:30
Tsuyoshi Ozawa
542d53a268 MAPREDUCE-6330. Fix typo in some Task Attempt API's URL. Contributed by Ryu Kobayashi. 
(cherry picked from commit e33594dd87f83953b59b7416c82992de760eb4cd)
 2015-04-23 03:22:02 +09:00
Gera Shegalov
6f06e3b7ea MAPREDUCE-6297. Task Id of the failed task in diagnostics should link to the task page. (Siqi Li via gera) 
(cherry picked from commit 89ded89e86e5d9a634d92a5d8a7c889744d97f94)
 2015-04-21 14:21:57 -07:00
Akira Ajisaka
51c458a5c4 MAPREDUCE-6300. Task list sort by task id broken. Contributed by Siqi Li. 
(cherry picked from commit 1fa8075794894b3e0053d251788277fb88e85533)
 2015-04-16 10:06:03 -05:00
Akira Ajisaka
4801b0b478 Revert "MAPREDUCE-6300. Task list sort by task id broken. Contributed by Siqi Li." 
This reverts commit 3de0bf9a35094f3c2ac216992d861729251b6a3d.
 2015-04-11 12:37:44 +09:00
Akira Ajisaka
3de0bf9a35 MAPREDUCE-6300. Task list sort by task id broken. Contributed by Siqi Li. 
(cherry picked from commit 4cec9975c0e2c2fcca090888dc2c5e149c5be1e9)
 2015-04-11 01:31:44 +09:00
Akira Ajisaka
f603b9ccca MAPREDUCE-6307. Remove property mapreduce.tasktracker.taskmemorymanager.monitoringinterval. Contributed by J.Andreina. 
(cherry picked from commit 83979e61a49daa67a05eed1ba64c24d31a41610a)

Conflicts:
	hadoop-mapreduce-project/CHANGES.txt
 2015-04-10 20:23:46 +09:00
Robert Kanter
0fdd5c23c3 MAPREDUCE-6266. Job#getTrackingURL should consistently return a proper URL (rchiang via rkanter) 
(cherry picked from commit c4986b2d00d327f18d0c7e2f8805b69a4e07a19b)
 2015-04-09 13:49:03 -07:00
Vinod Kumar Vavilapalli
defae1d399 Revert "MAPREDUCE-6288. Changed permissions on JobHistory server's done directory so that user's client can load the conf files directly. Contributed by Robert Kanter." 
This reverts commit 5358b83167777a7108b32c9900fb0d01ca0fe961.

(cherry picked from commit cc0a01c50308fe599b4bdd5a80068d6244062e4f)
 2015-03-30 14:52:36 -07:00
Vinod Kumar Vavilapalli
cc130a033a MAPREDUCE-6288. Changed permissions on JobHistory server's done directory so that user's client can load the conf files directly. Contributed by Robert Kanter. 
(cherry picked from commit 5358b83167777a7108b32c9900fb0d01ca0fe961)
 2015-03-30 10:28:28 -07:00
Robert Kanter
308460654a MAPREDUCE-6282. Reuse historyFileAbsolute.getFileSystem in CompletedJob#loadFullHistoryData for code optimization. (zxu via rkanter) 
(cherry picked from commit d4f7e2507f4bb024444d172f94e74431bc2f319c)
 2015-03-20 13:12:34 -07:00
Haohui Mai
e75e6c66ea HADOOP-11633. Convert remaining branch-2 .apt.vm files to markdown. Contributed by Masatake Iwasaki. 2015-03-11 14:23:44 -07:00
Vinod Kumar Vavilapalli
d9416317a9 Preparing for 2.8.0 development. Changing SNAPSHOT version to be 2.8 on branch-2. 2015-03-08 20:32:51 -07:00
Tsuyoshi Ozawa
b1e7f9c588 MAPREDUCE-6268. Fix typo in Task Attempt API's URL. Contributed by Ryu Kobayashi. 
(cherry picked from commit 742f9d90c00f823ad7fea7e79702fcf238fa5721)

Conflicts:
	hadoop-mapreduce-project/CHANGES.txt
	hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-hs/src/site/markdown/HistoryServerRest.md
 2015-03-03 16:24:27 +09:00
Konstantin V Shvachko
9e67f2cb0d YARN-3255. RM, NM, JobHistoryServer, and WebAppProxyServer's main() should support generic options. Contributed by Konstantin Shvachko. 2015-02-26 17:26:08 -08:00
Devaraj K
a00c9362e6 MAPREDUCE-6253. Update use of Iterator to Iterable. Contributed by Ray 
Chiang.

(cherry picked from commit 76e309ead01f02b32335330cd920536f907fb71f)
 2015-02-12 13:47:19 +05:30
Jason Lowe
2dc4af2b93 MAPREDUCE-6059. Speed up history server startup time (Siqi Li via aw) 
(cherry picked from commit fd57ab2002f97dcc83d455a5e0c770c8efde77a4)

Conflicts:

	hadoop-mapreduce-project/CHANGES.txt
 2015-02-05 16:40:09 +00:00