Apache/hadoop
1
0
Fork 0
Code Issues Pull Requests Packages Projects Releases Wiki Activity
10,322 Commits 326 Branches 371 Tags
Commit Graph

1386 Commits

Author SHA1 Message Date
Jian He
a4ff03b200 YARN-3840. Resource Manager web ui issue when sorting application by id (with application having id > 9999) Contributed by Mohammad Shahid Khan 
(cherry picked from commit 8fbea531d7f7b665f6f55af54c8ebf330118ff37)
 2015-11-09 11:05:12 -08:00
Tsuyoshi Ozawa
1c0c77e7f0 MAPREDUCE-5763. Warn message about httpshuffle in NM logs. Contributed by Akira AJISAKA. 
(cherry picked from commit 668e897538e317d96784f75b4e1f979c72225db7)
 2015-11-09 15:26:02 +09:00
Jason Lowe
a7d8895b60 MAPREDUCE-6528. Memory leak for HistoryFileManager.getJobSummary(). Contributed by Junping Du 
(cherry picked from commit 6344b6a7694c70f296392b6462dba452ff762109)
 2015-10-30 15:32:48 +00:00
Tsuyoshi Ozawa
1c80473a41 YARN-4320. TestJobHistoryEventHandler fails as AHS in MiniYarnCluster no longer binds to default port 8188. Contributed by Varun Saxena. 
(cherry picked from commit ce31b22739512804da38cf87e0ce1059e3128da3)
 2015-10-30 18:00:35 +09:00
Xuan
b505bf9e58 YARN-4313. Race condition in MiniMRYarnCluster when getting history 
server address. Contributed by Jian He

(cherry picked from commit 7412ff48eeb967c972c19c1370c77a41c5b3b81f)
 2015-10-29 17:37:24 -07:00
Jason Lowe
145058003a MAPREDUCE-6515. Update Application priority in AM side from AM-RM heartbeat. Contributed by Sunil G 
(cherry picked from commit cf953b6258b036fc482456b4591cfb98054f48f2)
 2015-10-29 18:05:56 +00:00
Tsuyoshi Ozawa
3138b43f16 YARN-3573. MiniMRYarnCluster constructor that starts the timeline server using a boolean should be marked deprecated. Contributed by Brahma Reddy Battula. 
(cherry picked from commit 96677bef00b03057038157efeb3c2ad4702914da)
 2015-10-27 12:35:20 +09:00
Junping Du
801ce47e28 MAPREDUCE-6508. TestNetworkedJob fails consistently due to delegation token changes on RM. Contributed by Akira AJISAKA 
(cherry picked from commit eb6379ca25e1bb6d3978bd3a021723c38c95bec9)
 2015-10-23 10:07:41 -07:00
Jason Lowe
7c002e2416 MAPREDUCE-6518. Set SO_KEEPALIVE on shuffle connections. Contributed by Chang Li 
(cherry picked from commit ee9b80acf84718b952cbb82380cd0a4dbe22dc79)
 2015-10-21 21:45:51 +00:00
Jason Lowe
be72ed1c75 MAPREDUCE-6489. Fail fast rogue tasks that write too much to local disk. Contributed by Maysam Yabandeh 2015-10-21 14:11:48 +00:00
Robert Kanter
492142097b MAPREDUCE-6495. Docs for archive-logs tool (rkanter) 
(cherry picked from commit 0c4af0f99811a7138954391df3761aef9ff09155)
 2015-10-20 17:34:56 -07:00
Sangjin Lee
180efe6677 YARN-2556. Tool to measure the performance of the timeline server (Chang Li via sjlee) 
(cherry picked from commit 58590fef49bf45fc97c81277560e08da6b753f95)
 2015-10-16 16:57:32 -07:00
Haohui Mai
ee04add696 HDFS-9167. Update pom.xml in other modules to depend on hdfs-client instead of hdfs. Contributed by Mingliang Liu. 2015-10-13 15:24:29 -07:00
Karthik Kambatla
2d8f9e3fd9 MAPREDUCE-6302. Incorrect headroom can lead to a deadlock between map and reduce allocations. (kasha) 
(cherry picked from commit 4aa9b3e75ca86917125e56e1b438668273a5d87f)

Conflicts:
	hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-app/src/main/java/org/apache/hadoop/mapreduce/v2/app/rm/RMContainerAllocator.java
 2015-10-09 07:54:47 -07:00
Jason Lowe
0139517811 YARN-261. Ability to fail AM attempts. Contributed by Andrey Klochkov and Rohith Sharma K S 
(cherry picked from commit a0bca2b5ad2344fda5411d910a3730c85f12a0df)
 2015-10-09 14:19:41 +00:00
Akira Ajisaka
22ff37f5ce MAPREDUCE-6479. Add missing mapred job command options in mapreduce document. Contributed by nijel. 
(cherry picked from commit 584cf3bb35b96749f711b9e7145cf3fb71897cd9)
 2015-10-09 10:58:15 +09:00
Xuan
53bddc410f YARN-1897. CLI and core support for signal container functionality. Contributed by Ming Ma 
(cherry picked from commit 8f08532bde153811368e1b8336446fba4743f9d2)
 2015-10-02 18:53:04 -07:00
rohithsharmaks
7dd9ebdebc MAPREDUCE-6485. Create a new task attempt with failed map task priority if in-progress attempts are unassigned. (Xianyin Xin via rohithsharmaks) 
(cherry picked from commit 439f43ad3defbac907eda2d139a793f153544430)
 2015-10-02 20:36:20 +05:30
Akira Ajisaka
e42f576232 MAPREDUCE-6497. Fix wrong value of JOB_FINISHED event in JobHistoryEventHandler. Contributed by Shinichi Yamashita. 
(cherry picked from commit 195793c6f3e53a5c0527020477fe9c9158576f77)
 2015-10-01 10:26:00 +02:00
Jason Lowe
3c4c52a095 MAPREDUCE-6492. AsyncDispatcher exit with NPE on TaskAttemptImpl#sendJHStartEventForAssignedFailTask. Contributed by Bibin A Chundatt 
(cherry picked from commit ab11085b81353e1617875deb10f3c8e2a8b91a1e)
 2015-09-28 22:13:03 +00:00
Jian He
d458805690 YARN-1651. CapacityScheduler side changes to support container resize. Contributed by Wangda Tan 
(cherry picked from commit 89cab1ba5f0671f8ef30dbe7432079c18362b434)

Conflicts:
	hadoop-tools/hadoop-sls/src/main/java/org/apache/hadoop/yarn/sls/scheduler/SLSCapacityScheduler.java
 2015-09-24 10:30:04 -07:00
Jian He
c19e8ffdc7 YARN-1449. AM-NM protocol changes to support container resizing. Contributed by Meng Ding & Wangda Tan) 
(cherry picked from commit 83a18add10ee937a04e833a66e0a4642e776e510)
 2015-09-24 10:23:25 -07:00
Jian He
89dc37450c YARN-3866. AM-RM protocol changes to support container resizing. Contributed by Meng Ding 
(cherry picked from commit dfe2cb849fcb273d32e840d8562eaaee904af054)
 2015-09-24 10:23:13 -07:00
Zhihai Xu
e5ba1a0521 MAPREDUCE-6484. Yarn Client uses local address instead of RM address as token renewer in a secure cluster when RM HA is enabled. Contributed by Zhihai Xu 
(cherry picked from commit 97a08807ec328a6f455d941fe3d9a469d885caa6)
 2015-09-24 08:14:11 -07:00
Tsuyoshi Ozawa
52cc2ca488 HADOOP-12428. Fix inconsistency between log-level guards and statements. Contributed by Jagadesh Kiran N and Jackie Chang. 
(cherry picked from commit dfd807afab0fae3839c9cc5d552aa0304444f956)
 2015-09-22 12:54:44 +09:00
Zhihai Xu
88a3a07636 MAPREDUCE-6460. TestRMContainerAllocator.testAttemptNotFoundCausesRMCommunicatorException fails. Contributed by Zhihai Xu. 
(cherry picked from commit e3ace31e2b7e3a8dc6cf3dc044d5ef2a9aa12e29)
 2015-09-19 00:12:15 -07:00
Haohui Mai
0971503522 MAPREDUCE-6483. Replace deprecated method NameNode.getUri() with DFSUtilClient.getNNUri() in TestMRCredentials. Contributed by Mingliang Liu. 2015-09-18 13:36:22 -07:00
Wangda Tan
6041fc75d8 MAPREDUCE-6478. Add an option to skip cleanupJob stage or ignore cleanup failure during commitJob. (Junping Du via wangda) 2015-09-18 10:19:32 -07:00
Jason Lowe
f27f1ef44c MAPREDUCE-5982. Task attempts that fail from the ASSIGNED state can disappear. Contributed by Chang Li 
(cherry picked from commit ee4ee6af6a5a6299d27462adb6944206039bbbae)
 2015-09-17 21:38:44 +00:00
Jason Lowe
4861fadb17 MAPREDUCE-5002. AM could potentially allocate a reduce container to a map attempt. Contributed by Chang Li 
(cherry picked from commit 3f82f582e51c514cc0f052c828c2f58c6e8927ad)
 2015-09-17 18:18:28 +00:00
Jason Lowe
316296e35f MAPREDUCE-6481. LineRecordReader may give incomplete record and wrong position/key information for uncompressed input sometimes. Contributed by Zhihai Xu 
(cherry picked from commit 58d1a02b8d66b1d2a6ac2158be32bd35ad2e69bd)
 2015-09-17 14:31:57 +00:00
Haohui Mai
ac2ed09be7 MAPREDUCE-6477. Replace usage of deprecated NameNode.DEFAULT_PORT in TestFileSystem. Contributed by Mingliang Liu. 2015-09-16 13:06:38 -07:00
Jason Lowe
52f56524d4 MAPREDUCE-6472. MapReduce AM should have java.io.tmpdir=./tmp to be consistent with tasks. Contributed by Naganarasimha G R 
(cherry picked from commit 62943b8e3aff3b274c439f72a8bb86094c1ab0e8)
 2015-09-15 19:44:17 +00:00
Jason Lowe
7909462c3a MAPREDUCE-6474. ShuffleHandler can possibly exhaust nodemanager file descriptors. Contributed by Kuhu Shukla 
(cherry picked from commit 8e615588d5216394d0251a9c97bd706537856c6d)
 2015-09-10 16:01:27 +00:00
Tsuyoshi Ozawa
8bf5362014 MAPREDUCE-6442. Stack trace is missing when error occurs in client protocol provider's constructor Contributed by Chang Li. 
(cherry picked from commit 9b685773ec1031e90cc2b8aedb42670721757e22)
 2015-09-05 11:21:35 +09:00
Andrew Wang
355398e441 HADOOP-12358. Add -safely flag to rm to prompt when deleting many files. Contributed by Xiaoyu Yao. 
(cherry picked from commit e1feaf6db03451068c660a863926032b35a569f8)
 2015-09-04 13:48:42 -07:00
Akira Ajisaka
a61c0f5ca1 Revert "MAPREDUCE-6363. [NNBench] Lease mismatch error when running with multiple mappers. Contributed by Brahma Reddy Battula." 
This reverts commit 98446ee695f922ab079654bbc6d613ab334e371f.
 2015-09-02 12:26:25 +09:00
Zhihai Xu
297bda600e MAPREDUCE-6452. NPE when intermediate encrypt enabled for LocalRunner. Contributed by Zhihai Xu 
(cherry picked from commit cbb249534aa72ff6c290c4f99766415aeea9d6f5)
 2015-08-28 12:20:49 -07:00
Jian He
387076894f YARN-4014. Support user cli interface in for Application Priority. Contributed by Rohith Sharma K S 
(cherry picked from commit 57c7ae1affb2e1821fbdc3f47738d7e6fd83c7c1)
 2015-08-24 20:37:18 -07:00
Akira Ajisaka
29819e7240 MAPREDUCE-6357. MultipleOutputs.write() API should document that output committing is not utilized when input path is absolute. Contributed by Dustin Cote. 
(cherry picked from commit 2ba90c93d71aa2d30ee9ed431750c10c685e5599)
 2015-08-21 10:43:05 +09:00
Vinod Kumar Vavilapalli
9da81a3955 MAPREDUCE-6454. Fixed MapReduce to modify HADOOP_CLASSPATH to have distributed cache files so that child processes running hadoop scripts can access these files. Contributed by Junping Du. 2015-08-20 17:35:24 -07:00
Karthik Kambatla
20734320a1 MAPREDUCE-6439. AM may fail instead of retrying if RM shuts down during the allocate call. (Anubhav Dhoot via kasha) 
(cherry picked from commit 8dfec7a1979e8f70f8355c096874921d368342ef)
 2015-08-15 00:53:13 -07:00
Karthik Kambatla
b826168173 MAPREDUCE-5817. Mappers get rescheduled on node transition even after all reducers are completed. (Sangjin Lee via kasha) 
(cherry picked from commit 27d24f96ab8d17e839a1ef0d7076efc78d28724a)
 2015-08-14 12:40:38 -07:00
Junping Du
c8469357ba MAPREDUCE-6443. Add JvmPauseMonitor to JobHistoryServer. Contributed by Robert Kanter. 2015-08-06 07:00:42 -07:00
Colin Patrick Mccabe
07a042aa3c HADOOP-7824. NativeIO.java flags and identifiers must be set correctly for each platform, not hardcoded to their Linux values (Martin Walsh via Colin P. McCabe) 
(cherry picked from commit 21d10ccc6e463cf250414264c78acb4a6e7c83e3)
 2015-07-31 15:02:01 -07:00
Jason Lowe
508da4d461 MAPREDUCE-6394. Speed up Task processing loop in HsTasksBlock#render(). Contributed by Ray Chiang 
(cherry picked from commit d0e0ba8010b72f58ddede5303f2b88404263d4bf)
 2015-07-31 18:19:52 +00:00
Zhihai Xu
9d40eead81 MAPREDUCE-6433. launchTime may be negative. Contributed by Zhihai Xu 
(cherry picked from commit 93d50b782494af7eef980c4d596a59ff4e11646e)
 2015-07-30 23:12:30 -07:00
Akira Ajisaka
4da8ff60ef MAPREDUCE-5762. Port MAPREDUCE-3223 and MAPREDUCE-4695 (Remove MRv1 config from mapred-default.xml) to branch-2. (aajisaka) 2015-07-16 03:43:49 +09:00
Chris Douglas
2a0e551cd8 MAPREDUCE-6427. Fix typo in JobHistoryEventHandler. Contributed by Ray Chiang 2015-07-14 14:56:21 -07:00
Akira Ajisaka
0ba2defb9e YARN-3381. Fix typo InvalidStateTransitonException. Contributed by Brahma Reddy Battula. 
(cherry picked from commit 19295b36d90e26616accee73b1f7743aab5df692)
 2015-07-13 17:53:27 +09:00