Apache/hadoop
1
0
Fork 0
Code Issues Pull Requests Packages Projects Releases Wiki Activity
10,452 Commits 326 Branches 371 Tags
Commit Graph

1398 Commits

Author SHA1 Message Date
Jian He
387076894f YARN-4014. Support user cli interface in for Application Priority. Contributed by Rohith Sharma K S 
(cherry picked from commit 57c7ae1affb2e1821fbdc3f47738d7e6fd83c7c1)
 2015-08-24 20:37:18 -07:00
Akira Ajisaka
29819e7240 MAPREDUCE-6357. MultipleOutputs.write() API should document that output committing is not utilized when input path is absolute. Contributed by Dustin Cote. 
(cherry picked from commit 2ba90c93d71aa2d30ee9ed431750c10c685e5599)
 2015-08-21 10:43:05 +09:00
Vinod Kumar Vavilapalli
9da81a3955 MAPREDUCE-6454. Fixed MapReduce to modify HADOOP_CLASSPATH to have distributed cache files so that child processes running hadoop scripts can access these files. Contributed by Junping Du. 2015-08-20 17:35:24 -07:00
Karthik Kambatla
20734320a1 MAPREDUCE-6439. AM may fail instead of retrying if RM shuts down during the allocate call. (Anubhav Dhoot via kasha) 
(cherry picked from commit 8dfec7a1979e8f70f8355c096874921d368342ef)
 2015-08-15 00:53:13 -07:00
Karthik Kambatla
b826168173 MAPREDUCE-5817. Mappers get rescheduled on node transition even after all reducers are completed. (Sangjin Lee via kasha) 
(cherry picked from commit 27d24f96ab8d17e839a1ef0d7076efc78d28724a)
 2015-08-14 12:40:38 -07:00
Junping Du
c8469357ba MAPREDUCE-6443. Add JvmPauseMonitor to JobHistoryServer. Contributed by Robert Kanter. 2015-08-06 07:00:42 -07:00
Colin Patrick Mccabe
07a042aa3c HADOOP-7824. NativeIO.java flags and identifiers must be set correctly for each platform, not hardcoded to their Linux values (Martin Walsh via Colin P. McCabe) 
(cherry picked from commit 21d10ccc6e463cf250414264c78acb4a6e7c83e3)
 2015-07-31 15:02:01 -07:00
Jason Lowe
508da4d461 MAPREDUCE-6394. Speed up Task processing loop in HsTasksBlock#render(). Contributed by Ray Chiang 
(cherry picked from commit d0e0ba8010b72f58ddede5303f2b88404263d4bf)
 2015-07-31 18:19:52 +00:00
Zhihai Xu
9d40eead81 MAPREDUCE-6433. launchTime may be negative. Contributed by Zhihai Xu 
(cherry picked from commit 93d50b782494af7eef980c4d596a59ff4e11646e)
 2015-07-30 23:12:30 -07:00
Akira Ajisaka
4da8ff60ef MAPREDUCE-5762. Port MAPREDUCE-3223 and MAPREDUCE-4695 (Remove MRv1 config from mapred-default.xml) to branch-2. (aajisaka) 2015-07-16 03:43:49 +09:00
Chris Douglas
2a0e551cd8 MAPREDUCE-6427. Fix typo in JobHistoryEventHandler. Contributed by Ray Chiang 2015-07-14 14:56:21 -07:00
Akira Ajisaka
0ba2defb9e YARN-3381. Fix typo InvalidStateTransitonException. Contributed by Brahma Reddy Battula. 
(cherry picked from commit 19295b36d90e26616accee73b1f7743aab5df692)
 2015-07-13 17:53:27 +09:00
Devaraj K
3074d7082e MAPREDUCE-6426. TestShuffleHandler#testGetMapOutputInfo is failing. 
Contributed by zhihai xu.

(cherry picked from commit fffb15bb431fcdd7ca4ce0e249f9d45f4968497b)
 2015-07-09 15:02:09 +05:30
Chris Douglas
5fbf91b453 MAPREDUCE-6038. A boolean may be set error in the Word Count v2.0 in MapReduce Tutorial. Contributed by Tsuyoshi Ozawa 
(cherry picked from commit 4990ccfdc742be9c0517557a6256fefcae0df67d)
 2015-07-07 10:26:57 -07:00
Devaraj K
415dd64461 MAPREDUCE-6425. ShuffleHandler passes wrong "base" parameter to 
getMapOutputInfo if mapId is not in the cache. Contributed by zhihai xu.

(cherry picked from commit bff67dfe2f811654ffb1bbcbd87509c185f452b6)
 2015-07-06 13:47:54 +05:30
Jason Lowe
8a4970e845 MAPREDUCE-6418. MRApp should not shutdown LogManager during shutdown. Contributed by Chang Li 
(cherry picked from commit eac1d1894354e90d314087af8e7fb168ddef9a3d)
 2015-07-01 18:02:17 +00:00
Jason Lowe
f3195eda27 MAPREDUCE-6420. Interrupted Exception in LocalContainerLauncher should be logged in warn/info level. Contributed by Chang Li 
(cherry picked from commit 826715622e6937887a4e20b3ce327d7e2fd89009)
 2015-07-01 17:50:47 +00:00
Jason Lowe
df4e1e4965 MAPREDUCE-6376. Add avro binary support for jhist files. Contributed by Ray Chiang 
(cherry picked from commit 2ac87df578accb6e612f70ded76271cb5082ee10)

Conflicts:

	hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-core/src/main/resources/mapred-default.xml
 2015-07-01 16:07:54 +00:00
Robert Kanter
e5b0439bc1 MAPREDUCE-6121. JobResourceUpdater#compareFs() doesn't handle HA namespaces (rchiang via rkanter) 
(cherry picked from commit 3a72bfd08281fd271bef4f41289125d39c41928c)
 2015-06-30 16:50:48 -07:00
Jason Lowe
fdfd5be2b3 MAPREDUCE-6384. Add the last reporting reducer info for too many fetch failure diagnostics. Contributed by Chang Li 
(cherry picked from commit b6ba56457c6b01dae795c11d587c3fe3855ee707)
 2015-06-30 21:24:38 +00:00
Jason Lowe
31fd7a51d8 MAPREDUCE-6413. TestLocalJobSubmission is failing with unknown host. Contributed by zhihai xu 
(cherry picked from commit aa5b15b03be61ebb76a226e0de485d5228c8e3d0)
 2015-06-25 19:52:23 +00:00
Jason Lowe
2762affbb2 MAPREDUCE-6400. Multiple shuffle transfer fails because input is closed too early. Contributed by Brahma Reddy Battula, Akira AJISAKA, and Gera Shegalov. 
(cherry picked from commit 72d08a0e41efda635baa985d55d67cb059a7c07c)
 2015-06-24 15:30:09 +00:00
Vinod Kumar Vavilapalli
8c9e2cf10a MAPREDUCE-6410. Fixed MapReduce JobHistory server to use the right (login) UGI to refresh log and cleaner settings. Contributed by Varun Saxena. 
(cherry picked from commit d481684c7c9293a94f54ef622a92753531c6acc7)
 2015-06-23 11:41:05 -07:00
Jason Lowe
3bf44b862b MAPREDUCE-5948. org.apache.hadoop.mapred.LineRecordReader does not handle multibyte record delimiters well. Contributed by Vinayakumar B, Rushabh Shah, and Akira AJISAKA 
(cherry picked from commit 077250d8d7b4b757543a39a6ce8bb6e3be356c6f)
 2015-06-22 22:00:20 +00:00
Gera Shegalov
d8cc8f54dd MAPREDUCE-6408. Queue name and user name should be printed on the job page. (Siqi Li via gera) 
(cherry picked from commit 11ac84820746e43f1e7072e08f62ddf81713aeb2)
 2015-06-22 14:42:31 -07:00
Akira Ajisaka
c4747158b7 MAPREDUCE-6403. Fix typo in the usage of NNBench. Contributed by Jagadesh Kiran N. 
(cherry picked from commit 445b13240cb632ee8bc718f0576f783a7350af2e)
 2015-06-22 15:34:34 +09:00
Gera Shegalov
ee63c3e831 MAPREDUCE-6305. AM/Task log page should be able to link back to the job. (Siqi Li via gera) 
(cherry picked from commit 055cd5a9a11bd3e544a8a67c6c44b53efcb3a9ed)
 2015-06-20 12:40:59 -07:00
Gera Shegalov
34e744ce1a MAPREDUCE-6316. Task Attempt List entries should link to the task overview. (Siqi Li via gera) 
(cherry picked from commit 63d40d528654707b3f56619a4eb20e21cdeadc7e)
 2015-06-20 11:52:11 -07:00
Devaraj K
5c3070dc16 MAPREDUCE-6405. NullPointerException in App Attempts page. Contributed by 
Siqi Li and Gera Shegalov.

(cherry picked from commit 311a4179cc53a2754d1185b1cb10e6adb1b95d3d)
 2015-06-20 10:37:42 +05:30
Gera Shegalov
01991876ed MAPREDUCE-6395. Improve the commit failure messages in MRAppMaster recovery. (Brahma Reddy Battula via gera) 
(cherry picked from commit 10107243be66bae2212a2cd8575f9f5ade13fe9e)
 2015-06-19 02:29:31 -07:00
Devaraj K
c4c7fd0bf1 MAPREDUCE-6373. The logger reports total input paths but it is referring 
to input files. Contributed by Bibin A Chundatt.

(cherry picked from commit 1babe50a2cbaae3c8165229347e743d0dc94e979)
 2015-06-18 11:45:11 +05:30
Akira Ajisaka
98446ee695 MAPREDUCE-6363. [NNBench] Lease mismatch error when running with multiple mappers. Contributed by Brahma Reddy Battula. 
(cherry picked from commit 75a2560e51387ea31ef4609ef434475bbbc628f7)
 2015-06-15 14:53:28 -07:00
Devaraj K
5426e3e929 MAPREDUCE-6350. JobHistory doesn't support fully-functional search. 
Contributed by Siqi Li.
 2015-06-10 12:43:31 +05:30
cnauroth
5a002aef12 MAPREDUCE-6392. Document mapred class path options. Contributed by Brahma Reddy Battula. 
(cherry picked from commit 025a3a8be07ba2b0c27ee39034b506d91aa486fc)
 2015-06-08 16:22:12 -07:00
Tsuyoshi Ozawa
c6cdecc6b3 MAPREDUCE-6388. Remove deprecation warnings from JobHistoryServer classes. Contributed by Ray Chiang. 
(cherry picked from commit 6afe20a7a4bbfa30fce3e3c9873ad43201987998)
 2015-06-08 15:06:23 -07:00
Jason Lowe
588c22e5a0 MAPREDUCE-6354. ShuffleHandler should be able to log shuffle connections. Contributed by Chang Li 
(cherry picked from commit b3ffa870034d06608a1946e2d9ce7dbd535a2c53)
 2015-06-05 22:39:52 +00:00
Karthik Kambatla
1a2e6e8091 MAPREDUCE-6387. Serialize the recently added Task#encryptedSpillKey field at the end. (Arun Suresh via kasha) 
(cherry picked from commit 6786daab33792c9c426adf0fc6b361f193d356a8)
 2015-06-05 09:17:34 -07:00
Devaraj K
7f075f2b75 MAPREDUCE-6377. JHS sorting on state column not working in webUi. 
Contributed by zhihai xu.

(cherry picked from commit 790a861766dcd60212d83f444f2f96d3acf20a94)

Conflicts:
	hadoop-mapreduce-project/CHANGES.txt
 2015-06-05 16:03:02 +05:30
Vinod Kumar Vavilapalli
6044b5e5f6 MAPREDUCE-5232. Add a configuration to be able to log classpath and other system properties on mapreduce JVMs startup. Contributed by Sangjin Lee. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1482643 13f79535-47bb-0310-9956-ffa450edef68
(cherry picked from commit 4d8e350750748b919ee2158690a44cd9fd80dcae)

Conflicts:
	hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-app/src/main/java/org/apache/hadoop/mapreduce/v2/app/MRAppMaster.java
	hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-common/src/test/java/org/apache/hadoop/mapreduce/v2/util/TestMRApps.java
 2015-06-04 11:15:30 -07:00
Akira Ajisaka
1cccd1eeb3 HADOOP-12058. Fix dead links to DistCp and Hadoop Archives pages. Contributed by Kazuho Fujii. 
(cherry picked from commit dbed757cbaf3ddff0c9ae2d905d42828aadedf2a)
 2015-06-05 01:46:33 +09:00
Gera Shegalov
bae1ec5f29 MAPREDUCE-6382. Don't escape HTML links in Diagnostics in JHS job overview. (Siqi Li via gera) 
(cherry picked from commit e8302071ff16247a3d0cfc2dcca02da1bbf4db18)
 2015-06-03 23:52:16 -07:00
Gera Shegalov
25c1e54d3f MAPREDUCE-6174. Combine common stream code into parent class for InMemoryMapOutput and OnDiskMapOutput. (Eric Payne via gera) 
(cherry picked from commit d90c13e2da8867661bf19a802add70145ab9a462)
 2015-06-03 16:57:54 -07:00
Zhijie Shen
2fcd3fe6ba Revert "YARN-1462. Made RM write application tags to timeline server and exposed them to users via generic history web UI and REST API. Contributed by Xuan Gong." 
This reverts commit 4e2ed34d2023a7cd3842e41aba1995f5d0f53cf2.
 2015-06-03 14:16:51 -07:00
Jason Lowe
5ecc647ae0 MAPREDUCE-6374. Distributed Cache File visibility should check permission of full path. Contributed by Chang Li 
(cherry picked from commit 107da29ff9b6efd02b21c96df60c58ac51590ebd)
 2015-06-03 20:20:47 +00:00
Zhijie Shen
4e2ed34d20 YARN-1462. Made RM write application tags to timeline server and exposed them to users via generic history web UI and REST API. Contributed by Xuan Gong. 
(cherry picked from commit a9f986bc5a41ab74171da1076d10e83c5adc284f)
 2015-05-30 21:06:32 -07:00
Wangda Tan
7cd0874dfd MAPREDUCE-6304. Specifying node labels when submitting MR jobs. (Naganarasimha G R via wangda) 
(cherry picked from commit 3164e7d83875aa6b7435d1dfe61ac280aa277f1c)

Conflicts:
	hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-app/src/main/java/org/apache/hadoop/mapreduce/v2/app/rm/RMContainerAllocator.java
 2015-05-27 14:43:20 -07:00
cnauroth
6ed8989a6f YARN-3626. On Windows localized resources are not moved to the front of the classpath when they should be. Contributed by Craig Welch. 
(cherry picked from commit 7bc33b63c78c6ee9a04c0c1511ef16e63f7f3074)
 2015-05-27 14:21:15 -07:00
Colin Patrick Mccabe
3dec58dd78 HADOOP-11969. ThreadLocal initialization in several classes is not thread safe (Sean Busbey via Colin P. McCabe) 
(cherry picked from commit 7dba7005b79994106321b0f86bc8f4ea51a3c185)
 2015-05-26 12:40:41 -07:00
Tsuyoshi Ozawa
e694c3339e MAPREDUCE-6364. Add a Kill link to Task Attempts page. Contributed by Ryu Kobayashi. 
(cherry picked from commit 022f49d59e404678d0f322bb1b67208f3c17b7ef)
 2015-05-26 23:59:20 +09:00
Tsuyoshi Ozawa
6042b53a74 MAPREDUCE-6204. TestJobCounters should use new properties instead of JobConf.MAPRED_TASK_JAVA_OPTS. 
(cherry picked from commit b27ce3c5318a40bc79565783961f6675dcdcb14a)
 2015-05-22 23:30:12 +09:00