Apache/hadoop
1
0
Fork 0
Code Issues Pull Requests Packages Projects Releases Wiki Activity
9,685 Commits 326 Branches 371 Tags
Commit Graph

1694 Commits

Author SHA1 Message Date
Jason Lowe
508da4d461 MAPREDUCE-6394. Speed up Task processing loop in HsTasksBlock#render(). Contributed by Ray Chiang 
(cherry picked from commit d0e0ba8010b72f58ddede5303f2b88404263d4bf)
 2015-07-31 18:19:52 +00:00
Zhihai Xu
9d40eead81 MAPREDUCE-6433. launchTime may be negative. Contributed by Zhihai Xu 
(cherry picked from commit 93d50b782494af7eef980c4d596a59ff4e11646e)
 2015-07-30 23:12:30 -07:00
Akira Ajisaka
4da8ff60ef MAPREDUCE-5762. Port MAPREDUCE-3223 and MAPREDUCE-4695 (Remove MRv1 config from mapred-default.xml) to branch-2. (aajisaka) 2015-07-16 03:43:49 +09:00
Chris Douglas
2a0e551cd8 MAPREDUCE-6427. Fix typo in JobHistoryEventHandler. Contributed by Ray Chiang 2015-07-14 14:56:21 -07:00
Akira Ajisaka
0e4d251fb2 MAPREDUCE-6421. Fix findbugs warning in RMContainerAllocator.reduceNodeLabelExpression. Contributed by Brahma Reddy Battula. 
(cherry picked from commit e04faf8e462a24a3ddfd8f22050e74eda4aa19ff)
 2015-07-13 14:29:39 +09:00
Devaraj K
3074d7082e MAPREDUCE-6426. TestShuffleHandler#testGetMapOutputInfo is failing. 
Contributed by zhihai xu.

(cherry picked from commit fffb15bb431fcdd7ca4ce0e249f9d45f4968497b)
 2015-07-09 15:02:09 +05:30
Chris Douglas
5fbf91b453 MAPREDUCE-6038. A boolean may be set error in the Word Count v2.0 in MapReduce Tutorial. Contributed by Tsuyoshi Ozawa 
(cherry picked from commit 4990ccfdc742be9c0517557a6256fefcae0df67d)
 2015-07-07 10:26:57 -07:00
Vinod Kumar Vavilapalli (I am also known as @tshooter.)
a8bf34cad8 Release process for 2.7.1: Set the release date for 2.7.1. 
(cherry picked from commit bf89ddb9b8ca27a34074b415f85599dd48b8fc50)
 2015-07-06 16:41:00 -07:00
Devaraj K
415dd64461 MAPREDUCE-6425. ShuffleHandler passes wrong "base" parameter to 
getMapOutputInfo if mapId is not in the cache. Contributed by zhihai xu.

(cherry picked from commit bff67dfe2f811654ffb1bbcbd87509c185f452b6)
 2015-07-06 13:47:54 +05:30
Jason Lowe
8a4970e845 MAPREDUCE-6418. MRApp should not shutdown LogManager during shutdown. Contributed by Chang Li 
(cherry picked from commit eac1d1894354e90d314087af8e7fb168ddef9a3d)
 2015-07-01 18:02:17 +00:00
Jason Lowe
f3195eda27 MAPREDUCE-6420. Interrupted Exception in LocalContainerLauncher should be logged in warn/info level. Contributed by Chang Li 
(cherry picked from commit 826715622e6937887a4e20b3ce327d7e2fd89009)
 2015-07-01 17:50:47 +00:00
Jason Lowe
df4e1e4965 MAPREDUCE-6376. Add avro binary support for jhist files. Contributed by Ray Chiang 
(cherry picked from commit 2ac87df578accb6e612f70ded76271cb5082ee10)

Conflicts:

	hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-core/src/main/resources/mapred-default.xml
 2015-07-01 16:07:54 +00:00
Robert Kanter
e5b0439bc1 MAPREDUCE-6121. JobResourceUpdater#compareFs() doesn't handle HA namespaces (rchiang via rkanter) 
(cherry picked from commit 3a72bfd08281fd271bef4f41289125d39c41928c)
 2015-06-30 16:50:48 -07:00
Jason Lowe
fdfd5be2b3 MAPREDUCE-6384. Add the last reporting reducer info for too many fetch failure diagnostics. Contributed by Chang Li 
(cherry picked from commit b6ba56457c6b01dae795c11d587c3fe3855ee707)
 2015-06-30 21:24:38 +00:00
Vinod Kumar Vavilapalli
4155bb565e Adding release 2.7.2 to CHANGES.txt. 
(cherry picked from commit aad6a7d5dba5858d6e9845f18c4baec16c91911d)
 2015-06-28 16:30:27 -07:00
Jason Lowe
31fd7a51d8 MAPREDUCE-6413. TestLocalJobSubmission is failing with unknown host. Contributed by zhihai xu 
(cherry picked from commit aa5b15b03be61ebb76a226e0de485d5228c8e3d0)
 2015-06-25 19:52:23 +00:00
Jason Lowe
2762affbb2 MAPREDUCE-6400. Multiple shuffle transfer fails because input is closed too early. Contributed by Brahma Reddy Battula, Akira AJISAKA, and Gera Shegalov. 
(cherry picked from commit 72d08a0e41efda635baa985d55d67cb059a7c07c)
 2015-06-24 15:30:09 +00:00
Vinod Kumar Vavilapalli
8c9e2cf10a MAPREDUCE-6410. Fixed MapReduce JobHistory server to use the right (login) UGI to refresh log and cleaner settings. Contributed by Varun Saxena. 
(cherry picked from commit d481684c7c9293a94f54ef622a92753531c6acc7)
 2015-06-23 11:41:05 -07:00
Jason Lowe
3bf44b862b MAPREDUCE-5948. org.apache.hadoop.mapred.LineRecordReader does not handle multibyte record delimiters well. Contributed by Vinayakumar B, Rushabh Shah, and Akira AJISAKA 
(cherry picked from commit 077250d8d7b4b757543a39a6ce8bb6e3be356c6f)
 2015-06-22 22:00:20 +00:00
Gera Shegalov
d8cc8f54dd MAPREDUCE-6408. Queue name and user name should be printed on the job page. (Siqi Li via gera) 
(cherry picked from commit 11ac84820746e43f1e7072e08f62ddf81713aeb2)
 2015-06-22 14:42:31 -07:00
Akira Ajisaka
c4747158b7 MAPREDUCE-6403. Fix typo in the usage of NNBench. Contributed by Jagadesh Kiran N. 
(cherry picked from commit 445b13240cb632ee8bc718f0576f783a7350af2e)
 2015-06-22 15:34:34 +09:00
Gera Shegalov
ee63c3e831 MAPREDUCE-6305. AM/Task log page should be able to link back to the job. (Siqi Li via gera) 
(cherry picked from commit 055cd5a9a11bd3e544a8a67c6c44b53efcb3a9ed)
 2015-06-20 12:40:59 -07:00
Gera Shegalov
34e744ce1a MAPREDUCE-6316. Task Attempt List entries should link to the task overview. (Siqi Li via gera) 
(cherry picked from commit 63d40d528654707b3f56619a4eb20e21cdeadc7e)
 2015-06-20 11:52:11 -07:00
Devaraj K
5c3070dc16 MAPREDUCE-6405. NullPointerException in App Attempts page. Contributed by 
Siqi Li and Gera Shegalov.

(cherry picked from commit 311a4179cc53a2754d1185b1cb10e6adb1b95d3d)
 2015-06-20 10:37:42 +05:30
Gera Shegalov
01991876ed MAPREDUCE-6395. Improve the commit failure messages in MRAppMaster recovery. (Brahma Reddy Battula via gera) 
(cherry picked from commit 10107243be66bae2212a2cd8575f9f5ade13fe9e)
 2015-06-19 02:29:31 -07:00
Devaraj K
c4c7fd0bf1 MAPREDUCE-6373. The logger reports total input paths but it is referring 
to input files. Contributed by Bibin A Chundatt.

(cherry picked from commit 1babe50a2cbaae3c8165229347e743d0dc94e979)
 2015-06-18 11:45:11 +05:30
Akira Ajisaka
98446ee695 MAPREDUCE-6363. [NNBench] Lease mismatch error when running with multiple mappers. Contributed by Brahma Reddy Battula. 
(cherry picked from commit 75a2560e51387ea31ef4609ef434475bbbc628f7)
 2015-06-15 14:53:28 -07:00
Devaraj K
b117527bf3 MAPREDUCE-6389. Fix BaileyBorweinPlouffe CLI usage message. Contributed by 
Brahma Reddy Battula.

(cherry picked from commit c7729efee8727b59f2c78cd5a3ad23fa84139068)
 2015-06-10 21:06:35 +05:30
Devaraj K
5426e3e929 MAPREDUCE-6350. JobHistory doesn't support fully-functional search. 
Contributed by Siqi Li.
 2015-06-10 12:43:31 +05:30
cnauroth
5a002aef12 MAPREDUCE-6392. Document mapred class path options. Contributed by Brahma Reddy Battula. 
(cherry picked from commit 025a3a8be07ba2b0c27ee39034b506d91aa486fc)
 2015-06-08 16:22:12 -07:00
Tsuyoshi Ozawa
c6cdecc6b3 MAPREDUCE-6388. Remove deprecation warnings from JobHistoryServer classes. Contributed by Ray Chiang. 
(cherry picked from commit 6afe20a7a4bbfa30fce3e3c9873ad43201987998)
 2015-06-08 15:06:23 -07:00
Jason Lowe
588c22e5a0 MAPREDUCE-6354. ShuffleHandler should be able to log shuffle connections. Contributed by Chang Li 
(cherry picked from commit b3ffa870034d06608a1946e2d9ce7dbd535a2c53)
 2015-06-05 22:39:52 +00:00
Karthik Kambatla
1a2e6e8091 MAPREDUCE-6387. Serialize the recently added Task#encryptedSpillKey field at the end. (Arun Suresh via kasha) 
(cherry picked from commit 6786daab33792c9c426adf0fc6b361f193d356a8)
 2015-06-05 09:17:34 -07:00
Devaraj K
37ad3b3fef MAPREDUCE-6383. Pi job (QuasiMonteCarlo) should not try to read the 
results file if its job fails. Contributed by Harsh J.

(cherry picked from commit 09b8241e6e369ee00241efcc2e048e710dbcccd1)
 2015-06-05 21:23:48 +05:30
Devaraj K
7f075f2b75 MAPREDUCE-6377. JHS sorting on state column not working in webUi. 
Contributed by zhihai xu.

(cherry picked from commit 790a861766dcd60212d83f444f2f96d3acf20a94)

Conflicts:
	hadoop-mapreduce-project/CHANGES.txt
 2015-06-05 16:03:02 +05:30
Vinod Kumar Vavilapalli
6044b5e5f6 MAPREDUCE-5232. Add a configuration to be able to log classpath and other system properties on mapreduce JVMs startup. Contributed by Sangjin Lee. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1482643 13f79535-47bb-0310-9956-ffa450edef68
(cherry picked from commit 4d8e350750748b919ee2158690a44cd9fd80dcae)

Conflicts:
	hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-app/src/main/java/org/apache/hadoop/mapreduce/v2/app/MRAppMaster.java
	hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-common/src/test/java/org/apache/hadoop/mapreduce/v2/util/TestMRApps.java
 2015-06-04 11:15:30 -07:00
Gera Shegalov
bae1ec5f29 MAPREDUCE-6382. Don't escape HTML links in Diagnostics in JHS job overview. (Siqi Li via gera) 
(cherry picked from commit e8302071ff16247a3d0cfc2dcca02da1bbf4db18)
 2015-06-03 23:52:16 -07:00
Robert Kanter
ee3002933a MAPREDUCE-5965. Hadoop streaming throws error if list of input files is high. Error is: "error=7, Argument list too long at if number of input file is high" (wilfreds via rkanter) 
(cherry picked from commit cc70df98e74142331043a611a3bd8a53ff6a2242)
 2015-06-03 18:42:16 -07:00
Gera Shegalov
25c1e54d3f MAPREDUCE-6174. Combine common stream code into parent class for InMemoryMapOutput and OnDiskMapOutput. (Eric Payne via gera) 
(cherry picked from commit d90c13e2da8867661bf19a802add70145ab9a462)
 2015-06-03 16:57:54 -07:00
Jason Lowe
5ecc647ae0 MAPREDUCE-6374. Distributed Cache File visibility should check permission of full path. Contributed by Chang Li 
(cherry picked from commit 107da29ff9b6efd02b21c96df60c58ac51590ebd)
 2015-06-03 20:20:47 +00:00
Wangda Tan
7cd0874dfd MAPREDUCE-6304. Specifying node labels when submitting MR jobs. (Naganarasimha G R via wangda) 
(cherry picked from commit 3164e7d83875aa6b7435d1dfe61ac280aa277f1c)

Conflicts:
	hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-app/src/main/java/org/apache/hadoop/mapreduce/v2/app/rm/RMContainerAllocator.java
 2015-05-27 14:43:20 -07:00
Tsuyoshi Ozawa
e694c3339e MAPREDUCE-6364. Add a Kill link to Task Attempts page. Contributed by Ryu Kobayashi. 
(cherry picked from commit 022f49d59e404678d0f322bb1b67208f3c17b7ef)
 2015-05-26 23:59:20 +09:00
Tsuyoshi Ozawa
6042b53a74 MAPREDUCE-6204. TestJobCounters should use new properties instead of JobConf.MAPRED_TASK_JAVA_OPTS. 
(cherry picked from commit b27ce3c5318a40bc79565783961f6675dcdcb14a)
 2015-05-22 23:30:12 +09:00
Junping Du
dafe33efbf Moving MAPREDUCE-6361 to 2.7.1 CHANGES.txt 2015-05-19 11:53:46 -07:00
Devaraj K
b9865cbf3d MAPREDUCE-6273. HistoryFileManager should check whether summaryFile exists 
to avoid FileNotFoundException causing HistoryFileInfo into MOVE_FAILED
state. Contributed by zhihai xu.

(cherry picked from commit 1ea90144d58443a7431ff33ba49ee19278ebe42b)
 2015-05-15 12:34:08 +05:30
Devaraj K
454236ec19 MAPREDUCE-5708. Duplicate String.format in 
YarnOutputFiles.getSpillFileForWrite. Contributed by Konstantin Weitz.

(cherry picked from commit 05ff54c66c49301c4ec2549704d9d459e784572c)
 2015-05-14 22:11:15 +05:30
Tsuyoshi Ozawa
780d4f416e MAPREDUCE-6366. mapreduce.terasort.final.sync configuration in TeraSort doesn't work. Contributed by Takuya Fukudome. 
(cherry picked from commit e82067bfe680ce04acc0153693cce3cd385e5567)
 2015-05-13 16:45:46 +09:00
Akira Ajisaka
345e62c6d5 Revert "MAPREDUCE-5762. Port MAPREDUCE-3223 (Remove MRv1 config from mapred-default.xml) to branch-2. (aajisaka)" 
This reverts commit 853cc0466811c310486fff6042d7b76ab3b78947.
 2015-05-13 14:02:41 +09:00
Vinod Kumar Vavilapalli
d5755ba241 MAPREDUCE-6251. Added a new config for JobClient to retry JobStatus calls so that they don't fail on history-server backed by DFSes with not so strong guarantees. Contributed by Craig Welch. 
(cherry picked from commit f24452d14e9ba48cdb82e5e6e5c10ce5b1407308)
 2015-05-12 12:13:23 -07:00
Tsuyoshi Ozawa
fb5b0ebb45 MAPREDUCE-6361. NPE issue in shuffle caused by concurrent issue between copySucceeded() in one thread and copyFailed() in another thread on the same host. Contributed by Junping Du. 
(cherry picked from commit f4e2b3cc0b1f4e49c306bc09a9dddd0495225bb2)
 2015-05-13 00:28:36 +09:00