Apache
/
hadoop
1
0
Fork 0
Code Issues Pull Requests Packages Projects Releases Wiki Activity
9,346 Commits 326 Branches 371 Tags 1.5 GiB
Commit Graph

1662 Commits

Author SHA1 Message Date
Karthik Kambatla 1a2e6e8091 MAPREDUCE-6387. Serialize the recently added Task#encryptedSpillKey field at the end. (Arun Suresh via kasha) 
(cherry picked from commit 6786daab33)
 2015-06-05 09:17:34 -07:00
Devaraj K 37ad3b3fef MAPREDUCE-6383. Pi job (QuasiMonteCarlo) should not try to read the 
results file if its job fails. Contributed by Harsh J.

(cherry picked from commit 09b8241e6e)
 2015-06-05 21:23:48 +05:30
Devaraj K 7f075f2b75 MAPREDUCE-6377. JHS sorting on state column not working in webUi. 
Contributed by zhihai xu.

(cherry picked from commit 790a861766)

Conflicts:
	hadoop-mapreduce-project/CHANGES.txt
 2015-06-05 16:03:02 +05:30
Vinod Kumar Vavilapalli 6044b5e5f6 MAPREDUCE-5232. Add a configuration to be able to log classpath and other system properties on mapreduce JVMs startup. Contributed by Sangjin Lee. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1482643 13f79535-47bb-0310-9956-ffa450edef68
(cherry picked from commit 4d8e350750)

Conflicts:
	hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-app/src/main/java/org/apache/hadoop/mapreduce/v2/app/MRAppMaster.java
	hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-common/src/test/java/org/apache/hadoop/mapreduce/v2/util/TestMRApps.java
 2015-06-04 11:15:30 -07:00
Gera Shegalov bae1ec5f29 MAPREDUCE-6382. Don't escape HTML links in Diagnostics in JHS job overview. (Siqi Li via gera) 
(cherry picked from commit e8302071ff)
 2015-06-03 23:52:16 -07:00
Robert Kanter ee3002933a MAPREDUCE-5965. Hadoop streaming throws error if list of input files is high. Error is: "error=7, Argument list too long at if number of input file is high" (wilfreds via rkanter) 
(cherry picked from commit cc70df98e7)
 2015-06-03 18:42:16 -07:00
Gera Shegalov 25c1e54d3f MAPREDUCE-6174. Combine common stream code into parent class for InMemoryMapOutput and OnDiskMapOutput. (Eric Payne via gera) 
(cherry picked from commit d90c13e2da)
 2015-06-03 16:57:54 -07:00
Jason Lowe 5ecc647ae0 MAPREDUCE-6374. Distributed Cache File visibility should check permission of full path. Contributed by Chang Li 
(cherry picked from commit 107da29ff9)
 2015-06-03 20:20:47 +00:00
Wangda Tan 7cd0874dfd MAPREDUCE-6304. Specifying node labels when submitting MR jobs. (Naganarasimha G R via wangda) 
(cherry picked from commit 3164e7d838)

Conflicts:
	hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-app/src/main/java/org/apache/hadoop/mapreduce/v2/app/rm/RMContainerAllocator.java
 2015-05-27 14:43:20 -07:00
Tsuyoshi Ozawa e694c3339e MAPREDUCE-6364. Add a Kill link to Task Attempts page. Contributed by Ryu Kobayashi. 
(cherry picked from commit 022f49d59e)
 2015-05-26 23:59:20 +09:00
Tsuyoshi Ozawa 6042b53a74 MAPREDUCE-6204. TestJobCounters should use new properties instead of JobConf.MAPRED_TASK_JAVA_OPTS. 
(cherry picked from commit b27ce3c531)
 2015-05-22 23:30:12 +09:00
Junping Du dafe33efbf Moving MAPREDUCE-6361 to 2.7.1 CHANGES.txt 2015-05-19 11:53:46 -07:00
Devaraj K b9865cbf3d MAPREDUCE-6273. HistoryFileManager should check whether summaryFile exists 
to avoid FileNotFoundException causing HistoryFileInfo into MOVE_FAILED
state. Contributed by zhihai xu.

(cherry picked from commit 1ea90144d5)
 2015-05-15 12:34:08 +05:30
Devaraj K 454236ec19 MAPREDUCE-5708. Duplicate String.format in 
YarnOutputFiles.getSpillFileForWrite. Contributed by Konstantin Weitz.

(cherry picked from commit 05ff54c66c)
 2015-05-14 22:11:15 +05:30
Tsuyoshi Ozawa 780d4f416e MAPREDUCE-6366. mapreduce.terasort.final.sync configuration in TeraSort doesn't work. Contributed by Takuya Fukudome. 
(cherry picked from commit e82067bfe6)
 2015-05-13 16:45:46 +09:00
Akira Ajisaka 345e62c6d5 Revert "MAPREDUCE-5762. Port MAPREDUCE-3223 (Remove MRv1 config from mapred-default.xml) to branch-2. (aajisaka)" 
This reverts commit 853cc04668.
 2015-05-13 14:02:41 +09:00
Vinod Kumar Vavilapalli d5755ba241 MAPREDUCE-6251. Added a new config for JobClient to retry JobStatus calls so that they don't fail on history-server backed by DFSes with not so strong guarantees. Contributed by Craig Welch. 
(cherry picked from commit f24452d14e)
 2015-05-12 12:13:23 -07:00
Tsuyoshi Ozawa fb5b0ebb45 MAPREDUCE-6361. NPE issue in shuffle caused by concurrent issue between copySucceeded() in one thread and copyFailed() in another thread on the same host. Contributed by Junping Du. 
(cherry picked from commit f4e2b3cc0b)
 2015-05-13 00:28:36 +09:00
Vinayakumar B cda1962996 MAPREDUCE-6360. TestMapreduceConfigFields is placed in wrong dir, introducing compile error (Contributed by Arshad Mohammad) 
(cherry picked from commit 360dff5903)
 2015-05-12 12:50:44 +05:30
Jason Lowe 53e9973afc MAPREDUCE-5465. Tasks are often killed before they exit on their own. Contributed by Ming Ma 2015-05-11 22:40:05 +00:00
Karthik Kambatla 52469b10dd MAPREDUCE-6353. Divide by zero error in MR AM when calculating available containers. (Anubhav Dhoot via kasha) 
(cherry picked from commit 1773aac780)
 2015-05-09 14:43:57 -07:00
Junping Du 39c188449e MAPREDUCE-6359. In RM HA setup, Cluster tab links populated with AM hostname instead of RM. Contributed by zhaoyunjiong. 
(cherry picked from commit df36ad0a08)
 2015-05-09 06:12:01 -07:00
Ravi Prakash 23065e6051 MAPREDUCE-4750. Enable NNBenchWithoutMR in MapredTestDriver (Liang Xie and Jason Lowe via raviprak) 
(cherry picked from commit 5aab014340)
 2015-05-08 15:58:29 -07:00
Jason Lowe 0f6c971282 MAPREDUCE-5248. Let NNBenchWithoutMR specify the replication factor for its test. Contributed by Erik Paulson 
(cherry picked from commit 30099a36c6)
 2015-05-08 22:30:21 +00:00
Jason Lowe dfc4a62f7d MAPREDUCE-3383. Duplicate job.getOutputValueGroupingComparator() in ReduceTask. Contributed by Binglin Chang 
(cherry picked from commit c39012f4a0)
 2015-05-08 21:17:05 +00:00
Devaraj K a957baeb71 MAPREDUCE-5981. Log levels of certain MR logs can be changed to DEBUG. 
Contributed by Varun Saxena.

(cherry picked from commit dc2b2ae31f)
 2015-05-08 21:28:52 +05:30
Devaraj K ade5165ecf MAPREDUCE-6342. Make POM project names consistent. Contributed by Rohith. 2015-05-08 15:20:35 +05:30
Akira Ajisaka 5cc036d473 MAPREDUCE-6079. Rename JobImpl#username to reporterUserName. Contributed by Tsuyoshi Ozawa. 
(cherry picked from commit ecfa052274)
 2015-05-08 18:17:35 +09:00
Tsuyoshi Ozawa 1ae8bf5b21 MAPREDUCE-6284. Add Task Attempt State API to MapReduce Application Master REST API. Contributed by Ryu Kobayashi. 
(cherry picked from commit d18f10ad1b)
 2015-05-08 16:00:41 +09:00
Jason Lowe 976a3c1f9d MAPREDUCE-6279. AM should explicity exit JVM after all services have stopped. Contributed by Eric Payne 
(cherry picked from commit f30065c8b6)
 2015-05-07 22:06:54 +00:00
Akira Ajisaka f7c011cbaa MAPREDUCE-6356. Misspelling of threshold in log4j.properties for tests. Contributed by Brahma Reddy Battula. 
(cherry picked from commit d335071398)
 2015-05-07 19:41:44 +09:00
Robert Kanter bd207b6cc0 MAPREDUCE-6192. Create unit test to automatically compare MR related classes and mapred-default.xml (rchiang via rkanter) 2015-05-05 14:47:52 -07:00
Tsuyoshi Ozawa 4e96175b33 MAPREDUCE-6165. [JDK8] TestCombineFileInputFormat failed on JDK8. Contributed by Akira AJISAKA. 
(cherry picked from commit 551615fa13)
 2015-05-05 10:23:25 +09:00
Jason Lowe 81f128f293 MAPREDUCE-6259. IllegalArgumentException due to missing job submit time. Contributed by zhihai xu 
(cherry picked from commit bf70c5ae28)
 2015-05-04 20:40:16 +00:00
Jason Lowe 87c2d915f1 MAPREDUCE-5649. Reduce cannot use more than 2G memory for the final merge. Contributed by Gera Shegalov 
(cherry picked from commit 7dc3c1203d)
 2015-05-04 19:03:43 +00:00
Tsuyoshi Ozawa 7e03bda3e7 MAPREDUCE-6349. Fix typo in property org.apache.hadoop.mapreduce.lib.chain.Chain.REDUCER_INPUT_VALUE_CLASS. Contributed by Ray Chiang. 
(cherry picked from commit bb6ef2984d)
 2015-05-04 15:36:53 +09:00
Tsuyoshi Ozawa 1035f550b9 MAPREDUCE-5905. CountersStrings.toEscapedCompactStrings outputs unnecessary null strings. Contributed by Akira AJISAKA. 
(cherry picked from commit 3ba18362f2)
 2015-05-04 15:02:40 +09:00
Gera Shegalov ff6492d603 MAPREDUCE-6345. Documentation fix for when CRLA is enabled for MRAppMaster logs. (Rohit Agarwal via gera) 
(cherry picked from commit f1a152cc0a)
 2015-05-01 19:07:14 -07:00
Devaraj K c3d083a9ff MAPREDUCE-6339. Job history file is not flushed correctly because 
isTimerActive flag is not set true when flushTimerTask is scheduled.
Contributed by zhihai xu.

(cherry picked from commit f5b38477f9)
 2015-04-30 12:36:00 +05:30
Jason Lowe ea828578a3 MAPREDUCE-6334. Fetcher#copyMapOutput is leaking usedMemory upon IOException during InMemoryMapOutput shuffle handler. Contributed by Eric Payne 
(cherry picked from commit bc1bd7e5c4)
 2015-04-28 20:19:52 +00:00
Devaraj K 1d03ac3533 Moving MAPREDUCE-6252 to the 2.7.1 CHANGES.txt 
(cherry picked from commit 99fe03e439)
 2015-04-28 12:44:07 +05:30
Vinod Kumar Vavilapalli 32dc13d907 MAPREDUCE-6324. Fixed MapReduce uber jobs to not fail the udpate of AM-RM tokens when they roll-over. Contributed by Jason Lowe. 
(cherry picked from commit 9fc32c5c4d)
 2015-04-27 14:59:57 -07:00
Tsuyoshi Ozawa fa9def5a2a MAPREDUCE-6341. Fix typo in mapreduce tutorial. Contributed by John Michael Luy. 
(cherry picked from commit 9a3dda3d34)

Conflicts:
	hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-core/src/site/markdown/MapReduceTutorial.md
 2015-04-28 01:58:12 +09:00
Devaraj K 84ae26573c MAPREDUCE-6252. JobHistoryServer should not fail when encountering a 
missing directory. Contributed by Craig Welch.

(cherry picked from commit 5e67c4d384)
 2015-04-27 15:02:44 +05:30
Gera Shegalov fa915f73e2 MAPREDUCE-6333. TestEvents,TestAMWebServicesTasks,TestAppController are broken due to MAPREDUCE-6297. (Siqi Li via gera) 
(cherry picked from commit 78c6b46241)
 2015-04-24 17:38:16 -07:00
Vinod Kumar Vavilapalli db1a33e956 Moving YARN-3351, YARN-3382, YARN-3472, MAPREDUCE-6238 to the 2.7.1 CHANGES.txt 
sections given the recent merge into branch-2.7.

(cherry picked from commit 2f82ae042a)
 2015-04-24 17:20:02 -07:00
Tsuyoshi Ozawa a479669e96 Fix a misleading summary of MAPREDUCE-6330 in CHANGES.txt. 
(cherry picked from commit 252e69f562)
 2015-04-23 03:28:46 +09:00
Tsuyoshi Ozawa 542d53a268 MAPREDUCE-6330. Fix typo in some Task Attempt API's URL. Contributed by Ryu Kobayashi. 
(cherry picked from commit e33594dd87)
 2015-04-23 03:22:02 +09:00
Gera Shegalov 6f06e3b7ea MAPREDUCE-6297. Task Id of the failed task in diagnostics should link to the task page. (Siqi Li via gera) 
(cherry picked from commit 89ded89e86)
 2015-04-21 14:21:57 -07:00
Gera Shegalov 6f5dc9f5af MAPREDUCE-6293. Set job classloader on uber-job's LocalContainerLauncher event thread. (Sangjin Lee via gera) 
(cherry picked from commit 725eb52ddc)
 2015-04-21 14:21:56 -07:00