Apache/hadoop
1
0
Fork 0
Code Issues Pull Requests Packages Projects Releases Wiki Activity
9,342 Commits 326 Branches 371 Tags
Commit Graph

1944 Commits

Author SHA1 Message Date
Devaraj K
37ad3b3fef MAPREDUCE-6383. Pi job (QuasiMonteCarlo) should not try to read the 
results file if its job fails. Contributed by Harsh J.

(cherry picked from commit 09b8241e6e369ee00241efcc2e048e710dbcccd1)
 2015-06-05 21:23:48 +05:30
Devaraj K
7f075f2b75 MAPREDUCE-6377. JHS sorting on state column not working in webUi. 
Contributed by zhihai xu.

(cherry picked from commit 790a861766dcd60212d83f444f2f96d3acf20a94)

Conflicts:
	hadoop-mapreduce-project/CHANGES.txt
 2015-06-05 16:03:02 +05:30
Vinod Kumar Vavilapalli
6044b5e5f6 MAPREDUCE-5232. Add a configuration to be able to log classpath and other system properties on mapreduce JVMs startup. Contributed by Sangjin Lee. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1482643 13f79535-47bb-0310-9956-ffa450edef68
(cherry picked from commit 4d8e350750748b919ee2158690a44cd9fd80dcae)

Conflicts:
	hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-app/src/main/java/org/apache/hadoop/mapreduce/v2/app/MRAppMaster.java
	hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-common/src/test/java/org/apache/hadoop/mapreduce/v2/util/TestMRApps.java
 2015-06-04 11:15:30 -07:00
Akira Ajisaka
1cccd1eeb3 HADOOP-12058. Fix dead links to DistCp and Hadoop Archives pages. Contributed by Kazuho Fujii. 
(cherry picked from commit dbed757cbaf3ddff0c9ae2d905d42828aadedf2a)
 2015-06-05 01:46:33 +09:00
Gera Shegalov
bae1ec5f29 MAPREDUCE-6382. Don't escape HTML links in Diagnostics in JHS job overview. (Siqi Li via gera) 
(cherry picked from commit e8302071ff16247a3d0cfc2dcca02da1bbf4db18)
 2015-06-03 23:52:16 -07:00
Robert Kanter
ee3002933a MAPREDUCE-5965. Hadoop streaming throws error if list of input files is high. Error is: "error=7, Argument list too long at if number of input file is high" (wilfreds via rkanter) 
(cherry picked from commit cc70df98e74142331043a611a3bd8a53ff6a2242)
 2015-06-03 18:42:16 -07:00
Gera Shegalov
25c1e54d3f MAPREDUCE-6174. Combine common stream code into parent class for InMemoryMapOutput and OnDiskMapOutput. (Eric Payne via gera) 
(cherry picked from commit d90c13e2da8867661bf19a802add70145ab9a462)
 2015-06-03 16:57:54 -07:00
Zhijie Shen
2fcd3fe6ba Revert "YARN-1462. Made RM write application tags to timeline server and exposed them to users via generic history web UI and REST API. Contributed by Xuan Gong." 
This reverts commit 4e2ed34d2023a7cd3842e41aba1995f5d0f53cf2.
 2015-06-03 14:16:51 -07:00
Jason Lowe
5ecc647ae0 MAPREDUCE-6374. Distributed Cache File visibility should check permission of full path. Contributed by Chang Li 
(cherry picked from commit 107da29ff9b6efd02b21c96df60c58ac51590ebd)
 2015-06-03 20:20:47 +00:00
Zhijie Shen
4e2ed34d20 YARN-1462. Made RM write application tags to timeline server and exposed them to users via generic history web UI and REST API. Contributed by Xuan Gong. 
(cherry picked from commit a9f986bc5a41ab74171da1076d10e83c5adc284f)
 2015-05-30 21:06:32 -07:00
Andrew Wang
bdaa0fed13 HADOOP-11885. hadoop-dist dist-layout-stitching.sh does not work with dash. (wang) 
(cherry picked from commit 7673d4f205b26a6a26cfc47d999ece96f3c42782)

 Conflicts:
	hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-nativetask/pom.xml
	hadoop-mapreduce-project/pom.xml
 2015-05-29 14:55:52 -07:00
Wangda Tan
7cd0874dfd MAPREDUCE-6304. Specifying node labels when submitting MR jobs. (Naganarasimha G R via wangda) 
(cherry picked from commit 3164e7d83875aa6b7435d1dfe61ac280aa277f1c)

Conflicts:
	hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-app/src/main/java/org/apache/hadoop/mapreduce/v2/app/rm/RMContainerAllocator.java
 2015-05-27 14:43:20 -07:00
cnauroth
6ed8989a6f YARN-3626. On Windows localized resources are not moved to the front of the classpath when they should be. Contributed by Craig Welch. 
(cherry picked from commit 7bc33b63c78c6ee9a04c0c1511ef16e63f7f3074)
 2015-05-27 14:21:15 -07:00
Colin Patrick Mccabe
3dec58dd78 HADOOP-11969. ThreadLocal initialization in several classes is not thread safe (Sean Busbey via Colin P. McCabe) 
(cherry picked from commit 7dba7005b79994106321b0f86bc8f4ea51a3c185)
 2015-05-26 12:40:41 -07:00
Tsuyoshi Ozawa
e694c3339e MAPREDUCE-6364. Add a Kill link to Task Attempts page. Contributed by Ryu Kobayashi. 
(cherry picked from commit 022f49d59e404678d0f322bb1b67208f3c17b7ef)
 2015-05-26 23:59:20 +09:00
Tsuyoshi Ozawa
6042b53a74 MAPREDUCE-6204. TestJobCounters should use new properties instead of JobConf.MAPRED_TASK_JAVA_OPTS. 
(cherry picked from commit b27ce3c5318a40bc79565783961f6675dcdcb14a)
 2015-05-22 23:30:12 +09:00
Vinayakumar B
19895a41ad HADOOP-11743. maven doesn't clean all the site files (Contributed by ramtin) 
(cherry picked from commit 55ed6558bad5d76ce4ebf958fc101a48032a0cd1)

Conflicts:
	hadoop-common-project/hadoop-common/pom.xml
 2015-05-22 16:46:50 +05:30
Wangda Tan
b5ba6979b7 YARN-3583. Support of NodeLabel object instead of plain String in YarnClient side. (Sunil G via wangda) 
(cherry picked from commit 563eb1ad2ae848a23bbbf32ebfaf107e8fa14e87)
(cherry picked from commit b0d22b0c606fad6b4ab5443c0aed07c829b46726)
 2015-05-19 17:37:24 -07:00
Junping Du
dafe33efbf Moving MAPREDUCE-6361 to 2.7.1 CHANGES.txt 2015-05-19 11:53:46 -07:00
Devaraj K
b9865cbf3d MAPREDUCE-6273. HistoryFileManager should check whether summaryFile exists 
to avoid FileNotFoundException causing HistoryFileInfo into MOVE_FAILED
state. Contributed by zhihai xu.

(cherry picked from commit 1ea90144d58443a7431ff33ba49ee19278ebe42b)
 2015-05-15 12:34:08 +05:30
Vinod Kumar Vavilapalli
87862970f1 Fixing MR intermediate spills. Contributed by Arun Suresh. 
(cherry picked from commit 6b710a42e00acca405e085724c89cda016cf7442)
 2015-05-14 16:56:56 -07:00
Devaraj K
454236ec19 MAPREDUCE-5708. Duplicate String.format in 
YarnOutputFiles.getSpillFileForWrite. Contributed by Konstantin Weitz.

(cherry picked from commit 05ff54c66c49301c4ec2549704d9d459e784572c)
 2015-05-14 22:11:15 +05:30
Xuan
487d9b0f3f YARN-3626. On Windows localized resources are not moved to the front of the classpath when they should be. Contributed by Craig Welch 
(cherry picked from commit 0f95921447ea547bdf9caf18f7fde46bc66031f8)
 2015-05-13 13:12:04 -07:00
cnauroth
ddc4afeff2 HADOOP-11966. Variable cygwin is undefined in hadoop-config.sh when executed through hadoop-daemon.sh. Contributed by Chris Nauroth. 2015-05-13 12:25:49 -07:00
Tsuyoshi Ozawa
780d4f416e MAPREDUCE-6366. mapreduce.terasort.final.sync configuration in TeraSort doesn't work. Contributed by Takuya Fukudome. 
(cherry picked from commit e82067bfe680ce04acc0153693cce3cd385e5567)
 2015-05-13 16:45:46 +09:00
Akira Ajisaka
345e62c6d5 Revert "MAPREDUCE-5762. Port MAPREDUCE-3223 (Remove MRv1 config from mapred-default.xml) to branch-2. (aajisaka)" 
This reverts commit 853cc0466811c310486fff6042d7b76ab3b78947.
 2015-05-13 14:02:41 +09:00
Vinod Kumar Vavilapalli
d5755ba241 MAPREDUCE-6251. Added a new config for JobClient to retry JobStatus calls so that they don't fail on history-server backed by DFSes with not so strong guarantees. Contributed by Craig Welch. 
(cherry picked from commit f24452d14e9ba48cdb82e5e6e5c10ce5b1407308)
 2015-05-12 12:13:23 -07:00
Tsuyoshi Ozawa
fb5b0ebb45 MAPREDUCE-6361. NPE issue in shuffle caused by concurrent issue between copySucceeded() in one thread and copyFailed() in another thread on the same host. Contributed by Junping Du. 
(cherry picked from commit f4e2b3cc0b1f4e49c306bc09a9dddd0495225bb2)
 2015-05-13 00:28:36 +09:00
Vinayakumar B
cda1962996 MAPREDUCE-6360. TestMapreduceConfigFields is placed in wrong dir, introducing compile error (Contributed by Arshad Mohammad) 
(cherry picked from commit 360dff5903085c3c7f02ccf9f17d71a842275e48)
 2015-05-12 12:50:44 +05:30
Jason Lowe
53e9973afc MAPREDUCE-5465. Tasks are often killed before they exit on their own. Contributed by Ming Ma 2015-05-11 22:40:05 +00:00
Junping Du
a1a7deebf8 YARN-3587. Fix the javadoc of DelegationTokenSecretManager in yarn, etc. projects. Contributed by Gabor Liptak. 
(cherry picked from commit 7e543c27fa2881aa65967be384a6203bd5b2304f)
 2015-05-11 05:28:25 -07:00
Karthik Kambatla
52469b10dd MAPREDUCE-6353. Divide by zero error in MR AM when calculating available containers. (Anubhav Dhoot via kasha) 
(cherry picked from commit 1773aac780585871072960a5863af461e112a030)
 2015-05-09 14:43:57 -07:00
Karthik Kambatla
e8dd0bc4fd YARN-1287. Consolidate MockClocks. (Sebastian Wong and Anubhav Dhoot via kasha) 
(cherry picked from commit 70fb37cd79a7eba6818313960624380bacfe0bb2)
 2015-05-09 14:35:13 -07:00
Junping Du
39c188449e MAPREDUCE-6359. In RM HA setup, Cluster tab links populated with AM hostname instead of RM. Contributed by zhaoyunjiong. 
(cherry picked from commit df36ad0a08261b03c250b6f745b27e5f83e4286e)
 2015-05-09 06:12:01 -07:00
Ravi Prakash
23065e6051 MAPREDUCE-4750. Enable NNBenchWithoutMR in MapredTestDriver (Liang Xie and Jason Lowe via raviprak) 
(cherry picked from commit 5aab014340b53ebc9363ee244b2cbea7a4c1f573)
 2015-05-08 15:58:29 -07:00
Jason Lowe
0f6c971282 MAPREDUCE-5248. Let NNBenchWithoutMR specify the replication factor for its test. Contributed by Erik Paulson 
(cherry picked from commit 30099a36c6b0f658d25fb505a9f3ce15d19f7ba6)
 2015-05-08 22:30:21 +00:00
Chris Douglas
4f301f92c1 MAPREDUCE-2094. LineRecordReader should not seek into non-splittable, compressed streams. 2015-05-08 14:30:56 -07:00
Jason Lowe
dfc4a62f7d MAPREDUCE-3383. Duplicate job.getOutputValueGroupingComparator() in ReduceTask. Contributed by Binglin Chang 
(cherry picked from commit c39012f4a0444f9e4b7d67957d5192127d143d90)
 2015-05-08 21:17:05 +00:00
Devaraj K
a957baeb71 MAPREDUCE-5981. Log levels of certain MR logs can be changed to DEBUG. 
Contributed by Varun Saxena.

(cherry picked from commit dc2b2ae31f2eb6dae324c2e14ed7660ce605a89b)
 2015-05-08 21:28:52 +05:30
Devaraj K
ade5165ecf MAPREDUCE-6342. Make POM project names consistent. Contributed by Rohith. 2015-05-08 15:20:35 +05:30
Akira Ajisaka
5cc036d473 MAPREDUCE-6079. Rename JobImpl#username to reporterUserName. Contributed by Tsuyoshi Ozawa. 
(cherry picked from commit ecfa05227489724f12775b94a9f9184b66e865af)
 2015-05-08 18:17:35 +09:00
Tsuyoshi Ozawa
1ae8bf5b21 MAPREDUCE-6284. Add Task Attempt State API to MapReduce Application Master REST API. Contributed by Ryu Kobayashi. 
(cherry picked from commit d18f10ad1b3e497fa1aaaeb85ba055f87d9849f7)
 2015-05-08 16:00:41 +09:00
Jason Lowe
976a3c1f9d MAPREDUCE-6279. AM should explicity exit JVM after all services have stopped. Contributed by Eric Payne 
(cherry picked from commit f30065c8b6099372f57015b505434120fe83c2b0)
 2015-05-07 22:06:54 +00:00
Akira Ajisaka
f7c011cbaa MAPREDUCE-6356. Misspelling of threshold in log4j.properties for tests. Contributed by Brahma Reddy Battula. 
(cherry picked from commit d335071398fdb14153c854b740814b4fd385b658)
 2015-05-07 19:41:44 +09:00
Robert Kanter
bd207b6cc0 MAPREDUCE-6192. Create unit test to automatically compare MR related classes and mapred-default.xml (rchiang via rkanter) 2015-05-05 14:47:52 -07:00
Xuan
02cdcaec58 YARN-2123. Progress bars in Web UI always at 100% due to non-US locale. 
Contributed by Akira AJISAKA

(cherry picked from commit b7dd3a4f04f712b7594c4e6e7ce50fd314f7c342)
 2015-05-05 10:45:18 -07:00
Tsuyoshi Ozawa
4e96175b33 MAPREDUCE-6165. [JDK8] TestCombineFileInputFormat failed on JDK8. Contributed by Akira AJISAKA. 
(cherry picked from commit 551615fa13f65ae996bae9c1bacff189539b6557)
 2015-05-05 10:23:25 +09:00
Jason Lowe
81f128f293 MAPREDUCE-6259. IllegalArgumentException due to missing job submit time. Contributed by zhihai xu 
(cherry picked from commit bf70c5ae2824a9139c1aa9d7c14020018881cec2)
 2015-05-04 20:40:16 +00:00
Jason Lowe
87c2d915f1 MAPREDUCE-5649. Reduce cannot use more than 2G memory for the final merge. Contributed by Gera Shegalov 
(cherry picked from commit 7dc3c1203d1ab14c09d0aaf0869a5bcdfafb0a5a)
 2015-05-04 19:03:43 +00:00
Tsuyoshi Ozawa
7e03bda3e7 MAPREDUCE-6349. Fix typo in property org.apache.hadoop.mapreduce.lib.chain.Chain.REDUCER_INPUT_VALUE_CLASS. Contributed by Ray Chiang. 
(cherry picked from commit bb6ef2984d8f117711b806c4ebdc757bd182c06e)
 2015-05-04 15:36:53 +09:00