bc18c6f842
HADOOP-10005. Merge 1532907 from trunk
...
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/branches/branch-2@1532908 13f79535-47bb-0310-9956-ffa450edef68
2013-10-16 21:05:23 +00:00
1f77877bd9
MAPREDUCE-5463. Deprecate SLOTS_MILLIS counters. (Tzuyoshi Ozawa via Sandy Ryza)
...
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/branches/branch-2@1531823 13f79535-47bb-0310-9956-ffa450edef68
2013-10-14 09:12:11 +00:00
5ff9f8bed2
merge MAPREDUCE-5329. Allow MR applications to use additional AuxServices, which are compatible with the default MapReduce shuffle. Contributed by Avner BenHanoch.
...
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/branches/branch-2@1531742 13f79535-47bb-0310-9956-ffa450edef68
2013-10-13 20:19:06 +00:00
82c4a9438d
YARN-1283. Fixed RM to give a fully-qualified proxy URL for an application so that clients don't need to do scheme-mangling. Contributed by Omkar Vinit Joshi.
...
svn merge --ignore-ancestry -c 1530819 ../../trunk/
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/branches/branch-2@1530820 13f79535-47bb-0310-9956-ffa450edef68
2013-10-09 22:49:50 +00:00
9b5a023fc2
svn merge -c 1530816 FIXES: MAPREDUCE-5102. fix coverage org.apache.hadoop.mapreduce.lib.db and org.apache.hadoop.mapred.lib.db. Contributed by Aleksey Gorshkov, Andrey Klochkov, and Nathan Roberts
...
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/branches/branch-2@1530818 13f79535-47bb-0310-9956-ffa450edef68
2013-10-09 22:45:34 +00:00
d5f32b0fab
svn merge -c 1530683 FIXES: MAPREDUCE-5569. FloatSplitter is not generating correct splits. Contributed by Nathan Roberts
...
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/branches/branch-2@1530684 13f79535-47bb-0310-9956-ffa450edef68
2013-10-09 15:44:47 +00:00
2031005eeb
HADOOP-9470. eliminate duplicate FQN tests in different Hadoop modules (Ivan A. Veselovsky via daryn)
...
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/branches/branch-2@1530671 13f79535-47bb-0310-9956-ffa450edef68
2013-10-09 15:10:43 +00:00
ad9515fa9b
MAPREDUCE-5562. Fixed MR App Master to perform pending tasks like staging-dir cleanup, sending job-end notification correctly when unregister with RM fails. Contributed by Zhijie Shen.
...
svn merge --ignore-ancestry -c 1529682 ../../trunk/
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/branches/branch-2@1529683 13f79535-47bb-0310-9956-ffa450edef68
2013-10-06 20:54:10 +00:00
4b32d6ad9a
YARN-1277. Added a policy based configuration for http/https in common HttpServer and using the same in YARN - related
...
to per project https config support via HADOOP-10022. Contributed by Suresh Srinivas and Omkar Vinit Joshi.
svn merge --ignore-ancestry -c 1529662 ../../trunk/
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/branches/branch-2@1529663 13f79535-47bb-0310-9956-ffa450edef68
2013-10-06 18:44:27 +00:00
856907b8db
MAPREDUCE-5533. Fixed MR speculation code to track any TaskAttempts that aren't heart-beating for a while, so that we can aggressively speculate instead of waiting for task-timeout. Contributed by Xuan Gong.
...
svn merge --ignore-ancestry -c 1529229 ../../trunk/
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/branches/branch-2@1529230 13f79535-47bb-0310-9956-ffa450edef68
2013-10-04 17:29:21 +00:00
658905cd44
MAPREDUCE-5442. Merging change r1529077 from trunk to branch-2.
...
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/branches/branch-2@1529079 13f79535-47bb-0310-9956-ffa450edef68
2013-10-04 05:42:58 +00:00
2cc1005b92
Merge 1529068 from trunk to branch-2 for YARN-1131. logs command should return an appropriate error message if YARN application is still running. Contributed by Siddharth Seth.
...
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/branches/branch-2@1529069 13f79535-47bb-0310-9956-ffa450edef68
2013-10-04 02:38:17 +00:00
8a93fa55b6
YARN-1199. Make NM/RM Versions Available (Mit Desai via jeagles)
...
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/branches/branch-2@1529008 13f79535-47bb-0310-9956-ffa450edef68
2013-10-03 21:18:18 +00:00
53df9bdb14
Merge r1529005 from trunk to branch-2 for MAPREDUCE-5489. MR jobs hangs as it does not use the node-blacklisting feature in RM requests (Zhijie Shen via bikas)
...
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/branches/branch-2@1529007 13f79535-47bb-0310-9956-ffa450edef68
2013-10-03 21:18:06 +00:00
c738c106e9
MAPREDUCE-5554. hdfs-site.xml included in hadoop-mapreduce-client-jobclient tests jar is breaking tests for downstream components (Robert Kanter via Sandy Ryza)
...
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/branches/branch-2@1528647 13f79535-47bb-0310-9956-ffa450edef68
2013-10-02 22:00:56 +00:00
ce990cd911
Merge -c 1528626 from trunk to branch-2 to fix MAPREDUCE-5459. Update documentation on how to run MRv1 examples on YARN. Contributed by Zhijie Shen.
...
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/branches/branch-2@1528627 13f79535-47bb-0310-9956-ffa450edef68
2013-10-02 21:09:59 +00:00
f49d9cf7a4
Merge -c 1528533 from trunk to branch-2 to fix MAPREDUCE-5530. Fix compat with hadoop-1 in mapred.lib.CombinFileInputFormat by re-introducing isSplittable(FileSystem, Path) api and ensuring semantic compatibility. Contributed by Robert Kanter.
...
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/branches/branch-2@1528535 13f79535-47bb-0310-9956-ffa450edef68
2013-10-02 16:09:53 +00:00
6daca6a93d
MAPREDUCE-5536. Fixed MR AM and JHS to respect mapreduce.jobhistory.webapp.https.address. Contributed by Omkar Vinit Joshi.
...
svn merge --ignore-ancestry -c 1528251 ../../trunk/
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/branches/branch-2@1528253 13f79535-47bb-0310-9956-ffa450edef68
2013-10-01 22:54:35 +00:00
7e812e7126
svn merge -c 1528237 FIXES: MAPREDUCE-4421. Run MapReduce framework via the distributed cache. Contributed by Jason Lowe
...
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/branches/branch-2@1528242 13f79535-47bb-0310-9956-ffa450edef68
2013-10-01 22:40:01 +00:00
06e8660f2b
MAPREDUCE-5544. JobClient#getJob loads job conf twice. (Sandy Ryza)
...
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/branches/branch-2@1528197 13f79535-47bb-0310-9956-ffa450edef68
2013-10-01 20:05:09 +00:00
85f5626076
Merge -c 1527848 from trunk to branch-2 to fix MAPREDUCE-5551. Fix compat with hadoop-1 in SequenceFileAsBinaryOutputFormat.WritableValueBytes by re-introducing missing constructors. Contributed by Zhijie Shen.
...
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/branches/branch-2@1527850 13f79535-47bb-0310-9956-ffa450edef68
2013-10-01 01:25:31 +00:00
3c3b52b80b
MAPREDUCE-5538. Fixed MR AppMaster to send job-notification URL only after the job is really done - a bug caused by MAPREDUCE-5505. Contributed by Zhijie Shen.
...
svn merge --ignore-ancestry -c 1527219 ../../trunk/
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/branches/branch-2@1527221 13f79535-47bb-0310-9956-ffa450edef68
2013-09-28 19:14:46 +00:00
c03040a498
Merge -c 1527048 from trunk to branch-2 to fix MAPREDUCE-5529. Fix compat with hadoop-1 in mapred.TotalOrderPartitioner by re-introducing (get,set)PartitionFile which takes in JobConf. Contributed by Robert Kanter.
...
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/branches/branch-2@1527054 13f79535-47bb-0310-9956-ffa450edef68
2013-09-27 20:04:31 +00:00
d36dec185e
svn commit -c 1527037 FIXES: MAPREDUCE-5545. org.apache.hadoop.mapred.TestTaskAttemptListenerImpl.testCommitWindow times out. Contributed by Robert Kanter
...
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/branches/branch-2@1527039 13f79535-47bb-0310-9956-ffa450edef68
2013-09-27 19:17:24 +00:00
146d9264a9
Merge -c 1527033 from trunk to branch-2 to fix MAPREDUCE-5531. Fix compat with hadoop-1 in mapreduce.(TaskID, TaskAttemptID) by re-introducing missing constructors. Contributed by Robert Kanter.
...
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/branches/branch-2@1527034 13f79535-47bb-0310-9956-ffa450edef68
2013-09-27 19:10:07 +00:00
d8e9a472e7
svn merge -c 1527015 FIXES: MAPREDUCE-5332. Support token-preserving restart of history server. Contributed by Jason Lowe
...
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/branches/branch-2@1527016 13f79535-47bb-0310-9956-ffa450edef68
2013-09-27 18:26:09 +00:00
78045b7abc
Merge -c 1526735 from trunk to branch-2 to fix MAPREDUCE-5514. Fix TestRMContainerAllocator. Contributed by Zhijie Shen.
...
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/branches/branch-2@1526736 13f79535-47bb-0310-9956-ffa450edef68
2013-09-26 23:03:25 +00:00
2b7ae8dd5d
svn merge -c 1526688 FIXES: MAPREDUCE-5513. ConcurrentModificationException in JobControl. Contributed by Robert Parker
...
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/branches/branch-2@1526702 13f79535-47bb-0310-9956-ffa450edef68
2013-09-26 21:21:34 +00:00
6bdd4791db
MAPREDUCE-5525. Merging change r1526624 from trunk to branch-2.
...
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/branches/branch-2@1526625 13f79535-47bb-0310-9956-ffa450edef68
2013-09-26 17:47:45 +00:00
fa2a3c94f2
HADOOP-9998. Provide methods to clear only part of the DNSToSwitchMapping. (Junping Du via Colin Patrick McCabe)
...
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/branches/branch-2@1526568 13f79535-47bb-0310-9956-ffa450edef68
2013-09-26 15:24:40 +00:00
6e132b1a2b
MAPREDUCE-5170. Fixed a wrong log message in CombineFileInputFormat class. Contributed by Sangjin Lee.
...
svn merge --ignore-ancestry -c 1526377 ../../trunk/
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/branches/branch-2@1526379 13f79535-47bb-0310-9956-ffa450edef68
2013-09-26 04:30:36 +00:00
b41137c73a
MAPREDUCE-5503. Fixed a test issue in TestMRJobClient. Contributed by Jian He.
...
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/branches/branch-2@1526364 13f79535-47bb-0310-9956-ffa450edef68
2013-09-26 03:02:06 +00:00
a5ecb4de41
Merge r1526071 from trunk to branch-2 for MAPREDUCE-5505. Clients should be notified job finished only after job successfully unregistered (Zhijie Shen via bikas)
...
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/branches/branch-2@1526072 13f79535-47bb-0310-9956-ffa450edef68
2013-09-25 00:53:10 +00:00
e23d5672a7
merge YARN-1229 from trunk. Define constraints on Auxiliary Service names. Change ShuffleHandler service name from mapreduce.shuffle to mapreduce_shuffle. Contributed by Xuan Gong.
...
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/branches/branch-2@1526066 13f79535-47bb-0310-9956-ffa450edef68
2013-09-25 00:37:14 +00:00
2daddb610e
YARN-1204. Added separate configuration properties for https for RM and NM without which servers enabled with https will also start on http ports. Contributed by Omkar Vinit Joshi.
...
MAPREDUCE-5523. Added separate configuration properties for https for JHS without which even when https is enabled, it starts on http port itself. Contributed by Omkar Vinit Joshi.
svn merge --ignore-ancestry -c 1525947 ../../trunk/
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/branches/branch-2@1525948 13f79535-47bb-0310-9956-ffa450edef68
2013-09-24 17:20:48 +00:00
e2bfdfc585
svn merge -c 1525670 FIXES: MAPREDUCE-5522. Incorrect oreder expected from JobQueueInfo (Jinghui Wang via bobby)
...
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/branches/branch-2@1525673 13f79535-47bb-0310-9956-ffa450edef68
2013-09-23 18:39:47 +00:00
d71bb8385b
YARN-1203. Changed YARN web-app proxy to handle http and https URLs from AM registration and finish correctly. Contributed by Omkar Vinit Joshi.
...
MAPREDUCE-5515. Fixed MR AM's webapp to depend on a new config mapreduce.ssl.enabled to enable https and disabling it by default as MR AM needs
to set up its own certificates etc and not depend on clusters'. Contributed by Omkar Vinit Joshi.
svn merge --ignore-ancestry -c 1524864 ../../trunk/
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/branches/branch-2@1524867 13f79535-47bb-0310-9956-ffa450edef68
2013-09-19 23:51:14 +00:00
d8ed9ef4b8
MAPREDUCE-5488. Changed MR client to keep trying to reach the application when it sees that on attempt's AM is down. Contributed by Jian He.
...
svn merge --ignore-ancestry -c 1524856 ../../trunk/
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/branches/branch-2@1524857 13f79535-47bb-0310-9956-ffa450edef68
2013-09-19 22:36:58 +00:00
11d0fe9722
MAPREDUCE-5504. mapred queue -info inconsistent with types (Kousuke Saruta via tgraves)
...
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/branches/branch-2@1524843 13f79535-47bb-0310-9956-ffa450edef68
2013-09-19 21:00:40 +00:00
f0e79b8157
MAPREDUCE-5487. In task processes, JobConf is unnecessarily loaded again in Limits (Sandy Ryza)
...
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/branches/branch-2@1524417 13f79535-47bb-0310-9956-ffa450edef68
2013-09-18 13:58:37 +00:00
73c69decf1
Merge -c 1523660 from trunk to branch-2 to fix MAPREDUCE-5493. Cleanup in-memory & on-disk segments to prevent leak on shuffle completion. Contributed by Jason Lowe.
...
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/branches/branch-2@1523661 13f79535-47bb-0310-9956-ffa450edef68
2013-09-16 14:19:33 +00:00
c061a0da55
Merge -c 1523652 from trunk to branch-2 to fix YARN-1170. YARN & MapReduce proto definitions fixed to specify protobuf package as hadoop.yarn and hadoop.mapreduce respectively. Contributed by Binglin Chang.
...
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/branches/branch-2@1523653 13f79535-47bb-0310-9956-ffa450edef68
2013-09-16 14:06:27 +00:00
7993c89ea8
MAPREDUCE-5379. Include token tracking ids in jobconf. (kkambatl via tucu)
...
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/branches/branch-2@1523607 13f79535-47bb-0310-9956-ffa450edef68
2013-09-16 12:20:30 +00:00
c5ef30971e
Merge r1523376 from trunk to branch-2 for YARN-540. Race condition causing RM to potentially relaunch already unregistered AMs on RM restart (Jian He via bikas)
...
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/branches/branch-2@1523377 13f79535-47bb-0310-9956-ffa450edef68
2013-09-15 01:07:11 +00:00
87d09e5ebe
Merge r1523146 from trunk to branch-2 YARN-1116. Populate AMRMTokens back to AMRMTokenSecretManager after RM restarts (Jian He via bikas)
...
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/branches/branch-2@1523149 13f79535-47bb-0310-9956-ffa450edef68
2013-09-13 23:57:56 +00:00
90f492e532
MAPREDUCE-5497. Changed MRAppMaster to sleep only after doing everything else but just before ClientService to avoid race conditions during RM restart. Contributed by Jian He.
...
svn merge --ignore-ancestry -c 1521699 ../../trunk/
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/branches/branch-2@1521700 13f79535-47bb-0310-9956-ffa450edef68
2013-09-11 00:39:21 +00:00
a574ef2864
MAPREDUCE-5020. Compile failure with JDK8 (Trevor Robinson via tgraves)
...
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/branches/branch-2@1521584 13f79535-47bb-0310-9956-ffa450edef68
2013-09-10 18:53:29 +00:00
d997ca22f9
MAPREDUCE-5414. TestTaskAttempt fails in JDK7 with NPE. Contributed by Nemon Lou.
...
svn merge --ignore-ancestry -c 1520964 ../../trunk
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/branches/branch-2@1520965 13f79535-47bb-0310-9956-ffa450edef68
2013-09-09 06:46:16 +00:00
5ddee655e4
svn merge -c 1520156 FIXES: MAPREDUCE-5475. MRClientService does not verify ACLs properly. Contributed by Jason Lowe
...
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/branches/branch-2@1520157 13f79535-47bb-0310-9956-ffa450edef68
2013-09-04 22:32:35 +00:00
ab02296741
HADOOP-9918. Add addIfService to CompositeService (Karthik Kambatla via Sandy Ryza)
...
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/branches/branch-2@1519130 13f79535-47bb-0310-9956-ffa450edef68
2013-08-31 02:13:11 +00:00
Suresh Srinivas
Sanford Ryza
Siddharth Seth
Vinod Kumar Vavilapalli
Jason Darrell Lowe
Daryn Sharp
Chris Nauroth
Hitesh Shah
Jonathan Turner Eagles
Bikas Saha
Arun Murthy
Colin McCabe
Robert Joseph Evans
Thomas Graves
Alejandro Abdelnur
Devarajulu K