44ea6e3ad5
Back out revision 1503499 for MAPREDUCE-5317. Stale files left behind for failed jobs
...
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1503744 13f79535-47bb-0310-9956-ffa450edef68
2013-07-16 15:05:16 +00:00
8a037edbfb
MAPREDUCE-5317. Stale files left behind for failed jobs. Contributed by Ravi Prakash
...
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1503499 13f79535-47bb-0310-9956-ffa450edef68
2013-07-15 22:24:37 +00:00
413bddf596
MAPREDUCE-4374. Fix child task environment variable config and add support for Windows. Contributed by Chuan Liu.
...
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1502046 13f79535-47bb-0310-9956-ffa450edef68
2013-07-10 22:59:08 +00:00
e2cce7bd26
YARN-727, MAPREDUCE-5325. ClientRMProtocol.getAllApplications should accept ApplicationType as a parameter. Contributed by Xuan Gong.
...
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1501599 13f79535-47bb-0310-9956-ffa450edef68
2013-07-09 23:01:01 +00:00
5e4f6ad1d9
YARN-791. Changed RM APIs and web-services related to nodes to ensure that both are consistent with each other. Contributed by Sandy Ryza.
...
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1500994 13f79535-47bb-0310-9956-ffa450edef68
2013-07-08 22:30:52 +00:00
4175e41548
MAPREDUCE-5366. TestMRAsyncDiskService fails on Windows. Contributed by Chuan Liu.
...
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1500842 13f79535-47bb-0310-9956-ffa450edef68
2013-07-08 17:07:38 +00:00
e01cbdac25
MAPREDUCE-5360. TestMRJobClient fails on Windows due to path format. Contributed by Chuan Liu.
...
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1500194 13f79535-47bb-0310-9956-ffa450edef68
2013-07-06 03:34:23 +00:00
cf4d0cf189
MAPREDUCE-5358. MRAppMaster throws invalid transitions for JobImpl. Contributed by Devaraj K
...
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1499425 13f79535-47bb-0310-9956-ffa450edef68
2013-07-03 14:41:18 +00:00
5b64e3c2be
MAPREDUCE-5357. Job staging directory owner checking could fail on Windows.
...
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1499210 13f79535-47bb-0310-9956-ffa450edef68
2013-07-03 05:15:21 +00:00
06ea8ffcf2
MAPREDUCE-5359. JobHistory should not use File.separator to match timestamp in path. Contributed by Chuan Liu.
...
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1499153 13f79535-47bb-0310-9956-ffa450edef68
2013-07-02 23:57:30 +00:00
f047f9cac6
MAPREDUCE-5355. MiniMRYarnCluster with localFs does not work on Windows. Contributed by Chuan Liu.
...
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1499148 13f79535-47bb-0310-9956-ffa450edef68
2013-07-02 23:19:36 +00:00
e846c98397
MAPREDUCE-3193. FileInputFormat doesn't read files recursively in the input path dir. Contributed by Devaraj K
...
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1499125 13f79535-47bb-0310-9956-ffa450edef68
2013-07-02 21:47:17 +00:00
27a50a2bef
MAPREDUCE-5333. Add test that verifies MRAM works correctly when sending requests with non-normalized capabilities. (ywskycn via tucu)
...
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1497877 13f79535-47bb-0310-9956-ffa450edef68
2013-06-28 18:53:19 +00:00
11bcd2ed12
MAPREDUCE-5326. Added version to shuffle header. Contributed by Zhijie Shen.
...
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1496741 13f79535-47bb-0310-9956-ffa450edef68
2013-06-26 05:49:31 +00:00
1a06175440
YARN-851. Share NMTokens using NMTokenCache (api-based) between AMRMClient and NMClient instead of memory based approach which is used currently. Contributed by Omkar Vinit Joshi.
...
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1495247 13f79535-47bb-0310-9956-ffa450edef68
2013-06-21 00:08:13 +00:00
84862b54aa
MAPREDUCE-5334. Fix failing unit tests - TestContainerLauncher, TestContainerLauncherImpl. Contributed by Vinod Kumar Vavilapalli.
...
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1494820 13f79535-47bb-0310-9956-ffa450edef68
2013-06-20 00:35:51 +00:00
1ab4659b6b
MAPREDUCE-5084. fix coverage org.apache.hadoop.mapreduce.v2.app.webapp and org.apache.hadoop.mapreduce.v2.hs.webapp (Aleksey Gorshkov via jeagles)
...
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1494676 13f79535-47bb-0310-9956-ffa450edef68
2013-06-19 16:22:12 +00:00
d3198dddc8
YARN-553. Replaced YarnClient.getNewApplication with YarnClient.createApplication which provides a directly usable ApplicationSubmissionContext to simplify the api. Contributed by Karthik Kambatla.
...
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1494476 13f79535-47bb-0310-9956-ffa450edef68
2013-06-19 07:06:54 +00:00
16616f9e5b
MAPREDUCE-5194. Heed interrupts during Fetcher shutdown.
...
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1494416 13f79535-47bb-0310-9956-ffa450edef68
2013-06-19 02:12:12 +00:00
243bcd367f
YARN-694. Starting to use NMTokens to authenticate all communication with NodeManagers. Contributed by Omkar Vinit Joshi.
...
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1494369 13f79535-47bb-0310-9956-ffa450edef68
2013-06-18 23:19:49 +00:00
52fe4e6be9
MAPREDUCE-5316. job -list-attempt-ids command does not handle illegal task-state. Contributed by Ashwin Shankar
...
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1494285 13f79535-47bb-0310-9956-ffa450edef68
2013-06-18 20:25:00 +00:00
f4d80e91ae
YARN-841. Move Auxiliary service to yarn-api, annotate and document it. Contributed by Vinod Kumar Vavilapalli.
...
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1494031 13f79535-47bb-0310-9956-ffa450edef68
2013-06-18 06:20:37 +00:00
acc0d3eb52
YARN-840. Moved ProtoUtils to yarn.api.records.pb.impl. Contributed by Jian He.
...
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1494027 13f79535-47bb-0310-9956-ffa450edef68
2013-06-18 05:47:54 +00:00
7ef54faad4
YARN-834. Fixed annotations for yarn-client module, reorganized packages and clearly differentiated *Async apis. Contributed by Arun C Murthy and Zhijie Shen.
...
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1494017 13f79535-47bb-0310-9956-ffa450edef68
2013-06-18 04:02:47 +00:00
2b14656ab5
YARN-805. Fix javadoc and annotations on classes in the yarn-api package. Contributed by Jian He.
...
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1493992 13f79535-47bb-0310-9956-ffa450edef68
2013-06-18 03:10:11 +00:00
5d1b453b85
YARN-610. ClientToken is no longer set in the environment of the Containers. Contributed by Omkar Vinit Joshi.
...
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1493968 13f79535-47bb-0310-9956-ffa450edef68
2013-06-17 23:13:17 +00:00
7d0c43d7cf
YARN-822. Renamed ApplicationToken to be AMRMToken, and similarly the corresponding TokenSelector and SecretManager. Contributed by Omkar Vinit Joshi.
...
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1493889 13f79535-47bb-0310-9956-ffa450edef68
2013-06-17 19:11:29 +00:00
cd30058193
YARN-837. Moved yarn.ClusterInfo into MapReduce project as it doesn't belong to YARN. Contributed by Zhijie Shen.
...
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1493676 13f79535-47bb-0310-9956-ffa450edef68
2013-06-17 08:13:48 +00:00
4662b8b443
YARN-833. Moved Graph and VisualizeStateMachine into yarn.state package. Contributed by Zhijie Shen.
...
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1493654 13f79535-47bb-0310-9956-ffa450edef68
2013-06-17 06:51:51 +00:00
b9efe6bd4a
HADOOP-9649. Promoted YARN service life-cycle libraries into Hadoop Common for usage across all Hadoop projects. Contributed by Zhijie Shen.
...
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1493650 13f79535-47bb-0310-9956-ffa450edef68
2013-06-17 06:39:33 +00:00
d4a811edb2
YARN-826. Moved Clock and SystemClock into yarn.util package. Contributed by Zhijie Shen.
...
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1493639 13f79535-47bb-0310-9956-ffa450edef68
2013-06-17 04:49:13 +00:00
77e134d13f
YARN-387. Renamed YARN protocols for consistency. Contributed by Vinod K V.
...
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1493623 13f79535-47bb-0310-9956-ffa450edef68
2013-06-17 02:27:18 +00:00
789c03438c
MAPREDUCE-5171. Expose blacklisted nodes from the MR AM REST API. (sandyr via tucu)
...
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1493600 13f79535-47bb-0310-9956-ffa450edef68
2013-06-16 22:15:37 +00:00
e3769ea1d2
YARN-829. Renamed RMTokenSelector to be RMDelegationTokenSelector. Contributed by Zhijie Shen.
...
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1493577 13f79535-47bb-0310-9956-ffa450edef68
2013-06-16 20:33:30 +00:00
7f96128e43
MAPREDUCE-5184. Document compatibility for MapReduce applications in hadoop-2 vis-a-vis hadoop-1. Contributed by Zhijie Shen.
...
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1493570 13f79535-47bb-0310-9956-ffa450edef68
2013-06-16 19:09:48 +00:00
98d97d316c
YARN-787. Removed minimum resource from RegisterApplicationMasterResponse. Contributed by Alejandro Abdelnur.
...
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1493509 13f79535-47bb-0310-9956-ffa450edef68
2013-06-16 13:59:41 +00:00
1a389305b2
MAPREDUCE-5192. Allow for alternate resolutions of TaskCompletionEvents. Contributed by Chris Douglas.
...
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1493445 13f79535-47bb-0310-9956-ffa450edef68
2013-06-16 03:07:12 +00:00
c2d0cf1b20
YARN-821. Renamed setFinishApplicationStatus to setFinalApplicationStatus in FinishApplicationMasterRequest for consistency. Contributed by Jian He.
...
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1493315 13f79535-47bb-0310-9956-ffa450edef68
2013-06-15 04:57:04 +00:00
079ed1c9e1
MAPREDUCE-5310. MRAM should not normalize allocation request capabilities. (tucu)
...
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1493207 13f79535-47bb-0310-9956-ffa450edef68
2013-06-14 19:06:50 +00:00
fdc9412a81
YARN-806. Moved ContainerExitStatus from yarn.api to yarn.api.records. Contributed by Jian He.
...
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1493138 13f79535-47bb-0310-9956-ffa450edef68
2013-06-14 16:10:37 +00:00
2166fa201e
MAPREDUCE-5319. Set user.name in job.xml. Contributed by Xuan Gong.
...
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1492962 13f79535-47bb-0310-9956-ffa450edef68
2013-06-14 06:26:55 +00:00
c0cd68d8fb
YARN-773. Moved YarnRuntimeException from package api.yarn to api.yarn.exceptions. Contributed by Jian He.
...
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1492903 13f79535-47bb-0310-9956-ffa450edef68
2013-06-13 23:25:14 +00:00
8dc0d5af43
MAPREDUCE-4019. -list-attempt-ids is not working. Contributed by Ashwin Shankar, Devaraj K, and B Anil Kumar
...
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1492868 13f79535-47bb-0310-9956-ffa450edef68
2013-06-13 21:01:37 +00:00
b64572b06b
MAPREDUCE-5199. Removing ApplicationTokens file as it is no longer needed. Contributed by Daryn Sharp.
...
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1492848 13f79535-47bb-0310-9956-ffa450edef68
2013-06-13 20:20:33 +00:00
0928502029
YARN-530. Defined Service model strictly, implemented AbstractService for robust subclassing and migrated yarn-common services. Contributed by Steve Loughran.
...
YARN-117. Migrated rest of YARN to the new service model. Contributed by Steve Louhran.
MAPREDUCE-5298. Moved MapReduce services to YARN-530 stricter lifecycle. Contributed by Steve Loughran.
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1492718 13f79535-47bb-0310-9956-ffa450edef68
2013-06-13 15:54:38 +00:00
b090bc1c21
MAPREDUCE-5259. TestTaskLog fails on Windows because of path separators missmatch. Contributed by Ivan Mitic.
...
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1492364 13f79535-47bb-0310-9956-ffa450edef68
2013-06-12 18:54:56 +00:00
c9fce677b3
MAPREDUCE-5304. mapreduce.Job killTask/failTask/getTaskCompletionEvents methods have incompatible signature changes. (kkambatl via tucu)
...
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1492360 13f79535-47bb-0310-9956-ffa450edef68
2013-06-12 18:47:33 +00:00
c0737b0bdb
MAPREDUCE-5308. Shuffling to memory can get out-of-sync when fetching multiple compressed map outputs. Contributed by Nathan Roberts
...
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1491611 13f79535-47bb-0310-9956-ffa450edef68
2013-06-10 21:39:34 +00:00
d00d968c49
MAPREDUCE-5312. TestRMNMInfo is failing. (sandyr via tucu)
...
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1491043 13f79535-47bb-0310-9956-ffa450edef68
2013-06-08 19:40:59 +00:00
9fcfbf5f51
YARN-759. Create Command enum in AllocateResponse (bikas)
...
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1490470 13f79535-47bb-0310-9956-ffa450edef68
2013-06-06 23:33:48 +00:00
2051fd5ee2
YARN-750. Allow for black-listing resources in YARN API and Impl in CS (acmurthy via bikas)
...
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1490392 13f79535-47bb-0310-9956-ffa450edef68
2013-06-06 18:46:59 +00:00
66de4ccf60
YARN-724. Moved ProtoBase from api.records to api.records.impl.pb. Contributed by Jian He.
...
MAPREDUCE-5303. Changed MR app after moving ProtoBase to package impl.pb via YARN-724. Contributed by Jian He.
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1489658 13f79535-47bb-0310-9956-ffa450edef68
2013-06-04 22:43:19 +00:00
978012b9b6
YARN-755. Renamed AllocateResponse.reboot to AllocateResponse.resync. Contributed by Bikas Saha.
...
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1489295 13f79535-47bb-0310-9956-ffa450edef68
2013-06-04 05:53:47 +00:00
5c097a134e
YARN-635 and MAPREDUCE-5301. Bug fix as the committed patch ended up being stale.
...
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1489287 13f79535-47bb-0310-9956-ffa450edef68
2013-06-04 04:41:55 +00:00
a83fb61ac0
YARN-635. Renamed YarnRemoteException to YarnException. Contributed by Siddharth Seth.
...
MAPREDUCE-5301. Updated MR code to work with YARN-635 changes of renaming YarnRemoteException to YarnException. Contributed by Siddharth Seth
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1489283 13f79535-47bb-0310-9956-ffa450edef68
2013-06-04 04:05:50 +00:00
78daceaea4
MAPREDUCE-5283. Over 10 different tests have near identical implementations of AppContext. Committed by Sandy Ryza
...
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1489271 13f79535-47bb-0310-9956-ffa450edef68
2013-06-04 01:50:15 +00:00
d33534c4fb
YARN-748. Moved BuilderUtils from yarn-common to yarn-server-common for eventual retirement. Contributed by Jian He.
...
MAPREDUCE-5297. Updated MR App since BuilderUtils is no longer public after YARN-748. Contributed by Jian He.
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1489257 13f79535-47bb-0310-9956-ffa450edef68
2013-06-04 00:34:18 +00:00
72e1568be3
MAPREDUCE-5245. Added back constants to JobConf to fix incompatibilities. Contributed by Zhijie Shen.
...
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1489236 13f79535-47bb-0310-9956-ffa450edef68
2013-06-03 23:56:33 +00:00
78331cfeca
MAPREDUCE-5300. Fix backward incompatibility for o.a.h.mapreduce.filecache.DistributedCache. Contributed by Zhijie Shen.
...
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1489227 13f79535-47bb-0310-9956-ffa450edef68
2013-06-03 23:42:28 +00:00
1caf3e363f
MAPREDUCE-5296. Fix backward incompatibility for JobControl by adding the omitted addJob. Contributed by Zhijie Shen.
...
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1489219 13f79535-47bb-0310-9956-ffa450edef68
2013-06-03 22:54:48 +00:00
0add6b4edc
MAPREDUCE-5220. Fix backward incompatibility for TaskCompletionEvent by adding back setTaskID. Contributed by Zhijie Shen.
...
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1489187 13f79535-47bb-0310-9956-ffa450edef68
2013-06-03 21:35:04 +00:00
9a3379f6b1
MAPREDUCE-5036. Default shuffle handler port should not be 8080. Contributed by Sandy Ryza.
...
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1489060 13f79535-47bb-0310-9956-ffa450edef68
2013-06-03 16:53:13 +00:00
ac44e0a0d0
MAPREDUCE-5268. Improve history server startup performance. Contributed by Karthik Kambatla
...
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1489012 13f79535-47bb-0310-9956-ffa450edef68
2013-06-03 14:46:30 +00:00
a2205a3b72
YARN-720 and MAPREDUCE-5291. container-log4j.properties should not refer to mapreduce properties. Update MRApp to use YARN properties for log setup. Contributed by Zhijie Shen.
...
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1488829 13f79535-47bb-0310-9956-ffa450edef68
2013-06-03 02:35:34 +00:00
f27e484636
YARN-749. Rename ResourceRequest.(get,set)HostName to ResourceRequest.(get,set)ResourceName. Contributed by Arun C. Murthy.
...
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1488806 13f79535-47bb-0310-9956-ffa450edef68
2013-06-02 22:53:58 +00:00
05cdca070e
MAPREDUCE-5274. Bring back SecureShuffleUtils.toHex in mapreduce for binary compatibility with 1.x APIs. Contributed by Mayank Bansal.
...
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1488625 13f79535-47bb-0310-9956-ffa450edef68
2013-06-01 22:21:43 +00:00
8267117e10
MAPREDUCE-5229. Put back FileOutputCommitter.TEMP_DIR_NAME in mapreduce for binary compatibility with 1.x APIs. Contributed by Zhijie Shen.
...
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1488621 13f79535-47bb-0310-9956-ffa450edef68
2013-06-01 22:10:42 +00:00
a2c4233004
YARN-717. Put object creation factories for Token in the class itself and remove useless derivations for specific tokens. Contributed by Jian He.
...
MAPREDUCE-5289. Updated MR App to use Token directly after YARN-717. Contributed by Jian He.
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1488616 13f79535-47bb-0310-9956-ffa450edef68
2013-06-01 21:43:17 +00:00
a77030abdc
MAPREDUCE-5280. Bring back removed constructor and a method in mapreduce ClusterMetrics for binary compatibility with 1.x APIs. Contributed by Mayank Bansal.
...
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1488458 13f79535-47bb-0310-9956-ffa450edef68
2013-06-01 03:57:40 +00:00
b8e9f06774
MAPREDUCE-5273. Bring back a couple of protected variables in mapred and mapreduce CombineFileRecordReader for binary compatibility with 1.x APIs. Contributed by Mayank Bansal.
...
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1488442 13f79535-47bb-0310-9956-ffa450edef68
2013-06-01 00:33:49 +00:00
39f019f413
YARN-735. Make ApplicationAttemptId, ContainerId and NodeId immutable. Contributed by Jian He.
...
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1488439 13f79535-47bb-0310-9956-ffa450edef68
2013-06-01 00:14:31 +00:00
b65e7ed920
MAPREDUCE-5231. Bring back a constructor in mapred's DBInputFormat.DBRecordReader for binary compatibility with 1.x mapred APIs. Contributed by Zhijie Shen.
...
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1488436 13f79535-47bb-0310-9956-ffa450edef68
2013-05-31 23:56:28 +00:00
73b88956f5
MAPREDUCE-5275. Bring back a couple of APIs in mapreduce.security.TokenCache for binary compatibility with 1.x mapreduce APIs. Contributed by Mayank Bansal.
...
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1488369 13f79535-47bb-0310-9956-ffa450edef68
2013-05-31 19:48:03 +00:00
0a452a30ab
MAPREDUCE-5176. Add annotation for tagging tasks as responsive to
...
preemption. Contributed by Carlo Curino, cdouglas
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1488103 13f79535-47bb-0310-9956-ffa450edef68
2013-05-31 06:07:18 +00:00
f3f7a11336
MAPREDUCE-5286. Change MapReduce to use ContainerTokenIdentifier instead of the entire Container in the startContainer call - YARN-684. Contributed by Vinod Kumar Vavilapalli.
...
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1488087 13f79535-47bb-0310-9956-ffa450edef68
2013-05-31 04:17:33 +00:00
4394e5edb0
MAPREDUCE-5228. Bring back FileInputFormat.Counter and FileOuputFormat.Counter for binary compatibility with 1.x mapred APIs. Contributed by Mayank Bansal.
...
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1488060 13f79535-47bb-0310-9956-ffa450edef68
2013-05-30 23:53:21 +00:00
131bfc91a6
MAPREDUCE-5261. Reverting the patch as it is no longer needed.
...
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1488032 13f79535-47bb-0310-9956-ffa450edef68
2013-05-30 21:53:34 +00:00
982753dc8e
YARN-716. Making ApplicationID immutable. Contributed by Siddharth Seth.
...
MAPREDUCE-5282. Updating MR App to use immutable ApplicationID after YARN-716. Contributed by Siddharth Seth.
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1487994 13f79535-47bb-0310-9956-ffa450edef68
2013-05-30 20:18:39 +00:00
b16c5638b5
YARN-719. Move RMIdentifier from Container to ContainerTokenIdentifier. Contributed by Vinod Kumar Vavilapalli.
...
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1487741 13f79535-47bb-0310-9956-ffa450edef68
2013-05-30 04:59:05 +00:00
a720a524e9
MAPREDUCE-5263. Bring back old methods and fields in filecache.DistributedCache for binary compatibility with mapred in 1.x. Contributed by Zhijie Shen.
...
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1487695 13f79535-47bb-0310-9956-ffa450edef68
2013-05-30 01:23:16 +00:00
0edae7a96a
MAPREDUCE-5237. Add methods that were removed from ClusterStatus back into 2.x so as to be compatible with 1.x. Contributed by Zhijie Shen.
...
Incompatible as ClusterStatus.UNINITIALIZED_MEMORY_VALUE is a long now and so breaks 0.23.x but it shouldn't be a big deal in reality.
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1487175 13f79535-47bb-0310-9956-ffa450edef68
2013-05-29 00:43:13 +00:00
643155cbee
MAPREDUCE-5270. Migrated MR app from using BuilderUtil factory methods to individual record factory methods. Contributed by Jian He.
...
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1486271 13f79535-47bb-0310-9956-ffa450edef68
2013-05-25 01:46:14 +00:00
259edf8dca
YARN-571. Remove user from ContainerLaunchContext. Contributed by Omkar Vinit Joshi.
...
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1485928 13f79535-47bb-0310-9956-ffa450edef68
2013-05-24 03:22:54 +00:00
a791527dd1
MAPREDUCE-5230. Bring back NLineInputFormat.createFileSplit for binary compatibility with mapred in 1.x Contributed by Mayank Bansal.
...
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1485906 13f79535-47bb-0310-9956-ffa450edef68
2013-05-24 00:42:22 +00:00
43876770d9
YARN-563. Add the concept of an application-type for each application. Contributed by Mayank Bansal.
...
MAPREDUCE-5246. Specify application-type at the time of job submission after YARN-563. Contributed by Mayank Bansal.
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1485790 13f79535-47bb-0310-9956-ffa450edef68
2013-05-23 18:02:04 +00:00
9b284150f1
MAPREDUCE-5233. Add methods that are changed or removed from JobControl.Job when compared to 1.x. This breaks 0.23.x users of one API in Job. Contributed by Mayank Bansal.
...
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1485491 13f79535-47bb-0310-9956-ffa450edef68
2013-05-22 23:12:45 +00:00
27e6673ec3
MAPREDUCE-5156. Change hadoop examples ProgramDriver to be able to run 1.x examples jar on top of YARN. This change breaks 0.23.x direct usages of ProgramDriver. Contributed by Zhijie Shen.
...
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1485085 13f79535-47bb-0310-9956-ffa450edef68
2013-05-22 06:36:24 +00:00
4a4e42d919
MAPREDUCE-5261. Fix issues in TestRMContainerAllocator after YARN-617. Contributed by Omkar Vinit Joshi.
...
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1485079 13f79535-47bb-0310-9956-ffa450edef68
2013-05-22 06:24:17 +00:00
6377bfc505
MAPREDUCE-5235. Bring back old fields and exceptions in Counters for binary compatibility with mapred in 1.x. Contributed by Mayank Bansal
...
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1484992 13f79535-47bb-0310-9956-ffa450edef68
2013-05-21 22:00:08 +00:00
6517c3f714
MAPREDUCE-5191. TestQueue#testQueue fails with timeout on Windows. (Contributed by Ivan Mitic)
...
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1484575 13f79535-47bb-0310-9956-ffa450edef68
2013-05-20 20:02:07 +00:00
9cf436aef5
MAPREDUCE-5257. Fix issues in TestContainerLauncherImpl after YARN-617. Contributed by Omkar Vinit Joshi.
...
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1484349 13f79535-47bb-0310-9956-ffa450edef68
2013-05-19 21:15:11 +00:00
ca2265b581
MAPREDUCE-4927. Historyserver 500 error due to NPE when accessing specific counters page for failed job. Contributed by Ashwin Shankar
...
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1483974 13f79535-47bb-0310-9956-ffa450edef68
2013-05-17 20:19:48 +00:00
0b668c21b2
MAPREDUCE-5234. Change mapred.TaskReport and mapreduce.TaskReport for binary compatibility with mapred in 1.x but incompatible with 0.23.x. Contributed by Mayank Bansal.
...
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1483940 13f79535-47bb-0310-9956-ffa450edef68
2013-05-17 18:16:33 +00:00
ca80246731
YARN-617. Made ContainerTokens to be used for validation at NodeManager also in unsecure mode to prevent AMs from faking resource requirements in unsecure mode. Contributed by Omkar Vinit Joshi.
...
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1483667 13f79535-47bb-0310-9956-ffa450edef68
2013-05-17 06:36:33 +00:00
4d8e350750
MAPREDUCE-5232. Add a configuration to be able to log classpath and other system properties on mapreduce JVMs startup. Contributed by Sangjin Lee.
...
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1482643 13f79535-47bb-0310-9956-ffa450edef68
2013-05-14 23:43:53 +00:00
3472800eb6
MAPREDUCE-5244. Two functions changed their visibility in JobStatus. (zjshen via tucu)
...
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1482622 13f79535-47bb-0310-9956-ffa450edef68
2013-05-14 22:12:40 +00:00
febf951220
MAPREDUCE-5222. Bring back some methods and constants in Jobclient for binary compatibility with mapred in 1.x. Contributed by Karthik Kambatla.
...
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1482208 13f79535-47bb-0310-9956-ffa450edef68
2013-05-14 05:52:11 +00:00
1a119f87b4
YARN-615. Rename ContainerLaunchContext.containerTokens to tokens. Contributed by Vinod Kumar Vavilapalli.
...
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1482199 13f79535-47bb-0310-9956-ffa450edef68
2013-05-14 04:11:55 +00:00
14d856be57
MAPREDUCE-5240. Fix a bug in MRAppMaster because of which OutputCommitter could not access credentials set by the user. Contributed by Vinod Kumar Vavilapalli.
...
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1482175 13f79535-47bb-0310-9956-ffa450edef68
2013-05-14 02:48:53 +00:00
2103c80b77
MAPREDUCE-5220. Setter methods in TaskCompletionEvent are public in MR1 and protected in MR2. (sandyr via tucu)
...
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1482142 13f79535-47bb-0310-9956-ffa450edef68
2013-05-13 23:08:10 +00:00
7359dc32d3
MAPREDUCE-4067. Changed MRClientProtocol api to throw IOException only (Xuan Gong via vinodkv)
...
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1481695 13f79535-47bb-0310-9956-ffa450edef68
2013-05-13 03:34:25 +00:00
47d1ca402f
MAPREDUCE-5208. Modified ShuffleHandler to use SecureIOUtils for reading local files. Contributed by Omkar Vinit Joshi.
...
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1481657 13f79535-47bb-0310-9956-ffa450edef68
2013-05-12 21:59:16 +00:00
2638bc67a4
YARN-634. Modified YarnRemoteException to be not backed by PB and introduced a separate SerializedException record. Contributed by Siddharth Seth.
...
MAPREDUCE-5239. Updated MR App to reflect YarnRemoteException changes after YARN-634. Contributed by Siddharth Seth.
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1481205 13f79535-47bb-0310-9956-ffa450edef68
2013-05-10 21:49:28 +00:00
d303f01297
MAPREDUCE-4942. mapreduce.Job has a bunch of methods that throw InterruptedException so its incompatible with MR1. (rkanter via tucu)
...
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1480748 13f79535-47bb-0310-9956-ffa450edef68
2013-05-09 20:04:32 +00:00
9c4f86879c
YARN-630. Changed AMRMProtocol api to throw IOException and YarnRemoteException. Contributed by Xuan Gong.
...
MAPREDUCE-5226. Handling YarnRemoteException separately from IOException in MR App's use of AMRMProtocol after YARN-630. Contributed by Xuan Gong.
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1480529 13f79535-47bb-0310-9956-ffa450edef68
2013-05-09 05:32:04 +00:00
0727ecaf50
YARN-631. Changed ClientRMProtocol api to throw IOException and YarnRemoteException. Contributed by Xuan Gong.
...
MAPREDUCE-5212. Handling YarnRemoteException separately from IOException in MR App after YARN-631. Contributed by Xuan Gong.
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1480494 13f79535-47bb-0310-9956-ffa450edef68
2013-05-08 22:53:01 +00:00
f6167ef6c6
MAPREDUCE-5157. Bring back old sampler related code so that we can support binary compatibility with hadoop-1 sorter example. Contributed by Zhijie Shen.
...
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1480474 13f79535-47bb-0310-9956-ffa450edef68
2013-05-08 21:42:47 +00:00
67d7d8f4d3
MAPREDUCE-5209. Fix units in a ShuffleScheduler log message.
...
Contributed by Tsuyoshi OZAWA
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1480464 13f79535-47bb-0310-9956-ffa450edef68
2013-05-08 21:12:27 +00:00
8888d3fc49
MAPREDUCE-5159. Change ValueAggregatorJob to add APIs which can support binary compatibility with hadoop-1 examples. Contributed by Zhijie Shen.
...
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1480394 13f79535-47bb-0310-9956-ffa450edef68
2013-05-08 18:28:22 +00:00
92b7165a71
YARN-629. Make YarnRemoteException not be rooted at IOException. Contributed by Xuan Gong.
...
MAPREDUCE-5204. Handling YarnRemoteException separately from IOException in MR app after YARN-629. Contributed by Xuan Gong.
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1479680 13f79535-47bb-0310-9956-ffa450edef68
2013-05-06 19:04:31 +00:00
6e37a7e5e1
MAPREDUCE-5205. Fixed MR App to load tokens correctly. Contributed by Vinod Kumar Vavilapalli.
...
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1479016 13f79535-47bb-0310-9956-ffa450edef68
2013-05-04 00:27:47 +00:00
fc275fed66
YARN-625. Move the utility method unwrapAndThrowException from YarnRemoteExceptionPBImpl to RPCUtil. Contributed by Siddharth Seth.
...
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1478594 13f79535-47bb-0310-9956-ffa450edef68
2013-05-02 22:44:12 +00:00
9b97df9abe
MAPREDUCE-5036. Default shuffle handler port should not be 8080. Contributed by Sandy Ryza.
...
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1478422 13f79535-47bb-0310-9956-ffa450edef68
2013-05-02 15:58:53 +00:00
ca2c38fba1
MAPREDUCE-5193. A few MR tests use block sizes which are smaller than the default minimum block size. Contributed by Andrew Wang.
...
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1478250 13f79535-47bb-0310-9956-ffa450edef68
2013-05-02 01:59:36 +00:00
71be8ad325
MAPREDUCE-5145. Changed default max-attempts to be more than one for MR jobs inline with YARN. Contributed by Zhijie Shen.
...
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1477889 13f79535-47bb-0310-9956-ffa450edef68
2013-05-01 00:42:32 +00:00
8f7ce62085
MAPREDUCE-5177. Use common utils FileUtil#setReadable/Writable/Executable & FileUtil#canRead/Write/Execute. Contributed by Ivan Mitic.
...
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1477403 13f79535-47bb-0310-9956-ffa450edef68
2013-04-29 23:00:39 +00:00
ae393ab26e
MAPREDUCE-5179. Fix unit test in TestHSWebServices which fails when versionInfo has parantheses like when running on a git checkout. Contributed by Hitesh Shah.
...
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1476835 13f79535-47bb-0310-9956-ffa450edef68
2013-04-28 21:02:25 +00:00
fe3853484e
HADOOP-9290. Some tests cannot load native library on windows. Contributed by Chris Nauroth.
...
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1476586 13f79535-47bb-0310-9956-ffa450edef68
2013-04-27 14:24:02 +00:00
fbb55784d9
YARN-562. Modified NM to reject any containers allocated by a previous ResourceManager. Contributed by Jian He.
...
MAPREDUCE-5167. Update MR App after YARN-562 to use the new builder API for the container. Contributed by Jian He.
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1476034 13f79535-47bb-0310-9956-ffa450edef68
2013-04-26 03:50:20 +00:00
448df18646
YARN-577. Add application-progress also to ApplicationReport. Contributed by Hitesh Shah.
...
MAPREDUCE-5178. Update MR App to set progress in ApplicationReport after YARN-577. Contributed by Hitesh Shah.
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1475636 13f79535-47bb-0310-9956-ffa450edef68
2013-04-25 05:42:22 +00:00
e96077aaf4
MAPREDUCE-5181. RMCommunicator should not use AMToken from the env. Contributed by Vinod Kumar Vavilapalli.
...
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1471816 13f79535-47bb-0310-9956-ffa450edef68
2013-04-25 02:50:16 +00:00
daaa73b657
MAPREDUCE-5166. Fix ConcurrentModificationException due to insufficient synchronization on updates to task Counters. Contributed by Sandy Ryza.
...
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1471796 13f79535-47bb-0310-9956-ffa450edef68
2013-04-25 00:54:04 +00:00
40e78c2ca2
MAPREDUCE-4737. Ensure that mapreduce APIs are semantically consistent with mapred API w.r.t Mapper.cleanup and Reducer.cleanup; in the sense that cleanup is now called even if there is an error. The old mapred API already ensures that Mapper.close and Reducer.close are invoked during error handling. Note that it is an incompatible change, however end-users can override Mapper.run and Reducer.run to get the old (inconsistent) behaviour. Contributed by Arun C. Murthy.
...
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1471556 13f79535-47bb-0310-9956-ffa450edef68
2013-04-24 17:38:43 +00:00
0777474cc2
MAPREDUCE-5069. add concrete common implementations of CombineFileInputFormat (Sangjin Lee via bobby)
...
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1471424 13f79535-47bb-0310-9956-ffa450edef68
2013-04-24 14:11:50 +00:00
27e8c86999
YARN-561. Modified NodeManager to set key information into the environment of every container that it launches. Contributed by Xuan Gong.
...
MAPREDUCE-5175. Updated MR App to not set envs that will be set by NMs anyways after YARN-561. Contributed by Xuan Gong.
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1471156 13f79535-47bb-0310-9956-ffa450edef68
2013-04-23 21:39:51 +00:00
009af54d51
MAPREDUCE-5146. application classloader may be used too early to load classes. Contributed by Sangjin Lee.
...
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1470694 13f79535-47bb-0310-9956-ffa450edef68
2013-04-22 20:07:06 +00:00
794f9bb3e4
MAPREDUCE-5066. Added a timeout for the job.end.notification.url. Contributed by Ivan Mitic.
...
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1470216 13f79535-47bb-0310-9956-ffa450edef68
2013-04-20 19:18:24 +00:00
febc52be37
MAPREDUCE-5147. Maven build should create hadoop-mapreduce-client-app-VERSION.jar directly (Robert Parker via tgraves)
...
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1470035 13f79535-47bb-0310-9956-ffa450edef68
2013-04-19 20:50:48 +00:00
c1ce3c5ad2
MAPREDUCE-4987. TestMRJobs#testDistributedCache fails on Windows due to classpath problems and unexpected behavior of symlinks (Chris Nauroth via bikas)
...
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1470003 13f79535-47bb-0310-9956-ffa450edef68
2013-04-19 19:32:33 +00:00
8e1c2823fc
YARN-441. Removed unused utility methods for collections from two API records. Contributed by Xuan Gong.
...
MAPREDUCE-5163. Update MR App to not use API utility methods for collections after YARN-441. Contributed by Xuan Gong.
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1469657 13f79535-47bb-0310-9956-ffa450edef68
2013-04-19 01:33:13 +00:00
e222d1e783
MAPREDUCE-4932. mapreduce.job#getTaskCompletionEvents incompatible with Hadoop 1. (rkanter via tucu)
...
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1469587 13f79535-47bb-0310-9956-ffa450edef68
2013-04-18 21:42:03 +00:00
cb78a65a15
MAPREDUCE-5152. Make MR App to simply pass through the container from RM instead of extracting and populating information itself to start any container. Contributed by Vinod Kumar Vavilapalli.
...
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1469544 13f79535-47bb-0310-9956-ffa450edef68
2013-04-18 20:13:40 +00:00
d12c463121
MAPREDUCE-4898. FileOutputFormat.checkOutputSpecs and FileOutputFormat.setOutputPath incompatible with MR1. (rkanter via tucu)
...
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1469523 13f79535-47bb-0310-9956-ffa450edef68
2013-04-18 18:51:45 +00:00
cd27afa71a
MAPREDUCE-5128. mapred-default.xml is missing a bunch of history server configs. (sandyr via tucu)
...
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1469095 13f79535-47bb-0310-9956-ffa450edef68
2013-04-17 23:14:34 +00:00
9e650d36b7
MAPREDUCE-5140. MR part of YARN-514 (Zhijie Shen via bikas)
...
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1469060 13f79535-47bb-0310-9956-ffa450edef68
2013-04-17 20:25:21 +00:00
9ea20fc537
MAPREDUCE-5015. Coverage fix for org.apache.hadoop.mapreduce.tools.CLI (Aleksey Gorshkov via tgraves)
...
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1468483 13f79535-47bb-0310-9956-ffa450edef68
2013-04-16 16:11:35 +00:00
0e01f26821
YARN-444. Moved special container exit codes from YarnConfiguration to API where they belong. Contributed by Sandy Ryza.
...
MAPREDUCE-5151. Updated MR AM to use standard exit codes from the API after YARN-444. Contributed by Sandy Ryza.
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1468276 13f79535-47bb-0310-9956-ffa450edef68
2013-04-15 23:54:09 +00:00
947e97f354
MAPREDUCE-4974. Optimising the LineRecordReader initialize() method (Gelesh via bobby)
...
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1468232 13f79535-47bb-0310-9956-ffa450edef68
2013-04-15 21:33:47 +00:00
8e838bd03c
MAPREDUCE-4985. Add compression option to TestDFSIO usage. Contributed by Plamen Jeliazkov.
...
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1467145 13f79535-47bb-0310-9956-ffa450edef68
2013-04-11 23:49:16 +00:00
3c8bd60616
MAPREDUCE-5094. Disabled memory monitoring by default in MiniMRYarnCluster to avoid some downstream tests failing. Contributed by Siddharth Seth.
...
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1467124 13f79535-47bb-0310-9956-ffa450edef68
2013-04-11 22:54:02 +00:00
5fd460e688
MAPREUDUCE-5059. Change average merge time on Job overview page to be the time delta between the end of the shuffle and the start of the reduce. Contributed by Omkar Vinit Joshi.
...
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1467120 13f79535-47bb-0310-9956-ffa450edef68
2013-04-11 22:27:45 +00:00
e4c55e17fe
YARN-486. Changed NM's startContainer API to accept Container record given by RM as a direct parameter instead of as part of the ContainerLaunchContext record. Contributed by Xuan Gong.
...
MAPREDUCE-5139. Update MR AM to use the modified startContainer API after YARN-486. Contributed by Xuan Gong.
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1467063 13f79535-47bb-0310-9956-ffa450edef68
2013-04-11 19:28:51 +00:00
8e86ffd91e
MAPREDUCE-5136. TestJobImpl->testJobNoTasks fails with IBM JAVA. Contributed by Amir Sanjar
...
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1467061 13f79535-47bb-0310-9956-ffa450edef68
2013-04-11 19:12:59 +00:00
d04e65b797
MAPREDUCE-5137. AM web UI: clicking on Map Task results in 500 error. Contributed by Thomas Graves
...
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1466948 13f79535-47bb-0310-9956-ffa450edef68
2013-04-11 16:27:24 +00:00
6a1c41111e
MAPREDUCE-5079. Changes job recovery to restore state directly from job history, instaed of simulating state machine events. Contributed by Jason Lowe and Robert Parker.
...
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1466767 13f79535-47bb-0310-9956-ffa450edef68
2013-04-11 04:52:38 +00:00
3a54a5653b
YARN-112. Fixed a race condition during localization that fails containers. Contributed by Omkar Vinit Joshi.
...
MAPREDUCE-5138. Fix LocalDistributedCacheManager after YARN-112. Contributed by Omkar Vinit Joshi.
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1466196 13f79535-47bb-0310-9956-ffa450edef68
2013-04-09 19:56:10 +00:00
7f13207ed1
MAPREDUCE-5129. Allow tags to JobHistory for deeper analytics. Contributed by Billie Rinaldi.
...
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1465188 13f79535-47bb-0310-9956-ffa450edef68
2013-04-06 05:30:56 +00:00
a5734cd38a
MAPREDUCE-5007. fix coverage org.apache.hadoop.mapreduce.v2.hs (Aleksey Gorshkov via tgraves)
...
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1465017 13f79535-47bb-0310-9956-ffa450edef68
2013-04-05 15:43:53 +00:00
fc75d3f3dc
MAPREDUCE-5086. MR app master deletes staging dir when sent a reboot command from the RM. Contributed by Jian He
...
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1464255 13f79535-47bb-0310-9956-ffa450edef68
2013-04-04 01:56:49 +00:00
d18cc69d4e
MAPREDUCE-5088. MR Client gets an renewer token exception while Oozie is submitting a job (daryn)
...
merge -c1463804 from branch-2.0.4-alpha
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1464153 13f79535-47bb-0310-9956-ffa450edef68
2013-04-03 18:45:15 +00:00
38678cc3d9
MAPREDUCE-5117. Changed MRClientProtocolPBClientImpl to be closeable and thus fix failures in renewal of HistoryServer's delegations tokens. Contributed by Siddharth Seth.
...
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1463828 13f79535-47bb-0310-9956-ffa450edef68
2013-04-03 05:37:10 +00:00
6f0c4dca74
Reverted MAPREDUCE-4974 because of test failures.
...
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1463359 13f79535-47bb-0310-9956-ffa450edef68
2013-04-02 02:13:18 +00:00
b55756dd03
MAPREDUCE-4974. Optimising the LineRecordReader initialize() method (Gelesh via bobby)
...
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1463221 13f79535-47bb-0310-9956-ffa450edef68
2013-04-01 17:45:07 +00:00
520033b1cd
YARN-450. Define value for * in the scheduling protocol (Zhijie Shen via bikas)
...
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1462271 13f79535-47bb-0310-9956-ffa450edef68
2013-03-28 19:44:28 +00:00
375584ae0f
MAPREDUCE-5006. Fix failing streaming tests due to MAPREDUCE-4994. Contributed by Sandy Ryza.
...
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1461541 13f79535-47bb-0310-9956-ffa450edef68
2013-03-27 11:47:06 +00:00
d41e67b966
MAPREDUCE-5077. Remove mapreduce.util.ResourceCalculatorPlugin and related code. Contributed by Karthik Kambatla.
...
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1461251 13f79535-47bb-0310-9956-ffa450edef68
2013-03-26 18:15:40 +00:00
a0a844222d
MAPREDUCE-4875. coverage fixing for org.apache.hadoop.mapred (Aleksey Gorshkov via bobby)
...
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1461146 13f79535-47bb-0310-9956-ffa450edef68
2013-03-26 14:16:13 +00:00
46315a2d91
MAPREDUCE-5062. Fix MR AM to read max-retries from the RM. Contributed by *Zhijie Shen.
...
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1460923 13f79535-47bb-0310-9956-ffa450edef68
2013-03-25 22:33:43 +00:00
b8c74a8a25
MAPREDUCE-5083. MiniMRCluster should use a random component when creating an actual cluster. Contributed by Siddharth Seth.
...
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1459932 13f79535-47bb-0310-9956-ffa450edef68
2013-03-22 19:11:01 +00:00
cd2c5fab8b
MAPREDUCE-3872. Fix an event handling races in ContainerLauncherImpl. (Contributed by Robert Kanter)
...
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1459547 13f79535-47bb-0310-9956-ffa450edef68
2013-03-21 21:55:31 +00:00
6610212dd3
MAPREDUCE-5078. TestMRAppMaster fails on Windows due to mismatched path separators. (Contributed by Chris Nauroth)
...
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1459544 13f79535-47bb-0310-9956-ffa450edef68
2013-03-21 21:44:13 +00:00
1bd345d6e3
YARN-396. Rationalize AllocateResponse in RM Scheduler API. Contributed by Zhijie Shen.
...
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1459040 13f79535-47bb-0310-9956-ffa450edef68
2013-03-20 20:44:35 +00:00
f7df0cb7df
MAPREDUCE-4972. Coverage fixing for org.apache.hadoop.mapreduce.jobhistory (Aleksey Gorshkov via bobby)
...
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1458906 13f79535-47bb-0310-9956-ffa450edef68
2013-03-20 16:05:33 +00:00
c19633da5b
Reverting MAPREDUCE-5028 (commit 1457918)
...
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1458433 13f79535-47bb-0310-9956-ffa450edef68
2013-03-19 17:56:29 +00:00
523d3daac0
MAPREDUCE-5053. java.lang.InternalError from decompression codec cause reducer to fail (Robert Parker via jeagles)
...
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1458350 13f79535-47bb-0310-9956-ffa450edef68
2013-03-19 15:45:35 +00:00
53e7aaa6dd
MAPREDUCE-5028. Maps fail when io.sort.mb is set to high value. (kkambatl via tucu)
...
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1457918 13f79535-47bb-0310-9956-ffa450edef68
2013-03-18 18:56:40 +00:00
5ac6abe107
MAPREDUCE-4990. Construct debug strings conditionally in ShuffleHandler.Shuffle#sendMapOutput(). (kkambatl via tucu)
...
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1457914 13f79535-47bb-0310-9956-ffa450edef68
2013-03-18 18:45:55 +00:00
64741f4635
HDFS-4521. Invalid network toploogies should not be cached. Contributed by Colin Patrick McCabe.
...
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1457878 13f79535-47bb-0310-9956-ffa450edef68
2013-03-18 17:20:13 +00:00
7d7553c4eb
MAPREDUCE-5042. Reducer unable to fetch for a map task that was recovered (Jason Lowe via bobby)
...
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1457119 13f79535-47bb-0310-9956-ffa450edef68
2013-03-15 21:09:25 +00:00
8c8ff3f5e0
MAPREDUCE-4716. TestHsWebServicesJobsQuery.testJobsQueryStateInvalid fails with jdk7. (tgraves via tucu)
...
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1457065 13f79535-47bb-0310-9956-ffa450edef68
2013-03-15 18:55:22 +00:00
d412a1a1a8
MAPREDUCE-4571. TestHsWebServicesJobs fails on jdk7. (tgraves via tucu)
...
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1457061 13f79535-47bb-0310-9956-ffa450edef68
2013-03-15 18:50:51 +00:00
1b6f0582fb
YARN-237. Refreshing the RM page forgets how many rows I had in my Datatables (jian he via bobby)
...
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1456536 13f79535-47bb-0310-9956-ffa450edef68
2013-03-14 16:56:10 +00:00
57803245ec
MAPREDUCE-5060. Fetch failures that time out only count against the first map task. Contributed by Robert Joseph Evans
...
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1455740 13f79535-47bb-0310-9956-ffa450edef68
2013-03-12 22:51:28 +00:00
7122cb5bf4
MAPREDUCE-5023. History Server Web Services missing Job Counters (Ravi Prakash via tgraves)
...
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1454156 13f79535-47bb-0310-9956-ffa450edef68
2013-03-07 23:02:28 +00:00
6942fd1db5
HADOOP-9372. Fix bad timeout annotations on tests. Contributed by Arpit Agarwal.
...
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1453637 13f79535-47bb-0310-9956-ffa450edef68
2013-03-07 01:09:53 +00:00
638801cce1
HADOOP-8952. Enhancements to support Hadoop on Windows Server and Windows Azure environments. Contributed by Ivan Mitic, Chuan Liu, Ramya Sunil, Bikas Saha, Kanna Karanam, John Gordon, Brandon Li, Chris Nauroth, David Lao, Sumadhur Reddy Bolli, Arpit Agarwal, Ahmed El Baz, Mike Liddell, Jing Zhao, Thejas Nair, Steve Maine, Ganeshan Iyer, Raja Aluri, Giridharan Kesavan, Ramya Bharathi Nimmagadda.
...
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1453486 13f79535-47bb-0310-9956-ffa450edef68
2013-03-06 19:15:18 +00:00
df68c56267
MAPREDUCE-3685. Fix bugs in MergeManager to ensure compression codec is appropriately used and that on-disk segments are correctly sorted on file-size. Contributed by Anty Rao and Ravi Prakash.
...
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1453365 13f79535-47bb-0310-9956-ffa450edef68
2013-03-06 15:02:45 +00:00
e1062b8b78
MAPREDUCE-5027. Shuffle does not limit number of outstanding connections (Robert Parker via jeagles)
...
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1453098 13f79535-47bb-0310-9956-ffa450edef68
2013-03-06 00:03:41 +00:00
ec13f1eb3a
MAPREDUCE-5043. Fetch failure processing can cause AM event queue to backup and eventually OOM (Jason Lowe via bobby)
...
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1452372 13f79535-47bb-0310-9956-ffa450edef68
2013-03-04 16:32:18 +00:00
38c4e17ec1
MAPREDUCE-4794. DefaultSpeculator generates error messages on normal shutdown (Jason Lowe via jeagles)
...
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1451826 13f79535-47bb-0310-9956-ffa450edef68
2013-03-02 03:30:58 +00:00
5d6eca08bd
MAPREDUCE-4896. mapred queue -info spits out ugly exception when queue does not exist. (sandyr via tucu)
...
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1451452 13f79535-47bb-0310-9956-ffa450edef68
2013-03-01 01:28:18 +00:00
892e088773
HADOOP-9349. Confusing output when running hadoop version from one hadoop installation when HADOOP_HOME points to another. (sandyr via tucu)
...
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1451448 13f79535-47bb-0310-9956-ffa450edef68
2013-03-01 01:20:53 +00:00
892846dc04
MAPREDUCE-4871. AM uses mapreduce.jobtracker.split.metainfo.maxsize but mapred-default has mapreduce.job.split.metainfo.maxsize (Jason Lowe via jeagles)
...
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1451318 13f79535-47bb-0310-9956-ffa450edef68
2013-02-28 19:55:10 +00:00
979fb054f8
MAPREDUCE-4693. Historyserver should provide counters for failed tasks. Contributed by Xuan Gong.
...
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1450956 13f79535-47bb-0310-9956-ffa450edef68
2013-02-27 21:03:19 +00:00
0b9ed2364a
MAPREDUCE-4892. Modify CombineFileInputFormat to not skew input slits' allocation on small clusters. Contributed by Bikas Saha.
...
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1450912 13f79535-47bb-0310-9956-ffa450edef68
2013-02-27 18:49:37 +00:00
14089f1e57
MAPREDUCE-5008. Merger progress miscounts with respect to EOF_MARKER. Contributed by Sandy Ryza.
...
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1450723 13f79535-47bb-0310-9956-ffa450edef68
2013-02-27 10:40:00 +00:00
70b1c5ca75
HADOOP-9334. Upgrade netty version. Contributed by Nicolas Liochon.
...
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1450463 13f79535-47bb-0310-9956-ffa450edef68
2013-02-26 21:54:25 +00:00
0b73dde6ce
MAPREDUCE-4951. Container preemption interpreted as task failure. Contributed by Sandy Ryza.
...
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1448615 13f79535-47bb-0310-9956-ffa450edef68
2013-02-21 11:56:08 +00:00
87035a004c
MAPREDUCE-5013. mapred.JobStatus compatibility: MR2 missing constructors from MR1. Contributed by Sandy Ryza.
...
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1448602 13f79535-47bb-0310-9956-ffa450edef68
2013-02-21 11:42:48 +00:00
7555b4df74
MAPREDUCE-4846. Some JobQueueInfo methods are public in MR1 but protected in MR2. Contributed by Sandy Ryza.
...
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1448597 13f79535-47bb-0310-9956-ffa450edef68
2013-02-21 11:34:09 +00:00
c8f35bc3d2
MAPREDUCE-5009. Killing the Task Attempt slated for commit does not clear the value from the Task commitAttempt member (Robert Parker via jeagles)
...
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1447965 13f79535-47bb-0310-9956-ffa450edef68
2013-02-19 23:35:56 +00:00
1c49e77998
MAPREDUCE-5012. Typo in javadoc for IdentityMapper class. Contributed by Adam Monsen.
...
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1447865 13f79535-47bb-0310-9956-ffa450edef68
2013-02-19 18:45:22 +00:00
0a8cfa9016
HADOOP-9304. remove addition of avro genreated-sources dirs to build. (tucu)
...
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1446296 13f79535-47bb-0310-9956-ffa450edef68
2013-02-14 18:48:14 +00:00
ab330b33bb
MAPREDUCE-4994. Addendum fixing testcases failures. (sandyr via tucu)
...
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1446042 13f79535-47bb-0310-9956-ffa450edef68
2013-02-14 04:22:04 +00:00
6bcbf492a2
HADOOP-9117. replace protoc ant plugin exec with a maven plugin. (tucu)
...
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1445956 13f79535-47bb-0310-9956-ffa450edef68
2013-02-13 22:13:44 +00:00
43204f0c83
MAPREDUCE-5000. Fixes getCounters when speculating by fixing the selection of the best attempt for a task. Contributed by Jason Lowe.
...
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1445871 13f79535-47bb-0310-9956-ffa450edef68
2013-02-13 19:20:31 +00:00
390d1fbef8
MAPREDUCE-4992. AM hangs in RecoveryService when recovering tasks with speculative attempts. Contributed by Robert Parker
...
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1445456 13f79535-47bb-0310-9956-ffa450edef68
2013-02-13 02:11:36 +00:00
c2d6407070
MAPREDUCE-4989. JSONify DataTables input data for Attempts page. Contributed by Ravi Prakash
...
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1445448 13f79535-47bb-0310-9956-ffa450edef68
2013-02-13 01:15:48 +00:00
7e615c7f75
MAPREDUCE-4994. -jt generic command line option does not work. (sandyr via tucu)
...
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1445330 13f79535-47bb-0310-9956-ffa450edef68
2013-02-12 20:16:08 +00:00
35832053bf
MAPREDUCE-4671. AM does not tell the RM about container requests which are no longer needed. Contributed by Bikas Saha.
...
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1443323 13f79535-47bb-0310-9956-ffa450edef68
2013-02-07 07:07:11 +00:00
Jason Darrell Lowe
Chris Nauroth
Hitesh Shah
Vinod Kumar Vavilapalli
Alejandro Abdelnur
Arun Murthy
Siddharth Seth
Jonathan Turner Eagles
Christopher Douglas
Bikas Saha
Thomas White
Aaron Myers
Suresh Srinivas
Robert Joseph Evans
Thomas Graves
Konstantin Shvachko
Konstantin Boudnik