96677bef00
YARN-3573. MiniMRYarnCluster constructor that starts the timeline server using a boolean should be marked deprecated. Contributed by Brahma Reddy Battula.
2015-10-27 12:35:01 +09:00
eb6379ca25
MAPREDUCE-6508. TestNetworkedJob fails consistently due to delegation token changes on RM. Contributed by Akira AJISAKA
2015-10-23 10:05:46 -07:00
ee9b80acf8
MAPREDUCE-6518. Set SO_KEEPALIVE on shuffle connections. Contributed by Chang Li
2015-10-21 21:44:43 +00:00
cb26cd4bee
MAPREDUCE-6489. Fail fast rogue tasks that write too much to local disk. Contributed by Maysam Yabandeh
2015-10-21 14:01:23 +00:00
0c4af0f998
MAPREDUCE-6495. Docs for archive-logs tool (rkanter)
2015-10-20 17:34:34 -07:00
58590fef49
YARN-2556. Tool to measure the performance of the timeline server (Chang Li via sjlee)
2015-10-16 16:53:20 -07:00
da8441d0fe
HDFS-9167. Update pom.xml in other modules to depend on hdfs-client instead of hdfs. Contributed by Mingliang Liu.
2015-10-13 15:24:24 -07:00
4aa9b3e75c
MAPREDUCE-6302. Incorrect headroom can lead to a deadlock between map and reduce allocations. (kasha)
2015-10-09 07:37:39 -07:00
a0bca2b5ad
YARN-261. Ability to fail AM attempts. Contributed by Andrey Klochkov and Rohith Sharma K S
2015-10-09 14:17:38 +00:00
584cf3bb35
MAPREDUCE-6479. Add missing mapred job command options in mapreduce document. Contributed by nijel.
2015-10-09 10:54:12 +09:00
8f08532bde
YARN-1897. CLI and core support for signal container functionality. Contributed by Ming Ma
2015-10-02 18:50:47 -07:00
439f43ad3d
MAPREDUCE-6485. Create a new task attempt with failed map task priority if in-progress attempts are unassigned. (Xianyin Xin via rohithsharmaks)
2015-10-02 20:34:34 +05:30
195793c6f3
MAPREDUCE-6497. Fix wrong value of JOB_FINISHED event in JobHistoryEventHandler. Contributed by Shinichi Yamashita.
2015-10-01 10:22:45 +02:00
ab11085b81
MAPREDUCE-6492. AsyncDispatcher exit with NPE on TaskAttemptImpl#sendJHStartEventForAssignedFailTask. Contributed by Bibin A Chundatt
2015-09-28 22:12:03 +00:00
97a08807ec
MAPREDUCE-6484. Yarn Client uses local address instead of RM address as token renewer in a secure cluster when RM HA is enabled. Contributed by Zhihai Xu
2015-09-24 08:12:07 -07:00
89cab1ba5f
YARN-1651. CapacityScheduler side changes to support container resize. Contributed by Wangda Tan
2015-09-23 13:29:38 -07:00
83a18add10
YARN-1449. AM-NM protocol changes to support container resizing. Contributed by Meng Ding & Wangda Tan)
2015-09-23 13:29:36 -07:00
dfe2cb849f
YARN-3866. AM-RM protocol changes to support container resizing. Contributed by Meng Ding
2015-09-23 13:29:36 -07:00
dfd807afab
HADOOP-12428. Fix inconsistency between log-level guards and statements. Contributed by Jagadesh Kiran N and Jackie Chang.
2015-09-22 12:54:29 +09:00
e3ace31e2b
MAPREDUCE-6460. TestRMContainerAllocator.testAttemptNotFoundCausesRMCommunicatorException fails. Contributed by Zhihai Xu.
2015-09-19 00:10:06 -07:00
602335dfe6
MAPREDUCE-6483. Replace deprecated method NameNode.getUri() with DFSUtilClient.getNNUri() in TestMRCredentials. Contributed by Mingliang Liu.
2015-09-18 13:36:09 -07:00
372ad270a0
MAPREDUCE-6478. Add an option to skip cleanupJob stage or ignore cleanup failure during commitJob. (Junping Du via wangda)
2015-09-18 10:10:32 -07:00
ee4ee6af6a
MAPREDUCE-5982. Task attempts that fail from the ASSIGNED state can disappear. Contributed by Chang Li
2015-09-17 21:37:39 +00:00
3f82f582e5
MAPREDUCE-5002. AM could potentially allocate a reduce container to a map attempt. Contributed by Chang Li
2015-09-17 18:17:29 +00:00
58d1a02b8d
MAPREDUCE-6481. LineRecordReader may give incomplete record and wrong position/key information for uncompressed input sometimes. Contributed by Zhihai Xu
2015-09-17 14:30:18 +00:00
941001bfa6
MAPREDUCE-6477. Replace usage of deprecated NameNode.DEFAULT_PORT in TestFileSystem. Contributed by Mingliang Liu.
2015-09-16 13:06:26 -07:00
62943b8e3a
MAPREDUCE-6472. MapReduce AM should have java.io.tmpdir=./tmp to be consistent with tasks. Contributed by Naganarasimha G R
2015-09-15 19:43:20 +00:00
8e615588d5
MAPREDUCE-6474. ShuffleHandler can possibly exhaust nodemanager file descriptors. Contributed by Kuhu Shukla
2015-09-10 16:00:17 +00:00
9b685773ec
MAPREDUCE-6442. Stack trace is missing when error occurs in client protocol provider's constructor Contributed by Chang Li.
2015-09-05 11:21:03 +09:00
e1feaf6db0
HADOOP-12358. Add -safely flag to rm to prompt when deleting many files. Contributed by Xiaoyu Yao.
2015-09-04 13:42:55 -07:00
00804e2457
Revert "MAPREDUCE-6363. [NNBench] Lease mismatch error when running with multiple mappers. Contributed by Brahma Reddy Battula."
...
This reverts commit 75a2560e51
2015-09-02 12:22:16 +09:00
cbb249534a
MAPREDUCE-6452. NPE when intermediate encrypt enabled for LocalRunner. Contributed by Zhihai Xu
2015-08-28 12:13:23 -07:00
50fa383704
Revert "MAPREDUCE-6455. Unable to use surefire > 2.18. (Charlie Helin via kasha)"
...
This reverts commit 61bf9cae6f
2015-08-27 10:15:19 -07:00
57c7ae1aff
YARN-4014. Support user cli interface in for Application Priority. Contributed by Rohith Sharma K S
2015-08-24 20:36:44 -07:00
61bf9cae6f
MAPREDUCE-6455. Unable to use surefire > 2.18. (Charlie Helin via kasha)
2015-08-22 00:39:04 -07:00
2ba90c93d7
MAPREDUCE-6357. MultipleOutputs.write() API should document that output committing is not utilized when input path is absolute. Contributed by Dustin Cote.
2015-08-21 10:41:54 +09:00
8dfec7a197
MAPREDUCE-6439. AM may fail instead of retrying if RM shuts down during the allocate call. (Anubhav Dhoot via kasha)
2015-08-15 00:52:11 -07:00
27d24f96ab
MAPREDUCE-5817. Mappers get rescheduled on node transition even after all reducers are completed. (Sangjin Lee via kasha)
2015-08-14 12:29:50 -07:00
fb1be0b310
MAPREDUCE-6257. Document encrypted spills (Bibin A Chundatt via aw)
2015-08-06 10:13:51 -07:00
e73a928a63
MAPREDUCE-6443. Add JvmPauseMonitor to JobHistoryServer. Contributed by Robert Kanter.
...
(cherry picked from commit c8469357ba
2015-08-06 07:03:46 -07:00
21d10ccc6e
HADOOP-7824. NativeIO.java flags and identifiers must be set correctly for each platform, not hardcoded to their Linux values (Martin Walsh via Colin P. McCabe)
2015-07-31 15:01:34 -07:00
d0e0ba8010
MAPREDUCE-6394. Speed up Task processing loop in HsTasksBlock#render(). Contributed by Ray Chiang
2015-07-31 18:17:54 +00:00
93d50b7824
MAPREDUCE-6433. launchTime may be negative. Contributed by Zhihai Xu
2015-07-30 23:07:31 -07:00
cf74772064
MAPREDUCE-5801. Uber mode's log message is missing a vcore reason (Steven Wong via aw)
2015-07-21 10:58:52 -07:00
f4ccdb11dc
MAPREDUCE-6427. Fix typo in JobHistoryEventHandler. Contributed by Ray Chiang
2015-07-14 14:55:43 -07:00
19295b36d9
YARN-3381. Fix typo InvalidStateTransitonException. Contributed by Brahma Reddy Battula.
2015-07-13 17:52:13 +09:00
fffb15bb43
MAPREDUCE-6426. TestShuffleHandler#testGetMapOutputInfo is failing.
...
Contributed by zhihai xu.
2015-07-09 15:00:26 +05:30
7e2fe8c9f2
MAPREDUCE-6038. A boolean may be set error in the Word Count v2.0 in MapReduce Tutorial. Contributed by Tsuyoshi Ozawa
2015-07-07 10:27:04 -07:00
bff67dfe2f
MAPREDUCE-6425. ShuffleHandler passes wrong "base" parameter to
...
getMapOutputInfo if mapId is not in the cache. Contributed by zhihai xu.
2015-07-06 13:46:37 +05:30
eac1d18943
MAPREDUCE-6418. MRApp should not shutdown LogManager during shutdown. Contributed by Chang Li
2015-07-01 18:01:22 +00:00
826715622e
MAPREDUCE-6420. Interrupted Exception in LocalContainerLauncher should be logged in warn/info level. Contributed by Chang Li
2015-07-01 17:50:02 +00:00
2ac87df578
MAPREDUCE-6376. Add avro binary support for jhist files. Contributed by Ray Chiang
2015-07-01 16:00:03 +00:00
3a72bfd082
MAPREDUCE-6121. JobResourceUpdater#compareFs() doesn't handle HA namespaces (rchiang via rkanter)
2015-06-30 16:49:27 -07:00
9c63825f10
MAPREDUCE-6407. Migrate MAPREDUCE native build to new CMake framework (Alan Burlison via Colin P. McCabe)
2015-06-30 16:17:06 -07:00
b6ba56457c
MAPREDUCE-6384. Add the last reporting reducer info for too many fetch failure diagnostics. Contributed by Chang Li
2015-06-30 21:22:30 +00:00
aa5b15b03b
MAPREDUCE-6413. TestLocalJobSubmission is failing with unknown host. Contributed by zhihai xu
2015-06-25 19:50:07 +00:00
72d08a0e41
MAPREDUCE-6400. Multiple shuffle transfer fails because input is closed too early. Contributed by Brahma Reddy Battula, Akira AJISAKA, and Gera Shegalov.
2015-06-24 15:29:11 +00:00
d481684c7c
MAPREDUCE-6410. Fixed MapReduce JobHistory server to use the right (login) UGI to refresh log and cleaner settings. Contributed by Varun Saxena.
2015-06-23 11:39:55 -07:00
41ae7768eb
MAPREDUCE-6406. Update
...
FileOutputCommitter.FILEOUTPUTCOMMITTER_ALGORITHM_VERSION_DEFAULT to match
mapred-default.xml. Contributed by Ray Chiang.
2015-06-23 10:23:44 +05:30
077250d8d7
MAPREDUCE-5948. org.apache.hadoop.mapred.LineRecordReader does not handle multibyte record delimiters well. Contributed by Vinayakumar B, Rushabh Shah, and Akira AJISAKA
2015-06-22 21:59:20 +00:00
11ac848207
MAPREDUCE-6408. Queue name and user name should be printed on the job page. (Siqi Li via gera)
2015-06-22 14:40:32 -07:00
445b13240c
MAPREDUCE-6403. Fix typo in the usage of NNBench. Contributed by Jagadesh Kiran N.
2015-06-22 15:32:39 +09:00
055cd5a9a1
MAPREDUCE-6305. AM/Task log page should be able to link back to the job. (Siqi Li via gera)
2015-06-20 12:38:36 -07:00
63d40d5286
MAPREDUCE-6316. Task Attempt List entries should link to the task overview. (Siqi Li via gera)
2015-06-20 11:38:21 -07:00
311a4179cc
MAPREDUCE-6405. NullPointerException in App Attempts page. Contributed by
...
Siqi Li and Gera Shegalov.
2015-06-20 10:35:04 +05:30
10107243be
MAPREDUCE-6395. Improve the commit failure messages in MRAppMaster recovery. (Brahma Reddy Battula via gera)
2015-06-19 02:22:59 -07:00
1babe50a2c
MAPREDUCE-6373. The logger reports total input paths but it is referring
...
to input files. Contributed by Bibin A Chundatt.
2015-06-18 11:42:22 +05:30
85cc644f92
MAPREDUCE-6396. TestPipeApplication fails by NullPointerException. Contributed by Brahma Reddy Battula.
2015-06-15 15:28:31 -07:00
75a2560e51
MAPREDUCE-6363. [NNBench] Lease mismatch error when running with multiple mappers. Contributed by Brahma Reddy Battula.
2015-06-15 14:52:26 -07:00
6785661e55
MAPREDUCE-6350. JobHistory doesn't support fully-functional search.
...
Contributed by Siqi Li.
2015-06-10 12:40:29 +05:30
025a3a8be0
MAPREDUCE-6392. Document mapred class path options. Contributed by Brahma Reddy Battula.
2015-06-08 16:22:03 -07:00
6afe20a7a4
MAPREDUCE-6388. Remove deprecation warnings from JobHistoryServer classes. Contributed by Ray Chiang.
2015-06-08 15:06:10 -07:00
b3ffa87003
MAPREDUCE-6354. ShuffleHandler should be able to log shuffle connections. Contributed by Chang Li
2015-06-05 22:38:31 +00:00
6786daab33
MAPREDUCE-6387. Serialize the recently added Task#encryptedSpillKey field at the end. (Arun Suresh via kasha)
2015-06-05 09:14:06 -07:00
790a861766
MAPREDUCE-6377. JHS sorting on state column not working in webUi.
...
Contributed by zhihai xu.
2015-06-05 15:50:16 +05:30
dbed757cba
HADOOP-12058. Fix dead links to DistCp and Hadoop Archives pages. Contributed by Kazuho Fujii.
2015-06-05 01:45:34 +09:00
e8302071ff
MAPREDUCE-6382. Don't escape HTML links in Diagnostics in JHS job overview. (Siqi Li via gera)
2015-06-03 23:46:06 -07:00
d90c13e2da
MAPREDUCE-6174. Combine common stream code into parent class for InMemoryMapOutput and OnDiskMapOutput. (Eric Payne via gera)
2015-06-03 16:50:26 -07:00
bc85959edd
Revert "YARN-1462. Made RM write application tags to timeline server and exposed them to users via generic history web UI and REST API. Contributed by Xuan Gong."
...
This reverts commit 4a9ec1a824
2015-06-03 14:15:56 -07:00
107da29ff9
MAPREDUCE-6374. Distributed Cache File visibility should check permission of full path. Contributed by Chang Li
2015-06-03 20:19:27 +00:00
4a9ec1a824
YARN-1462. Made RM write application tags to timeline server and exposed them to users via generic history web UI and REST API. Contributed by Xuan Gong.
2015-05-30 21:05:36 -07:00
7673d4f205
HADOOP-11885. hadoop-dist dist-layout-stitching.sh does not work with dash. (wang)
2015-05-29 14:54:18 -07:00
788406e90c
MAPREDUCE-6336. Enable v2 FileOutputCommitter by default. (Siqi Li via gera)
2015-05-27 15:14:46 -07:00
4102e5882e
YARN-3626. On Windows localized resources are not moved to the front of the classpath when they should be. Contributed by Craig Welch.
2015-05-27 14:31:49 -07:00
3164e7d838
MAPREDUCE-6304. Specifying node labels when submitting MR jobs. (Naganarasimha G R via wangda)
2015-05-27 14:26:03 -07:00
7dba7005b7
HADOOP-11969. ThreadLocal initialization in several classes is not thread safe (Sean Busbey via Colin P. McCabe)
2015-05-26 12:15:46 -07:00
022f49d59e
MAPREDUCE-6364. Add a Kill link to Task Attempts page. Contributed by Ryu Kobayashi.
2015-05-26 23:59:02 +09:00
b27ce3c531
MAPREDUCE-6204. TestJobCounters should use new properties instead of JobConf.MAPRED_TASK_JAVA_OPTS.
2015-05-22 23:29:53 +09:00
55ed6558ba
HADOOP-11743. maven doesn't clean all the site files (Contributed by ramtin)
2015-05-22 16:45:19 +05:30
563eb1ad2a
YARN-3583. Support of NodeLabel object instead of plain String in YarnClient side. (Sunil G via wangda)
2015-05-19 16:54:38 -07:00
1ea90144d5
MAPREDUCE-6273. HistoryFileManager should check whether summaryFile exists
...
to avoid FileNotFoundException causing HistoryFileInfo into MOVE_FAILED
state. Contributed by zhihai xu.
2015-05-15 12:31:32 +05:30
6b710a42e0
Fixing MR intermediate spills. Contributed by Arun Suresh.
2015-05-14 16:07:56 -07:00
05ff54c66c
MAPREDUCE-5708. Duplicate String.format in
...
YarnOutputFiles.getSpillFileForWrite. Contributed by Konstantin Weitz.
2015-05-14 22:09:54 +05:30
0f95921447
YARN-3626. On Windows localized resources are not moved to the front of the classpath when they should be. Contributed by Craig Welch
2015-05-13 13:10:53 -07:00
f24452d14e
MAPREDUCE-6251. Added a new config for JobClient to retry JobStatus calls so that they don't fail on history-server backed by DFSes with not so strong guarantees. Contributed by Craig Welch.
2015-05-12 12:11:42 -07:00
f4e2b3cc0b
MAPREDUCE-6361. NPE issue in shuffle caused by concurrent issue between copySucceeded() in one thread and copyFailed() in another thread on the same host. Contributed by Junping Du.
2015-05-13 00:28:17 +09:00
360dff5903
MAPREDUCE-6360. TestMapreduceConfigFields is placed in wrong dir, introducing compile error (Contributed by Arshad Mohammad)
2015-05-12 12:49:16 +05:30
444836b3dc
MAPREDUCE-5465. Tasks are often killed before they exit on their own. Contributed by Ming Ma
2015-05-11 22:37:35 +00:00
7e543c27fa
YARN-3587. Fix the javadoc of DelegationTokenSecretManager in yarn, etc. projects. Contributed by Gabor Liptak.
2015-05-11 05:27:07 -07:00
1773aac780
MAPREDUCE-6353. Divide by zero error in MR AM when calculating available containers. (Anubhav Dhoot via kasha)
2015-05-09 14:43:18 -07:00
70fb37cd79
YARN-1287. Consolidate MockClocks. (Sebastian Wong and Anubhav Dhoot via kasha)
2015-05-09 14:34:54 -07:00
df36ad0a08
MAPREDUCE-6359. In RM HA setup, Cluster tab links populated with AM hostname instead of RM. Contributed by zhaoyunjiong.
2015-05-09 06:11:13 -07:00
5aab014340
MAPREDUCE-4750. Enable NNBenchWithoutMR in MapredTestDriver (Liang Xie and Jason Lowe via raviprak)
2015-05-08 15:56:29 -07:00
30099a36c6
MAPREDUCE-5248. Let NNBenchWithoutMR specify the replication factor for its test. Contributed by Erik Paulson
2015-05-08 22:29:47 +00:00
2edcf931d7
MAPREDUCE-2094. LineRecordReader should not seek into non-splittable, compressed streams.
2015-05-08 14:31:11 -07:00
c39012f4a0
MAPREDUCE-3383. Duplicate job.getOutputValueGroupingComparator() in ReduceTask. Contributed by Binglin Chang
2015-05-08 20:48:44 +00:00
bdbd10fde1
MAPREDUCE-2632. Avoid calling the partitioner when the numReduceTasks is 1. (Ravi Teja Ch N V and Sunil G via kasha)
2015-05-08 11:54:37 -07:00
dc2b2ae31f
MAPREDUCE-5981. Log levels of certain MR logs can be changed to DEBUG.
...
Contributed by Varun Saxena.
2015-05-08 21:27:54 +05:30
2401ff76be
MAPREDUCE-6342. Make POM project names consistent. Contributed by Rohith.
2015-05-08 15:17:52 +05:30
ecfa052274
MAPREDUCE-6079. Rename JobImpl#username to reporterUserName. Contributed by Tsuyoshi Ozawa.
2015-05-08 18:16:58 +09:00
d18f10ad1b
MAPREDUCE-6284. Add Task Attempt State API to MapReduce Application Master REST API. Contributed by Ryu Kobayashi.
2015-05-08 15:59:46 +09:00
f30065c8b6
MAPREDUCE-6279. AM should explicity exit JVM after all services have stopped. Contributed by Eric Payne
2015-05-07 22:05:12 +00:00
d335071398
MAPREDUCE-6356. Misspelling of threshold in log4j.properties for tests. Contributed by Brahma Reddy Battula.
2015-05-07 19:39:24 +09:00
9809a16d3c
MAPREDUCE-6192. Create unit test to automatically compare MR related classes and mapred-default.xml (rchiang via rkanter)
2015-05-05 14:43:28 -07:00
b7dd3a4f04
YARN-2123. Progress bars in Web UI always at 100% due to non-US locale.
...
Contributed by Akira AJISAKA
2015-05-05 10:40:16 -07:00
551615fa13
MAPREDUCE-6165. [JDK8] TestCombineFileInputFormat failed on JDK8. Contributed by Akira AJISAKA.
2015-05-05 10:23:13 +09:00
bf70c5ae28
MAPREDUCE-6259. IllegalArgumentException due to missing job submit time. Contributed by zhihai xu
2015-05-04 20:39:18 +00:00
7dc3c1203d
MAPREDUCE-5649. Reduce cannot use more than 2G memory for the final merge. Contributed by Gera Shegalov
2015-05-04 19:02:39 +00:00
bb6ef2984d
MAPREDUCE-6349. Fix typo in property org.apache.hadoop.mapreduce.lib.chain.Chain.REDUCER_INPUT_VALUE_CLASS. Contributed by Ray Chiang.
2015-05-04 15:36:33 +09:00
3ba18362f2
MAPREDUCE-5905. CountersStrings.toEscapedCompactStrings outputs unnecessary null strings. Contributed by Akira AJISAKA.
2015-05-04 15:02:21 +09:00
f1a152cc0a
MAPREDUCE-6345. Documentation fix for when CRLA is enabled for MRAppMaster logs. (Rohit Agarwal via gera)
2015-05-01 18:18:56 -07:00
f5b38477f9
MAPREDUCE-6339. Job history file is not flushed correctly because
...
isTimerActive flag is not set true when flushTimerTask is scheduled.
Contributed by zhihai xu.
2015-04-30 12:33:22 +05:30
519092322d
MAPREDUCE-6343. JobConf.parseMaximumHeapSizeMB() fails to parse value greater than 2GB expressed in bytes. (Hao Xia via kasha)
2015-04-28 14:05:26 -07:00
bc1bd7e5c4
MAPREDUCE-6334. Fetcher#copyMapOutput is leaking usedMemory upon IOException during InMemoryMapOutput shuffle handler. Contributed by Eric Payne
2015-04-28 20:19:05 +00:00
9fc32c5c4d
MAPREDUCE-6324. Fixed MapReduce uber jobs to not fail the udpate of AM-RM tokens when they roll-over. Contributed by Jason Lowe.
2015-04-27 14:58:16 -07:00
9a3dda3d34
MAPREDUCE-6341. Fix typo in mapreduce tutorial. Contributed by John Michael Luy.
2015-04-28 01:47:37 +09:00
5e67c4d384
MAPREDUCE-6252. JobHistoryServer should not fail when encountering a
...
missing directory. Contributed by Craig Welch.
2015-04-27 15:01:42 +05:30
618ba707f0
MAPREDUCE-6057. Remove obsolete entries from mapred-default.xml (Ray Chiang via aw)
2015-04-26 20:31:40 -07:00
78c6b46241
MAPREDUCE-6333. TestEvents,TestAMWebServicesTasks,TestAppController are broken due to MAPREDUCE-6297. (Siqi Li via gera)
2015-04-24 17:31:10 -07:00
ac281e3fc8
HADOOP-11627. Remove io.native.lib.available. Contributed by Brahma Reddy Battula.
2015-04-24 08:08:55 +09:00
f5fe35e297
YARN-3413. Changed Nodelabel attributes (like exclusivity) to be settable only via addToClusterNodeLabels but not changeable at runtime. (Wangda Tan via vinodkv)
2015-04-23 11:19:55 -07:00
e33594dd87
MAPREDUCE-6330. Fix typo in some Task Attempt API's URL. Contributed by Ryu Kobayashi.
2015-04-23 03:21:42 +09:00
89ded89e86
MAPREDUCE-6297. Task Id of the failed task in diagnostics should link to the task page. (Siqi Li via gera)
2015-04-21 13:57:23 -07:00
725eb52ddc
MAPREDUCE-6293. Set job classloader on uber-job's LocalContainerLauncher event thread. (Sangjin Lee via gera)
2015-04-21 13:57:22 -07:00
d50e8f0928
MAPREDUCE-6238. MR2 can't run local jobs with -libjars command options which is a regression from MR1 (zxu via rkanter)
2015-04-20 14:14:08 -07:00
bb6dde68f1
YARN-3021. YARN's delegation-token handling disallows certain trust setups to operate properly over DistCp. Contributed by Yongjun Zhang
2015-04-16 19:43:37 -07:00
1fa8075794
MAPREDUCE-6300. Task list sort by task id broken. Contributed by Siqi Li.
2015-04-16 10:04:09 -05:00
2c17da4208
Revert "MAPREDUCE-6300. Task list sort by task id broken. Contributed by Siqi Li."
...
This reverts commit 6ac98b4804
2015-04-11 12:37:12 +09:00
577d755e4b
YARN-3348. Add a 'yarn top' tool to help understand cluster usage. Contributed by Varun Vasudev
2015-04-10 09:57:17 -07:00
6ac98b4804
MAPREDUCE-6300. Task list sort by task id broken. Contributed by Siqi Li.
2015-04-11 01:31:31 +09:00
83979e61a4
MAPREDUCE-6307. Remove property mapreduce.tasktracker.taskmemorymanager.monitoringinterval. Contributed by J.Andreina.
2015-04-10 20:21:00 +09:00
c91b2de68a
MAPREDUCE-6314. TestPipeApplication fails on trunk. Contributed by Varun Vasudev.
2015-04-10 14:02:26 +05:30
c4986b2d00
MAPREDUCE-6266. Job#getTrackingURL should consistently return a proper URL (rchiang via rkanter)
2015-04-09 13:48:14 -07:00
265ed1fe80
YARN-2890. MiniYarnCluster should turn on timeline service if configured to do so. Contributed by Mit Desai.
2015-04-08 14:13:10 -07:00
a0795c1e81
MAPREDUCE-4844. Counters / AbstractCounters have constant references not declared final. (Brahma Reddy Battula via gera)
2015-04-03 10:28:52 -07:00
eccb7d46ef
MAPREDUCE-6303. Read timeout when retrying a fetch error can be fatal to a reducer. Contributed by Jason Lowe.
2015-04-02 12:13:03 -07:00
cc0a01c503
Revert "MAPREDUCE-6288. Changed permissions on JobHistory server's done directory so that user's client can load the conf files directly. Contributed by Robert Kanter."
...
This reverts commit 5358b83167
2015-03-30 14:48:21 -07:00
5358b83167
MAPREDUCE-6288. Changed permissions on JobHistory server's done directory so that user's client can load the conf files directly. Contributed by Robert Kanter.
2015-03-30 10:27:19 -07:00
c358368f51
YARN-3304. Cleaning up ResourceCalculatorProcessTree APIs for public use and removing inconsistencies in the default values. Contributed by Junping Du and Karthik Kambatla.
2015-03-30 10:09:40 -07:00
27d49e6714
MAPREDUCE-6291. Correct mapred queue usage command. Contributed by Brahma Reddy Battula.
2015-03-28 11:58:17 +05:30
05499b1093
MAPREDUCE-6294. Remove an extra parameter described in Javadoc of TockenCache. Contributed by Brahma Reddy Battula.
2015-03-28 00:08:35 +09:00
c770df49b4
MAPREDUCE-6292. Use org.junit package instead of junit.framework in TestCombineFileInputFormat. (aajisaka)
2015-03-25 19:00:35 +09:00
4170c99147
MAPREDUCE-6285. ClientServiceDelegate should not retry upon AuthenticationException. Contributed by Jonathan Eagles.
2015-03-25 00:56:26 +09:00
7e6f384dd7
MAPREDUCE-6242. Progress report log is incredibly excessive in application
...
master. Contributed by Varun Saxena.
2015-03-23 22:51:20 +05:30
4cd54d9a29
MAPREDUCE-5448. Addendum fix to remove deprecation warning by junit.Assert import in TestFileOutputCommitter.
2015-03-22 10:33:15 +05:30
1d5c796d65
MAPREDUCE-5190. Unnecessary condition test in RandomSampler. Contributed by Jingguo Yao.
2015-03-22 10:03:25 +05:30
b46c2bb51a
MAPREDUCE-5448. MapFileOutputFormat#getReaders bug with invisible files/folders. Contributed by Maysam Yabandeh.
2015-03-22 09:45:48 +05:30
433542904a
MAPREDUCE-6286. A typo in HistoryViewer makes some code useless, which causes counter limits are not reset correctly. Contributed by Zhihai Xu.
2015-03-22 02:51:02 +05:30
e1e09052e8
MAPREDUCE-6213. NullPointerException caused by job history server addr not resolvable. Contributed by Peng Zhang.
2015-03-22 02:44:36 +05:30
7a678db3ac
MAPREDUCE-6239. Consolidate TestJobConf classes in hadoop-mapreduce-client-jobclient and hadoop-mapreduce-client-core. Contributed by Varun Saxena.
2015-03-21 09:43:29 +05:30
d4f7e2507f
MAPREDUCE-6282. Reuse historyFileAbsolute.getFileSystem in CompletedJob#loadFullHistoryData for code optimization. (zxu via rkanter)
2015-03-20 13:11:58 -07:00
8041267f02
MAPREDUCE-4424. 'mapred job -list' command should show the job name as well. Contributed by Avinash Kujur.
2015-03-20 18:46:51 +09:00
91baca145a
MAPREDUCE-6275. Race condition in FileOutputCommitter v2 for user-specified task output subdirs. Contributed by Gera Shegalov and Siqi Li
2015-03-19 21:39:21 +00:00
30da99cbaf
MAPREDUCE-6277. Job can post multiple history files if attempt loses connection to the RM. Contributed by Chang Li
2015-03-18 19:30:37 +00:00
bb243cea93
MAPREDUCE-5755. MapTask.MapOutputBuffer#compare/swap should have @Override annotation. (ozawa)
2015-03-17 14:55:15 +09:00
f222bde273
MAPREDUCE-6100. replace mapreduce.job.credentials.binary with MRJobConfig.MAPREDUCE_JOB_CREDENTIALS_BINARY for better readability. Contributed by Zhihai Xu.
2015-03-17 11:06:35 +05:30
685dbafbe2
MAPREDUCE-6105. nconsistent configuration in property mapreduce.reduce.shuffle.merge.percent. Contributed by Ray Chiang.
2015-03-17 02:28:09 +05:30
587d8be17b
MAPREDUCE-4414. Add main methods to JobConf and YarnConfiguration, for debug purposes. Contributed by Plamen Jeliazkov.
2015-03-17 01:03:08 +05:30
9d38520c8e
MAPREDUCE-6265. Make ContainerLauncherImpl.INITIAL_POOL_SIZE configurable to better control to launch/kill containers. Contributed by Zhihai Xu
2015-03-14 16:44:02 +09:00
8180e676ab
YARN-3267. Timelineserver applies the ACL rules after applying the limit on the number of records (Chang Li via jeagles)
2015-03-13 12:04:30 -05:00
aa92b764a7
MAPREDUCE-4815. Speed up FileOutputCommitter#commitJob for many output files. (Siqi Li via gera)
2015-03-10 11:32:08 -07:00
20b8ee1350
MAPREDUCE-4742. Fix typo in nnbench#displayUsage. Contributed by Liang Xie.
2015-03-11 00:41:12 +09:00
d39bc903a0
Configurable timeout between YARNRunner terminate the application and forcefully kill. Contributed by Eric Payne.
2015-03-10 06:21:59 -07:00
348208014b
MAPREDUCE-6136. MRAppMaster doesn't shutdown file systems. Contributed by Brahma Reddy Battula.
2015-03-05 14:12:47 +09:00
c66c3ac6bf
MAPREDUCE-6267. Refactor JobSubmitter#copyAndConfigureFiles into it's own class. (Chris Trezzo via kasha)
2015-03-04 14:42:07 -08:00
4228de9402
MAPREDUCE-5583. Ability to limit running map and reduce tasks. Contributed by Jason Lowe.
2015-03-03 02:02:28 -08:00
9ae7f9eb7b
MAPREDUCE-5657. Fix Javadoc errors caused by incorrect or illegal tags in doc comments. Contributed by Akira AJISAKA.
2015-03-03 18:06:26 +09:00
742f9d90c0
MAPREDUCE-6268. Fix typo in Task Attempt API's URL. Contributed by Ryu Kobayashi.
2015-03-03 16:21:16 +09:00
d1c6accb6f
HADOOP-11602. Fix toUpperCase/toLowerCase to use Locale.ENGLISH. (ozawa)
2015-03-03 14:17:52 +09:00
8ca0d957c4
YARN-3255. RM, NM, JobHistoryServer, and WebAppProxyServer's main() should support generic options. Contributed by Konstantin Shvachko.
2015-02-26 17:12:19 -08:00
bfbf076b7d
MAPREDUCE-5612. Add javadoc for TaskCompletionEvent.Status. Contributed by Chris Palmer.
2015-02-26 15:05:17 -08:00
1047c883ba
MAPREDUCE-6223. TestJobConf#testNegativeValueForTaskVmem failures. (Varun Saxena via kasha)
2015-02-26 14:24:19 -08:00
9a37247a67
MAPREDUCE-2815. JavaDoc does not generate correctly for MultithreadedMapRunner. Contributed by Chris Palmer.
2015-02-24 12:10:17 -08:00
0d7d508c82
MAPREDUCE-6264. Remove httpclient dependency from hadoop-mapreduce-client. Contributed by Brahma Reddy Battula.
2015-02-24 11:34:26 -08:00
9cedad11d8
Revert "HADOOP-11602. Fix toUpperCase/toLowerCase to use Locale.ENGLISH. (ozawa)"
...
This reverts commit 946456c6d8
2015-02-25 00:32:04 +09:00
d49ae725d5
YARN-3076. Add API/Implementation to YarnClient to retrieve label-to-node mapping (Varun Saxena via wangda)
2015-02-19 11:00:57 -08:00
a19820f2fb
MAPREDUCE-6228. Add truncate operation to SLive. Constributed by Plamen Jeliazkov.
2015-02-19 00:02:49 -08:00
946456c6d8
HADOOP-11602. Fix toUpperCase/toLowerCase to use Locale.ENGLISH. (ozawa)
2015-02-19 13:06:53 +09:00
4981d082d4
MAPREDUCE-6261. NullPointerException if MapOutputBuffer.flush invoked twice. Contributed by Tsuyoshi OZAWA
2015-02-18 19:28:02 +00:00
7c782047c6
MAPREDUCE-4286. TestClientProtocolProviderImpls passes on failure conditions. Contributed by Devaraj K.
2015-02-18 15:45:52 +09:00
8b787e2fdb
MAPREDUCE-6260. Convert site documentation to markdown (Masatake Iwasaki via aw)
2015-02-17 06:52:14 -10:00
814afa46ef
MAPREDUCE-6225. Fix new findbug warnings in hadoop-mapreduce-client-core. Contributed by Varun Saxena
2015-02-16 09:38:05 -08:00
3338f6d907
MAPREDUCE-6256. Removed unused private methods in o.a.h.mapreduce.Job.java. Contributed by Naganarasimha G R.
2015-02-15 10:09:42 +09:00
ba3c80a5ca
MAPREDUCE-6255. Fix JobCounter's format to use grouping separator. Contributed by Ryu Kobayashi.
2015-02-13 16:11:00 +09:00
9b0ba59b82
MAPREDUCE-6221. Stringifier is left unclosed in Chain#getChainElementConf(). Contributed by Ted Yu.
2015-02-13 02:27:54 +09:00
ac8d52bf50
MAPREDUCE-4431. mapred command should print the reason on killing already completed jobs. Contributed by Devaraj K.
2015-02-12 20:35:57 +09:00
b42d09eb62
MAPREDUCE-5335. Rename Job Tracker terminology in ShuffleSchedulerImpl. Contributed by Devaraj K.
2015-02-12 20:08:29 +09:00
76e309ead0
MAPREDUCE-6253. Update use of Iterator to Iterable. Contributed by Ray
...
Chiang.
2015-02-12 13:45:19 +05:30
241336ca2b
MAPREDUCE-6237. Multiple mappers with DBInputFormat don't work because of reusing conections. Contributed by Kannan Rajah.
2015-02-10 03:52:42 +09:00
ef01768333
MAPREDUCE-6227. DFSIO for truncate. (shv via yliu)
2015-02-08 02:43:43 +08:00
e2ee2ff7d7
MAPREDUCE-6233. org.apache.hadoop.mapreduce.TestLargeSort.testLargeSort failed in trunk (zxu via rkanter)
2015-02-05 14:20:28 -08:00
e1990ab427
MAPREDUCE-6186. Redundant call to requireJob() while displaying the conf page. Contributed by Rohit Agarwal
2015-02-05 20:13:31 +00:00
9112f093cd
HADOOP-10976. moving the source code of hadoop-tools docs to the directory under hadoop-tools (Masatake Iwasaki via aw)
2015-02-04 17:57:34 -08:00
cc6bbfceae
MAPREDUCE-5988. Fix dead links to the javadocs in mapreduce project. (aajisaka)
2015-02-04 17:46:01 -08:00
fd57ab2002
MAPREDUCE-6059. Speed up history server startup time (Siqi Li via aw)
2015-02-04 17:36:28 -08:00
bd69fb2d44
MAPREDUCE-5800. Use Job#getInstance instead of deprecated constructors. (aajisaka)
2015-02-03 14:30:09 -08:00
8acc5e9b4b
MAPREDUCE-6143. add configuration for mapreduce speculative execution in MR2 (zxu via rkanter)
2015-02-02 13:51:08 -08:00
342efa110a
HADOOP-9907. Webapp http://hostname:port/metrics link is not working. (aajisaka)
2015-01-30 02:49:10 +09:00
cff05bff1f
MAPREDUCE-6230. Fixed RMContainerAllocator to update the new AMRMToken service name properly. Contributed by Jason Lowe
2015-01-28 15:51:30 -08:00
0a05ae1782
HDFS-7566. Remove obsolete entries from hdfs-default.xml (Ray Chiang via aw)
2015-01-27 14:31:27 -08:00
56b7ec71a6
MAPREDUCE-6141. History server leveldb recovery store. Contributed by Jason Lowe
2015-01-26 16:28:55 +00:00
a003f71cac
MAPREDUCE-5785. Derive heap size or mapreduce.*.memory.mb automatically. (Gera Shegalov and Karthik Kambatla via gera)
2015-01-21 18:42:03 -08:00
220a49de47
HADOOP-11484. hadoop-mapreduce-client-nativetask fails to build on ARM AARCH64 due to x86 asm statements (Edward Nevill via Colin P. McCabe)
2015-01-21 11:24:09 -08:00
42715547f0
HADOOP-11256. Some site docs have inconsistent appearance (Masatake Iwasaki via aw)
2015-01-21 08:44:22 -08:00
d336d13678
MAPREDUCE-6210. Use getApplicationAttemptId() instead of getApplicationID() for logging AttemptId in RMContainerAllocator.java (Contributed by Leitao Guo)
2015-01-14 17:38:35 +09:00
10ac5abb86
MAPREDUCE-6173. Document the configuration of deploying MR over
...
distributed cache with enabling wired encryption at the same time.
Contributed by Junping Du.
2015-01-13 10:04:31 -08:00
09d31bc630
HADOOP-9992. Modify the NN loadGenerator to optionally run as a MapReduce job. Contributed by Akshay Radia
2015-01-09 17:24:22 -08:00
a6ed4894b5
HADOOP-11032. Replace use of Guava's Stopwatch with Hadoop's StopWatch. (ozawa)
2015-01-08 14:51:57 +09:00
9886396400
Revert "Replace use of Guava's Stopwatch with Hadoop's StopWatch. (ozawa)" because of missing JIRA's number.
...
This reverts commit 2eba7eb9af
2015-01-08 14:35:08 +09:00
2eba7eb9af
Replace use of Guava's Stopwatch with Hadoop's StopWatch. (ozawa)
2015-01-08 14:21:30 +09:00
a0aeed100b
MAPREDUCE-6206. TestAggregatedTransferRate fails on non-US systems. Contributed by Jens Rabe
2015-01-06 23:39:43 +00:00
d02fb53750
HADOOP-11445. Bzip2Codec: Data block is skipped when position of newly created stream is equal to start of split. Contributed by Ankit Kamboj
2015-01-06 21:19:10 +00:00
6621c3598e
MAPREDUCE-6149. Document override log4j.properties in MR job. Contributed by Junping Du.
2014-12-30 23:12:51 +05:30
808cba3821
YARN-2952. Fixed incorrect version check in StateStore. Contributed by Rohith Sharmaks
2014-12-19 16:56:30 -08:00
390a7c12f5
MAPREDUCE-6199. AbstractCounters are not reset completely on deserialization (adhoot via rkanter)
2014-12-19 15:43:48 -08:00
6f1e3667cf
HADOOP-11213. Typos in html pages: SecureMode and EncryptedShuffle. (Wei Yan via kasha)
2014-12-19 15:00:28 -08:00
d9e4d67d18
MAPREDUCE-6045. need close the DataInputStream after open it in TestMapReduce.java (zxu via rkanter)
2014-12-19 11:59:22 -08:00
c379e102dd
MAPREDUCE-6191. Improve clearing stale state of Java serialization
...
testcase. (Sam Liu via Eric Yang)
2014-12-15 19:39:41 -08:00
af006937e8
MAPREDUCE-6166. Reducers do not validate checksum of map outputs when fetching directly to disk. (Eric Payne via gera)
2014-12-15 19:08:59 -08:00
a4f2995b9e
Revert "YARN-2890. MiniYARNCluster should start the timeline server based on the configuration. Contributed by Mit Desai."
...
This reverts commit 51af8d367d
2014-12-15 11:04:09 -08:00
298d09c9b5
MAPREDUCE-6194. Bubble up final exception in failures during creation of output collectors. Contributed by Varun Saxena.
2014-12-15 14:26:43 +05:30
0bd0229110
MAPREDUCE-6046. Change the class name for logs in RMCommunicator.
...
Contributed by Sahil Takiar.
2014-12-12 11:42:03 +05:30
0bcea111e5
HADOOP-11211. mapreduce.job.classloader.system.classes semantics should be order-independent. (Yitong Zhou via gera)
2014-12-11 13:12:13 -08:00
cb99f43305
MAPREDUCE-5420. Remove mapreduce.task.tmp.dir from mapred-default.xml. Contributed by James Carman. (harsh)
2014-12-11 09:45:49 +05:30
d777a1e4ca
HADOOP-11369. Fix new findbugs warnings in hadoop-mapreduce-client, non-core directories. Contributed by Li Lu.
2014-12-09 10:46:13 -08:00
8963515b88
MAPREDUCE-6177. Minor typo in the EncryptedShuffle document about ssl-client.xml. Contributed by Yangping Wu. (harsh)
2014-12-08 15:57:52 +05:30
03ab24aa01
MAPREDUCE-5932. Provide an option to use a dedicated reduce-side shuffle log. Contributed by Gera Shegalov
2014-12-03 17:02:14 +00:00
51af8d367d
YARN-2890. MiniYARNCluster should start the timeline server based on the configuration. Contributed by Mit Desai.
2014-12-01 19:04:04 -08:00
0c588904f8
MAPREDUCE-6160. Potential NullPointerException in MRClientProtocol interface implementation. Contributed by Rohith
2014-12-01 22:39:22 +00:00
2b30fb1053
MAPREDUCE-6172. TestDbClasses timeouts are too aggressive. Contributed by Varun Saxena
2014-12-01 21:51:05 +00:00
a655973e78
Revert "MAPREDUCE-5785. Derive heap size or mapreduce.*.memory.mb automatically. (Gera Shegalov and Karthik Kambatla via kasha)"
...
This reverts commit a4df9eed05
2014-11-25 16:24:26 -08:00
78f7cdbfd6
MAPREDUCE-5568. Fixed CompletedJob in JHS to show progress percentage correctly in case the number of mappers or reducers is zero. Contributed by MinJi Kim
2014-11-25 13:02:55 -08:00
a4df9eed05
MAPREDUCE-5785. Derive heap size or mapreduce.*.memory.mb automatically. (Gera Shegalov and Karthik Kambatla via kasha)
2014-11-21 18:36:41 -08:00
c298a9a845
YARN-2375. Allow enabling/disabling timeline server per framework. (Mit Desai via jeagles)
2014-11-20 23:34:35 -06:00
90194ca1cb
MAPREDUCE-6169. MergeQueue should release reference to the current item from key and value at the end of the iteration to save memory. (Zhihai Xu via kasha)
2014-11-20 15:36:57 -08:00
7250b0bf91
MAPREDUCE-6049. AM JVM does not exit if MRClientService graceful shutdown
...
fails. Contributed by Rohith.
2014-11-18 22:05:13 +05:30
09b3dee122
MAPREDUCE-6162. mapred hsadmin fails on a secure cluster. Contributed by Jason Lowe
2014-11-17 22:59:59 +00:00
1a1dcce827
MAPREDUCE-5918. LineRecordReader can return the same decompressor to CodecPool multiple times (Sergey Murylev via raviprak)
2014-11-14 03:45:53 -08:00
177e8090f5
MAPREDUCE-6156. Fetcher - connect() doesn't handle connection refused correctly. Contributed by Junping Du
2014-11-13 15:42:25 +00:00
0abb973f09
HADOOP-7984. Add hadoop --loglevel option to change log level. Contributed by Aikira AJISAKA.
2014-11-12 21:41:19 -08:00
43cd07b408
YARN-2830. Add backwords compatible ContainerId.newInstance constructor. Contributed by Jonathan Eagles.
2014-11-09 14:57:37 -08:00
6caa8100d5
HADOOP-11286. Copied LimitInputStream from guava-0.14 to hadoop to avoid issues with newer versions of guava in applications. Contributed by Christopher Tubbs.
2014-11-08 15:39:56 -08:00
8f701ae07a
MAPREDUCE-5958. Wrong reduce task progress if map output is compressed. Contributed by Emilio Coppa and Jason Lowe.
2014-11-06 15:53:40 -06:00
10f9f5101c
MAPREDUCE-5960. JobSubmitter's check whether job.jar is local is incorrect with no authority in job jar path. Contributed by Gera Shegalov
2014-11-06 15:10:40 +00:00
73068f677b
MAPREDUCE-6048. Fixed TestJavaSerialization failure. Contributed by Varun Vasudev
2014-11-04 20:19:54 -08:00
d794f785de
HADOOP-11230. Add missing dependency of bouncycastle for kms, httpfs, hdfs, MR and YARN. Contributed by Robert Kanter.
2014-11-04 17:52:03 -08:00
ed63b11646
MAPREDUCE-6052. Supported overriding the default container-log4j.properties file per job. Contributed by Junping Du.
2014-11-01 00:47:57 -07:00
e0233c16eb
YARN-2698. Moved some node label APIs to be correctly placed in client protocol. Contributed by Wangda Tan.
2014-10-30 22:59:31 -07:00
3f1441245d
MAPREDUCE-6142. Fixed test failures in TestJobHistoryEventHandler and TestMRTimelineEventHandling. Contributed by Zhijie Shen.
2014-10-29 09:57:39 -07:00
3c5f5af118
HADOOP-11243. SSLFactory shouldn't allow SSLv3. (Wei Yan via kasha)
2014-10-28 18:03:00 -07:00
971e91c8c0
MAPREDUCE-6018. Added an MR specific config to enable emitting job history data to the timeline server. Contributed by Robert Kanter.
2014-10-27 21:03:53 -07:00
6b2f11b54b
MAPREDUCE-5933. Enabled MR AM to post history events to the timeline server. Contributed by Robert Kanter.
2014-10-27 20:35:40 -07:00
0f3b6900be
YARN-2209. Replaced AM resync/shutdown command with corresponding exceptions and made related MR changes. Contributed by Jian He.
2014-10-23 21:56:03 -07:00
3b12fd6cfb
YARN-2198. Remove the need to run NodeManager as privileged account for Windows Secure Container Executor. Contributed by Remus Rusanu
2014-10-22 15:57:46 -07:00
209b1699fc
MAPREDUCE-5542. Killing a job just as it finishes can generate an NPE in client. Contributed by Rohith
2014-10-17 19:51:10 +00:00
0af1a2b5bc
YARN-2312. Deprecated old ContainerId#getId API and updated MapReduce to use ContainerId#getContainerId instead. Contributed by Tsuyoshi OZAWA
2014-10-15 15:22:07 -07:00
f19771a24c
MAPREDUCE-5970. Provide a boolean switch to enable MR-AM profiling. Contributed by Gera Shegalov
2014-10-15 17:50:31 +00:00
b9edad6403
MAPREDUCE-5873. Shuffle bandwidth computation includes time spent waiting for maps. Contributed by Siqi Li
2014-10-15 15:52:53 +00:00
793dbf2465
MAPREDUCE-6115. TestPipeApplication#testSubmitter fails in trunk. Contributed by Binglin Chang
2014-10-13 16:04:14 +00:00
bbe80cdc7b
MAPREDUCE-6125. TestContainerLauncherImpl sometimes fails. Contributed by Mit Desai
2014-10-13 15:08:23 +00:00
e8a31f2e1c
MAPREDUCE-5875. Make Counter limits consistent across JobClient, MRAppMaster, and YarnChild. (Gera Shegalov via kasha)
2014-10-11 22:49:06 -07:00
777877d079
MAPREDUCE-6123. TestCombineFileInputFormat incorrectly starts 2 MiniDFSCluster instances. Contributed by Chris Nauroth.
2014-10-09 22:35:58 -07:00
684170d5ef
MAPREDUCE-6122. TestLineRecordReader may fail due to test data files checked out of git with incorrect line endings. Contributed by Chris Nauroth.
2014-10-09 22:25:17 -07:00
d8d628d1e0
HADOOP-11184. Update Hadoop's lz4 to r123 (cmccabe)
2014-10-09 17:05:35 -07:00
2e789eb226
MAPREDUCE-6029. TestCommitterEventHandler fails in trunk. Contributed by Mit Desai
2014-10-07 14:10:39 +00:00
3f282762d1
MAPREDUCE-6103.Adding reservation APIs to MR resource manager delegate. Contributed by Subru Krishnan and Carlo Curino.
...
(cherry picked from commit aa92dd45f2
2014-10-03 15:42:59 -07:00
23fbfe36c3
MAPREDUCE-6094. TestMRCJCFileInputFormat.testAddInputPath() fails on trunk. Contributed by Akira AJISAKA
2014-09-29 18:12:59 +00:00
cf4631d7c9
HADOOP-11049. javax package system class default is too broad. Contributed by Sangjin Lee
2014-09-29 14:36:29 +00:00
5f16c98ad6
MAPREDUCE-6087. Fixed wrong config name of MRJobConfig#MR_CLIENT_TO_AM_IPC_MAX_RETRIES_ON_TIMEOUTS. Contributed by Akira AJISAKA
2014-09-26 18:05:26 -07:00
c7c8e388a9
MAPREDUCE-5945. Update the description of GenericOptionsParser -jt option (Akira AJISAKA via aw)
2014-09-26 14:25:49 -07:00
b40f433cf8
MAPREDUCE-5796. Use current version of the archive name in DistributedCacheDeploy document (Akira AJISAKA via aw)
2014-09-26 12:59:20 -07:00
1147b9ae16
MAPREDUCE-6073. Description of mapreduce.job.speculative.slowtaskthreshold in mapred-default should be moved into description tags (Tsuyoshi OZAWA via aw)
2014-09-26 11:16:12 -07:00
662fc11ae7
MAPREDUCE-5831. Make MR client ignore unknown counters received from AM. Contributed by Junping Du.
2014-09-25 23:45:07 -07:00
6c54308423
MAPREDUCE-6093. minor distcp doc edits (Charles Lamb via aw)
2014-09-25 11:31:48 -07:00
6c3cebda3f
MAPREDUCE-6078. native-task: fix gtest build on macosx (Binglin Chang)
2014-09-25 07:57:01 -07:00
7af4c3888b
MAPREDUCE-6104. TestJobHistoryParsing.testPartialJob fails in branch-2. Contributed by Mit Desai
2014-09-24 16:09:33 +00:00
376233cdd4
MAPREDUCE-5279. Made MR headroom calculation honor cpu dimension when YARN scheduler resource type is memory plus cpu. Contributed by Peng Zhang and Varun Vasudev.
2014-09-22 09:28:47 -07:00
7039b98e1c
MAPREDUCE-6095. Enable DistributedCache for uber-mode Jobs. Contributed by Gera Shegalov
2014-09-22 15:20:59 +00:00
aa1052c34b
YARN-2460. Remove obsolete entries from yarn-default.xml (Ray Chiang via aw)
2014-09-19 14:07:38 -07:00
951847ba94
MAPREDUCE-6091. YARNRunner.getJobStatus() fails with ApplicationNotFoundException if the job rolled off the RM view. Contributed by Sangjin Lee
2014-09-19 20:15:54 +00:00
6fe5c6b746
YARN-2568. Fixed the potential test failures due to race conditions when RM work-preserving recovery is enabled. Contributed by Jian He.
2014-09-18 21:56:56 -07:00
52945a33cc
MAPREDUCE-6086. mapreduce.job.credentials.binary should allow all URIs. (Zhihai Xu via kasha)
2014-09-18 15:46:52 -07:00
2c3da25fd7
MAPREDUCE-5891. Improved shuffle error handling across NM restarts. Contributed by Junping Du
2014-09-18 22:00:11 +00:00
fe2f54d4cb
HADOOP-11106. Document considerations of HAR and Encryption. (clamb via wang)
2014-09-18 14:51:08 -07:00
1cf3198047
MAPREDUCE-6090. mapred hsadmin getGroups fails to connect in some cases. Contributed by Robert Kanter
2014-09-18 21:00:53 +00:00
14e2639fd0
YARN-611. Added an API to let apps specify an interval beyond which AM failures should be ignored towards counting max-attempts. Contributed by Xuan Gong.
2014-09-13 18:04:05 -07:00
8a3a327cd3
MAPREDUCE-2841. Merge remote-tracking branch 'apache/MR-2841' into trunk
2014-09-12 18:31:53 -07:00
3122daa802
YARN-2229. Changed the integer field of ContainerId to be long type. Contributed by Tsuyoshi OZAWA
2014-09-12 10:33:33 -07:00
9e28e089c6
MAPREDUCE-6070. yarn.app.am.resource.mb/cpu-vcores affects uber mode but is not documented. Contributed by Tsuyoshi OZAWA
2014-09-11 16:19:23 +00:00
7f80e14209
MAPREDUCE-6075. HistoryServerFileSystemStateStore can create zero-length files. Contributed by Jason Lowe
2014-09-10 22:22:39 +00:00
d989ac0444
MAPREDUCE-5972. Fix typo 'programatically' in job.xml (and a few other places) (Akira AJISAKA via aw)
2014-09-08 13:15:46 -07:00
52a8b4db92
MAPREDUCE-6077. native-task: Remove CustomModule examples in nativetask (seanzhong)
2014-09-06 11:46:07 +08:00
7c91f9b148
MAPREDUCE-6074. native-task: Fix release audit warnings
2014-09-05 19:58:41 -07:00
4b3f1e2ce4
Merge remote-tracking branch 'apache/trunk' into MR-2841
2014-09-05 10:47:27 -07:00
683987be7c
MAPREDUCE-6069. Lint/style fixes and removal of unused code. Contributed by Todd Lipcon.
2014-09-05 10:44:38 -07:00
45efc966ee
MAPREDUCE-6071. JobImpl#makeUberDecision doesn't log that Uber mode is disabled because of too much CPUs. Contributed by Tsuyoshi OZAWA
2014-09-05 13:42:00 +00:00
00322161b5
MAPREDUCE-6067. native-task: fix some counter issues (Binglin Chang)
2014-09-05 14:20:39 +08:00
a18424e724
MAPREDUCE-6063. Correct spill size calculation for spills wrapping the circular buffer. Contributed by zhihai xu.
2014-09-03 17:01:59 -07:00
1081d9cee2
MAPREDUCE-6055. native-task: findbugs, interface annotations, and other misc cleanup
2014-09-03 12:22:38 -07:00
cce7d1e2f9
Merge remote-tracking branch 'origin/trunk' into MR-2841
...
Conflicts:
hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-core/src/main/java/org/apache/hadoop/mapred/MapTask.java
2014-09-02 15:55:54 -07:00
7ade9b04ea
MAPREDUCE-6056. native-task: move system test working dir to target dir and cleanup test config xml files (Manu Zhang via bchang)
2014-09-02 16:26:09 +08:00
17cd0faace
MAPREDUCE-6058. native-task: KVTest and LargeKVTest should check mr job is sucessful (Contributed by Binglin Chang)
2014-09-02 11:00:48 +08:00
ea1c6f31c2
HADOOP-11021. Configurable replication factor in the hadoop archive command. Contributed by Zhe Zhang.
2014-08-29 14:44:37 -07:00
15366d9227
MAPREDUCE-5931. Validate SleepJob command line parameters. Contributed by Gera Shegalov
2014-08-29 19:50:15 +00:00
48aa3b7274
Fix typos in log messages. Contributed by Ray Chiang
2014-08-28 16:29:35 -07:00
bfd1d75d87
MAPREDUCE-6054. native-task: Speed up tests. Contributed by Todd Lipcon.
2014-08-27 12:25:49 -07:00
fad4524c85
MAPREDUCE-5977. Fix or suppress native-task gcc warnings. Contributed by Manu Zhang.
2014-08-27 12:25:42 -07:00
812bd0c0e5
MAPREDUCE-5885. build/test/test.mapred.spill causes release audit warnings. Contributed by Chen He
2014-08-27 15:14:54 +00:00
524a63e59e
MAPREDUCE-6044. Fully qualified intermediate done dir path breaks per-user dir creation on Windows. Contributed by Zhijie Shen.
...
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1619863 13f79535-47bb-0310-9956-ffa450edef68
2014-08-22 17:06:23 +00:00
5109157ed1
MAPREDUCE-5130. Add missing job config options to mapred-default.xml (Ray Chiang via Sandy Ryza)
...
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1619626 13f79535-47bb-0310-9956-ffa450edef68
2014-08-21 23:28:44 +00:00
da4ba50269
HADOOP-10893. isolated classloader on the client side. Contributed by Sangjin Lee
...
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1619604 13f79535-47bb-0310-9956-ffa450edef68
2014-08-21 21:38:16 +00:00
e5e9d792c7
MAPREDUCE-5974. Allow specifying multiple MapOutputCollectors with fallback. (Todd Lipcon via kasha)
...
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1619492 13f79535-47bb-0310-9956-ffa450edef68
2014-08-21 17:37:34 +00:00
ef32d09030
[post-HADOOP-9902] mapred version is missing (Akira AJISAKA via aw)
...
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1619201 13f79535-47bb-0310-9956-ffa450edef68
2014-08-20 18:47:10 +00:00
d2a39b61aa
Merge from trunk to branch.
...
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/branches/fs-encryption@1619018 13f79535-47bb-0310-9956-ffa450edef68
2014-08-20 01:22:42 +00:00
0cc08f6da4
Merge from trunk to branch.
...
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/branches/fs-encryption@1618700 13f79535-47bb-0310-9956-ffa450edef68
2014-08-18 18:41:31 +00:00
73325f23f6
MAPREDUCE-6012. DBInputSplit creates invalid ranges on Oracle. (Wei Yan via kasha)
...
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1618694 13f79535-47bb-0310-9956-ffa450edef68
2014-08-18 18:25:50 +00:00
d51f81c3b6
MAPREDUCE-6036. TestJobEndNotifier fails intermittently in branch-2. Contributed by chang li
...
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1618691 13f79535-47bb-0310-9956-ffa450edef68
2014-08-18 18:18:22 +00:00
f8e871d01b
MAPREDUCE-6024. Shortened the time when Fetcher is stuck in retrying before concluding the failure by configuration. Contributed by Yunjiong Zhao.
...
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1618677 13f79535-47bb-0310-9956-ffa450edef68
2014-08-18 17:57:48 +00:00
08d6201b34
MAPREDUCE-6033. Updated access check for displaying job information
...
(Yu Gao via Eric Yang)
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1618648 13f79535-47bb-0310-9956-ffa450edef68
2014-08-18 16:13:56 +00:00
84bc2fe402
MAPREDUCE-6032. Made MR jobs write job history files on the default FS when the current context’s FS is different. Contributed by Benjamin Zhitomirsky.
...
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1618269 13f79535-47bb-0310-9956-ffa450edef68
2014-08-15 20:17:45 +00:00
7378af7bca
MAPREDUCE-5878. some standard JDK APIs are not part of system classes defaults. Contributed by Sangjin Lee
...
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1618080 13f79535-47bb-0310-9956-ffa450edef68
2014-08-14 23:17:33 +00:00
2d904b3fa3
MAPREDUCE-5906. Inconsistent configuration in property "mapreduce.reduce.shuffle.input.buffer.percent" (Akira AJISAKA via aw)
...
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1618007 13f79535-47bb-0310-9956-ffa450edef68
2014-08-14 18:07:06 +00:00
e1f15d2cdf
MAPREDUCE-5999. Fix dead link in InputFormat javadoc (Akira AJISAKA via aw)
...
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1618003 13f79535-47bb-0310-9956-ffa450edef68
2014-08-14 17:56:53 +00:00
7557f7bb2d
MAPREDUCE-5998. CompositeInputFormat javadoc is broken (Akira AJISAKA via aw)
...
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1617995 13f79535-47bb-0310-9956-ffa450edef68
2014-08-14 17:18:14 +00:00
6be98091f7
MAPREDUCE-5950. incorrect description in distcp2 document (Akira AJISAKA via aw)
...
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1617994 13f79535-47bb-0310-9956-ffa450edef68
2014-08-14 17:11:39 +00:00
92054f2a06
MAPREDUCE-5597. Missing alternatives in javadocs for deprecated constructors in mapreduce.Job (Akira AJISAKA via aw)
...
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1617986 13f79535-47bb-0310-9956-ffa450edef68
2014-08-14 16:00:18 +00:00
9579554988
MAPREDUCE-6010. HistoryServerFileSystemStateStore fails to update tokens. Contributed by Jason Lowe
...
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1617984 13f79535-47bb-0310-9956-ffa450edef68
2014-08-14 15:55:03 +00:00
e7b540eced
MAPREDUCE-5363. Fix doc and spelling for TaskCompletionEvent#getTaskStatus and getStatus (Akira AJISAKA via aw)
...
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1617982 13f79535-47bb-0310-9956-ffa450edef68
2014-08-14 15:49:10 +00:00
1e016e6451
MAPREDUCE-4791. Javadoc for KeyValueTextInputFormat should include default separator and how to change it (Akira AJISAKA via aw)
...
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1617979 13f79535-47bb-0310-9956-ffa450edef68
2014-08-14 15:43:28 +00:00
b3df973adb
MAPREDUCE-883. harchive: Document how to unarchive (Akira AJISAKA and Koji Noguchi via aw)
...
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1617977 13f79535-47bb-0310-9956-ffa450edef68
2014-08-14 15:39:52 +00:00
808bf8bac1
MAPREDUCE-6026. native-task: fix logging. Contributed by Manu Zhang.
...
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/branches/MR-2841@1617878 13f79535-47bb-0310-9956-ffa450edef68
2014-08-14 04:53:36 +00:00
886338ff80
MAPREDUCE-6006. native-task: add native tests to maven and fix bug in pom.xml. Contributed by Binglin Chang.
...
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/branches/MR-2841@1617877 13f79535-47bb-0310-9956-ffa450edef68
2014-08-14 04:47:20 +00:00
e51a5d691b
MAPREDUCE-5943. Separate mapred commands from CommandManual.apt.vm (Akira AJISAKA via aw)
...
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1617806 13f79535-47bb-0310-9956-ffa450edef68
2014-08-13 19:34:26 +00:00
74f7be0887
merge from trunk r1617527
...
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/branches/fs-encryption@1617532 13f79535-47bb-0310-9956-ffa450edef68
2014-08-12 17:02:07 +00:00
eeb4acd955
YARN-2212: ApplicationMaster needs to find a way to update the AMRMToken periodically. Contributed by Xuan Gong
...
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1616892 13f79535-47bb-0310-9956-ffa450edef68
2014-08-08 21:38:24 +00:00
041b8326a1
MAPREDUCE-6007. Add support to distcp to preserve raw.* namespace extended attributes. (clamb)
...
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/branches/fs-encryption@1616657 13f79535-47bb-0310-9956-ffa450edef68
2014-08-08 01:32:18 +00:00
311d2f0773
Merge from trunk to branch.
...
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/branches/fs-encryption@1616589 13f79535-47bb-0310-9956-ffa450edef68
2014-08-07 20:26:52 +00:00
de2595833c
MAPREDUCE-6021. MR AM should have working directory in LD_LIBRARY_PATH. Contributed by Jason Lowe
...
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1616585 13f79535-47bb-0310-9956-ffa450edef68
2014-08-07 20:15:56 +00:00
4d4fb1723f
MAPREDUCE-5978. native-task: remove test case for not supported codec Bzip2Codec and DefaultCodec (Manu Zhang)
...
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/branches/MR-2841@1616116 13f79535-47bb-0310-9956-ffa450edef68
2014-08-06 07:40:24 +00:00
432f641bc2
MAPREDUCE-5976. native-task: should not fail to build if snappy is missing (Manu Zhang)
...
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/branches/MR-2841@1616115 13f79535-47bb-0310-9956-ffa450edef68
2014-08-06 07:32:49 +00:00
7ecaa81d27
MAPREDUCE-5984. native-task: Reuse lz4 sources in hadoop-common. Contributed by Binglin Chang
...
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/branches/MR-2841@1616105 13f79535-47bb-0310-9956-ffa450edef68
2014-08-06 06:01:12 +00:00
857d134fda
MAPREDUCE-6014. New task status field in task attempts table can lead to an empty web page. Contributed by Mit Desai
...
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1616018 13f79535-47bb-0310-9956-ffa450edef68
2014-08-05 21:52:22 +00:00
ac73d416f3
Merge from trunk to branch
...
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/branches/fs-encryption@1615844 13f79535-47bb-0310-9956-ffa450edef68
2014-08-05 02:30:54 +00:00
83a396733e
MAPREDUCE-6005. native-task: Fix some valgrind errors (Binglin Chang)
...
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/branches/MR-2841@1615489 13f79535-47bb-0310-9956-ffa450edef68
2014-08-04 06:19:51 +00:00
e52f67e389
YARN-1994. Expose YARN/MR endpoints on multiple interfaces. Contributed by Craig Welch, Milan Potocnik,and Arpit Agarwal
...
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1614981 13f79535-47bb-0310-9956-ffa450edef68
2014-07-31 20:06:02 +00:00
1d6e178144
YARN-2347. Consolidated RMStateVersion and NMDBSchemaVersion into Version in yarn-server-common. Contributed by Junping Du.
...
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1614838 13f79535-47bb-0310-9956-ffa450edef68
2014-07-31 09:27:43 +00:00
78d86a9839
MAPREDUCE-5995. native-task: Revert changes to Text internals. Contributed by Todd Lipcon.
...
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/branches/MR-2841@1613828 13f79535-47bb-0310-9956-ffa450edef68
2014-07-27 19:04:14 +00:00
1d3e9ec935
merge from trunk r1613787
...
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/branches/fs-encryption@1613788 13f79535-47bb-0310-9956-ffa450edef68
2014-07-27 13:32:51 +00:00
549bcc2c02
MAPREDUCE-6002. Made MR task avoid reporting error to AM when the task process is shutting down. Contributed by Wangda Tan.
...
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1613743 13f79535-47bb-0310-9956-ffa450edef68
2014-07-27 01:37:51 +00:00
ea9e5b7baa
MAPREDUCE-5991. native-task should not run unit tests if native profile is not enabled. Contributed by Binglin Chang.
...
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/branches/MR-2841@1613072 13f79535-47bb-0310-9956-ffa450edef68
2014-07-24 11:44:09 +00:00
6635180324
MAPREDUCE-6000. native-task: Simplify ByteBufferDataReader/Writer. Contributed by Todd Lipcon.
...
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/branches/MR-2841@1613036 13f79535-47bb-0310-9956-ffa450edef68
2014-07-24 08:24:05 +00:00
77acc70df5
MAPREDUCE-5997. native-task: Use DirectBufferPool from Hadoop Common. Contributed by Todd Lipcon.
...
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/branches/MR-2841@1613034 13f79535-47bb-0310-9956-ffa450edef68
2014-07-24 08:20:25 +00:00
b2cba48f36
MAPREDUCE-5996. native-task: Rename system tests into standard directory layout. Contributed by Todd Lipcon.
...
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/branches/MR-2841@1613004 13f79535-47bb-0310-9956-ffa450edef68
2014-07-24 06:17:33 +00:00
d468a92447
MAPREDUCE-5994. Simplify ByteUtils and fix failing test. Contributed by Todd Lipcon.
...
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/branches/MR-2841@1613003 13f79535-47bb-0310-9956-ffa450edef68
2014-07-24 06:14:27 +00:00
22ba00de60
Merge from trunk to branch pt 2
...
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/branches/fs-encryption@1612941 13f79535-47bb-0310-9956-ffa450edef68
2014-07-23 20:45:10 +00:00
341695e731
Merge trunk into branch
...
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/branches/MR-2841@1612742 13f79535-47bb-0310-9956-ffa450edef68
2014-07-23 01:47:28 +00:00
e788e8b6ee
MAPREDUCE-5985. native-task: Fix build on macosx. Contributed by Binglin Chang
...
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/branches/MR-2841@1612659 13f79535-47bb-0310-9956-ffa450edef68
2014-07-22 19:55:03 +00:00
60b1e835e0
MAPREDUCE-5963. ShuffleHandler DB schema should be versioned with compatible/incompatible changes. Contributed by Junping Du
...
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1612652 13f79535-47bb-0310-9956-ffa450edef68
2014-07-22 19:22:24 +00:00
cb20bb6117
Merge from trunk to branch
...
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/branches/fs-encryption@1612403 13f79535-47bb-0310-9956-ffa450edef68
2014-07-21 21:44:50 +00:00
c2174a5536
MAPREDUCE-5756. CombineFileInputFormat.getSplits() including directories in its results. Contributed by Jason Dere
...
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1612400 13f79535-47bb-0310-9956-ffa450edef68
2014-07-21 21:24:15 +00:00
9e62bcca4e
MAPREDUCE-5957. AM throws ClassNotFoundException with job classloader enabled if custom output format/committer is used. Contributed by Sangjin Lee
...
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1612358 13f79535-47bb-0310-9956-ffa450edef68
2014-07-21 17:54:46 +00:00
875592220f
MAPREDUCE-5910. Make MR AM resync with RM in case of work-preserving RM-restart. Contributed by Rohith
...
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1611434 13f79535-47bb-0310-9956-ffa450edef68
2014-07-17 18:46:20 +00:00
b2551c06a0
Import initial code for MAPREDUCE-2841 (native output collector)
...
Contributed by:
Binglin Chang
Yang Dong
Sean Zhong
Manu Zhang
Zhongliang Zhu
Vincent Wang
Yan Dong
Cheng Lian
Xusen Yin
Fangqin Dai
Jiang Weihua
Gansha Wu
Avik Dey
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/branches/MR-2841@1611413 13f79535-47bb-0310-9956-ffa450edef68
2014-07-17 17:44:55 +00:00
43a12f3c01
MAPREDUCE-5952. LocalContainerLauncher#renameMapOutputForReduce incorrectly assumes a single dir for mapOutIndex. (Gera Shegalov via kasha)
...
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1611196 13f79535-47bb-0310-9956-ffa450edef68
2014-07-16 21:24:51 +00:00
56c0bd4d37
Merge from trunk to branch
...
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/branches/fs-encryption@1610853 13f79535-47bb-0310-9956-ffa450edef68
2014-07-15 21:10:24 +00:00
b60ef8b91b
MAPREDUCE-5790. Made it easier to enable hprof profile options by default. Contributed by Gera Shegalov.
...
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1610578 13f79535-47bb-0310-9956-ffa450edef68
2014-07-15 04:49:01 +00:00
64306aa1b5
MAPREDUCE-5956. Made MR AM not use maxAttempts to determine if the current attempt is the last retry. Contributed by Wangda Tan.
...
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1609649 13f79535-47bb-0310-9956-ffa450edef68
2014-07-11 08:45:40 +00:00
95986dd2fb
MAPREDUCE-5890. Support for encrypting Intermediate data and spills in local filesystem. (asuresh via tucu)
...
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/branches/fs-encryption@1609597 13f79535-47bb-0310-9956-ffa450edef68
2014-07-11 00:43:03 +00:00
4b2ded8202
MAPREDUCE-5517. Fixed MapReduce ApplicationMaster to not validate reduce side resource configuration for deciding uber-mode on map-only jobs. Contributed by Siqi Li.
...
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1608595 13f79535-47bb-0310-9956-ffa450edef68
2014-07-07 20:14:01 +00:00
286ae1edd0
MAPREDUCE-5866. TestFixedLengthInputFormat fails in windows. Contributed by Varun Vasudev.
...
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1608585 13f79535-47bb-0310-9956-ffa450edef68
2014-07-07 19:56:44 +00:00
d85f90e5b3
MAPREDUCE-5868. Fixed an issue with TestPipeApplication that was causing the nightly builds to fail. Contributed by Akira Ajisaka.
...
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1608579 13f79535-47bb-0310-9956-ffa450edef68
2014-07-07 19:45:57 +00:00
c6a09d2110
MAPREDUCE-5900. Changed to the interpret container preemption exit code as a task attempt killing event. Contributed by Mayank Bansal.
...
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1607512 13f79535-47bb-0310-9956-ffa450edef68
2014-07-03 01:43:56 +00:00
cfd9344540
MAPREDUCE-5939. StartTime showing up as the epoch time in JHS UI after upgrade. Contributed by Chen He
...
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1605892 13f79535-47bb-0310-9956-ffa450edef68
2014-06-26 19:56:09 +00:00
e285b98f0f
YARN-2152. Added missing information into ContainerTokenIdentifier so that NodeManagers can report the same to RM when RM restarts. Contributed Jian He.
...
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1605205 13f79535-47bb-0310-9956-ffa450edef68
2014-06-24 21:43:22 +00:00
1355ff6132
MAPREDUCE-5830. Added back the private API HostUtil.getTaskLogUrl(..) for binary compatibility with older clients like Hive 0.13. Contributed by Akira Ajisaka.
...
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1604230 13f79535-47bb-0310-9956-ffa450edef68
2014-06-20 18:28:21 +00:00
7b9c074b76
MAPREDUCE-5844. Add a configurable delay to reducer-preemption. (Maysam Yabandeh via kasha)
...
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1603957 13f79535-47bb-0310-9956-ffa450edef68
2014-06-19 17:22:56 +00:00
bd23a2ff22
MAPREDUCE-5896. InputSplits should indicate which locations have the block cached in memory. (Sandy Ryza via kasha)
...
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1603670 13f79535-47bb-0310-9956-ffa450edef68
2014-06-18 23:28:50 +00:00
0abddac476
MAPREDUCE-5924. Changed TaskAttemptImpl to ignore TA_COMMIT_PENDING event at COMMIT_PENDING state. Contributed by Zhijie Shen
...
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1603338 13f79535-47bb-0310-9956-ffa450edef68
2014-06-18 00:51:39 +00:00
180ac8dc1d
MAPREDUCE-5912. Task.calculateOutputSize does not handle Windows files after MAPREDUCE-5196. Contributed by Remus Rusanu.
...
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1602282 13f79535-47bb-0310-9956-ffa450edef68
2014-06-12 20:02:45 +00:00
fc7c8f9bf2
HADOOP-10686. Writables are not always configured. (Abraham Elmahrek via kasha)
...
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1602079 13f79535-47bb-0310-9956-ffa450edef68
2014-06-12 07:23:20 +00:00
473e83ab71
MAPREDUCE-5920. Add Xattr option in DistCp docs. Contributed by Yi Liu.
...
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1601983 13f79535-47bb-0310-9956-ffa450edef68
2014-06-11 18:36:06 +00:00
40ba8c17c1
MAPREDUCE-5777. Support utf-8 text with Byte Order Marker. (Zhihai Xu via kasha)
...
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1600977 13f79535-47bb-0310-9956-ffa450edef68
2014-06-06 18:37:03 +00:00
806f4d11f2
HADOOP-10602. Documentation has broken "Go Back" hyperlinks. Contributed by Akira AJISAKA.
...
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1598337 13f79535-47bb-0310-9956-ffa450edef68
2014-05-29 17:26:50 +00:00
61c59c0704
MAPREDUCE-5895. Close streams properly to avoid leakage in TaskLog. Contributed by Kousuke Saruta.
...
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1598209 13f79535-47bb-0310-9956-ffa450edef68
2014-05-29 07:05:26 +00:00
4bb4de93d6
MAPREDUCE-5862. Line records longer than 2x split size aren't handled correctly. Contributed by bc Wong
...
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1598111 13f79535-47bb-0310-9956-ffa450edef68
2014-05-28 19:37:52 +00:00
1f8bb1001a
MAPREDUCE-5309. 2.0.4 JobHistoryParser can't parse certain failed job history files generated by 2.0.3 history server. Contributed by Rushabh S Shah
...
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1596295 13f79535-47bb-0310-9956-ffa450edef68
2014-05-20 15:46:54 +00:00
4a839f0cc1
Fix CRLFs in TestKillAMPreemptionPolicy.java
...
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1596058 13f79535-47bb-0310-9956-ffa450edef68
2014-05-19 21:18:55 +00:00
f74e446355
MAPREDUCE-5867. Fix NPE in KillAMPreemptionPolicy related to ProportionalCapacityPreemptionPolicy. Contributed by Sunil G.
...
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1595754 13f79535-47bb-0310-9956-ffa450edef68
2014-05-19 06:42:29 +00:00
c9377189a0
MAPREDUCE 5874. Creating MapReduce REST API section. Contributed by Tsuyoshi Ozawa
...
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1595265 13f79535-47bb-0310-9956-ffa450edef68
2014-05-16 16:50:58 +00:00
1e5d2c612b
MAPREDUCE-5861. finishedSubMaps field in LocalContainerLauncher does not need to be volatile. (Contributed by Tsuyoshi OZAWA)
...
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1594795 13f79535-47bb-0310-9956-ffa450edef68
2014-05-15 07:07:18 +00:00
Tsuyoshi Ozawa
Junping Du
Jason Lowe
Robert Kanter
Sangjin Lee
Haohui Mai
Karthik Kambatla
Akira Ajisaka
Xuan
rohithsharmaks
Zhihai Xu
Jian He
Wangda Tan
Andrew Wang
Allen Wittenauer
Colin Patrick Mccabe
Chris Douglas
Devaraj K
Robert Kanter
Vinod Kumar Vavilapalli
Gera Shegalov
cnauroth
Zhijie Shen
Andrew Wang
Vinayakumar B
Ravi Prakash
Harsh J
Hitesh Shah
Jonathan Eagles
Konstantin V Shvachko
Plamen Jeliazkov
yliu
Brandon Li
Eric Yang
Ravi Prakash
Arun C. Murthy
Kihwal Lee
subru
Todd Lipcon
Sean Zhong
Binglin Chang
Sanford Ryza
Charles Lamb
Sean Zhong
Alejandro Abdelnur