69c21d2db0
MAPREDUCE-6641. TestTaskAttempt fails in trunk. Contributed by Haibo Chen.
2016-06-23 13:26:10 +09:00
79a7289165
MAPREDUCE-6721. mapreduce.reduce.shuffle.memory.limit.percent=0.0 should be legal to enforce shuffle to disk. (Gera Shegalov via ozawa)
...
This closes #102
2016-06-22 17:20:13 -07:00
d8107fcd1c
MAPREDUCE-6197. Cache MapOutputLocations in ShuffleHandler. Contributed by Junping Du
2016-06-21 14:25:58 -07:00
605b4b6136
MAPREDUCE-6719. The list of -libjars archives should be replaced with a wildcard in the distributed cache to reduce the application footprint in the state store (Daniel Templeton via sjlee)
2016-06-21 11:25:11 -07:00
5d58858bb6
HADOOP-9613. [JDK8] Update jersey version to latest 1.x release.
2016-06-21 08:05:32 +09:00
51d497fa93
MAPREDUCE-6542. HistoryViewer uses SimpleDateFormat, but SimpleDateFormat is not threadsafe. Contributed by zhangyubiao.
2016-06-17 11:03:54 +09:00
c77a1095dc
YARN-1942. Deprecate toString/fromString methods from ConverterUtils and move them to records classes like ContainerId/ApplicationId, etc. (wangda)
2016-06-14 15:06:38 -07:00
e3ba9ad3f1
HADOOP-13270. BZip2CompressionInputStream finds the same compression marker twice in corner case, causing duplicate data blocks. Contributed by Kai Sasaki.
2016-06-14 10:18:41 +09:00
0b7b8a3776
MAPREDUCE-6741. Refactor UncompressedSplitLineReader.fillBuffer(). Contributed by Daniel Templeton.
2016-06-10 19:15:36 +09:00
0af96a1c08
MAPREDUCE-6240. Hadoop client displays confusing error message. (gera)
2016-06-08 12:59:37 -07:00
733f3f18d5
MAPREDUCE-6702. TestMiniMRChildTask.testTaskEnv and TestMiniMRChildTask.testTaskOldEnv are failing (ajisakaa via rkanter)
2016-06-07 15:46:06 -07:00
4a1cedc010
MAPREDUCE-5044. Have AM trigger jstack on task attempts that timeout before killing them. (Eric Payne and Gera Shegalov via mingma)
2016-06-06 14:30:51 -07:00
dc26601d8f
YARN-5180. Allow ResourceRequest to specify an enforceExecutionType flag. (asuresh)
2016-06-02 09:01:02 -07:00
93d8a7f2a2
Revert "HADOOP-12579. Deprecate and remove WriteableRPCEngine. Contributed by Kai Zheng"
...
This reverts commit a6c79f92d5
2016-06-01 08:41:15 +08:00
42f90ab885
YARN-4844. Add getMemorySize/getVirtualCoresSize to o.a.h.y.api.records.Resource. Contributed by Wangda Tan.
2016-05-29 21:24:16 +05:30
013532a95e
YARN-4957. Add getNewReservation in ApplicationClientProtocol (Sean Po via curino)
2016-05-25 16:55:49 -07:00
a6c79f92d5
HADOOP-12579. Deprecate and remove WriteableRPCEngine. Contributed by Kai Zheng
2016-05-27 04:29:42 +08:00
ae353ea969
MAPREDUCE-6703. Add flag to allow MapReduce AM to request for OPPORTUNISTIC containers. Contributed by Arun Suresh
2016-05-24 19:47:50 -07:00
3b3b63081b
MAPREDUCE-6607. Enable regex pattern matching when mapreduce.task.files.preserve.filepattern is set. Contributed by Kai Sasaki.
2016-05-23 07:24:31 +09:00
21d2b90213
MAPREDUCE-6696. Add a configuration to limit the number of map tasks allowed per job. Contributed by Zhihai Xu
2016-05-19 19:25:16 -07:00
22fcd819f0
YARN-5076. YARN web interfaces lack XFS protection. Contributed by Jonathan Maron.
...
(cherry picked from commit 2703ec6871
2016-05-19 14:15:21 -07:00
992a49353f
MAPREDUCE-6686. Add a way to download the job config from the mapred CLI (rkanter)
2016-05-18 12:16:21 -07:00
f6ef876fe1
MAPREDUCE-6657. Job history server can fail on startup when NameNode is in start phase. Contributed by Haibo Chen.
2016-05-17 14:41:47 -07:00
34fddd1e91
MAPREDUCE-6698. Increase timeout on TestUnnecessaryBlockingOnHistoryFileInfo.testTwoThreadsQueryingDifferentJobOfSameUser. Contributed by Haibo Chen
2016-05-17 15:01:33 +00:00
12fa4ec141
MAPREDUCE-6701. application master log can not be available when clicking jobhistory's am logs link. Contributed by Haibo Chen
2016-05-17 14:54:13 +00:00
b357930526
MAPREDUCE-6693. ArrayIndexOutOfBoundsException occurs when the length of the job name is equal to mapreduce.jobhistory.jobname.limit. Contributed by Ajith S.
2016-05-17 20:19:01 +09:00
730bc746f9
HADOOP-12930. Dynamic subcommands for hadoop shell scripts (aw)
...
This commit contains the following JIRA issues:
HADOOP-12931. bin/hadoop work for dynamic subcommands
HADOOP-12932. bin/yarn work for dynamic subcommands
HADOOP-12933. bin/hdfs work for dynamic subcommands
HADOOP-12934. bin/mapred work for dynamic subcommands
HADOOP-12935. API documentation for dynamic subcommands
HADOOP-12936. modify hadoop-tools to take advantage of dynamic subcommands
HADOOP-13086. enable daemonization of dynamic commands
HADOOP-13087. env var doc update for dynamic commands
HADOOP-13088. fix shellprofiles in hadoop-tools to allow replacement
HADOOP-13089. hadoop distcp adds client opts twice when dynamic
HADOOP-13094. hadoop-common unit tests for dynamic commands
HADOOP-13095. hadoop-hdfs unit tests for dynamic commands
HADOOP-13107. clean up how rumen is executed
HADOOP-13108. dynamic subcommands need a way to manipulate arguments
HADOOP-13110. add a streaming subcommand to mapred
HADOOP-13111. convert hadoop gridmix to be dynamic
HADOOP-13115. dynamic subcommand docs should talk about exit vs. continue program flow
HADOOP-13117. clarify daemonization and security vars for dynamic commands
HADOOP-13120. add a --debug message when dynamic commands have been used
HADOOP-13121. rename sub-project shellprofiles to match the rest of Hadoop
HADOOP-13129. fix typo in dynamic subcommand docs
HADOOP-13151. Underscores should be escaped in dynamic subcommands document
HADOOP-13153. fix typo in debug statement for dynamic subcommands
2016-05-16 17:54:45 -07:00
9227dfc25f
MAPREDUCE-6558. multibyte delimiters with compressed input files generate duplicate records. Contributed by Wilfred Spiegelenburg
2016-05-13 14:33:48 +00:00
3c5c57af28
HADOOP-13142. Change project version from 3.0.0 to 3.0.0-alpha1.
2016-05-12 18:27:28 -07:00
ca5613af91
Revert "Update project version to 3.0.0-alpha1-SNAPSHOT."
...
This reverts commit 6b53802cba
2016-05-12 15:32:45 -07:00
6b53802cba
Update project version to 3.0.0-alpha1-SNAPSHOT.
2016-05-12 11:05:05 -07:00
7eddecd357
MAPREDUCE-6639 Process hangs in LocatedFileStatusFetcher if FileSystem.get throws. Ryan Blue via stevel
2016-05-12 19:00:32 +01:00
5ffb54694b
MAPREDUCE-6684. High contention on scanning of user directory under immediate_done in Job History Server. Contributed by Haibo Chen
2016-05-10 16:03:49 +00:00
cd35b692de
MAPREDUCE-6678. Allow ShuffleHandler readahead without drop-behind. Contributed by Nathan Roberts.
2016-05-10 15:45:01 +00:00
c9bb96fa81
MAPREDUCE-6689. MapReduce job can infinitely increase number of reducer resource requests. Contributed by Wangda Tan
2016-05-06 22:25:47 +00:00
8d48266720
MAPREDUCE-6514. Fixed MapReduce ApplicationMaster to properly updated resources ask after ramping down of all reducers avoiding job hangs. Contributed by Varun Saxena and Wangda Tan.
2016-05-05 19:03:27 -07:00
9d3fcdfbb3
MAPREDUCE-6675. TestJobImpl.testUnusableNode failed (haibochen via rkanter)
2016-05-04 22:42:21 -07:00
ecce3b7d53
MAPREDUCE-6677. LocalContainerAllocator doesn't specify resource of the containers allocated (haibochen via rkanter)
2016-05-04 22:19:20 -07:00
4ee4e5ca2b
MAPREDUCE-6526. Remove usage of metrics v1 from hadoop-mapreduce. (aajisaka)
2016-05-03 10:46:11 +09:00
26ac0f8f0b
MAPREDUCE-6672. TestTeraSort fails on Windows. Contributed by Tibor Kiss.
2016-04-29 11:33:32 -07:00
63e5412f1a
HDFS-9427. HDFS should not default to ephemeral ports. Contributed by Xiaobing Zhou.
2016-04-22 15:14:40 -07:00
7da5847cf1
MAPREDUCE-2398. MRBench: setting the baseDir parameter has no effect. Contributed by Michael Noll and Wilfred Spiegelenburg.
2016-04-21 17:53:51 +05:30
1e48eefe58
MAPREDUCE-6680. JHS UserLogDir scan algorithm sometime could skip directory with update in CloudFS (Azure FileSystem, S3, etc. Contributed by Junping Du
2016-04-20 19:02:10 -07:00
e6c0742012
YARN-4965. Distributed shell AM failed due to ClientHandlerException thrown by jersey. Contributed by Junping Du
2016-04-16 19:39:18 -07:00
cc8b83a8e8
MAPREDUCE-6649. getFailureInfo not returning any failure info. Contributed by Eric Badger
2016-04-16 22:07:27 +00:00
8b2880c0b6
MAPREDUCE-6513. MR job got hanged forever when one NM unstable for some time. (Varun Saxena via wangda)
2016-04-14 11:00:53 -07:00
1fec06e037
MAPREDUCE-6633. AM should retry map attempts if the reduce task encounters commpression related errors. Contributed by Rushabh Shah
2016-04-09 16:51:57 +00:00
3be1ab485f
MAPREDUCE-6647. MR usage counters use the resources requested instead of the resources allocated (haibochen via rkanter)
2016-04-06 17:15:43 -07:00
de96d7c88a
MAPREDUCE-6670. TestJobListCache#testEviction sometimes fails on Windows with timeout. Contributed by Gergely Novák.
2016-04-06 08:32:35 -07:00
0a74610d1c
HADOOP-11393. Revert HADOOP_PREFIX, go back to HADOOP_HOME (aw)
2016-03-31 07:51:05 -07:00
Akira Ajisaka
Tsuyoshi Ozawa
Jian He
Sangjin Lee
Wangda Tan
Gera Shegalov
Robert Kanter
Ming Ma
Arun Suresh
Kai Zheng
Varun Vasudev
Carlo Curino
Junping Du
Jason Lowe
Allen Wittenauer
Andrew Wang
Steve Loughran
Eric Payne
Vinod Kumar Vavilapalli
Chris Nauroth
Jing Zhao
Harsh J
Xuan