4a1cedc010
MAPREDUCE-5044. Have AM trigger jstack on task attempts that timeout before killing them. (Eric Payne and Gera Shegalov via mingma)
2016-06-06 14:30:51 -07:00
dc26601d8f
YARN-5180. Allow ResourceRequest to specify an enforceExecutionType flag. (asuresh)
2016-06-02 09:01:02 -07:00
93d8a7f2a2
Revert "HADOOP-12579. Deprecate and remove WriteableRPCEngine. Contributed by Kai Zheng"
...
This reverts commit a6c79f92d5
2016-06-01 08:41:15 +08:00
42f90ab885
YARN-4844. Add getMemorySize/getVirtualCoresSize to o.a.h.y.api.records.Resource. Contributed by Wangda Tan.
2016-05-29 21:24:16 +05:30
013532a95e
YARN-4957. Add getNewReservation in ApplicationClientProtocol (Sean Po via curino)
2016-05-25 16:55:49 -07:00
a6c79f92d5
HADOOP-12579. Deprecate and remove WriteableRPCEngine. Contributed by Kai Zheng
2016-05-27 04:29:42 +08:00
ae353ea969
MAPREDUCE-6703. Add flag to allow MapReduce AM to request for OPPORTUNISTIC containers. Contributed by Arun Suresh
2016-05-24 19:47:50 -07:00
3b3b63081b
MAPREDUCE-6607. Enable regex pattern matching when mapreduce.task.files.preserve.filepattern is set. Contributed by Kai Sasaki.
2016-05-23 07:24:31 +09:00
21d2b90213
MAPREDUCE-6696. Add a configuration to limit the number of map tasks allowed per job. Contributed by Zhihai Xu
2016-05-19 19:25:16 -07:00
22fcd819f0
YARN-5076. YARN web interfaces lack XFS protection. Contributed by Jonathan Maron.
...
(cherry picked from commit 2703ec6871
2016-05-19 14:15:21 -07:00
992a49353f
MAPREDUCE-6686. Add a way to download the job config from the mapred CLI (rkanter)
2016-05-18 12:16:21 -07:00
f6ef876fe1
MAPREDUCE-6657. Job history server can fail on startup when NameNode is in start phase. Contributed by Haibo Chen.
2016-05-17 14:41:47 -07:00
34fddd1e91
MAPREDUCE-6698. Increase timeout on TestUnnecessaryBlockingOnHistoryFileInfo.testTwoThreadsQueryingDifferentJobOfSameUser. Contributed by Haibo Chen
2016-05-17 15:01:33 +00:00
12fa4ec141
MAPREDUCE-6701. application master log can not be available when clicking jobhistory's am logs link. Contributed by Haibo Chen
2016-05-17 14:54:13 +00:00
b357930526
MAPREDUCE-6693. ArrayIndexOutOfBoundsException occurs when the length of the job name is equal to mapreduce.jobhistory.jobname.limit. Contributed by Ajith S.
2016-05-17 20:19:01 +09:00
730bc746f9
HADOOP-12930. Dynamic subcommands for hadoop shell scripts (aw)
...
This commit contains the following JIRA issues:
HADOOP-12931. bin/hadoop work for dynamic subcommands
HADOOP-12932. bin/yarn work for dynamic subcommands
HADOOP-12933. bin/hdfs work for dynamic subcommands
HADOOP-12934. bin/mapred work for dynamic subcommands
HADOOP-12935. API documentation for dynamic subcommands
HADOOP-12936. modify hadoop-tools to take advantage of dynamic subcommands
HADOOP-13086. enable daemonization of dynamic commands
HADOOP-13087. env var doc update for dynamic commands
HADOOP-13088. fix shellprofiles in hadoop-tools to allow replacement
HADOOP-13089. hadoop distcp adds client opts twice when dynamic
HADOOP-13094. hadoop-common unit tests for dynamic commands
HADOOP-13095. hadoop-hdfs unit tests for dynamic commands
HADOOP-13107. clean up how rumen is executed
HADOOP-13108. dynamic subcommands need a way to manipulate arguments
HADOOP-13110. add a streaming subcommand to mapred
HADOOP-13111. convert hadoop gridmix to be dynamic
HADOOP-13115. dynamic subcommand docs should talk about exit vs. continue program flow
HADOOP-13117. clarify daemonization and security vars for dynamic commands
HADOOP-13120. add a --debug message when dynamic commands have been used
HADOOP-13121. rename sub-project shellprofiles to match the rest of Hadoop
HADOOP-13129. fix typo in dynamic subcommand docs
HADOOP-13151. Underscores should be escaped in dynamic subcommands document
HADOOP-13153. fix typo in debug statement for dynamic subcommands
2016-05-16 17:54:45 -07:00
9227dfc25f
MAPREDUCE-6558. multibyte delimiters with compressed input files generate duplicate records. Contributed by Wilfred Spiegelenburg
2016-05-13 14:33:48 +00:00
3c5c57af28
HADOOP-13142. Change project version from 3.0.0 to 3.0.0-alpha1.
2016-05-12 18:27:28 -07:00
ca5613af91
Revert "Update project version to 3.0.0-alpha1-SNAPSHOT."
...
This reverts commit 6b53802cba
2016-05-12 15:32:45 -07:00
6b53802cba
Update project version to 3.0.0-alpha1-SNAPSHOT.
2016-05-12 11:05:05 -07:00
7eddecd357
MAPREDUCE-6639 Process hangs in LocatedFileStatusFetcher if FileSystem.get throws. Ryan Blue via stevel
2016-05-12 19:00:32 +01:00
5ffb54694b
MAPREDUCE-6684. High contention on scanning of user directory under immediate_done in Job History Server. Contributed by Haibo Chen
2016-05-10 16:03:49 +00:00
cd35b692de
MAPREDUCE-6678. Allow ShuffleHandler readahead without drop-behind. Contributed by Nathan Roberts.
2016-05-10 15:45:01 +00:00
c9bb96fa81
MAPREDUCE-6689. MapReduce job can infinitely increase number of reducer resource requests. Contributed by Wangda Tan
2016-05-06 22:25:47 +00:00
8d48266720
MAPREDUCE-6514. Fixed MapReduce ApplicationMaster to properly updated resources ask after ramping down of all reducers avoiding job hangs. Contributed by Varun Saxena and Wangda Tan.
2016-05-05 19:03:27 -07:00
9d3fcdfbb3
MAPREDUCE-6675. TestJobImpl.testUnusableNode failed (haibochen via rkanter)
2016-05-04 22:42:21 -07:00
ecce3b7d53
MAPREDUCE-6677. LocalContainerAllocator doesn't specify resource of the containers allocated (haibochen via rkanter)
2016-05-04 22:19:20 -07:00
4ee4e5ca2b
MAPREDUCE-6526. Remove usage of metrics v1 from hadoop-mapreduce. (aajisaka)
2016-05-03 10:46:11 +09:00
26ac0f8f0b
MAPREDUCE-6672. TestTeraSort fails on Windows. Contributed by Tibor Kiss.
2016-04-29 11:33:32 -07:00
63e5412f1a
HDFS-9427. HDFS should not default to ephemeral ports. Contributed by Xiaobing Zhou.
2016-04-22 15:14:40 -07:00
7da5847cf1
MAPREDUCE-2398. MRBench: setting the baseDir parameter has no effect. Contributed by Michael Noll and Wilfred Spiegelenburg.
2016-04-21 17:53:51 +05:30
1e48eefe58
MAPREDUCE-6680. JHS UserLogDir scan algorithm sometime could skip directory with update in CloudFS (Azure FileSystem, S3, etc. Contributed by Junping Du
2016-04-20 19:02:10 -07:00
e6c0742012
YARN-4965. Distributed shell AM failed due to ClientHandlerException thrown by jersey. Contributed by Junping Du
2016-04-16 19:39:18 -07:00
cc8b83a8e8
MAPREDUCE-6649. getFailureInfo not returning any failure info. Contributed by Eric Badger
2016-04-16 22:07:27 +00:00
8b2880c0b6
MAPREDUCE-6513. MR job got hanged forever when one NM unstable for some time. (Varun Saxena via wangda)
2016-04-14 11:00:53 -07:00
1fec06e037
MAPREDUCE-6633. AM should retry map attempts if the reduce task encounters commpression related errors. Contributed by Rushabh Shah
2016-04-09 16:51:57 +00:00
3be1ab485f
MAPREDUCE-6647. MR usage counters use the resources requested instead of the resources allocated (haibochen via rkanter)
2016-04-06 17:15:43 -07:00
de96d7c88a
MAPREDUCE-6670. TestJobListCache#testEviction sometimes fails on Windows with timeout. Contributed by Gergely Novák.
2016-04-06 08:32:35 -07:00
0a74610d1c
HADOOP-11393. Revert HADOOP_PREFIX, go back to HADOOP_HOME (aw)
2016-03-31 07:51:05 -07:00
690d8a368d
MAPREDUCE-6663. [NNBench] Refactor nnbench as a Tool implementation. Contributed by Brahma Reddy Battula.
2016-03-30 11:42:54 +09:00
0050fa5f1c
MAPREDUCE-6543. Migrate MR client test cases part 2. Contributed by Dustin Cote.
2016-03-29 18:17:52 +09:00
0ef8bbfd87
MAPREDUCE-6662. Clear ASF Warnings on test data files. Contributed by Vinayakumar B
2016-03-29 10:32:21 +08:00
90fcb16c0f
MAPREDUCE-6655. Fix a typo (STRICT_IE6) in Encrypted Shuffle. Contributed by Wei-Chiu Chuang.
2016-03-28 05:30:42 -07:00
115be193df
Revert "MAPREDUCE-6543. Migrate MR client test cases part 2. Contributed by Dustin Cote."
...
This reverts commit 2c268cc936
2016-03-28 16:05:35 +09:00
3f622a143c
MAPREDUCE-6535. TaskID default constructor results in NPE on toString(). Contributed by Daniel Templeton
2016-03-25 20:04:32 +00:00
2c268cc936
MAPREDUCE-6543. Migrate MR client test cases part 2. Contributed by Dustin Cote.
2016-03-25 15:17:27 +09:00
738155063e
HADOOP-12857. rework hadoop-tools (aw)
2016-03-23 13:46:38 -07:00
368c77376a
MAPREDUCE-6656. [NNBench] OP_DELETE operation isn't working after MAPREDUCE-6363. Contributed by J.Andreina.
2016-03-23 23:22:43 +09:00
0bfe5a0167
MAPREDUCE-6580. Test failure: TestMRJobsWithProfiler. Contributed by Eric Badger.
2016-03-23 13:18:06 +09:00
57ead18a85
MAPREDUCE-6110. JobHistoryServer CLI throws NullPointerException with job ids that do not exist. (Kai Sasaki via gtcarrera9)
2016-03-22 14:27:32 -07:00
7fae4c68e6
YARN-4732. *ProcessTree classes have too many whitespace issues
2016-03-20 10:47:38 -07:00
80fa70c4e1
MAPREDUCE-6645. TestWordStats outputs logs under directories other than target/test-dir. Contributed by Gabor Liptak.
2016-03-17 18:08:31 +09:00
6529c87551
MAPREDUCE-6579. JobStatus#getFailureInfo should not output diagnostic information when the job is running. (Akira AJISAKA via wangda)
2016-03-15 17:28:40 -07:00
247a790609
MAPREDUCE-6520. Migrate MR Client test cases part 1. Contributed by Dustin Cote.
2016-03-11 22:51:20 +09:00
6876b9f9f9
Revert "MAPREDUCE-6520. Migrate MR Client test cases part 1."
...
This reverts commit ef68b44195
2016-03-11 22:50:57 +09:00
ef68b44195
MAPREDUCE-6520. Migrate MR Client test cases part 1.
2016-03-11 22:20:55 +09:00
89b16d27e2
HADOOP-12905. Clean up CHANGES.txt RAT exclusions from pom.xml files. Contributed by Chris Nauroth.
2016-03-08 21:22:06 -08:00
e51a8c1056
YARN-4737. Add CSRF filter support in YARN. Contributed by Jonathan Maron.
2016-03-07 15:26:44 +05:30
4f9fe3ac03
MAPREDUCE-6648. Add yarn.app.mapreduce.am.log.level to mapred-default.xml (harsh)
2016-03-07 13:09:57 +05:30
cbd31328a6
HADOOP-12470. In-page TOC of documentation should be automatically generated by doxia macro (iwasakims)
2016-03-04 14:11:36 +09:00
1bc454615e
HADOOP-11792. Remove all of the CHANGES.txt files.
2016-03-03 21:02:31 -08:00
ff0ee84d77
MAPREDUCE-4785. TestMRApp occasionally fails (haibochen via rkanter)
2016-03-03 16:38:07 -08:00
0f72da7e28
MAPREDUCE-6622. Add capability to set JHS job cache to a task-based limit (rchiang via rkanter)
2016-02-26 17:57:17 -08:00
c4d4df8de0
YARN-4701. When task logs are not available, port 8041 is referenced instead of port 8042 (haibochen via rkanter)
2016-02-25 12:48:02 -08:00
4343a4cf77
MAPREDUCE-6640. mapred job -history command should be able to take Job ID (rkanter)
2016-02-23 17:37:49 -08:00
c6f2d761d5
MAPREDUCE-6635. Unsafe long to int conversion in UncompressedSplitLineReader and IndexOutOfBoundsException. Contributed by Junping Du.
2016-02-23 13:05:18 +05:30
b68901d7dd
YARN-4708. Missing default mapper type in TimelineServer performance test tool usage. Contributed by Kai Sasaki.
2016-02-22 07:54:53 +09:00
6eae4337d1
MAPREDUCE-6613. Change mapreduce.jobhistory.jhist.format default from json to binary (rchiang via rkanter)
2016-02-19 17:15:54 -08:00
37577852ba
MAPREDUCE-6637. Testcase Failure : TestFileInputFormat.testSplitLocationInfo. Contributed by Brahma Reddy Battula.
2016-02-19 16:33:17 -08:00
8eee59ce6b
MAPREDUCE-6627. Add machine-readable output to mapred job -history command (rkanter)
2016-02-18 17:58:26 -08:00
a365a3941c
HADOOP-12817. Enable TLS v1.1 and 1.2 (rkanter)
2016-02-18 11:09:50 -08:00
2440671a11
MAPREDUCE-6634. Log uncaught exceptions/errors in various thread pools in mapreduce. Contributed by Sidharta Seethana.
2016-02-18 14:18:36 +05:30
453e7e0d10
MAPREDUCE-6341. addendum patch to fix typos. Contributed by John Michael Luy.
2016-02-17 14:17:32 +09:00
ec12ce8f48
Set the release date for 2.6.4.
2016-02-13 01:42:50 -08:00
a429f857b2
Update CHANGES.txt for commit of MAPREDUCE-6191 to other branches.
2016-02-10 16:52:13 +00:00
5e565ce7a7
Revert "HDFS-9686. Remove useless boxing/unboxing code. Contributed by Kousuke Saruta."
...
This reverts commit fe124da5ff
2016-02-10 23:43:33 +09:00
a0b1f10a30
MAPREDUCE-6626. Reuse ObjectMapper instance in MapReduce. Contributed by Lin Yiqun.
2016-02-10 03:05:07 +09:00
fba6e9f5ef
Update CHANGES.txt to reflect recent commits to branch-2.7 and branch-2.6.
2016-02-09 16:54:45 +00:00
fe124da5ff
HDFS-9686. Remove useless boxing/unboxing code. Contributed by Kousuke Saruta.
2016-02-06 19:31:01 +09:00
414cf07650
Adding release 2.6.5 to CHANGES.txt
2016-02-02 18:06:10 -08:00
ccbba4a4de
MAPREDUCE-6620. Jobs that did not start are shown as starting in 1969 in the JHS web UI (haibochen via rkanter)
2016-02-02 15:19:35 -08:00
43e669b22d
MAPREDUCE-6621. Memory Leak in JobClient#submitJobInternal(). Contributed by Xuan Gong
2016-02-02 19:30:45 +00:00
9875325d5c
YARN-4340. Add list API to reservation system. (Sean Po via wangda)
2016-02-02 10:17:33 +08:00
59a212b6e1
MAPREDUCE-6618. YarnClientProtocolProvider leaking the YarnClient thread. Contributed by Xuan Gong
2016-02-01 16:05:06 +00:00
df99ea8a92
MAPREDUCE-6616. Fail to create jobhistory file if there are some multibyte characters in the job name. Contributed by Kousuke Saruta.
2016-01-29 16:20:29 +09:00
41da9a0773
MAPREDUCE-6563. Streaming documentation contains a stray % character. Contributed by Chris Nauroth.
2016-01-28 14:46:01 -08:00
86560a4978
MAPREDUCE-6595. Fix findbugs warnings in OutputCommitter and FileOutputCommitter. Contributed by Akira AJISAKA.
2016-01-28 06:14:21 -08:00
dca0dc8ac2
MAPREDUCE-6431. JobClient should be an AutoClosable (haibochen via rkanter)
2016-01-27 17:11:07 -08:00
d0d7c22168
MAPREDUCE-6610. JobHistoryEventHandler should not swallow timeline response. Contributed by Li Lu
2016-01-25 23:01:03 -08:00
ec4d2d9f40
Release process for 2.7.2: Set the release date for 2.7.2
2016-01-25 15:45:57 -08:00
2b833297ce
Added MAPREDUCE-6614 to 2.8.0 in CHANGES.txt.
2016-01-25 11:41:19 +09:00
f5c8c857be
MAPREDUCE-6605. Fix typos mapreduce.map.skip.proc.count.autoincr and mapreduce.reduce.skip.proc.count.autoincr in mapred-default.xml. Contributed by Kai Sasaki.
2016-01-22 18:35:43 +09:00
d40859fab1
YARN-4526. Make SystemClock singleton so AppSchedulingInfo could use it. (kasha)
2016-01-18 10:58:14 +01:00
5af5f9575a
HADOOP-12714. Fix hadoop-mapreduce-client-nativetask unit test which fails because it is not able to open the "glibc bug spill" file. (cmccabe)
2016-01-15 13:53:05 -08:00
9fbd579ab5
MAPREDUCE-6554. MRAppMaster servicestart failing with NPE in MRAppMaster#parsePreviousJobHistory. Contributed by Bibin A Chundatt
2016-01-15 16:52:53 +00:00
b1ed28fa77
HADOOP-8887. Use a Maven plugin to build the native code using CMake (cmccabe)
2016-01-14 11:02:34 -08:00
5cc44d18aa
MAPREDUCE-6601. Fix typo in Job#setUseNewAPI. Contributed by Kai Sasaki.
2016-01-15 00:44:24 +09:00
7b0964f354
MAPREDUCE-6363. [NNBench] Lease mismatch error when running with multiple mappers. Contributed by Vlad Sharanhovich and Bibin A Chundatt.
2016-01-14 10:40:22 +09:00
9b33a03900
MAPREDUCE-6549. multibyte delimiters with LineRecordReader cause duplicate records (wilfreds via rkanter)
...
Removed multiple duplicate CHANGES.txt entries.
2016-01-13 15:02:51 -08:00
f657b54281
MAPREDUCE-6473. Job submission can take a long time during Cluster initialization. Contributed by Kuhu Shukla
2016-01-12 23:56:23 +00:00
9e792da014
YARN-4582. Label-related invalid resource request exception should be able to properly handled by application. (Bibin A Chundatt via wangda)
2016-01-12 12:53:31 +08:00
e5df366707
Add MR-5982 and MR-6492 to 2.6.4
2016-01-11 18:31:10 +00:00
8c1adeaa26
MAPREDUCE-6068. Illegal progress value warnings in map tasks. Contributed by Binglin Chang.
2016-01-11 06:12:19 -08:00
355c0ce723
MAPREDUCE-6577 addendum. Moved the changelog to 2.6.4 which is the earliest target.
2016-01-05 15:43:10 -08:00
f6f16118d3
MAPREDUCE-6577. MR AM unable to load native library without MR_AM_ADMIN_USER_ENV set (sjlee)
2016-01-05 15:22:50 -08:00
7dafee11d8
MAPREDUCE-6593. TestJobHistoryEventHandler.testTimelineEventHandling fails on trunk because of NPE. Contributed by Naganarasimha G R.
2016-01-04 16:30:55 +09:00
db99e30f67
Set release date for 2.6.3
2016-01-04 16:12:12 +09:00
2a59d01503
MAPREDUCE-6589. TestTaskLog outputs a log under directory other than target/test-dir. (aajisaka)
2015-12-30 01:33:47 +09:00
bee542817d
MAPREDUCE-6584. Remove trailing whitespaces from mapred-default.xml. (aajisaka)
2015-12-30 01:03:12 +09:00
32b3dc11eb
MAPREDUCE-6574. MR AM should print host of failed tasks. (Mohammad Shahid Khan via wangda)
2015-12-28 11:03:23 -08:00
a308e86621
MAPREDUCE-6419. JobHistoryServer doesn't sort properly based on Job ID when Job id's exceed 9999. (Mohammad Shahid Khan via rohithsharmaks)
2015-12-24 11:55:42 +05:30
7995a6ea4d
MAPREDUCE-6583. Clarify confusing sentence in MapReduce tutorial document. Contributed by Kai Sasaki.
2015-12-21 00:16:14 +09:00
85c2466048
YARN-4164. Changed updateApplicationPriority API to return the updated application priority. Contributed by Rohith Sharma K S
2015-12-18 14:13:48 -08:00
d4e766de93
MAPREDUCE-6566. Add retry support to mapreduce CLI tool. Contributed by Varun Vasudev
2015-12-16 15:26:03 -08:00
60fe6d53d9
Revert "MAPREDUCE-6566. Add retry support to mapreduce CLI tool. Contributed by Varun Vasudev"
...
This reverts commit fc470840a0
2015-12-16 15:11:38 -08:00
7092d47fc0
Update CHANGES.txt to move MAPREDUCE-6436 from YARN to MAPREDUCE
2015-12-15 21:43:19 -08:00
5b7078d069
MAPREDUCE-6436. JobHistory cache issue. Contributed by Kai Sasaki
2015-12-15 00:58:23 -08:00
fc470840a0
MAPREDUCE-6566. Add retry support to mapreduce CLI tool. Contributed by Varun Vasudev
2015-12-07 14:15:14 -08:00
65f395226b
HADOOP-12321. Make JvmPauseMonitor an AbstractService. (Sunil G via Stevel) [includes HDFS-8947 MAPREDUCE-6462 and YARN-4072]
2015-12-06 17:43:35 +00:00
9f77ccad73
YARN-3840. Resource Manager web ui issue when sorting application by id (with application having id > 9999). Contributed by Mohammad Shahid Khan and Varun Saxena
2015-12-03 12:48:50 -08:00
028894281b
Creating 2.6.4 entries in CHANGES.txt files.
2015-12-01 08:16:39 -08:00
43acf9ab8e
Update CHANGES.txt for commit of MR-6549 to branch-2.7 and branch-2.6.
2015-11-30 17:54:39 +00:00
c37c3f41b3
Revert "HADOOP-11505. Various native parts use bswap incorrectly and unportably (Alan Burlison via aw)"
...
This reverts commit 6725e7f1be
2015-11-28 17:21:11 -08:00
6725e7f1be
HADOOP-11505. Various native parts use bswap incorrectly and unportably (Alan Burlison via aw)
2015-11-27 18:25:45 -08:00
d57fd181c7
Adding release 2.8.0 to CHANGES.txt
2015-11-25 17:33:26 -08:00
6d84cc16b3
MAPREDUCE-6550. archive-logs tool changes log ownership to the Yarn user when using DefaultContainerExecutor (rkanter)
2015-11-25 17:12:40 -08:00
7fd00b3db4
MAPREDUCE-6549. multibyte delimiters with LineRecordReader cause duplicate records (wilfreds via rkanter)
2015-11-25 17:03:38 -08:00
15d577bfbb
Tests in mapreduce-client-app are writing outside of target. Contributed by Akira AJISAKA.
2015-11-25 09:15:26 -08:00
3e85542a7a
MAPREDUCE-6555. TestMRAppMaster fails on trunk. (Junping Du via ozawa)
2015-11-26 01:02:27 +09:00
177975e962
Fix indents in the 2.8.0 section of MapReduce CHANGES.txt.
2015-11-25 16:42:22 +09:00
5ec44fc35c
MAPREDUCE-6553. Replace '\u2b05' with '<-' in rendering job configuration. Contributed by Gabor Liptak.
2015-11-25 16:30:23 +09:00
f634505d48
MAPREDUCE-5870. Support for passing Job priority through Application Submission Context in Mapreduce Side. Contributed by Sunil G
2015-11-24 22:07:26 +00:00
cab3c7c889
MAPREDUCE-5883. "Total megabyte-seconds" in job counters is slightly misleading. Contributed by Nathan Roberts
2015-11-24 22:01:03 +00:00
8676a118a1
YARN-4349. Support CallerContext in YARN. Contributed by Wangda Tan
2015-11-23 17:19:48 -08:00
4bff073b4d
Revert "HADOOP-12572. Update Hadoop's lz4 to r131. Contributed by Kevin Bowling."
...
This reverts commit 4c061e607e
2015-11-23 15:00:56 -08:00
aa2009c37b
Update CHANGES.txt for recent commits to branch-2.6.
2015-11-23 21:57:18 +00:00
000e12f6fa
HDFS-9153. Pretty-format the output for DFSIO. Contributed by Kai Zheng.
2015-11-22 16:18:07 -08:00
4c061e607e
HADOOP-12572. Update Hadoop's lz4 to r131. Contributed by Kevin Bowling.
2015-11-22 15:18:24 -08:00
be9f3bba73
Update CHANGES.txt to reflect commit of MR-6377 to branch-2.7 and branch-2.6.
2015-11-18 17:20:15 +00:00
a7f776ffdf
MAPREDUCE-6499. Add elapsed time for retired job in JobHistoryServer WebUI. Contributed by Lin Yiqun.
2015-11-17 15:16:47 +09:00
fcd7888029
Revert "YARN-3840. Resource Manager web ui issue when sorting application by id (with application having id > 9999) Contributed by Mohammad Shahid Khan"
...
This reverts commit 8fbea531d7
2015-11-16 20:18:44 -08:00
6502d59e73
MAPREDUCE-5485. Allow repeating job commit by extending OutputCommitter API. Contributed by Junping Du
2015-11-16 17:06:41 -08:00
9ad708a0df
MAPREDUCE-6540. TestMRTimelineEventHandling fails (sjlee)
2015-11-11 15:37:24 -08:00
df68eac825
MAPREDUCE-6533. testDetermineCacheVisibilities of TestClientDistributedCacheManager is broken. Contributed by Chang Li
2015-11-11 17:26:16 +00:00
8fbea531d7
YARN-3840. Resource Manager web ui issue when sorting application by id (with application having id > 9999) Contributed by Mohammad Shahid Khan
2015-11-09 10:43:45 -08:00
668e897538
MAPREDUCE-5763. Warn message about httpshuffle in NM logs. Contributed by Akira AJISAKA.
2015-11-09 15:25:31 +09:00
b76b0ce51e
HADOOP-12366. expose calculated paths (aw)
2015-11-07 08:32:56 -08:00
19a0c2660c
MAPREDUCE-1125. SerialUtils.cc: deserializeFloat is out of sync with SerialUtils.hh (Simone Leo via aw)
2015-11-05 16:57:22 -08:00
fc7cd46faf
MAPREDUCE-3914. Mismatched free() / delete / delete [] in HadoopPipes (Joe Mudd via aw)
2015-11-05 16:53:32 -08:00
0383a3973b
Add 2.7.3 release to CHANGES.txt
2015-11-04 16:26:14 +00:00
73b9c7b82b
HADOOP-10787. Rename/remove non-HADOOP_*, etc from the shell scripts. Contributed by Allen Wittenauer.
2015-11-04 15:56:17 +05:30
04d97f8abb
MAPREDUCE-6525. Fix test failure of TestMiniMRClientCluster.testRestart. Contributed by Masatake Iwasaki.
2015-11-03 01:50:07 +09:00
a4a6b5b4b4
Updated the 2.6.2 final release date.
2015-10-30 18:47:16 -07:00
2868ca0328
MAPREDUCE-6451. DistCp has incorrect chunkFilePath for multiple jobs when strategy is dynamic. Contributed by Kuhu Shukla.
2015-10-30 14:56:41 -05:00
6344b6a769
MAPREDUCE-6528. Memory leak for HistoryFileManager.getJobSummary(). Contributed by Junping Du
2015-10-30 15:31:38 +00:00
6ae9efaf59
Update CHANGES.txt to reflect commit of MR-6273 to branch-2.6
2015-10-30 15:18:53 +00:00
eadf7b3096
Creating 2.6.3 entries in CHANGES.txt files.
2015-10-30 14:50:50 +00:00
ce31b22739
YARN-4320. TestJobHistoryEventHandler fails as AHS in MiniYarnCluster no longer binds to default port 8188. Contributed by Varun Saxena.
2015-10-30 18:00:20 +09:00
7412ff48ee
YARN-4313. Race condition in MiniMRYarnCluster when getting history
...
server address. Contributed by Jian He
2015-10-29 17:36:36 -07:00
ecad1253b7
Update CHANGES.txt to reflect commit of MR-6273 to branch-2.7
2015-10-29 18:32:25 +00:00
cf953b6258
MAPREDUCE-6515. Update Application priority in AM side from AM-RM heartbeat. Contributed by Sunil G
2015-10-29 18:05:01 +00:00
b0c818b60d
Set the release date for 2.6.2
2015-10-28 20:43:35 -07:00
68ce93c32e
MAPREDUCE-6435. MapReduce client assumes the world is x86 (Alan Burlison via aw)
2015-10-27 12:29:41 -07:00
5c24fe7f91
MAPREDUCE-6416. Not all platforms have d_type in struct dirent (Alan Burlison via aw)
2015-10-27 12:22:08 -07:00
ab99d953e4
MAPREDUCE-6412. Make hadoop-mapreduce-client Native code -Wall-clean (Alan Burlison via aw)
2015-10-27 12:15:11 -07:00
0e344902aa
fix changes.txt for MAPREDUCE-6391.
2015-10-27 12:07:18 -07:00
bcc4c746a8
MAPREDUCE-6391. util/Timer.cc completely misunderstands _POSIX_CPUTIME (Alan Burlison via aw)
2015-10-27 12:04:55 -07:00
96677bef00
YARN-3573. MiniMRYarnCluster constructor that starts the timeline server using a boolean should be marked deprecated. Contributed by Brahma Reddy Battula.
2015-10-27 12:35:01 +09:00
eb6379ca25
MAPREDUCE-6508. TestNetworkedJob fails consistently due to delegation token changes on RM. Contributed by Akira AJISAKA
2015-10-23 10:05:46 -07:00
ee9b80acf8
MAPREDUCE-6518. Set SO_KEEPALIVE on shuffle connections. Contributed by Chang Li
2015-10-21 21:44:43 +00:00
cb26cd4bee
MAPREDUCE-6489. Fail fast rogue tasks that write too much to local disk. Contributed by Maysam Yabandeh
2015-10-21 14:01:23 +00:00
0c4af0f998
MAPREDUCE-6495. Docs for archive-logs tool (rkanter)
2015-10-20 17:34:34 -07:00
58590fef49
YARN-2556. Tool to measure the performance of the timeline server (Chang Li via sjlee)
2015-10-16 16:53:20 -07:00
da8441d0fe
HDFS-9167. Update pom.xml in other modules to depend on hdfs-client instead of hdfs. Contributed by Mingliang Liu.
2015-10-13 15:24:24 -07:00
4aa9b3e75c
MAPREDUCE-6302. Incorrect headroom can lead to a deadlock between map and reduce allocations. (kasha)
2015-10-09 07:37:39 -07:00
a0bca2b5ad
YARN-261. Ability to fail AM attempts. Contributed by Andrey Klochkov and Rohith Sharma K S
2015-10-09 14:17:38 +00:00
584cf3bb35
MAPREDUCE-6479. Add missing mapred job command options in mapreduce document. Contributed by nijel.
2015-10-09 10:54:12 +09:00
1bca1bb517
MAPREDUCE-6503. archive-logs tool should use HADOOP_PREFIX instead of HADOOP_HOME (rkanter)
2015-10-06 16:49:50 -07:00
8f08532bde
YARN-1897. CLI and core support for signal container functionality. Contributed by Ming Ma
2015-10-02 18:50:47 -07:00
439f43ad3d
MAPREDUCE-6485. Create a new task attempt with failed map task priority if in-progress attempts are unassigned. (Xianyin Xin via rohithsharmaks)
2015-10-02 20:34:34 +05:30
195793c6f3
MAPREDUCE-6497. Fix wrong value of JOB_FINISHED event in JobHistoryEventHandler. Contributed by Shinichi Yamashita.
2015-10-01 10:22:45 +02:00
5db371f52f
MAPREDUCE-6494. Permission issue when running archive-logs tool as different users (rkanter)
2015-09-30 17:33:53 -07:00
ab11085b81
MAPREDUCE-6492. AsyncDispatcher exit with NPE on TaskAttemptImpl#sendJHStartEventForAssignedFailTask. Contributed by Bibin A Chundatt
2015-09-28 22:12:03 +00:00
66dad854c0
MAPREDUCE-6471. Document distcp incremental copy. Contributed by Neelesh Srinivas Salian.
2015-09-28 13:12:43 +05:30
d3c49e7662
MAPREDUCE-6480. archive-logs tool may miss applications (rkanter)
2015-09-25 15:02:42 -07:00
97a08807ec
MAPREDUCE-6484. Yarn Client uses local address instead of RM address as token renewer in a secure cluster when RM HA is enabled. Contributed by Zhihai Xu
2015-09-24 08:12:07 -07:00
1f707ecffd
Release process for 2.6.1: Set the release date for 2.6.1
2015-09-23 15:10:50 -07:00
89cab1ba5f
YARN-1651. CapacityScheduler side changes to support container resize. Contributed by Wangda Tan
2015-09-23 13:29:38 -07:00
83a18add10
YARN-1449. AM-NM protocol changes to support container resizing. Contributed by Meng Ding & Wangda Tan)
2015-09-23 13:29:36 -07:00
dfe2cb849f
YARN-3866. AM-RM protocol changes to support container resizing. Contributed by Meng Ding
2015-09-23 13:29:36 -07:00
cc2b473990
MAPREDUCE-5045. UtilTest#isCygwin method appears to be unused. Contributed by Neelesh Srinivas Salian.
2015-09-22 21:38:06 +05:30
dfd807afab
HADOOP-12428. Fix inconsistency between log-level guards and statements. Contributed by Jagadesh Kiran N and Jackie Chang.
2015-09-22 12:54:29 +09:00
e3ace31e2b
MAPREDUCE-6460. TestRMContainerAllocator.testAttemptNotFoundCausesRMCommunicatorException fails. Contributed by Zhihai Xu.
2015-09-19 00:10:06 -07:00
602335dfe6
MAPREDUCE-6483. Replace deprecated method NameNode.getUri() with DFSUtilClient.getNNUri() in TestMRCredentials. Contributed by Mingliang Liu.
2015-09-18 13:36:09 -07:00
f97dc034fe
Update CHANGES.txt to reflect commit of MR-5982 to 2.7.2
2015-09-18 19:24:10 +00:00
372ad270a0
MAPREDUCE-6478. Add an option to skip cleanupJob stage or ignore cleanup failure during commitJob. (Junping Du via wangda)
2015-09-18 10:10:32 -07:00
288c8854a6
Update CHANGES.txt to reflect LineRecordReader fixes committed to 2.7.2
2015-09-18 15:14:18 +00:00
ee4ee6af6a
MAPREDUCE-5982. Task attempts that fail from the ASSIGNED state can disappear. Contributed by Chang Li
2015-09-17 21:37:39 +00:00
3f82f582e5
MAPREDUCE-5002. AM could potentially allocate a reduce container to a map attempt. Contributed by Chang Li
2015-09-17 18:17:29 +00:00
58d1a02b8d
MAPREDUCE-6481. LineRecordReader may give incomplete record and wrong position/key information for uncompressed input sometimes. Contributed by Zhihai Xu
2015-09-17 14:30:18 +00:00
941001bfa6
MAPREDUCE-6477. Replace usage of deprecated NameNode.DEFAULT_PORT in TestFileSystem. Contributed by Mingliang Liu.
2015-09-16 13:06:26 -07:00
62943b8e3a
MAPREDUCE-6472. MapReduce AM should have java.io.tmpdir=./tmp to be consistent with tasks. Contributed by Naganarasimha G R
2015-09-15 19:43:20 +00:00
fbbb7ff1ed
Updating all CHANGES.txt files to move entires from future releases into 2.6.1
...
section given the large number of backports to 2.6.1.
2015-09-10 13:45:57 -07:00
8e615588d5
MAPREDUCE-6474. ShuffleHandler can possibly exhaust nodemanager file descriptors. Contributed by Kuhu Shukla
2015-09-10 16:00:17 +00:00
119cc75e7e
MAPREDUCE-6415. Create a tool to combine aggregated logs into HAR files. (Robert Kanter via kasha)
2015-09-09 17:45:19 -07:00
9b685773ec
MAPREDUCE-6442. Stack trace is missing when error occurs in client protocol provider's constructor Contributed by Chang Li.
2015-09-05 11:21:03 +09:00
e1feaf6db0
HADOOP-12358. Add -safely flag to rm to prompt when deleting many files. Contributed by Xiaoyu Yao.
2015-09-04 13:42:55 -07:00
00804e2457
Revert "MAPREDUCE-6363. [NNBench] Lease mismatch error when running with multiple mappers. Contributed by Brahma Reddy Battula."
...
This reverts commit 75a2560e51
2015-09-02 12:22:16 +09:00
cbb249534a
MAPREDUCE-6452. NPE when intermediate encrypt enabled for LocalRunner. Contributed by Zhihai Xu
2015-08-28 12:13:23 -07:00
50fa383704
Revert "MAPREDUCE-6455. Unable to use surefire > 2.18. (Charlie Helin via kasha)"
...
This reverts commit 61bf9cae6f
2015-08-27 10:15:19 -07:00
57c7ae1aff
YARN-4014. Support user cli interface in for Application Priority. Contributed by Rohith Sharma K S
2015-08-24 20:36:44 -07:00
61bf9cae6f
MAPREDUCE-6455. Unable to use surefire > 2.18. (Charlie Helin via kasha)
2015-08-22 00:39:04 -07:00
2ba90c93d7
MAPREDUCE-6357. MultipleOutputs.write() API should document that output committing is not utilized when input path is absolute. Contributed by Dustin Cote.
2015-08-21 10:41:54 +09:00
33474939a0
Creating 2.6.2 entries in CHANGES.txt files.
2015-08-20 17:21:06 -07:00
8dfec7a197
MAPREDUCE-6439. AM may fail instead of retrying if RM shuts down during the allocate call. (Anubhav Dhoot via kasha)
2015-08-15 00:52:11 -07:00
27d24f96ab
MAPREDUCE-5817. Mappers get rescheduled on node transition even after all reducers are completed. (Sangjin Lee via kasha)
2015-08-14 12:29:50 -07:00
fb1be0b310
MAPREDUCE-6257. Document encrypted spills (Bibin A Chundatt via aw)
2015-08-06 10:13:51 -07:00
e73a928a63
MAPREDUCE-6443. Add JvmPauseMonitor to JobHistoryServer. Contributed by Robert Kanter.
...
(cherry picked from commit c8469357ba
2015-08-06 07:03:46 -07:00
21d10ccc6e
HADOOP-7824. NativeIO.java flags and identifiers must be set correctly for each platform, not hardcoded to their Linux values (Martin Walsh via Colin P. McCabe)
2015-07-31 15:01:34 -07:00
666cafca8d
HADOOP-12249. pull argument parsing into a function (aw)
2015-07-31 14:32:21 -07:00
d0e0ba8010
MAPREDUCE-6394. Speed up Task processing loop in HsTasksBlock#render(). Contributed by Ray Chiang
2015-07-31 18:17:54 +00:00
93d50b7824
MAPREDUCE-6433. launchTime may be negative. Contributed by Zhihai Xu
2015-07-30 23:07:31 -07:00
cf74772064
MAPREDUCE-5801. Uber mode's log message is missing a vcore reason (Steven Wong via aw)
2015-07-21 10:58:52 -07:00
ee36f4f9b8
HADOOP-10979. Auto-entries in hadoop_usage (aw)
2015-07-16 16:58:11 -07:00
f4ccdb11dc
MAPREDUCE-6427. Fix typo in JobHistoryEventHandler. Contributed by Ray Chiang
2015-07-14 14:55:43 -07:00
19295b36d9
YARN-3381. Fix typo InvalidStateTransitonException. Contributed by Brahma Reddy Battula.
2015-07-13 17:52:13 +09:00
e04faf8e46
MAPREDUCE-6421. Fix findbugs warning in RMContainerAllocator.reduceNodeLabelExpression. Contributed by Brahma Reddy Battula.
2015-07-13 14:29:14 +09:00
fffb15bb43
MAPREDUCE-6426. TestShuffleHandler#testGetMapOutputInfo is failing.
...
Contributed by zhihai xu.
2015-07-09 15:00:26 +05:30
7e2fe8c9f2
MAPREDUCE-6038. A boolean may be set error in the Word Count v2.0 in MapReduce Tutorial. Contributed by Tsuyoshi Ozawa
2015-07-07 10:27:04 -07:00
bf89ddb9b8
Release process for 2.7.1: Set the release date for 2.7.1.
2015-07-06 16:39:59 -07:00
bff67dfe2f
MAPREDUCE-6425. ShuffleHandler passes wrong "base" parameter to
...
getMapOutputInfo if mapId is not in the cache. Contributed by zhihai xu.
2015-07-06 13:46:37 +05:30
eac1d18943
MAPREDUCE-6418. MRApp should not shutdown LogManager during shutdown. Contributed by Chang Li
2015-07-01 18:01:22 +00:00
826715622e
MAPREDUCE-6420. Interrupted Exception in LocalContainerLauncher should be logged in warn/info level. Contributed by Chang Li
2015-07-01 17:50:02 +00:00
2ac87df578
MAPREDUCE-6376. Add avro binary support for jhist files. Contributed by Ray Chiang
2015-07-01 16:00:03 +00:00
3a72bfd082
MAPREDUCE-6121. JobResourceUpdater#compareFs() doesn't handle HA namespaces (rchiang via rkanter)
2015-06-30 16:49:27 -07:00
9c63825f10
MAPREDUCE-6407. Migrate MAPREDUCE native build to new CMake framework (Alan Burlison via Colin P. McCabe)
2015-06-30 16:17:06 -07:00
b6ba56457c
MAPREDUCE-6384. Add the last reporting reducer info for too many fetch failure diagnostics. Contributed by Chang Li
2015-06-30 21:22:30 +00:00
aad6a7d5db
Adding release 2.7.2 to CHANGES.txt.
2015-06-28 16:29:12 -07:00
aa5b15b03b
MAPREDUCE-6413. TestLocalJobSubmission is failing with unknown host. Contributed by zhihai xu
2015-06-25 19:50:07 +00:00
72d08a0e41
MAPREDUCE-6400. Multiple shuffle transfer fails because input is closed too early. Contributed by Brahma Reddy Battula, Akira AJISAKA, and Gera Shegalov.
2015-06-24 15:29:11 +00:00
d481684c7c
MAPREDUCE-6410. Fixed MapReduce JobHistory server to use the right (login) UGI to refresh log and cleaner settings. Contributed by Varun Saxena.
2015-06-23 11:39:55 -07:00
41ae7768eb
MAPREDUCE-6406. Update
...
FileOutputCommitter.FILEOUTPUTCOMMITTER_ALGORITHM_VERSION_DEFAULT to match
mapred-default.xml. Contributed by Ray Chiang.
2015-06-23 10:23:44 +05:30
077250d8d7
MAPREDUCE-5948. org.apache.hadoop.mapred.LineRecordReader does not handle multibyte record delimiters well. Contributed by Vinayakumar B, Rushabh Shah, and Akira AJISAKA
2015-06-22 21:59:20 +00:00
11ac848207
MAPREDUCE-6408. Queue name and user name should be printed on the job page. (Siqi Li via gera)
2015-06-22 14:40:32 -07:00
445b13240c
MAPREDUCE-6403. Fix typo in the usage of NNBench. Contributed by Jagadesh Kiran N.
2015-06-22 15:32:39 +09:00
055cd5a9a1
MAPREDUCE-6305. AM/Task log page should be able to link back to the job. (Siqi Li via gera)
2015-06-20 12:38:36 -07:00
63d40d5286
MAPREDUCE-6316. Task Attempt List entries should link to the task overview. (Siqi Li via gera)
2015-06-20 11:38:21 -07:00
311a4179cc
MAPREDUCE-6405. NullPointerException in App Attempts page. Contributed by
...
Siqi Li and Gera Shegalov.
2015-06-20 10:35:04 +05:30
10107243be
MAPREDUCE-6395. Improve the commit failure messages in MRAppMaster recovery. (Brahma Reddy Battula via gera)
2015-06-19 02:22:59 -07:00
1babe50a2c
MAPREDUCE-6373. The logger reports total input paths but it is referring
...
to input files. Contributed by Bibin A Chundatt.
2015-06-18 11:42:22 +05:30
Ming Ma
Arun Suresh
Kai Zheng
Varun Vasudev
Carlo Curino
Jian He
Akira Ajisaka
Junping Du
Robert Kanter
Jason Lowe
Allen Wittenauer
Andrew Wang
Steve Loughran
Eric Payne
Vinod Kumar Vavilapalli
Chris Nauroth
Jing Zhao
Harsh J
Xuan
Wangda Tan
Rohith Sharma K S
Li Lu
Karthik Kambatla
Masatake Iwasaki
Robert Kanter
Tsuyoshi Ozawa
Karthik Kambatla
Colin Patrick Mccabe
Sangjin Lee
Zhihai Xu
Haohui Mai
Kihwal Lee
Chris Douglas
Devaraj K
Gera Shegalov