Commit Graph

467 Commits

Author SHA1 Message Date
Jian He e94e6be772 YARN-5270. Solve miscellaneous issues caused by YARN-4844. Contributed by Wangda Tan 2016-07-11 22:37:52 -07:00
Sangjin Lee 910742ad12 MAPREDUCE-6719. The list of -libjars archives should be replaced with a wildcard in the distributed cache to reduce the application footprint in the state store (Daniel Templeton via sjlee)
(cherry picked from commit 605b4b6136)
2016-06-21 11:26:55 -07:00
Wangda Tan 7eb524ddee YARN-4844. Addendum patch, revert few changes to public and evolving to avoid breaking downstream projects as much as possible. 2016-06-16 11:09:56 -07:00
Wangda Tan 863bfa4d6a YARN-1942. Deprecate toString/fromString methods from ConverterUtils and move them to records classes like ContainerId/ApplicationId, etc. (wangda) 2016-06-14 15:12:00 -07:00
Akira Ajisaka d219550e8b HADOOP-13270. BZip2CompressionInputStream finds the same compression marker twice in corner case, causing duplicate data blocks. Contributed by Kai Sasaki.
(cherry picked from commit e3ba9ad3f1)
2016-06-14 10:20:43 +09:00
Gera Shegalov 5a9929e19b MAPREDUCE-6240. Hadoop client displays confusing error message. (gera)
(cherry picked from commit 0af96a1c08)
2016-06-08 13:49:31 -07:00
Ming Ma 74e2b5efa2 MAPREDUCE-5044. Have AM trigger jstack on task attempts that timeout before killing them. (Eric Payne and Gera Shegalov via mingma)
(cherry picked from commit 4a1cedc010)
2016-06-06 14:34:47 -07:00
Varun Vasudev 4f36c3d214 YARN-4844. Add getMemoryLong/getVirtualCoreLong to o.a.h.y.api.records.Resource. Contributed by Wangda Tan. 2016-05-29 20:55:39 +05:30
Carlo Curino c656977961 YARN-4957. Add getNewReservation in ApplicationClientProtocol (Sean Po via curino)
(cherry picked from commit 013532a95e)
2016-05-25 16:58:19 -07:00
Jian He 396d222d7a MAPREDUCE-6703. Add flag to allow MapReduce AM to request for OPPORTUNISTIC containers. Contributed by Arun Suresh
(cherry picked from commit ae353ea969)
2016-05-24 19:48:11 -07:00
Jian He db3cf5b354 MAPREDUCE-6696. Add a configuration to limit the number of map tasks allowed per job. Contributed by Zhihai Xu
(cherry picked from commit 21d2b90213)
2016-05-19 19:25:48 -07:00
Robert Kanter 27aabcab45 MAPREDUCE-6686. Add a way to download the job config from the mapred CLI (rkanter)
(cherry picked from commit 992a49353f)
2016-05-18 12:16:45 -07:00
Robert Kanter cca8e9b90c MAPREDUCE-6677. LocalContainerAllocator doesn't specify resource of the containers allocated (haibochen via rkanter)
(cherry picked from commit ecce3b7d53)
2016-05-04 22:19:42 -07:00
Harsh J 2d7e4a8228 MAPREDUCE-2398. MRBench: setting the baseDir parameter has no effect. Contributed by Michael Noll and Wilfred Spiegelenburg.
(cherry picked from commit 7da5847cf1)
2016-04-21 17:54:24 +05:30
Akira Ajisaka 94b2ba89a4 MAPREDUCE-6663. [NNBench] Refactor nnbench as a Tool implementation. Contributed by Brahma Reddy Battula.
(cherry picked from commit 690d8a368d)
2016-03-30 11:44:33 +09:00
Akira Ajisaka 7b2cf29f4f MAPREDUCE-6543. Migrate MR client test cases part 2. Contributed by Dustin Cote.
(cherry picked from commit 0050fa5f1c)

Conflicts:
	hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-jobclient/src/test/java/org/apache/hadoop/mapreduce/TestMRJobClient.java
	hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-jobclient/src/test/java/org/apache/hadoop/mapreduce/lib/partition/TestTotalOrderPartitioner.java
2016-03-29 18:20:01 +09:00
Rohith Sharma K S f22ce96b50 MAPREDUCE-6662. Clear ASF Warnings on test data files. Contributed by Vinayakumar B
(cherry picked from commit 0ef8bbfd87)
2016-03-29 10:35:04 +08:00
Akira Ajisaka 2b1f066d2d Revert "MAPREDUCE-6543. Migrate MR client test cases part 2. Contributed by Dustin Cote."
This reverts commit 0dc1453626.
2016-03-28 16:05:55 +09:00
Akira Ajisaka 0dc1453626 MAPREDUCE-6543. Migrate MR client test cases part 2. Contributed by Dustin Cote.
(cherry picked from commit 2c268cc936)

Conflicts:
	hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-jobclient/src/test/java/org/apache/hadoop/mapreduce/TestMRJobClient.java
	hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-jobclient/src/test/java/org/apache/hadoop/mapreduce/lib/partition/TestTotalOrderPartitioner.java
2016-03-25 15:23:23 +09:00
Akira Ajisaka 7a3fd1bc1a MAPREDUCE-6656. [NNBench] OP_DELETE operation isn't working after MAPREDUCE-6363. Contributed by J.Andreina.
(cherry picked from commit 368c77376a)
2016-03-23 23:24:30 +09:00
Akira Ajisaka 12c1cc889a MAPREDUCE-6580. Test failure: TestMRJobsWithProfiler. Contributed by Eric Badger.
(cherry picked from commit 0bfe5a0167)
2016-03-23 13:19:08 +09:00
Wangda Tan abfb2957d1 MAPREDUCE-6579. JobStatus#getFailureInfo should not output diagnostic information when the job is running. (Akira AJISAKA via wangda)
(cherry picked from commit 6529c87551)
2016-03-15 17:30:37 -07:00
Akira Ajisaka c2e14d1bbb MAPREDUCE-6520. Migrate MR Client test cases part 1. Contributed by Dustin Cote.
(cherry picked from commit 247a790609)
2016-03-11 22:54:40 +09:00
Akira Ajisaka 99f61afcdd Revert "MAPREDUCE-6520. Migrate MR Client test cases part 1."
This reverts commit 7d665f0a98.
2016-03-11 22:52:10 +09:00
Akira Ajisaka 7d665f0a98 MAPREDUCE-6520. Migrate MR Client test cases part 1.
(cherry picked from commit ef68b44195)
2016-03-11 22:25:29 +09:00
Robert Kanter d9f01f28a0 MAPREDUCE-6640. mapred job -history command should be able to take Job ID (rkanter)
(cherry picked from commit 4343a4cf77)
2016-02-23 17:38:25 -08:00
Tsuyoshi Ozawa 929e869ae0 YARN-4708. Missing default mapper type in TimelineServer performance test tool usage. Contributed by Kai Sasaki.
(cherry picked from commit b68901d7dd)
2016-02-22 07:55:35 +09:00
Robert Kanter a33109a0b8 MAPREDUCE-6627. Add machine-readable output to mapred job -history command (rkanter)
(cherry picked from commit 8eee59ce6b)
2016-02-18 17:58:40 -08:00
Jason Lowe 303c650d0d MAPREDUCE-6191. Improve clearing stale state of Java serialization
testcase.  (Sam Liu via Eric Yang)
(cherry picked from commit c379e102dd)

Conflicts:

	hadoop-mapreduce-project/CHANGES.txt
2016-02-10 16:44:12 +00:00
Wangda Tan 9591363ad8 YARN-4340. Add list API to reservation system. (Sean Po via wangda)
(cherry picked from commit 9875325d5c)
2016-02-02 10:19:19 +08:00
Jason Lowe f5f6652137 MAPREDUCE-6618. YarnClientProtocolProvider leaking the YarnClient thread. Contributed by Xuan Gong
(cherry picked from commit 59a212b6e1)
2016-02-01 16:06:42 +00:00
Karthik Kambatla ef59521094 YARN-4526. Make SystemClock singleton so AppSchedulingInfo could use it. (kasha)
(cherry picked from commit d40859fab1)

Conflicts:
	hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-app/src/test/java/org/apache/hadoop/mapred/TestTaskAttemptListenerImpl.java
2016-01-18 10:59:34 +01:00
Akira Ajisaka 6efee8907a MAPREDUCE-6363. [NNBench] Lease mismatch error when running with multiple mappers. Contributed by Vlad Sharanhovich and Bibin A Chundatt.
(cherry picked from commit 7b0964f354)
2016-01-14 10:41:22 +09:00
Sangjin Lee 2dfa0613a1 MAPREDUCE-6577. MR AM unable to load native library without MR_AM_ADMIN_USER_ENV set (sjlee)
(cherry picked from commit f6f16118d3)
2016-01-05 15:25:47 -08:00
Jian He 1147b086d9 YARN-4164. Changed updateApplicationPriority API to return the updated application priority. Contributed by Rohith Sharma K S 2015-12-18 14:14:31 -08:00
Vinod Kumar Vavilapalli b20e77eba3 Preparing for 2.9.0 development: mvn versions:set -DnewVersion=2.9.0 2015-11-25 17:47:22 -08:00
Jason Lowe 2f18218508 MAPREDUCE-5870. Support for passing Job priority through Application Submission Context in Mapreduce Side. Contributed by Sunil G
(cherry picked from commit f634505d48)

Conflicts:

	hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-core/src/test/java/org/apache/hadoop/mapred/TestJobConf.java
2015-11-24 22:15:37 +00:00
Haohui Mai 469fef2a4f HDFS-9153. Pretty-format the output for DFSIO. Contributed by Kai Zheng. 2015-11-22 16:18:16 -08:00
Sangjin Lee 3214c2331f MAPREDUCE-6540. TestMRTimelineEventHandling fails (sjlee)
(cherry picked from commit 9ad708a0df)
2015-11-11 16:22:28 -08:00
Xuan b505bf9e58 YARN-4313. Race condition in MiniMRYarnCluster when getting history
server address. Contributed by Jian He

(cherry picked from commit 7412ff48ee)
2015-10-29 17:37:24 -07:00
Tsuyoshi Ozawa 3138b43f16 YARN-3573. MiniMRYarnCluster constructor that starts the timeline server using a boolean should be marked deprecated. Contributed by Brahma Reddy Battula.
(cherry picked from commit 96677bef00)
2015-10-27 12:35:20 +09:00
Junping Du 801ce47e28 MAPREDUCE-6508. TestNetworkedJob fails consistently due to delegation token changes on RM. Contributed by Akira AJISAKA
(cherry picked from commit eb6379ca25)
2015-10-23 10:07:41 -07:00
Sangjin Lee 180efe6677 YARN-2556. Tool to measure the performance of the timeline server (Chang Li via sjlee)
(cherry picked from commit 58590fef49)
2015-10-16 16:57:32 -07:00
Jason Lowe 0139517811 YARN-261. Ability to fail AM attempts. Contributed by Andrey Klochkov and Rohith Sharma K S
(cherry picked from commit a0bca2b5ad)
2015-10-09 14:19:41 +00:00
Xuan 53bddc410f YARN-1897. CLI and core support for signal container functionality. Contributed by Ming Ma
(cherry picked from commit 8f08532bde)
2015-10-02 18:53:04 -07:00
Haohui Mai 0971503522 MAPREDUCE-6483. Replace deprecated method NameNode.getUri() with DFSUtilClient.getNNUri() in TestMRCredentials. Contributed by Mingliang Liu. 2015-09-18 13:36:22 -07:00
Haohui Mai ac2ed09be7 MAPREDUCE-6477. Replace usage of deprecated NameNode.DEFAULT_PORT in TestFileSystem. Contributed by Mingliang Liu. 2015-09-16 13:06:38 -07:00
Jason Lowe 52f56524d4 MAPREDUCE-6472. MapReduce AM should have java.io.tmpdir=./tmp to be consistent with tasks. Contributed by Naganarasimha G R
(cherry picked from commit 62943b8e3a)
2015-09-15 19:44:17 +00:00
Andrew Wang 355398e441 HADOOP-12358. Add -safely flag to rm to prompt when deleting many files. Contributed by Xiaoyu Yao.
(cherry picked from commit e1feaf6db0)
2015-09-04 13:48:42 -07:00
Akira Ajisaka a61c0f5ca1 Revert "MAPREDUCE-6363. [NNBench] Lease mismatch error when running with multiple mappers. Contributed by Brahma Reddy Battula."
This reverts commit 98446ee695.
2015-09-02 12:26:25 +09:00