Wangda Tan
93b406b66d
YARN-4844. Addendum patch, revert few changes to public and evolving APIs to avoid breaking downstream projects as much as possible.
...
(cherry picked from commit 7eb524ddee
)
2016-06-16 11:10:46 -07:00
Wangda Tan
d838c6443d
YARN-1942. Deprecate toString/fromString methods from ConverterUtils and move them to records classes like ContainerId/ApplicationId, etc. (wangda)
2016-06-14 15:21:41 -07:00
Akira Ajisaka
8295351e54
HADOOP-13270. BZip2CompressionInputStream finds the same compression marker twice in corner case, causing duplicate data blocks. Contributed by Kai Sasaki.
...
(cherry picked from commit e3ba9ad3f1
)
(cherry picked from commit d219550e8b
)
2016-06-14 10:24:53 +09:00
Akira Ajisaka
3bea925cea
MAPREDUCE-6741. Refactor UncompressedSplitLineReader.fillBuffer(). Contributed by Daniel Templeton.
...
(cherry picked from commit 0b7b8a3776
)
(cherry picked from commit db2d96b378
)
2016-06-10 19:17:26 +09:00
Wangda Tan
19e578870d
YARN-4844. Add getMemorySize/getVirtualCoresSize to o.a.h.y.api.records.Resource. (wangda)
2016-06-07 12:41:50 -07:00
Ming Ma
ec4f9a14f9
MAPREDUCE-5044. Have AM trigger jstack on task attempts that timeout before killing them. (Eric Payne and Gera Shegalov via mingma)
...
(cherry picked from commit 4a1cedc010
)
(cherry picked from commit 74e2b5efa2
)
2016-06-06 14:49:43 -07:00
Wangda Tan
2f3e1d965d
Revert "YARn-4844. Add getMemoryLong/getVirtualCoreLong to o.a.h.y.api.records.Resource. Contributed by Wangda Tan."
...
This reverts commit 457884737f
.
2016-05-31 22:16:53 -07:00
Varun Vasudev
457884737f
YARn-4844. Add getMemoryLong/getVirtualCoreLong to o.a.h.y.api.records.Resource. Contributed by Wangda Tan.
2016-05-29 20:57:56 +05:30
Carlo Curino
7982933c09
YARN-4957. Add getNewReservation in ApplicationClientProtocol (Sean Po via curino)
...
(cherry picked from commit 013532a95e
)
(cherry picked from commit c656977961
)
2016-05-25 17:02:22 -07:00
Akira Ajisaka
4422beab72
MAPREDUCE-6607. Enable regex pattern matching when mapreduce.task.files.preserve.filepattern is set. Contributed by Kai Sasaki.
...
(cherry picked from commit a59f30272d
)
2016-05-23 07:26:14 +09:00
Rohith Sharma K S
726c1f14b8
YARN-3840. Resource Manager web ui issue when sorting application by id (with application having id > 9999). Contributed by Mohammad Shahid Khan and Varun Saxena
2016-05-19 10:50:32 +05:30
Jason Lowe
c983115e4c
MAPREDUCE-6698. Increase timeout on TestUnnecessaryBlockingOnHistoryFileInfo.testTwoThreadsQueryingDifferentJobOfSameUser. Contributed by Haibo Chen
...
(cherry picked from commit 34fddd1e91
)
2016-05-17 15:04:33 +00:00
Akira Ajisaka
fc221ecf3d
MAPREDUCE-6693. ArrayIndexOutOfBoundsException occurs when the length of the job name is equal to mapreduce.jobhistory.jobname.limit. Contributed by Ajith S.
...
(cherry picked from commit b357930526
)
(cherry picked from commit 19db965c27
)
2016-05-17 20:23:00 +09:00
Jason Lowe
ff8caccc87
MAPREDUCE-6558. multibyte delimiters with compressed input files generate duplicate records. Contributed by Wilfred Spiegelenburg
...
(cherry picked from commit 9227dfc25f
)
2016-05-13 14:36:13 +00:00
Steve Loughran
9b8f7a8fb2
MAPREDUCE-6639 Process hangs in LocatedFileStatusFetcher if FileSystem.get throws. Ryan Blue via stevel
2016-05-12 18:59:56 +01:00
Jason Lowe
da48923d3a
MAPREDUCE-6684. High contention on scanning of user directory under immediate_done in Job History Server. Contributed by Haibo Chen
...
(cherry picked from commit 5ffb54694b
)
Conflicts:
hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-hs/src/main/java/org/apache/hadoop/mapreduce/v2/hs/HistoryFileManager.java
2016-05-10 16:16:32 +00:00
Eric Payne
730ce76160
MAPREDUCE-6678. Allow ShuffleHandler readahead without drop-behind. Contributed by Nathan Roberts.
...
(cherry picked from commit cd35b692de
)
2016-05-10 16:07:16 +00:00
Jason Lowe
7fc44311d0
MAPREDUCE-6689. MapReduce job can infinitely increase number of reducer resource requests. Contributed by Wangda Tan
...
(cherry picked from commit c9bb96fa81
)
2016-05-06 22:28:03 +00:00
Vinod Kumar Vavilapalli
921562720e
MAPREDUCE-6514. Fixed MapReduce ApplicationMaster to properly updated resources ask after ramping down of all reducers avoiding job hangs. Contributed by Varun Saxena and Wangda Tan.
...
(cherry picked from commit 8d48266720
)
2016-05-05 20:43:55 -07:00
Jian He
0b008dc21c
MAPREDUCE-6680. JHS UserLogDir scan algorithm sometime could skip directory with update in CloudFS (Azure FileSystem, S3, etc. Contributed by Junping Du
...
(cherry picked from commit 1e48eefe58
)
2016-04-20 19:03:19 -07:00
Xuan
84ab382939
YARN-4965. Distributed shell AM failed due to ClientHandlerException thrown by jersey. Contributed by Junping Du
...
(cherry picked from commit e6c0742012
)
(cherry picked from commit 94e4d349b0
)
2016-04-16 19:40:46 -07:00
Eric Payne
1e33c05a7e
MAPREDUCE-6649. getFailureInfo not returning any failure info. Contributed by Eric Badger
2016-04-16 22:37:00 +00:00
Wangda Tan
c202ad6721
MAPREDUCE-6513. MR job got hanged forever when one NM unstable for some time. (Varun Saxena via wangda)
2016-04-14 17:19:26 -07:00
Robert Kanter
8da0a490c6
MAPREDUCE-4785. TestMRApp occasionally fails (haibochen via rkanter)
...
(cherry picked from commit ff0ee84d77
)
(cherry picked from commit f9f57265cf
)
Conflicts:
hadoop-mapreduce-project/CHANGES.txt
2016-04-14 14:54:42 -07:00
Junping Du
80c151b889
MAPREDUCE-6670. TestJobListCache#testEviction sometimes fails on Windows with timeout. Contributed by Gergely Novák.
...
(cherry picked from commit de96d7c88a
)
(cherry picked from commit 5574f82799
)
2016-04-13 08:58:24 -07:00
Eric Payne
3de0ff030c
MAPREDUCE-6633. AM should retry map attempts if the reduce task encounters commpression related errors. Contributed by Rushabh Shah
...
(cherry picked from commit 1fec06e037
)
2016-04-09 18:29:44 +00:00
Akira Ajisaka
73d51f5c47
MAPREDUCE-6663. [NNBench] Refactor nnbench as a Tool implementation. Contributed by Brahma Reddy Battula.
...
(cherry picked from commit 690d8a368d
)
(cherry picked from commit 94b2ba89a4
)
2016-03-30 11:44:49 +09:00
Akira Ajisaka
68cd8112e7
MAPREDUCE-6543. Migrate MR client test cases part 2. Contributed by Dustin Cote.
...
(cherry picked from commit 0050fa5f1c
)
Conflicts:
hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-jobclient/src/test/java/org/apache/hadoop/mapreduce/TestMRJobClient.java
hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-jobclient/src/test/java/org/apache/hadoop/mapreduce/lib/partition/TestTotalOrderPartitioner.java
(cherry picked from commit 7b2cf29f4f
)
Conflicts:
hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-jobclient/src/test/java/org/apache/hadoop/mapreduce/TestMRJobClient.java
2016-03-29 18:24:22 +09:00
Akira Ajisaka
0286ac2576
Revert "MAPREDUCE-6543. Migrate MR client test cases part 2. Contributed by Dustin Cote."
...
This reverts commit 416ea158c2
.
2016-03-28 16:06:10 +09:00
Akira Ajisaka
416ea158c2
MAPREDUCE-6543. Migrate MR client test cases part 2. Contributed by Dustin Cote.
...
(cherry picked from commit 2c268cc936
)
Conflicts:
hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-jobclient/src/test/java/org/apache/hadoop/mapreduce/TestMRJobClient.java
hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-jobclient/src/test/java/org/apache/hadoop/mapreduce/lib/partition/TestTotalOrderPartitioner.java
(cherry picked from commit 0dc1453626
)
Conflicts:
hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-jobclient/src/test/java/org/apache/hadoop/mapreduce/TestMRJobClient.java
2016-03-25 15:33:01 +09:00
Akira Ajisaka
f362b8a403
MAPREDUCE-6656. [NNBench] OP_DELETE operation isn't working after MAPREDUCE-6363. Contributed by J.Andreina.
...
(cherry picked from commit 368c77376a
)
(cherry picked from commit 7a3fd1bc1a
)
2016-03-23 23:24:40 +09:00
Akira Ajisaka
9d7c88f275
MAPREDUCE-6580. Test failure: TestMRJobsWithProfiler. Contributed by Eric Badger.
...
(cherry picked from commit 0bfe5a0167
)
(cherry picked from commit 12c1cc889a
)
2016-03-23 13:19:26 +09:00
Wangda Tan
2d9f42d685
MAPREDUCE-6579. JobStatus#getFailureInfo should not output diagnostic information when the job is running. (Akira AJISAKA via wangda)
...
(cherry picked from commit 6529c87551
)
2016-03-15 17:32:32 -07:00
Akira Ajisaka
cb97d8fa63
MAPREDUCE-6520. Migrate MR Client test cases part 1. Contributed by Dustin Cote.
...
(cherry picked from commit 247a790609
)
(cherry picked from commit c2e14d1bbb
)
2016-03-11 22:55:16 +09:00
Akira Ajisaka
097a03fb6d
Revert "MAPREDUCE-6520. Migrate MR Client test cases part 1."
...
This reverts commit 70e232f36e
.
2016-03-11 22:55:01 +09:00
Akira Ajisaka
70e232f36e
MAPREDUCE-6520. Migrate MR Client test cases part 1.
...
(cherry picked from commit ef68b44195
)
(cherry picked from commit 7d665f0a98
)
2016-03-11 22:26:03 +09:00
Harsh J
5f059e03f5
MAPREDUCE-6648. Add yarn.app.mapreduce.am.log.level to mapred-default.xml (harsh)
2016-03-07 13:13:43 +05:30
Robert Kanter
3d8ae9c577
MAPREDUCE-6622. Add capability to set JHS job cache to a task-based limit (rchiang via rkanter)
...
(cherry picked from commit 0f72da7e28
)
Conflicts:
hadoop-mapreduce-project/CHANGES.txt
2016-03-06 19:26:12 -08:00
Masatake Iwasaki
9d551a3adf
HADOOP-12470. In-page TOC of documentation should be automatically generated by doxia macro (iwasakims)
...
(cherry picked from commit cbd31328a6
)
Conflicts:
hadoop-common-project/hadoop-common/src/site/markdown/CommandsManual.md
hadoop-common-project/hadoop-common/src/site/markdown/UnixShellGuide.md
(cherry picked from commit 722182fdfb
)
2016-03-04 14:13:30 +09:00
Varun Vasudev
cb4f5ebd3e
MAPREDUCE-6635. Unsafe long to int conversion in UncompressedSplitLineReader and IndexOutOfBoundsException. Contributed by Junping Du.
...
(cherry picked from commit c6f2d761d5
)
(cherry picked from commit f1999fe275
)
2016-02-23 13:07:40 +05:30
Andrew Wang
d31660ef4a
MAPREDUCE-6637. Testcase Failure : TestFileInputFormat.testSplitLocationInfo. Contributed by Brahma Reddy Battula.
...
(cherry picked from commit 37577852ba
)
(cherry picked from commit 7f6737951a
)
2016-02-19 16:33:44 -08:00
Tsuyoshi Ozawa
7ccb0e0601
MAPREDUCE-6341. addendum patch to fix typos. Contributed by John Michael Luy.
...
(cherry picked from commit 453e7e0d10
)
2016-02-17 14:18:24 +09:00
Jason Lowe
83985eed2a
MAPREDUCE-6191. Improve clearing stale state of Java serialization
...
testcase. (Sam Liu via Eric Yang)
(cherry picked from commit c379e102dd
)
Conflicts:
hadoop-mapreduce-project/CHANGES.txt
(cherry picked from commit 303c650d0d
)
2016-02-10 16:46:58 +00:00
Akira Ajisaka
321da0604a
Revert "HDFS-9686. Remove useless boxing/unboxing code. Contributed by Kousuke Saruta."
...
This reverts commit 3f8a30c956
.
2016-02-10 23:45:44 +09:00
Akira Ajisaka
f943bf0e22
MAPREDUCE-6626. Reuse ObjectMapper instance in MapReduce. Contributed by Lin Yiqun.
...
(cherry picked from commit 10393d9a666419a5d8dc0cfb48da96e9c690c11e)
(cherry picked from commit 8cbe65a07dd0deb150e91b13e3a1cec13e3716ee)
2016-02-10 03:05:23 +09:00
Akira Ajisaka
3f8a30c956
HDFS-9686. Remove useless boxing/unboxing code. Contributed by Kousuke Saruta.
...
(cherry picked from commit fe124da5ff
)
(cherry picked from commit e4c01b8b1c
)
2016-02-06 19:33:11 +09:00
Jason Lowe
1d322ca5cc
MAPREDUCE-6621. Memory Leak in JobClient#submitJobInternal(). Contributed by Xuan Gong
...
(cherry picked from commit 43e669b22d
)
2016-02-02 19:32:04 +00:00
Wangda Tan
c487453b91
YARN-4340. Add list API to reservation system. (Sean Po via wangda)
...
(cherry picked from commit 9875325d5c
)
2016-02-02 10:23:25 +08:00
Jason Lowe
7bb48ed162
MAPREDUCE-6618. YarnClientProtocolProvider leaking the YarnClient thread. Contributed by Xuan Gong
...
(cherry picked from commit 59a212b6e1
)
2016-02-01 16:11:20 +00:00
Akira Ajisaka
ce5947916d
MAPREDUCE-6616. Fail to create jobhistory file if there are some multibyte characters in the job name. Contributed by Kousuke Saruta.
...
(cherry picked from commit d314a3f99d5d03c62a3b8fb8883886440ab7d1db)
(cherry picked from commit 79cde21064a1fe274a4dcbf3dec4ed0d743876cf)
2016-01-29 16:20:45 +09:00