Apache
/
hadoop
1
0
Fork 0
Code Issues Pull Requests Packages Projects Releases Wiki Activity
11,821 Commits 326 Branches 371 Tags 1.5 GiB
Commit Graph

1495 Commits

Author SHA1 Message Date
Akira Ajisaka 61128c50de MAPREDUCE-6795. Update the document for JobConf#setNumReduceTasks. Contributed by Yiqun Lin. 
(cherry picked from commit 235f39b7c4)
(cherry picked from commit a3897f77ae)
 2016-11-01 20:52:52 +09:00
Akira Ajisaka 757b38ea43 MAPREDUCE-6799. Document mapreduce.jobhistory.webapp.https.address in mapred-default.xml. Contributed by Yiqun Lin. 
(cherry picked from commit f646fe34be)
(cherry picked from commit ef13a5ef9d)
 2016-10-31 15:19:37 +09:00
Jason Lowe 0a405c4f71 MAPREDUCE-6724. Single shuffle to memory must not exceed Integer#MAX_VALUE. (Haibo Chen via gera) 
(cherry picked from commit 6890d5b472)

Conflicts:

	hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-core/src/test/java/org/apache/hadoop/mapreduce/task/reduce/TestMergeManager.java
 2016-10-28 14:40:28 +00:00
Naganarasimha 8a664eba7d MAPREDUCE-2631. Potential resource leaks in BinaryProtocol$TeeOutputStream.java. Contributed by Sunil G. 
(cherry picked from commit 28660f51af)
 2016-10-28 06:04:03 +05:30
Naganarasimha 27029393b7 MAPREDUCE-6541. Exclude scheduled reducer memory when calculating available mapper slots from headroom to avoid deadlock. Contributed by Varun Saxena 
(cherry picked from commit 060558c6f2)
 2016-10-27 18:06:34 +05:30
Karthik Kambatla c955a59d63 MAPREDUCE-6740. Enforce mapreduce.task.timeout to be at least mapreduce.task.progress-report.interval. (Haibo Chen via kasha) 2016-10-05 23:12:45 -07:00
Jason Lowe f1b74a3d9f MAPREDUCE-6741. Add MR support to redact job conf properties. Contributed by Haibo Chen 2016-10-05 15:24:10 +00:00
Jason Lowe db93a64eca MAPREDUCE-6771. RMContainerAllocator sends container diagnostics event after corresponding completion event. Contributed by Haibo Chen 
(cherry picked from commit c52ad9ee86)
 2016-09-29 15:51:36 +00:00
Wangda Tan 69f91d8c48 HADOOP-13544. JDiff reports unncessarily show unannotated APIs and cause confusion while our javadocs only show annotated and public APIs. (vinodkv via wangda) 
(cherry picked from commit 875062b5bc)
(cherry picked from commit 269401dc83)
 2016-09-27 11:46:49 -07:00
Naganarasimha fa2025316d YARN-3692. Allow REST API to set a user generated message when killing an application. Contributed by Rohith Sharma K S 2016-09-24 22:42:28 +05:30
Arun Suresh 979b29a03c YARN-5221. Expose UpdateResourceRequest API to allow AM to request for change in container properties. (asuresh) 
(cherry picked from commit d6d9cff21b)
(cherry picked from commit b279f42d79)
 2016-08-31 20:06:49 -07:00
Jason Lowe 9df118e5fe MAPREDUCE-4784. TestRecovery occasionally fails. Contributed by Haibo Chen 
(cherry picked from commit af508605a9)

Conflicts:

	hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-app/src/test/java/org/apache/hadoop/mapreduce/v2/app/TestRecovery.java
(cherry picked from commit f70f522e80)
 2016-08-30 14:09:45 +00:00
Jason Lowe f6578999d1 MAPREDUCE-6768. TestRecovery.testSpeculative failed with NPE. Contributed by Haibo Chen 
(cherry picked from commit 44d0af2611)
 2016-08-29 19:58:23 +00:00
Jason Lowe 00c8ab91ba MAPREDUCE-6761. Regression when handling providers - invalid configuration ServiceConfiguration causes Cluster initialization failure. Contributed by Peter Vary 
(cherry picked from commit 6fa9bf4407)
 2016-08-24 14:42:53 +00:00
Akira Ajisaka 854af9cee1 MAPREDUCE-6587. Remove unused params in connection-related methods of Fetcher. Contributed by Yiqun Lin. 
(cherry picked from commit 8cc4a67059)
(cherry picked from commit 41fe85f868)
 2016-08-23 17:12:32 +09:00
Varun Saxena e714d82196 MAPREDUCE-6750. Fix TestHSAdminServer#testRefreshSuperUserGroups (Kihwal Lee via Varun Saxena) 2016-08-22 11:05:01 -05:00
Wangda Tan e2bd490236 MAPREDUCE-6310. Add jdiff support to MapReduce. (Li Lu/vinodkv via wangda) 
(cherry picked from commit 3d937457ee)
(cherry picked from commit 65822e55fd)
 2016-08-19 16:49:54 -07:00
Varun Saxena 6f1a9f1b6a MAPREDUCE-6763. Shuffle server listen queue is too small (Jason Lowe via Varun Saxena) 2016-08-20 04:08:55 +05:30
Akira Ajisaka 59693ebbd5 MAPREDUCE-6730. Use StandardCharsets instead of String overload in TextOutputFormat. Contributed by Sahil Kang. 
This closes #114

(cherry picked from commit 70c2781152)
(cherry picked from commit e54de94e5a)
 2016-08-04 20:18:37 +09:00
Akira Ajisaka fa0b6ff97b MAPREDUCE-6729. Accurately compute the test execute time in DFSIO. Contributed by mingleizhang. 
This closes #112

(cherry picked from commit ce93595d7a)
(cherry picked from commit 96d5607e7a)
 2016-07-30 22:16:43 +09:00
Akira Ajisaka 19062df92f MAPREDUCE-6746. Replace org.apache.commons.io.Charsets with java.nio.charset.StandardCharsets. Contributed by Vincent Poon. 
(cherry picked from commit 3f0bffea2d)
(cherry picked from commit 0fc838540b)
 2016-07-30 11:47:33 +09:00
Jason Lowe e098a732ef MAPREDUCE-6744. Increase timeout on TestDFSIO tests. Contributed by Eric Badger 
(cherry picked from commit 703fdf86c6)
 2016-07-25 21:56:33 +00:00
Akira Ajisaka 3dcbd2df56 MAPREDUCE-6738. TestJobListCache.testAddExisting failed intermittently in slow VM testbed. Contributed by Junping Du. 
(cherry picked from commit ecff7d0359)
(cherry picked from commit f4d94f2246)
 2016-07-21 11:38:26 -07:00
Robert Kanter ba83b62d36 MAPREDUCE-6625. TestCLI#testGetJob fails occasionally (haibochen via rkanter) 
(cherry picked from commit ad256beb50)
 2016-07-12 16:31:24 -07:00
Jian He 23eb3c7ceb YARN-5270. Solve miscellaneous issues caused by YARN-4844. Contributed by Wangda Tan 2016-07-11 22:38:35 -07:00
Akira Ajisaka 5522a46f4c HADOOP-13320. Fix arguments check in documentation for WordCount v2.0. Contributed by niccolo becchi. 
(cherry picked from commit 9d46a49c74)
(cherry picked from commit b3b81b4f0e)
 2016-07-07 15:32:20 -07:00
Akira Ajisaka a0f6109907 MAPREDUCE-6542. HistoryViewer uses SimpleDateFormat, but SimpleDateFormat is not threadsafe. Contributed by zhangyubiao. 2016-06-27 15:19:21 -07:00
Akira Ajisaka ae5dc10ecd MAPREDUCE-6725. Javadoc for CLI#listEvents() contains no-existent param. Contributed by shenyinjie. 
(cherry picked from commit 0f23cd497b)
(cherry picked from commit 2830ee5f0f)
 2016-06-25 00:27:35 +09:00
Wangda Tan 93b406b66d YARN-4844. Addendum patch, revert few changes to public and evolving APIs to avoid breaking downstream projects as much as possible. 
(cherry picked from commit 7eb524ddee)
 2016-06-16 11:10:46 -07:00
Wangda Tan d838c6443d YARN-1942. Deprecate toString/fromString methods from ConverterUtils and move them to records classes like ContainerId/ApplicationId, etc. (wangda) 2016-06-14 15:21:41 -07:00
Akira Ajisaka 8295351e54 HADOOP-13270. BZip2CompressionInputStream finds the same compression marker twice in corner case, causing duplicate data blocks. Contributed by Kai Sasaki. 
(cherry picked from commit e3ba9ad3f1)
(cherry picked from commit d219550e8b)
 2016-06-14 10:24:53 +09:00
Akira Ajisaka 3bea925cea MAPREDUCE-6741. Refactor UncompressedSplitLineReader.fillBuffer(). Contributed by Daniel Templeton. 
(cherry picked from commit 0b7b8a3776)
(cherry picked from commit db2d96b378)
 2016-06-10 19:17:26 +09:00
Wangda Tan 19e578870d YARN-4844. Add getMemorySize/getVirtualCoresSize to o.a.h.y.api.records.Resource. (wangda) 2016-06-07 12:41:50 -07:00
Ming Ma ec4f9a14f9 MAPREDUCE-5044. Have AM trigger jstack on task attempts that timeout before killing them. (Eric Payne and Gera Shegalov via mingma) 
(cherry picked from commit 4a1cedc010)
(cherry picked from commit 74e2b5efa2)
 2016-06-06 14:49:43 -07:00
Wangda Tan 2f3e1d965d Revert "YARn-4844. Add getMemoryLong/getVirtualCoreLong to o.a.h.y.api.records.Resource. Contributed by Wangda Tan." 
This reverts commit 457884737f.
 2016-05-31 22:16:53 -07:00
Varun Vasudev 457884737f YARn-4844. Add getMemoryLong/getVirtualCoreLong to o.a.h.y.api.records.Resource. Contributed by Wangda Tan. 2016-05-29 20:57:56 +05:30
Carlo Curino 7982933c09 YARN-4957. Add getNewReservation in ApplicationClientProtocol (Sean Po via curino) 
(cherry picked from commit 013532a95e)
(cherry picked from commit c656977961)
 2016-05-25 17:02:22 -07:00
Akira Ajisaka 4422beab72 MAPREDUCE-6607. Enable regex pattern matching when mapreduce.task.files.preserve.filepattern is set. Contributed by Kai Sasaki. 
(cherry picked from commit a59f30272d)
 2016-05-23 07:26:14 +09:00
Rohith Sharma K S 726c1f14b8 YARN-3840. Resource Manager web ui issue when sorting application by id (with application having id > 9999). Contributed by Mohammad Shahid Khan and Varun Saxena 2016-05-19 10:50:32 +05:30
Jason Lowe c983115e4c MAPREDUCE-6698. Increase timeout on TestUnnecessaryBlockingOnHistoryFileInfo.testTwoThreadsQueryingDifferentJobOfSameUser. Contributed by Haibo Chen 
(cherry picked from commit 34fddd1e91)
 2016-05-17 15:04:33 +00:00
Akira Ajisaka fc221ecf3d MAPREDUCE-6693. ArrayIndexOutOfBoundsException occurs when the length of the job name is equal to mapreduce.jobhistory.jobname.limit. Contributed by Ajith S. 
(cherry picked from commit b357930526)
(cherry picked from commit 19db965c27)
 2016-05-17 20:23:00 +09:00
Jason Lowe ff8caccc87 MAPREDUCE-6558. multibyte delimiters with compressed input files generate duplicate records. Contributed by Wilfred Spiegelenburg 
(cherry picked from commit 9227dfc25f)
 2016-05-13 14:36:13 +00:00
Steve Loughran 9b8f7a8fb2 MAPREDUCE-6639 Process hangs in LocatedFileStatusFetcher if FileSystem.get throws. Ryan Blue via stevel 2016-05-12 18:59:56 +01:00
Jason Lowe da48923d3a MAPREDUCE-6684. High contention on scanning of user directory under immediate_done in Job History Server. Contributed by Haibo Chen 
(cherry picked from commit 5ffb54694b)

Conflicts:

	hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-hs/src/main/java/org/apache/hadoop/mapreduce/v2/hs/HistoryFileManager.java
 2016-05-10 16:16:32 +00:00
Eric Payne 730ce76160 MAPREDUCE-6678. Allow ShuffleHandler readahead without drop-behind. Contributed by Nathan Roberts. 
(cherry picked from commit cd35b692de)
 2016-05-10 16:07:16 +00:00
Jason Lowe 7fc44311d0 MAPREDUCE-6689. MapReduce job can infinitely increase number of reducer resource requests. Contributed by Wangda Tan 
(cherry picked from commit c9bb96fa81)
 2016-05-06 22:28:03 +00:00
Vinod Kumar Vavilapalli 921562720e MAPREDUCE-6514. Fixed MapReduce ApplicationMaster to properly updated resources ask after ramping down of all reducers avoiding job hangs. Contributed by Varun Saxena and Wangda Tan. 
(cherry picked from commit 8d48266720)
 2016-05-05 20:43:55 -07:00
Jian He 0b008dc21c MAPREDUCE-6680. JHS UserLogDir scan algorithm sometime could skip directory with update in CloudFS (Azure FileSystem, S3, etc. Contributed by Junping Du 
(cherry picked from commit 1e48eefe58)
 2016-04-20 19:03:19 -07:00
Xuan 84ab382939 YARN-4965. Distributed shell AM failed due to ClientHandlerException thrown by jersey. Contributed by Junping Du 
(cherry picked from commit e6c0742012)
(cherry picked from commit 94e4d349b0)
 2016-04-16 19:40:46 -07:00
Eric Payne 1e33c05a7e MAPREDUCE-6649. getFailureInfo not returning any failure info. Contributed by Eric Badger 2016-04-16 22:37:00 +00:00