Apache
/
hadoop
1
0
Fork 0
Code Issues Pull Requests Packages Projects Releases Wiki Activity
12,678 Commits 326 Branches 371 Tags 1.5 GiB
Commit Graph

1584 Commits

Author SHA1 Message Date
Robert Kanter 27aabcab45 MAPREDUCE-6686. Add a way to download the job config from the mapred CLI (rkanter) 
(cherry picked from commit 992a49353f)
 2016-05-18 12:16:45 -07:00
Junping Du 7e36b9159e MAPREDUCE-6657. Job history server can fail on startup when NameNode is in start phase. Contributed by Haibo Chen. 
(cherry picked from commit f6ef876fe1)
 2016-05-17 14:42:11 -07:00
Jason Lowe 9c47862d87 MAPREDUCE-6698. Increase timeout on TestUnnecessaryBlockingOnHistoryFileInfo.testTwoThreadsQueryingDifferentJobOfSameUser. Contributed by Haibo Chen 
(cherry picked from commit 34fddd1e91)
 2016-05-17 15:04:02 +00:00
Jason Lowe 51e06c085f MAPREDUCE-6701. application master log can not be available when clicking jobhistory's am logs link. Contributed by Haibo Chen 
(cherry picked from commit 12fa4ec141)
 2016-05-17 14:56:23 +00:00
Akira Ajisaka 19db965c27 MAPREDUCE-6693. ArrayIndexOutOfBoundsException occurs when the length of the job name is equal to mapreduce.jobhistory.jobname.limit. Contributed by Ajith S. 
(cherry picked from commit b357930526)
 2016-05-17 20:22:46 +09:00
Jason Lowe 08743edd70 MAPREDUCE-6558. multibyte delimiters with compressed input files generate duplicate records. Contributed by Wilfred Spiegelenburg 
(cherry picked from commit 9227dfc25f)
 2016-05-13 14:35:34 +00:00
Steve Loughran eab6e633ee MAPREDUCE-6639 Process hangs in LocatedFileStatusFetcher if FileSystem.get throws. Ryan Blue via stevel 2016-05-12 19:00:08 +01:00
Jason Lowe d3dfed3fd0 MAPREDUCE-6684. High contention on scanning of user directory under immediate_done in Job History Server. Contributed by Haibo Chen 
(cherry picked from commit 5ffb54694b)
 2016-05-10 16:06:26 +00:00
Eric Payne 77d147e0ec MAPREDUCE-6678. Allow ShuffleHandler readahead without drop-behind. Contributed by Nathan Roberts. 
(cherry picked from commit cd35b692de)
 2016-05-10 16:04:09 +00:00
Jason Lowe d1aa3e0f89 MAPREDUCE-6689. MapReduce job can infinitely increase number of reducer resource requests. Contributed by Wangda Tan 
(cherry picked from commit c9bb96fa81)
 2016-05-06 22:27:24 +00:00
Vinod Kumar Vavilapalli e9c7a17bc8 MAPREDUCE-6514. Fixed MapReduce ApplicationMaster to properly updated resources ask after ramping down of all reducers avoiding job hangs. Contributed by Varun Saxena and Wangda Tan. 
(cherry picked from commit 8d48266720)
 2016-05-05 19:05:44 -07:00
Robert Kanter f9d6092108 MAPREDUCE-6675. TestJobImpl.testUnusableNode failed (haibochen via rkanter) 
(cherry picked from commit 9d3fcdfbb3)
 2016-05-04 22:42:34 -07:00
Robert Kanter cca8e9b90c MAPREDUCE-6677. LocalContainerAllocator doesn't specify resource of the containers allocated (haibochen via rkanter) 
(cherry picked from commit ecce3b7d53)
 2016-05-04 22:19:42 -07:00
Harsh J 2d7e4a8228 MAPREDUCE-2398. MRBench: setting the baseDir parameter has no effect. Contributed by Michael Noll and Wilfred Spiegelenburg. 
(cherry picked from commit 7da5847cf1)
 2016-04-21 17:54:24 +05:30
Jian He 70a31458e0 MAPREDUCE-6680. JHS UserLogDir scan algorithm sometime could skip directory with update in CloudFS (Azure FileSystem, S3, etc. Contributed by Junping Du 
(cherry picked from commit 1e48eefe58)
 2016-04-20 19:02:58 -07:00
Xuan 94e4d349b0 YARN-4965. Distributed shell AM failed due to ClientHandlerException thrown by jersey. Contributed by Junping Du 
(cherry picked from commit e6c0742012)
 2016-04-16 19:40:11 -07:00
Eric Payne edb096eab7 MAPREDUCE-6649. getFailureInfo not returning any failure info. Contributed by Eric Badger 2016-04-16 22:22:42 +00:00
Wangda Tan ab23acdfda MAPREDUCE-6513. MR job got hanged forever when one NM unstable for some time. (Varun Saxena via wangda) 2016-04-14 11:16:37 -07:00
Eric Payne 17c4f34b3d MAPREDUCE-6633. AM should retry map attempts if the reduce task encounters commpression related errors. Contributed by Rushabh Shah 
(cherry picked from commit 1fec06e037)
 2016-04-09 18:27:03 +00:00
Robert Kanter 3584cbc355 MAPREDUCE-6647. MR usage counters use the resources requested instead of the resources allocated (haibochen via rkanter) 
(cherry picked from commit 3be1ab485f)
 2016-04-06 17:16:10 -07:00
Junping Du 5574f82799 MAPREDUCE-6670. TestJobListCache#testEviction sometimes fails on Windows with timeout. Contributed by Gergely Novák. 
(cherry picked from commit de96d7c88a)
 2016-04-06 08:35:51 -07:00
Akira Ajisaka 94b2ba89a4 MAPREDUCE-6663. [NNBench] Refactor nnbench as a Tool implementation. Contributed by Brahma Reddy Battula. 
(cherry picked from commit 690d8a368d)
 2016-03-30 11:44:33 +09:00
Akira Ajisaka 7b2cf29f4f MAPREDUCE-6543. Migrate MR client test cases part 2. Contributed by Dustin Cote. 
(cherry picked from commit 0050fa5f1c)

Conflicts:
	hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-jobclient/src/test/java/org/apache/hadoop/mapreduce/TestMRJobClient.java
	hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-jobclient/src/test/java/org/apache/hadoop/mapreduce/lib/partition/TestTotalOrderPartitioner.java
 2016-03-29 18:20:01 +09:00
Rohith Sharma K S f22ce96b50 MAPREDUCE-6662. Clear ASF Warnings on test data files. Contributed by Vinayakumar B 
(cherry picked from commit 0ef8bbfd87)
 2016-03-29 10:35:04 +08:00
Junping Du ae29b1a17b MAPREDUCE-6655. Fix a typo (STRICT_IE6) in Encrypted Shuffle. Contributed by Wei-Chiu Chuang. 
(cherry picked from commit 90fcb16c0f)
 2016-03-28 05:31:46 -07:00
Akira Ajisaka 2b1f066d2d Revert "MAPREDUCE-6543. Migrate MR client test cases part 2. Contributed by Dustin Cote." 
This reverts commit 0dc1453626.
 2016-03-28 16:05:55 +09:00
Jason Lowe 8360d98d6a MAPREDUCE-6535. TaskID default constructor results in NPE on toString(). Contributed by Daniel Templeton 
(cherry picked from commit 3f622a143c)
 2016-03-25 20:07:56 +00:00
Akira Ajisaka 0dc1453626 MAPREDUCE-6543. Migrate MR client test cases part 2. Contributed by Dustin Cote. 
(cherry picked from commit 2c268cc936)

Conflicts:
	hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-jobclient/src/test/java/org/apache/hadoop/mapreduce/TestMRJobClient.java
	hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-jobclient/src/test/java/org/apache/hadoop/mapreduce/lib/partition/TestTotalOrderPartitioner.java
 2016-03-25 15:23:23 +09:00
Akira Ajisaka 7a3fd1bc1a MAPREDUCE-6656. [NNBench] OP_DELETE operation isn't working after MAPREDUCE-6363. Contributed by J.Andreina. 
(cherry picked from commit 368c77376a)
 2016-03-23 23:24:30 +09:00
Akira Ajisaka 12c1cc889a MAPREDUCE-6580. Test failure: TestMRJobsWithProfiler. Contributed by Eric Badger. 
(cherry picked from commit 0bfe5a0167)
 2016-03-23 13:19:08 +09:00
Li Lu 8bbb5efbfb MAPREDUCE-6110. JobHistoryServer CLI throws NullPointerException with job ids that do not exist. (Kai Sasaki via gtcarrera9) 
(cherry picked from commit 57ead18a85)
 2016-03-22 14:29:12 -07:00
Karthik Kambatla 95b59c17a7 YARN-4732. *ProcessTree classes have too many whitespace issues 
(cherry picked from commit 7fae4c68e6)
 2016-03-20 10:47:56 -07:00
Wangda Tan abfb2957d1 MAPREDUCE-6579. JobStatus#getFailureInfo should not output diagnostic information when the job is running. (Akira AJISAKA via wangda) 
(cherry picked from commit 6529c87551)
 2016-03-15 17:30:37 -07:00
Akira Ajisaka c2e14d1bbb MAPREDUCE-6520. Migrate MR Client test cases part 1. Contributed by Dustin Cote. 
(cherry picked from commit 247a790609)
 2016-03-11 22:54:40 +09:00
Akira Ajisaka 99f61afcdd Revert "MAPREDUCE-6520. Migrate MR Client test cases part 1." 
This reverts commit 7d665f0a98.
 2016-03-11 22:52:10 +09:00
Akira Ajisaka 7d665f0a98 MAPREDUCE-6520. Migrate MR Client test cases part 1. 
(cherry picked from commit ef68b44195)
 2016-03-11 22:25:29 +09:00
Varun Vasudev e9a0ffc7f1 YARN-4737. Add CSRF filter support in YARN. Contributed by Jonathan Maron. 
(cherry picked from commit 43416187c07afb35e3267f94d0a41d8d3cfb5735)
 2016-03-07 15:23:36 +05:30
Harsh J ed22444a60 MAPREDUCE-6648. Add yarn.app.mapreduce.am.log.level to mapred-default.xml (harsh) 2016-03-07 13:13:05 +05:30
Masatake Iwasaki 722182fdfb HADOOP-12470. In-page TOC of documentation should be automatically generated by doxia macro (iwasakims) 
(cherry picked from commit cbd31328a6)

 Conflicts:
	hadoop-common-project/hadoop-common/src/site/markdown/CommandsManual.md
	hadoop-common-project/hadoop-common/src/site/markdown/UnixShellGuide.md
 2016-03-04 14:12:44 +09:00
Robert Kanter f9f57265cf MAPREDUCE-4785. TestMRApp occasionally fails (haibochen via rkanter) 
(cherry picked from commit ff0ee84d77)
 2016-03-03 16:38:25 -08:00
Robert Kanter 48cb4b9ba7 MAPREDUCE-6622. Add capability to set JHS job cache to a task-based limit (rchiang via rkanter) 
(cherry picked from commit 0f72da7e28)
 2016-02-26 17:57:45 -08:00
Robert Kanter ec48848711 YARN-4701. When task logs are not available, port 8041 is referenced instead of port 8042 (haibochen via rkanter) 
(cherry picked from commit c4d4df8de0)
 2016-02-25 12:48:29 -08:00
Robert Kanter d9f01f28a0 MAPREDUCE-6640. mapred job -history command should be able to take Job ID (rkanter) 
(cherry picked from commit 4343a4cf77)
 2016-02-23 17:38:25 -08:00
Varun Vasudev f1999fe275 MAPREDUCE-6635. Unsafe long to int conversion in UncompressedSplitLineReader and IndexOutOfBoundsException. Contributed by Junping Du. 
(cherry picked from commit c6f2d761d5)
 2016-02-23 13:05:52 +05:30
Tsuyoshi Ozawa 929e869ae0 YARN-4708. Missing default mapper type in TimelineServer performance test tool usage. Contributed by Kai Sasaki. 
(cherry picked from commit b68901d7dd)
 2016-02-22 07:55:35 +09:00
Andrew Wang 7f6737951a MAPREDUCE-6637. Testcase Failure : TestFileInputFormat.testSplitLocationInfo. Contributed by Brahma Reddy Battula. 
(cherry picked from commit 37577852ba)
 2016-02-19 16:33:40 -08:00
Robert Kanter a33109a0b8 MAPREDUCE-6627. Add machine-readable output to mapred job -history command (rkanter) 
(cherry picked from commit 8eee59ce6b)
 2016-02-18 17:58:40 -08:00
Robert Kanter f864faaafa HADOOP-12817. Enable TLS v1.1 and 1.2 (rkanter) 
(cherry picked from commit a365a3941c)
 2016-02-18 11:11:32 -08:00
Varun Vasudev 212c519ad3 MAPREDUCE-6634. Log uncaught exceptions/errors in various thread pools in mapreduce. Contributed by Sidharta Seethana. 
(cherry picked from commit c50d1e54000c76880a041ce5959a2eb23c86bd35)
 2016-02-18 14:19:01 +05:30
Tsuyoshi Ozawa a33cdfb875 MAPREDUCE-6341. addendum patch to fix typos. Contributed by John Michael Luy. 
(cherry picked from commit 453e7e0d10)
 2016-02-17 14:18:04 +09:00
Jason Lowe 303c650d0d MAPREDUCE-6191. Improve clearing stale state of Java serialization 
testcase.  (Sam Liu via Eric Yang)
(cherry picked from commit c379e102dd)

Conflicts:

	hadoop-mapreduce-project/CHANGES.txt
 2016-02-10 16:44:12 +00:00
Akira Ajisaka 15ef3a85d9 Revert "HDFS-9686. Remove useless boxing/unboxing code. Contributed by Kousuke Saruta." 
This reverts commit e4c01b8b1c.
 2016-02-10 23:45:15 +09:00
Akira Ajisaka 9a55d3a7cb MAPREDUCE-6626. Reuse ObjectMapper instance in MapReduce. Contributed by Lin Yiqun. 
(cherry picked from commit 10393d9a666419a5d8dc0cfb48da96e9c690c11e)
 2016-02-10 03:05:16 +09:00
Akira Ajisaka e4c01b8b1c HDFS-9686. Remove useless boxing/unboxing code. Contributed by Kousuke Saruta. 
(cherry picked from commit fe124da5ff)
 2016-02-06 19:32:50 +09:00
Robert Kanter 358c49d830 MAPREDUCE-6620. Jobs that did not start are shown as starting in 1969 in the JHS web UI (haibochen via rkanter) 
(cherry picked from commit ccbba4a4de)
 2016-02-02 15:21:51 -08:00
Jason Lowe 4ee0977687 MAPREDUCE-6621. Memory Leak in JobClient#submitJobInternal(). Contributed by Xuan Gong 
(cherry picked from commit 43e669b22d)
 2016-02-02 19:31:33 +00:00
Wangda Tan 9591363ad8 YARN-4340. Add list API to reservation system. (Sean Po via wangda) 
(cherry picked from commit 9875325d5c)
 2016-02-02 10:19:19 +08:00
Jason Lowe f5f6652137 MAPREDUCE-6618. YarnClientProtocolProvider leaking the YarnClient thread. Contributed by Xuan Gong 
(cherry picked from commit 59a212b6e1)
 2016-02-01 16:06:42 +00:00
Akira Ajisaka 6ce9b1240e MAPREDUCE-6616. Fail to create jobhistory file if there are some multibyte characters in the job name. Contributed by Kousuke Saruta. 
(cherry picked from commit d314a3f99d5d03c62a3b8fb8883886440ab7d1db)
 2016-01-29 16:20:37 +09:00
Robert Kanter a99fa74490 MAPREDUCE-6431. JobClient should be an AutoClosable (haibochen via rkanter) 
(cherry picked from commit dca0dc8ac2)
 2016-01-27 17:11:41 -08:00
Jian He da18bbedaf MAPREDUCE-6619. HADOOP_CLASSPATH is overwritten in MR container. Contributed by Junping Du 2016-01-27 13:16:59 -08:00
Jian He ad42229683 MAPREDUCE-6610. JobHistoryEventHandler should not swallow timeline response. Contributed by Li Lu 2016-01-25 23:01:29 -08:00
Akira Ajisaka a9386cae6e MAPREDUCE-6614. Remove unnecessary code in TestMapreduceConfigFields. Contributed by Kai Sasaki. 2016-01-25 11:39:42 +09:00
Akira Ajisaka 7f215ffdd5 MAPREDUCE-6605. Fix typos mapreduce.map.skip.proc.count.autoincr and mapreduce.reduce.skip.proc.count.autoincr in mapred-default.xml. Contributed by Kai Sasaki. 2016-01-22 18:37:17 +09:00
Karthik Kambatla ef59521094 YARN-4526. Make SystemClock singleton so AppSchedulingInfo could use it. (kasha) 
(cherry picked from commit d40859fab1)

Conflicts:
	hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-app/src/test/java/org/apache/hadoop/mapred/TestTaskAttemptListenerImpl.java
 2016-01-18 10:59:34 +01:00
Jason Lowe 5923f50797 MAPREDUCE-6554. MRAppMaster servicestart failing with NPE in MRAppMaster#parsePreviousJobHistory. Contributed by Bibin A Chundatt 
(cherry picked from commit 9fbd579ab5)
 2016-01-15 16:53:34 +00:00
Akira Ajisaka 7c4be15991 MAPREDUCE-6601. Fix typo in Job#setUseNewAPI. Contributed by Kai Sasaki. 
(cherry picked from commit 5cc44d18aa)
 2016-01-15 00:44:42 +09:00
Akira Ajisaka 6efee8907a MAPREDUCE-6363. [NNBench] Lease mismatch error when running with multiple mappers. Contributed by Vlad Sharanhovich and Bibin A Chundatt. 
(cherry picked from commit 7b0964f354)
 2016-01-14 10:41:22 +09:00
Jason Lowe 0af01092b6 MAPREDUCE-6473. Job submission can take a long time during Cluster initialization. Contributed by Kuhu Shukla 
(cherry picked from commit f657b54281)
 2016-01-12 23:58:01 +00:00
Wangda Tan 15e174fe79 YARN-4582. Label-related invalid resource request exception should be able to properly handled by application. (Bibin A Chundatt via wangda) 
(cherry picked from commit 9e792da014)
 2016-01-12 12:54:09 +08:00
Junping Du 51073c910c MAPREDUCE-6068. Illegal progress value warnings in map tasks. Contributed by Binglin Chang. 
(cherry picked from commit 8c1adeaa26)
 2016-01-11 06:13:53 -08:00
Sangjin Lee 2dfa0613a1 MAPREDUCE-6577. MR AM unable to load native library without MR_AM_ADMIN_USER_ENV set (sjlee) 
(cherry picked from commit f6f16118d3)
 2016-01-05 15:25:47 -08:00
Akira Ajisaka 6d235f0b91 MAPREDUCE-6593. TestJobHistoryEventHandler.testTimelineEventHandling fails on trunk because of NPE. Contributed by Naganarasimha G R. 
(cherry picked from commit 7dafee11d8)
 2016-01-04 16:31:28 +09:00
Akira Ajisaka a8314505d0 MAPREDUCE-6589. TestTaskLog outputs a log under directory other than target/test-dir. (aajisaka) 
(cherry picked from commit 2a59d01503)
 2015-12-30 01:34:15 +09:00
Akira Ajisaka 1a06949126 MAPREDUCE-6584. Remove trailing whitespaces from mapred-default.xml. (aajisaka) 2015-12-30 01:04:11 +09:00
Wangda Tan ca575e6add MAPREDUCE-6574. MR AM should print host of failed tasks. (Mohammad Shahid Khan via wangda) 
(cherry picked from commit 32b3dc11eb)
 2015-12-28 11:04:30 -08:00
Rohith Sharma K S 2d0348c907 MAPREDUCE-6419. JobHistoryServer doesn't sort properly based on Job ID when Job id's exceed 9999. (Mohammad Shahid Khan via rohithsharmaks) 
(cherry picked from commit a308e86621)
 2015-12-24 12:00:37 +05:30
Akira Ajisaka 8607cb6074 MAPREDUCE-6583. Clarify confusing sentence in MapReduce tutorial document. Contributed by Kai Sasaki. 
(cherry picked from commit 7995a6ea4d)
 2015-12-21 00:16:39 +09:00
Jian He 1147b086d9 YARN-4164. Changed updateApplicationPriority API to return the updated application priority. Contributed by Rohith Sharma K S 2015-12-18 14:14:31 -08:00
Xuan 5891d6ed55 MAPREDUCE-6566. Add retry support to mapreduce CLI tool. Contributed by Varun Vasudev 
(cherry picked from commit d4e766de93)
 2015-12-16 15:29:33 -08:00
Vinod Kumar Vavilapalli 74a011d73a Revert "MAPREDUCE-6566. Add retry support to mapreduce CLI tool. Contributed by Varun Vasudev" 
This reverts commit 3b10ff6a6f.
 2015-12-16 15:12:24 -08:00
Zhihai Xu da7bde60c0 MAPREDUCE-6436. JobHistory cache issue. Contributed by Kai Sasaki 
(cherry picked from commit 5b7078d069)
 2015-12-15 01:03:10 -08:00
Xuan 3b10ff6a6f MAPREDUCE-6566. Add retry support to mapreduce CLI tool. Contributed by Varun Vasudev 
(cherry picked from commit fc470840a0)
 2015-12-07 14:15:50 -08:00
Steve Loughran af14458de7 HADOOP-12321. Make JvmPauseMonitor an AbstractService. (Sunil G via Stevel) [includes HDFS-8947 MAPREDUCE-6462 and YARN-4072] 2015-12-06 17:42:56 +00:00
Jian He 23fe1abc02 YARN-3840. Resource Manager web ui issue when sorting application by id (with application having id > 9999). Contributed by Mohammad Shahid Khan and Varun Saxena 
(cherry picked from commit 9f77ccad73)
 2015-12-03 12:49:22 -08:00
Vinod Kumar Vavilapalli b20e77eba3 Preparing for 2.9.0 development: mvn versions:set -DnewVersion=2.9.0 2015-11-25 17:47:22 -08:00
Robert Kanter 8c6273d117 MAPREDUCE-6549. multibyte delimiters with LineRecordReader cause duplicate records (wilfreds via rkanter) 
(cherry picked from commit 7fd00b3db4)
 2015-11-25 17:03:52 -08:00
Junping Du 628c782870 Tests in mapreduce-client-app are writing outside of target. Contributed by Akira AJISAKA. 
(cherry picked from commit 15d577bfbb)
 2015-11-25 09:16:22 -08:00
Akira Ajisaka c6cf7910a4 MAPREDUCE-6553. Replace '\u2b05' with '<-' in rendering job configuration. Contributed by Gabor Liptak. 
(cherry picked from commit 5ec44fc35c)
 2015-11-25 16:33:18 +09:00
Jason Lowe 2f18218508 MAPREDUCE-5870. Support for passing Job priority through Application Submission Context in Mapreduce Side. Contributed by Sunil G 
(cherry picked from commit f634505d48)

Conflicts:

	hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-core/src/test/java/org/apache/hadoop/mapred/TestJobConf.java
 2015-11-24 22:15:37 +00:00
Jason Lowe ef3e01a1e4 MAPREDUCE-5883. "Total megabyte-seconds" in job counters is slightly misleading. Contributed by Nathan Roberts 
(cherry picked from commit cab3c7c889)
 2015-11-24 22:01:53 +00:00
Jian He b339a4b8d6 YARN-4349. Support CallerContext in YARN. Contributed by Wangda Tan 
(cherry picked from commit 8676a118a1)
 2015-11-23 17:20:24 -08:00
Haohui Mai 469fef2a4f HDFS-9153. Pretty-format the output for DFSIO. Contributed by Kai Zheng. 2015-11-22 16:18:16 -08:00
Akira Ajisaka 8de7911eb3 MAPREDUCE-6499. Add elapsed time for retired job in JobHistoryServer WebUI. Contributed by Lin Yiqun. 
(cherry picked from commit a7f776ffdf)
 2015-11-17 15:18:32 +09:00
Jian He 2cf084ac76 Revert "YARN-3840. Resource Manager web ui issue when sorting application by id (with application having id > 9999) Contributed by Mohammad Shahid Khan" 
This reverts commit a4ff03b200.

Conflicts:
	hadoop-yarn-project/CHANGES.txt
 2015-11-16 20:19:31 -08:00
Jian He f562c155d2 MAPREDUCE-5485. Allow repeating job commit by extending OutputCommitter API. Contributed by Junping Du 2015-11-16 17:08:22 -08:00
Sangjin Lee 3214c2331f MAPREDUCE-6540. TestMRTimelineEventHandling fails (sjlee) 
(cherry picked from commit 9ad708a0df)
 2015-11-11 16:22:28 -08:00
Jason Lowe ee5c20ddef MAPREDUCE-6533. testDetermineCacheVisibilities of TestClientDistributedCacheManager is broken. Contributed by Chang Li 
(cherry picked from commit df68eac825)
 2015-11-11 17:27:03 +00:00
Jian He a4ff03b200 YARN-3840. Resource Manager web ui issue when sorting application by id (with application having id > 9999) Contributed by Mohammad Shahid Khan 
(cherry picked from commit 8fbea531d7)
 2015-11-09 11:05:12 -08:00
Tsuyoshi Ozawa 1c0c77e7f0 MAPREDUCE-5763. Warn message about httpshuffle in NM logs. Contributed by Akira AJISAKA. 
(cherry picked from commit 668e897538)
 2015-11-09 15:26:02 +09:00
Jason Lowe a7d8895b60 MAPREDUCE-6528. Memory leak for HistoryFileManager.getJobSummary(). Contributed by Junping Du 
(cherry picked from commit 6344b6a769)
 2015-10-30 15:32:48 +00:00
Tsuyoshi Ozawa 1c80473a41 YARN-4320. TestJobHistoryEventHandler fails as AHS in MiniYarnCluster no longer binds to default port 8188. Contributed by Varun Saxena. 
(cherry picked from commit ce31b22739)
 2015-10-30 18:00:35 +09:00
Xuan b505bf9e58 YARN-4313. Race condition in MiniMRYarnCluster when getting history 
server address. Contributed by Jian He

(cherry picked from commit 7412ff48ee)
 2015-10-29 17:37:24 -07:00
Jason Lowe 145058003a MAPREDUCE-6515. Update Application priority in AM side from AM-RM heartbeat. Contributed by Sunil G 
(cherry picked from commit cf953b6258)
 2015-10-29 18:05:56 +00:00
Tsuyoshi Ozawa 3138b43f16 YARN-3573. MiniMRYarnCluster constructor that starts the timeline server using a boolean should be marked deprecated. Contributed by Brahma Reddy Battula. 
(cherry picked from commit 96677bef00)
 2015-10-27 12:35:20 +09:00
Junping Du 801ce47e28 MAPREDUCE-6508. TestNetworkedJob fails consistently due to delegation token changes on RM. Contributed by Akira AJISAKA 
(cherry picked from commit eb6379ca25)
 2015-10-23 10:07:41 -07:00
Jason Lowe 7c002e2416 MAPREDUCE-6518. Set SO_KEEPALIVE on shuffle connections. Contributed by Chang Li 
(cherry picked from commit ee9b80acf8)
 2015-10-21 21:45:51 +00:00
Jason Lowe be72ed1c75 MAPREDUCE-6489. Fail fast rogue tasks that write too much to local disk. Contributed by Maysam Yabandeh 2015-10-21 14:11:48 +00:00
Robert Kanter 492142097b MAPREDUCE-6495. Docs for archive-logs tool (rkanter) 
(cherry picked from commit 0c4af0f998)
 2015-10-20 17:34:56 -07:00
Sangjin Lee 180efe6677 YARN-2556. Tool to measure the performance of the timeline server (Chang Li via sjlee) 
(cherry picked from commit 58590fef49)
 2015-10-16 16:57:32 -07:00
Haohui Mai ee04add696 HDFS-9167. Update pom.xml in other modules to depend on hdfs-client instead of hdfs. Contributed by Mingliang Liu. 2015-10-13 15:24:29 -07:00
Karthik Kambatla 2d8f9e3fd9 MAPREDUCE-6302. Incorrect headroom can lead to a deadlock between map and reduce allocations. (kasha) 
(cherry picked from commit 4aa9b3e75c)

Conflicts:
	hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-app/src/main/java/org/apache/hadoop/mapreduce/v2/app/rm/RMContainerAllocator.java
 2015-10-09 07:54:47 -07:00
Jason Lowe 0139517811 YARN-261. Ability to fail AM attempts. Contributed by Andrey Klochkov and Rohith Sharma K S 
(cherry picked from commit a0bca2b5ad)
 2015-10-09 14:19:41 +00:00
Akira Ajisaka 22ff37f5ce MAPREDUCE-6479. Add missing mapred job command options in mapreduce document. Contributed by nijel. 
(cherry picked from commit 584cf3bb35)
 2015-10-09 10:58:15 +09:00
Xuan 53bddc410f YARN-1897. CLI and core support for signal container functionality. Contributed by Ming Ma 
(cherry picked from commit 8f08532bde)
 2015-10-02 18:53:04 -07:00
rohithsharmaks 7dd9ebdebc MAPREDUCE-6485. Create a new task attempt with failed map task priority if in-progress attempts are unassigned. (Xianyin Xin via rohithsharmaks) 
(cherry picked from commit 439f43ad3d)
 2015-10-02 20:36:20 +05:30
Akira Ajisaka e42f576232 MAPREDUCE-6497. Fix wrong value of JOB_FINISHED event in JobHistoryEventHandler. Contributed by Shinichi Yamashita. 
(cherry picked from commit 195793c6f3)
 2015-10-01 10:26:00 +02:00
Jason Lowe 3c4c52a095 MAPREDUCE-6492. AsyncDispatcher exit with NPE on TaskAttemptImpl#sendJHStartEventForAssignedFailTask. Contributed by Bibin A Chundatt 
(cherry picked from commit ab11085b81)
 2015-09-28 22:13:03 +00:00
Jian He d458805690 YARN-1651. CapacityScheduler side changes to support container resize. Contributed by Wangda Tan 
(cherry picked from commit 89cab1ba5f)

Conflicts:
	hadoop-tools/hadoop-sls/src/main/java/org/apache/hadoop/yarn/sls/scheduler/SLSCapacityScheduler.java
 2015-09-24 10:30:04 -07:00
Jian He c19e8ffdc7 YARN-1449. AM-NM protocol changes to support container resizing. Contributed by Meng Ding & Wangda Tan) 
(cherry picked from commit 83a18add10)
 2015-09-24 10:23:25 -07:00
Jian He 89dc37450c YARN-3866. AM-RM protocol changes to support container resizing. Contributed by Meng Ding 
(cherry picked from commit dfe2cb849f)
 2015-09-24 10:23:13 -07:00
Zhihai Xu e5ba1a0521 MAPREDUCE-6484. Yarn Client uses local address instead of RM address as token renewer in a secure cluster when RM HA is enabled. Contributed by Zhihai Xu 
(cherry picked from commit 97a08807ec)
 2015-09-24 08:14:11 -07:00
Tsuyoshi Ozawa 52cc2ca488 HADOOP-12428. Fix inconsistency between log-level guards and statements. Contributed by Jagadesh Kiran N and Jackie Chang. 
(cherry picked from commit dfd807afab)
 2015-09-22 12:54:44 +09:00
Zhihai Xu 88a3a07636 MAPREDUCE-6460. TestRMContainerAllocator.testAttemptNotFoundCausesRMCommunicatorException fails. Contributed by Zhihai Xu. 
(cherry picked from commit e3ace31e2b)
 2015-09-19 00:12:15 -07:00
Haohui Mai 0971503522 MAPREDUCE-6483. Replace deprecated method NameNode.getUri() with DFSUtilClient.getNNUri() in TestMRCredentials. Contributed by Mingliang Liu. 2015-09-18 13:36:22 -07:00
Wangda Tan 6041fc75d8 MAPREDUCE-6478. Add an option to skip cleanupJob stage or ignore cleanup failure during commitJob. (Junping Du via wangda) 2015-09-18 10:19:32 -07:00
Jason Lowe f27f1ef44c MAPREDUCE-5982. Task attempts that fail from the ASSIGNED state can disappear. Contributed by Chang Li 
(cherry picked from commit ee4ee6af6a)
 2015-09-17 21:38:44 +00:00
Jason Lowe 4861fadb17 MAPREDUCE-5002. AM could potentially allocate a reduce container to a map attempt. Contributed by Chang Li 
(cherry picked from commit 3f82f582e5)
 2015-09-17 18:18:28 +00:00
Jason Lowe 316296e35f MAPREDUCE-6481. LineRecordReader may give incomplete record and wrong position/key information for uncompressed input sometimes. Contributed by Zhihai Xu 
(cherry picked from commit 58d1a02b8d)
 2015-09-17 14:31:57 +00:00
Haohui Mai ac2ed09be7 MAPREDUCE-6477. Replace usage of deprecated NameNode.DEFAULT_PORT in TestFileSystem. Contributed by Mingliang Liu. 2015-09-16 13:06:38 -07:00
Jason Lowe 52f56524d4 MAPREDUCE-6472. MapReduce AM should have java.io.tmpdir=./tmp to be consistent with tasks. Contributed by Naganarasimha G R 
(cherry picked from commit 62943b8e3a)
 2015-09-15 19:44:17 +00:00
Jason Lowe 7909462c3a MAPREDUCE-6474. ShuffleHandler can possibly exhaust nodemanager file descriptors. Contributed by Kuhu Shukla 
(cherry picked from commit 8e615588d5)
 2015-09-10 16:01:27 +00:00
Tsuyoshi Ozawa 8bf5362014 MAPREDUCE-6442. Stack trace is missing when error occurs in client protocol provider's constructor Contributed by Chang Li. 
(cherry picked from commit 9b685773ec)
 2015-09-05 11:21:35 +09:00
Andrew Wang 355398e441 HADOOP-12358. Add -safely flag to rm to prompt when deleting many files. Contributed by Xiaoyu Yao. 
(cherry picked from commit e1feaf6db0)
 2015-09-04 13:48:42 -07:00
Akira Ajisaka a61c0f5ca1 Revert "MAPREDUCE-6363. [NNBench] Lease mismatch error when running with multiple mappers. Contributed by Brahma Reddy Battula." 
This reverts commit 98446ee695.
 2015-09-02 12:26:25 +09:00
Zhihai Xu 297bda600e MAPREDUCE-6452. NPE when intermediate encrypt enabled for LocalRunner. Contributed by Zhihai Xu 
(cherry picked from commit cbb249534a)
 2015-08-28 12:20:49 -07:00
Jian He 387076894f YARN-4014. Support user cli interface in for Application Priority. Contributed by Rohith Sharma K S 
(cherry picked from commit 57c7ae1aff)
 2015-08-24 20:37:18 -07:00
Akira Ajisaka 29819e7240 MAPREDUCE-6357. MultipleOutputs.write() API should document that output committing is not utilized when input path is absolute. Contributed by Dustin Cote. 
(cherry picked from commit 2ba90c93d7)
 2015-08-21 10:43:05 +09:00
Vinod Kumar Vavilapalli 9da81a3955 MAPREDUCE-6454. Fixed MapReduce to modify HADOOP_CLASSPATH to have distributed cache files so that child processes running hadoop scripts can access these files. Contributed by Junping Du. 2015-08-20 17:35:24 -07:00
Karthik Kambatla 20734320a1 MAPREDUCE-6439. AM may fail instead of retrying if RM shuts down during the allocate call. (Anubhav Dhoot via kasha) 
(cherry picked from commit 8dfec7a197)
 2015-08-15 00:53:13 -07:00
Karthik Kambatla b826168173 MAPREDUCE-5817. Mappers get rescheduled on node transition even after all reducers are completed. (Sangjin Lee via kasha) 
(cherry picked from commit 27d24f96ab)
 2015-08-14 12:40:38 -07:00
Junping Du c8469357ba MAPREDUCE-6443. Add JvmPauseMonitor to JobHistoryServer. Contributed by Robert Kanter. 2015-08-06 07:00:42 -07:00
Colin Patrick Mccabe 07a042aa3c HADOOP-7824. NativeIO.java flags and identifiers must be set correctly for each platform, not hardcoded to their Linux values (Martin Walsh via Colin P. McCabe) 
(cherry picked from commit 21d10ccc6e)
 2015-07-31 15:02:01 -07:00
Jason Lowe 508da4d461 MAPREDUCE-6394. Speed up Task processing loop in HsTasksBlock#render(). Contributed by Ray Chiang 
(cherry picked from commit d0e0ba8010)
 2015-07-31 18:19:52 +00:00
Zhihai Xu 9d40eead81 MAPREDUCE-6433. launchTime may be negative. Contributed by Zhihai Xu 
(cherry picked from commit 93d50b7824)
 2015-07-30 23:12:30 -07:00
Akira Ajisaka 4da8ff60ef MAPREDUCE-5762. Port MAPREDUCE-3223 and MAPREDUCE-4695 (Remove MRv1 config from mapred-default.xml) to branch-2. (aajisaka) 2015-07-16 03:43:49 +09:00
Chris Douglas 2a0e551cd8 MAPREDUCE-6427. Fix typo in JobHistoryEventHandler. Contributed by Ray Chiang 2015-07-14 14:56:21 -07:00
Akira Ajisaka 0ba2defb9e YARN-3381. Fix typo InvalidStateTransitonException. Contributed by Brahma Reddy Battula. 
(cherry picked from commit 19295b36d9)
 2015-07-13 17:53:27 +09:00
Devaraj K 3074d7082e MAPREDUCE-6426. TestShuffleHandler#testGetMapOutputInfo is failing. 
Contributed by zhihai xu.

(cherry picked from commit fffb15bb43)
 2015-07-09 15:02:09 +05:30
Chris Douglas 5fbf91b453 MAPREDUCE-6038. A boolean may be set error in the Word Count v2.0 in MapReduce Tutorial. Contributed by Tsuyoshi Ozawa 
(cherry picked from commit 4990ccfdc742be9c0517557a6256fefcae0df67d)
 2015-07-07 10:26:57 -07:00