Apache
/
hadoop
1
0
Fork 0
Code Issues Pull Requests Packages Projects Releases Wiki Activity
9,255 Commits 326 Branches 371 Tags 1.5 GiB
Commit Graph

1306 Commits

Author SHA1 Message Date
Vinod Kumar Vavilapalli 9f51a3f430 MAPREDUCE-6514. Fixed MapReduce ApplicationMaster to properly updated resources ask after ramping down of all reducers avoiding job hangs. Contributed by Varun Saxena and Wangda Tan. 
Updated the 2.7 branch patch to fix a minor issue in TestRMContainerAllocator.java

(cherry picked from commit 8d48266720)
 2016-05-05 19:39:25 -07:00
Jian He f1e4f4d83a MAPREDUCE-6680. JHS UserLogDir scan algorithm sometime could skip directory with update in CloudFS (Azure FileSystem, S3, etc. Contributed by Junping Du 
(cherry picked from commit 1e48eefe58)
 2016-04-20 19:03:53 -07:00
Karthik Kambatla cc6ae6fa7e MAPREDUCE-5817. Mappers get rescheduled on node transition even after all reducers are completed. (Sangjin Lee via kasha) 
(cherry picked from commit 27d24f96ab)

(cherry picked from commit b826168173)

Conflicts:
	hadoop-mapreduce-project/CHANGES.txt
 2016-04-18 21:58:53 -07:00
Eric Payne 2ec5bb0fe8 MAPREDUCE-6633. AM should retry map attempts if the reduce task encounters commpression related errors. Contributed by Rushabh Shah 
(cherry picked from commit 1fec06e037)
 2016-04-18 16:12:53 +00:00
Robert Kanter c9ba6fa9ba MAPREDUCE-4785. TestMRApp occasionally fails (haibochen via rkanter) 
(cherry picked from commit ff0ee84d77)
(cherry picked from commit f9f57265cf)

Conflicts:
	hadoop-mapreduce-project/CHANGES.txt
 2016-04-14 14:58:45 -07:00
Junping Du 683f313c1f MAPREDUCE-6670. TestJobListCache#testEviction sometimes fails on Windows with timeout. Contributed by Gergely Novák. 
(cherry picked from commit de96d7c88a)
(cherry picked from commit 5574f82799)
 2016-04-13 08:59:22 -07:00
Wangda Tan 801c95c1cc MAPREDUCE-6302. Preempt reducers after a configurable timeout irrespective of headroom. (kasha via wangda) 2016-04-08 14:35:28 -07:00
Akira Ajisaka a112c0138a MAPREDUCE-6656. [NNBench] OP_DELETE operation isn't working after MAPREDUCE-6363. Contributed by J.Andreina. 
(cherry picked from commit 368c77376a)
(cherry picked from commit 7a3fd1bc1a)
 2016-03-23 23:26:33 +09:00
Akira Ajisaka 04cc99152f MAPREDUCE-6580. Test failure: TestMRJobsWithProfiler. Contributed by Eric Badger. 
(cherry picked from commit 0bfe5a0167)
(cherry picked from commit 12c1cc889a)
 2016-03-23 13:23:29 +09:00
Robert Kanter 66e1e92ffd MAPREDUCE-6622. Add capability to set JHS job cache to a task-based limit (rchiang via rkanter) 
(cherry picked from commit 0f72da7e28)

Conflicts:
	hadoop-mapreduce-project/CHANGES.txt
 2016-03-06 18:34:46 -08:00
Varun Vasudev 0edc764184 MAPREDUCE-6635. Unsafe long to int conversion in UncompressedSplitLineReader and IndexOutOfBoundsException. Contributed by Junping Du. 
(cherry picked from commit c6f2d761d5)
(cherry picked from commit f1999fe275)

 Conflicts:
	hadoop-mapreduce-project/CHANGES.txt
 2016-02-23 14:45:07 +05:30
Andrew Wang 403934c011 MAPREDUCE-6637. Testcase Failure : TestFileInputFormat.testSplitLocationInfo. Contributed by Brahma Reddy Battula. 
(cherry picked from commit 37577852ba)
(cherry picked from commit 7f6737951a)
 2016-02-19 16:37:28 -08:00
Jason Lowe 717b1a1f99 MAPREDUCE-6460. TestRMContainerAllocator.testAttemptNotFoundCausesRMCommunicatorException fails. Contributed by Zhihai Xu. 
(cherry picked from commit e3ace31e2b)

Conflicts:

	hadoop-mapreduce-project/CHANGES.txt
 2016-02-10 16:58:57 +00:00
Jason Lowe 9d574368b5 MAPREDUCE-6191. Improve clearing stale state of Java serialization 
testcase.  (Sam Liu via Eric Yang)
(cherry picked from commit c379e102dd)

Conflicts:

	hadoop-mapreduce-project/CHANGES.txt
(cherry picked from commit 303c650d0d)
 2016-02-10 16:48:06 +00:00
Jason Lowe 6257ed8431 MAPREDUCE-6413. TestLocalJobSubmission is failing with unknown host. Contributed by zhihai xu 
(cherry picked from commit aa5b15b03b)

Conflicts:

	hadoop-mapreduce-project/CHANGES.txt
 2016-02-09 16:33:07 +00:00
Jason Lowe 7f6d150c9a MAPREDUCE-6621. Memory Leak in JobClient#submitJobInternal(). Contributed by Xuan Gong 
(cherry picked from commit 43e669b22d)
 2016-02-02 19:32:37 +00:00
Jason Lowe 69c61fae0c MAPREDUCE-6618. YarnClientProtocolProvider leaking the YarnClient thread. Contributed by Xuan Gong 
(cherry picked from commit 59a212b6e1)

Conflicts:

	hadoop-mapreduce-project/CHANGES.txt
 2016-02-01 16:10:56 +00:00
Jian He a1b75ff8c5 MAPREDUCE-6619. HADOOP_CLASSPATH is overwritten in MR container. Contributed by Junping Du 
(cherry picked from commit da18bbedaf)
 2016-01-27 13:20:16 -08:00
Jason Lowe 296ecbd5ec MAPREDUCE-6554. MRAppMaster servicestart failing with NPE in MRAppMaster#parsePreviousJobHistory. Contributed by Bibin A Chundatt 2016-01-15 16:59:44 +00:00
Akira Ajisaka b1ee278428 MAPREDUCE-6363. [NNBench] Lease mismatch error when running with multiple mappers. Contributed by Vlad Sharanhovich and Bibin A Chundatt. 
(cherry picked from commit 7b0964f354)
(cherry picked from commit 6efee8907a)
 2016-01-14 10:42:02 +09:00
Sangjin Lee 28fd4c70ca MAPREDUCE-6577. MR AM unable to load native library without MR_AM_ADMIN_USER_ENV set (sjlee) 
(cherry picked from commit f6f16118d3)
 2016-01-05 16:01:09 -08:00
Akira Ajisaka e94a8aea57 MAPREDUCE-6583. Clarify confusing sentence in MapReduce tutorial document. Contributed by Kai Sasaki. 
(cherry picked from commit 7995a6ea4d)
(cherry picked from commit 8607cb6074)
 2015-12-21 00:19:11 +09:00
Vinod Kumar Vavilapalli cab5d86738 Revert "MAPREDUCE-6566. Add retry support to mapreduce CLI tool. Contributed by Varun Vasudev" 
This reverts commit ba02e8f6d1.
 2015-12-16 15:17:01 -08:00
Zhihai Xu 35a5b8929e MAPREDUCE-6436. JobHistory cache issue. Contributed by Kai Sasaki 
(cherry picked from commit 5b7078d069)

Conflicts:
	hadoop-yarn-project/CHANGES.txt
 2015-12-15 01:24:03 -08:00
Xuan ba02e8f6d1 MAPREDUCE-6566. Add retry support to mapreduce CLI tool. Contributed by Varun Vasudev 
(cherry picked from commit fc470840a0)
(cherry picked from commit 3b10ff6a6f)
 2015-12-07 14:18:26 -08:00
Jian He 6a30d93aa4 YARN-3840. Resource Manager web ui issue when sorting application by id (with application having id > 9999). Contributed by Mohammad Shahid Khan and Varun Saxena 
(cherry picked from commit 9f77ccad73)

Conflicts:
	hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-app/src/main/java/org/apache/hadoop/mapreduce/v2/app/webapp/TaskPage.java
	hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-applicationhistoryservice/src/test/java/org/apache/hadoop/yarn/server/applicationhistoryservice/webapp/TestAHSWebApp.java
	hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-common/src/main/java/org/apache/hadoop/yarn/server/webapp/WebPageUtils.java
 2015-12-03 14:11:48 -08:00
Jason Lowe e1b6a5413f MAPREDUCE-6549. multibyte delimiters with LineRecordReader cause duplicate records (wilfreds via rkanter) 
(cherry picked from commit 7fd00b3db4)

Conflicts:

	hadoop-mapreduce-project/CHANGES.txt
 2015-11-30 17:45:27 +00:00
Jason Lowe 4ba5667aed MAPREDUCE-5883. "Total megabyte-seconds" in job counters is slightly misleading. Contributed by Nathan Roberts 
(cherry picked from commit cab3c7c889)
 2015-11-24 22:02:21 +00:00
Jason Lowe b96377334c MAPREDUCE-6377. JHS sorting on state column not working in webUi. 
Contributed by zhihai xu.
(cherry picked from commit 790a861766)

Conflicts:

	hadoop-mapreduce-project/CHANGES.txt
 2015-11-18 17:12:50 +00:00
Jian He 27b786660e Revert "YARN-3840. Resource Manager web ui issue when sorting application by id (with application having id > 9999) Contributed by Mohammad Shahid Khan" 
This reverts commit cb4d4d5918.

Conflicts:
	hadoop-yarn-project/CHANGES.txt
 2015-11-16 20:20:46 -08:00
Jian He f95f416fad MAPREDUCE-5485. Allow repeating job commit by extending OutputCommitter API. Contributed by Junping Du 2015-11-16 17:13:49 -08:00
Sangjin Lee adede3e53d MAPREDUCE-6540. TestMRTimelineEventHandling fails (sjlee) 
(cherry picked from commit 9ad708a0df)
 2015-11-11 16:35:46 -08:00
Jian He cb4d4d5918 YARN-3840. Resource Manager web ui issue when sorting application by id (with application having id > 9999) Contributed by Mohammad Shahid Khan 
(cherry picked from commit 8fbea531d7)
(cherry picked from commit a4ff03b200)

Conflicts:
	hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-app/src/main/java/org/apache/hadoop/mapreduce/v2/app/webapp/TaskPage.java
	hadoop-yarn-project/CHANGES.txt
	hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-applicationhistoryservice/src/test/java/org/apache/hadoop/yarn/server/applicationhistoryservice/webapp/TestAHSWebApp.java
	hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-common/src/main/java/org/apache/hadoop/yarn/server/webapp/WebPageUtils.java
 2015-11-09 14:30:47 -08:00
Vinod Kumar Vavilapalli 09348c4bce Preparing for 2.7.3 development 2015-11-03 15:18:10 -08:00
Jason Lowe 431f48f653 MAPREDUCE-6528. Memory leak for HistoryFileManager.getJobSummary(). Contributed by Junping Du 
(cherry picked from commit 6344b6a769)
 2015-10-30 15:33:08 +00:00
Tsuyoshi Ozawa c84ec0657f YARN-4320. TestJobHistoryEventHandler fails as AHS in MiniYarnCluster no longer binds to default port 8188. Contributed by Varun Saxena. 
(cherry picked from commit ce31b22739)
 2015-10-30 18:00:45 +09:00
Xuan ceefabc44a YARN-4313. Race condition in MiniMRYarnCluster when getting history 
server address. Contributed by Jian He

(cherry picked from commit 7412ff48ee)
(cherry picked from commit b505bf9e58)
 2015-10-29 17:38:09 -07:00
Jason Lowe 7499981d3c MAPREDUCE-6273. HistoryFileManager should check whether summaryFile exists 
to avoid FileNotFoundException causing HistoryFileInfo into MOVE_FAILED
state. Contributed by zhihai xu.

(cherry picked from commit 1ea90144d5)
(cherry picked from commit b9865cbf3d)

Conflicts:

	hadoop-mapreduce-project/CHANGES.txt
 2015-10-29 18:29:45 +00:00
Jason Lowe 0dc69afdc7 MAPREDUCE-6518. Set SO_KEEPALIVE on shuffle connections. Contributed by Chang Li 
(cherry picked from commit ee9b80acf8)
 2015-10-21 21:46:59 +00:00
Akira Ajisaka 3096d3e1a1 MAPREDUCE-6497. Fix wrong value of JOB_FINISHED event in JobHistoryEventHandler. Contributed by Shinichi Yamashita. 
(cherry picked from commit 195793c6f3)
(cherry picked from commit e42f576232)
 2015-10-01 10:26:24 +02:00
Jason Lowe fc10c99839 MAPREDUCE-6492. AsyncDispatcher exit with NPE on TaskAttemptImpl#sendJHStartEventForAssignedFailTask. Contributed by Bibin A Chundatt 
(cherry picked from commit ab11085b81)
 2015-09-28 22:13:40 +00:00
Jason Lowe 7429438c85 MAPREDUCE-5982. Task attempts that fail from the ASSIGNED state can disappear. Contributed by Chang Li 2015-09-18 19:21:35 +00:00
Jason Lowe 521196874c MAPREDUCE-6481. LineRecordReader may give incomplete record and wrong position/key information for uncompressed input sometimes. Contributed by Zhihai Xu 
(cherry picked from commit 58d1a02b8d)

Conflicts:

	hadoop-mapreduce-project/CHANGES.txt
 2015-09-18 15:10:38 +00:00
Jason Lowe 850d6a3fd3 MAPREDUCE-5948. org.apache.hadoop.mapred.LineRecordReader does not handle multibyte record delimiters well. Contributed by Vinayakumar B, Rushabh Shah, and Akira AJISAKA 
(cherry picked from commit 077250d8d7)

Conflicts:

	hadoop-mapreduce-project/CHANGES.txt
 2015-09-18 15:04:02 +00:00
Jason Lowe ca6d5b4b92 MAPREDUCE-6472. MapReduce AM should have java.io.tmpdir=./tmp to be consistent with tasks. Contributed by Naganarasimha G R 
(cherry picked from commit 62943b8e3a)
 2015-09-15 19:45:06 +00:00
Jason Lowe 4ea42b84b7 MAPREDUCE-5649. Reduce cannot use more than 2G memory for the final merge. Contributed by Gera Shegalov 
(cherry picked from commit 7dc3c1203d)

(cherry picked from commit 87c2d915f1)
 2015-09-10 11:00:00 -07:00
Hitesh Shah f6b7dc24f0 YARN-2890. MiniYarnCluster should turn on timeline service if configured to do so. Contributed by Mit Desai. 
(cherry picked from commit 265ed1fe80)
(cherry picked from commit 55b794e7fa)
 2015-09-10 10:55:23 -07:00
Jason Lowe d1186dbafa MAPREDUCE-6474. ShuffleHandler can possibly exhaust nodemanager file descriptors. Contributed by Kuhu Shukla 
(cherry picked from commit 8e615588d5)
 2015-09-10 16:05:48 +00:00
Tsuyoshi Ozawa e8a15495bc MAPREDUCE-6442. Stack trace is missing when error occurs in client protocol provider's constructor Contributed by Chang Li. 
(cherry picked from commit 9b685773ec)
 2015-09-05 11:24:30 +09:00
Vinod Kumar Vavilapalli f333e367e1 MAPREDUCE-6454. Fixed MapReduce to modify HADOOP_CLASSPATH to have distributed cache files so that child processes running hadoop scripts can access these files. Contributed by Junping Du. 
(cherry picked from commit 9da81a3955)
 2015-08-20 17:39:43 -07:00