Apache
/
hadoop
1
0
Fork 0
Code Issues Pull Requests Packages Projects Releases Wiki Activity
9,246 Commits 326 Branches 371 Tags 1.5 GiB
Commit Graph

1305 Commits

Author SHA1 Message Date
Jian He f1e4f4d83a MAPREDUCE-6680. JHS UserLogDir scan algorithm sometime could skip directory with update in CloudFS (Azure FileSystem, S3, etc. Contributed by Junping Du 
(cherry picked from commit 1e48eefe58)
 2016-04-20 19:03:53 -07:00
Karthik Kambatla cc6ae6fa7e MAPREDUCE-5817. Mappers get rescheduled on node transition even after all reducers are completed. (Sangjin Lee via kasha) 
(cherry picked from commit 27d24f96ab)

(cherry picked from commit b826168173)

Conflicts:
	hadoop-mapreduce-project/CHANGES.txt
 2016-04-18 21:58:53 -07:00
Eric Payne 2ec5bb0fe8 MAPREDUCE-6633. AM should retry map attempts if the reduce task encounters commpression related errors. Contributed by Rushabh Shah 
(cherry picked from commit 1fec06e037)
 2016-04-18 16:12:53 +00:00
Robert Kanter c9ba6fa9ba MAPREDUCE-4785. TestMRApp occasionally fails (haibochen via rkanter) 
(cherry picked from commit ff0ee84d77)
(cherry picked from commit f9f57265cf)

Conflicts:
	hadoop-mapreduce-project/CHANGES.txt
 2016-04-14 14:58:45 -07:00
Junping Du 683f313c1f MAPREDUCE-6670. TestJobListCache#testEviction sometimes fails on Windows with timeout. Contributed by Gergely Novák. 
(cherry picked from commit de96d7c88a)
(cherry picked from commit 5574f82799)
 2016-04-13 08:59:22 -07:00
Wangda Tan 801c95c1cc MAPREDUCE-6302. Preempt reducers after a configurable timeout irrespective of headroom. (kasha via wangda) 2016-04-08 14:35:28 -07:00
Akira Ajisaka a112c0138a MAPREDUCE-6656. [NNBench] OP_DELETE operation isn't working after MAPREDUCE-6363. Contributed by J.Andreina. 
(cherry picked from commit 368c77376a)
(cherry picked from commit 7a3fd1bc1a)
 2016-03-23 23:26:33 +09:00
Akira Ajisaka 04cc99152f MAPREDUCE-6580. Test failure: TestMRJobsWithProfiler. Contributed by Eric Badger. 
(cherry picked from commit 0bfe5a0167)
(cherry picked from commit 12c1cc889a)
 2016-03-23 13:23:29 +09:00
Robert Kanter 66e1e92ffd MAPREDUCE-6622. Add capability to set JHS job cache to a task-based limit (rchiang via rkanter) 
(cherry picked from commit 0f72da7e28)

Conflicts:
	hadoop-mapreduce-project/CHANGES.txt
 2016-03-06 18:34:46 -08:00
Varun Vasudev 0edc764184 MAPREDUCE-6635. Unsafe long to int conversion in UncompressedSplitLineReader and IndexOutOfBoundsException. Contributed by Junping Du. 
(cherry picked from commit c6f2d761d5)
(cherry picked from commit f1999fe275)

 Conflicts:
	hadoop-mapreduce-project/CHANGES.txt
 2016-02-23 14:45:07 +05:30
Andrew Wang 403934c011 MAPREDUCE-6637. Testcase Failure : TestFileInputFormat.testSplitLocationInfo. Contributed by Brahma Reddy Battula. 
(cherry picked from commit 37577852ba)
(cherry picked from commit 7f6737951a)
 2016-02-19 16:37:28 -08:00
Jason Lowe 717b1a1f99 MAPREDUCE-6460. TestRMContainerAllocator.testAttemptNotFoundCausesRMCommunicatorException fails. Contributed by Zhihai Xu. 
(cherry picked from commit e3ace31e2b)

Conflicts:

	hadoop-mapreduce-project/CHANGES.txt
 2016-02-10 16:58:57 +00:00
Jason Lowe 9d574368b5 MAPREDUCE-6191. Improve clearing stale state of Java serialization 
testcase.  (Sam Liu via Eric Yang)
(cherry picked from commit c379e102dd)

Conflicts:

	hadoop-mapreduce-project/CHANGES.txt
(cherry picked from commit 303c650d0d)
 2016-02-10 16:48:06 +00:00
Jason Lowe 6257ed8431 MAPREDUCE-6413. TestLocalJobSubmission is failing with unknown host. Contributed by zhihai xu 
(cherry picked from commit aa5b15b03b)

Conflicts:

	hadoop-mapreduce-project/CHANGES.txt
 2016-02-09 16:33:07 +00:00
Jason Lowe 7f6d150c9a MAPREDUCE-6621. Memory Leak in JobClient#submitJobInternal(). Contributed by Xuan Gong 
(cherry picked from commit 43e669b22d)
 2016-02-02 19:32:37 +00:00
Jason Lowe 69c61fae0c MAPREDUCE-6618. YarnClientProtocolProvider leaking the YarnClient thread. Contributed by Xuan Gong 
(cherry picked from commit 59a212b6e1)

Conflicts:

	hadoop-mapreduce-project/CHANGES.txt
 2016-02-01 16:10:56 +00:00
Jian He a1b75ff8c5 MAPREDUCE-6619. HADOOP_CLASSPATH is overwritten in MR container. Contributed by Junping Du 
(cherry picked from commit da18bbedaf)
 2016-01-27 13:20:16 -08:00
Jason Lowe 296ecbd5ec MAPREDUCE-6554. MRAppMaster servicestart failing with NPE in MRAppMaster#parsePreviousJobHistory. Contributed by Bibin A Chundatt 2016-01-15 16:59:44 +00:00
Akira Ajisaka b1ee278428 MAPREDUCE-6363. [NNBench] Lease mismatch error when running with multiple mappers. Contributed by Vlad Sharanhovich and Bibin A Chundatt. 
(cherry picked from commit 7b0964f354)
(cherry picked from commit 6efee8907a)
 2016-01-14 10:42:02 +09:00
Sangjin Lee 28fd4c70ca MAPREDUCE-6577. MR AM unable to load native library without MR_AM_ADMIN_USER_ENV set (sjlee) 
(cherry picked from commit f6f16118d3)
 2016-01-05 16:01:09 -08:00
Akira Ajisaka e94a8aea57 MAPREDUCE-6583. Clarify confusing sentence in MapReduce tutorial document. Contributed by Kai Sasaki. 
(cherry picked from commit 7995a6ea4d)
(cherry picked from commit 8607cb6074)
 2015-12-21 00:19:11 +09:00
Vinod Kumar Vavilapalli cab5d86738 Revert "MAPREDUCE-6566. Add retry support to mapreduce CLI tool. Contributed by Varun Vasudev" 
This reverts commit ba02e8f6d1.
 2015-12-16 15:17:01 -08:00
Zhihai Xu 35a5b8929e MAPREDUCE-6436. JobHistory cache issue. Contributed by Kai Sasaki 
(cherry picked from commit 5b7078d069)

Conflicts:
	hadoop-yarn-project/CHANGES.txt
 2015-12-15 01:24:03 -08:00
Xuan ba02e8f6d1 MAPREDUCE-6566. Add retry support to mapreduce CLI tool. Contributed by Varun Vasudev 
(cherry picked from commit fc470840a0)
(cherry picked from commit 3b10ff6a6f)
 2015-12-07 14:18:26 -08:00
Jian He 6a30d93aa4 YARN-3840. Resource Manager web ui issue when sorting application by id (with application having id > 9999). Contributed by Mohammad Shahid Khan and Varun Saxena 
(cherry picked from commit 9f77ccad73)

Conflicts:
	hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-app/src/main/java/org/apache/hadoop/mapreduce/v2/app/webapp/TaskPage.java
	hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-applicationhistoryservice/src/test/java/org/apache/hadoop/yarn/server/applicationhistoryservice/webapp/TestAHSWebApp.java
	hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-common/src/main/java/org/apache/hadoop/yarn/server/webapp/WebPageUtils.java
 2015-12-03 14:11:48 -08:00
Jason Lowe e1b6a5413f MAPREDUCE-6549. multibyte delimiters with LineRecordReader cause duplicate records (wilfreds via rkanter) 
(cherry picked from commit 7fd00b3db4)

Conflicts:

	hadoop-mapreduce-project/CHANGES.txt
 2015-11-30 17:45:27 +00:00
Jason Lowe 4ba5667aed MAPREDUCE-5883. "Total megabyte-seconds" in job counters is slightly misleading. Contributed by Nathan Roberts 
(cherry picked from commit cab3c7c889)
 2015-11-24 22:02:21 +00:00
Jason Lowe b96377334c MAPREDUCE-6377. JHS sorting on state column not working in webUi. 
Contributed by zhihai xu.
(cherry picked from commit 790a861766)

Conflicts:

	hadoop-mapreduce-project/CHANGES.txt
 2015-11-18 17:12:50 +00:00
Jian He 27b786660e Revert "YARN-3840. Resource Manager web ui issue when sorting application by id (with application having id > 9999) Contributed by Mohammad Shahid Khan" 
This reverts commit cb4d4d5918.

Conflicts:
	hadoop-yarn-project/CHANGES.txt
 2015-11-16 20:20:46 -08:00
Jian He f95f416fad MAPREDUCE-5485. Allow repeating job commit by extending OutputCommitter API. Contributed by Junping Du 2015-11-16 17:13:49 -08:00
Sangjin Lee adede3e53d MAPREDUCE-6540. TestMRTimelineEventHandling fails (sjlee) 
(cherry picked from commit 9ad708a0df)
 2015-11-11 16:35:46 -08:00
Jian He cb4d4d5918 YARN-3840. Resource Manager web ui issue when sorting application by id (with application having id > 9999) Contributed by Mohammad Shahid Khan 
(cherry picked from commit 8fbea531d7)
(cherry picked from commit a4ff03b200)

Conflicts:
	hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-app/src/main/java/org/apache/hadoop/mapreduce/v2/app/webapp/TaskPage.java
	hadoop-yarn-project/CHANGES.txt
	hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-applicationhistoryservice/src/test/java/org/apache/hadoop/yarn/server/applicationhistoryservice/webapp/TestAHSWebApp.java
	hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-common/src/main/java/org/apache/hadoop/yarn/server/webapp/WebPageUtils.java
 2015-11-09 14:30:47 -08:00
Vinod Kumar Vavilapalli 09348c4bce Preparing for 2.7.3 development 2015-11-03 15:18:10 -08:00
Jason Lowe 431f48f653 MAPREDUCE-6528. Memory leak for HistoryFileManager.getJobSummary(). Contributed by Junping Du 
(cherry picked from commit 6344b6a769)
 2015-10-30 15:33:08 +00:00
Tsuyoshi Ozawa c84ec0657f YARN-4320. TestJobHistoryEventHandler fails as AHS in MiniYarnCluster no longer binds to default port 8188. Contributed by Varun Saxena. 
(cherry picked from commit ce31b22739)
 2015-10-30 18:00:45 +09:00
Xuan ceefabc44a YARN-4313. Race condition in MiniMRYarnCluster when getting history 
server address. Contributed by Jian He

(cherry picked from commit 7412ff48ee)
(cherry picked from commit b505bf9e58)
 2015-10-29 17:38:09 -07:00
Jason Lowe 7499981d3c MAPREDUCE-6273. HistoryFileManager should check whether summaryFile exists 
to avoid FileNotFoundException causing HistoryFileInfo into MOVE_FAILED
state. Contributed by zhihai xu.

(cherry picked from commit 1ea90144d5)
(cherry picked from commit b9865cbf3d)

Conflicts:

	hadoop-mapreduce-project/CHANGES.txt
 2015-10-29 18:29:45 +00:00
Jason Lowe 0dc69afdc7 MAPREDUCE-6518. Set SO_KEEPALIVE on shuffle connections. Contributed by Chang Li 
(cherry picked from commit ee9b80acf8)
 2015-10-21 21:46:59 +00:00
Akira Ajisaka 3096d3e1a1 MAPREDUCE-6497. Fix wrong value of JOB_FINISHED event in JobHistoryEventHandler. Contributed by Shinichi Yamashita. 
(cherry picked from commit 195793c6f3)
(cherry picked from commit e42f576232)
 2015-10-01 10:26:24 +02:00
Jason Lowe fc10c99839 MAPREDUCE-6492. AsyncDispatcher exit with NPE on TaskAttemptImpl#sendJHStartEventForAssignedFailTask. Contributed by Bibin A Chundatt 
(cherry picked from commit ab11085b81)
 2015-09-28 22:13:40 +00:00
Jason Lowe 7429438c85 MAPREDUCE-5982. Task attempts that fail from the ASSIGNED state can disappear. Contributed by Chang Li 2015-09-18 19:21:35 +00:00
Jason Lowe 521196874c MAPREDUCE-6481. LineRecordReader may give incomplete record and wrong position/key information for uncompressed input sometimes. Contributed by Zhihai Xu 
(cherry picked from commit 58d1a02b8d)

Conflicts:

	hadoop-mapreduce-project/CHANGES.txt
 2015-09-18 15:10:38 +00:00
Jason Lowe 850d6a3fd3 MAPREDUCE-5948. org.apache.hadoop.mapred.LineRecordReader does not handle multibyte record delimiters well. Contributed by Vinayakumar B, Rushabh Shah, and Akira AJISAKA 
(cherry picked from commit 077250d8d7)

Conflicts:

	hadoop-mapreduce-project/CHANGES.txt
 2015-09-18 15:04:02 +00:00
Jason Lowe ca6d5b4b92 MAPREDUCE-6472. MapReduce AM should have java.io.tmpdir=./tmp to be consistent with tasks. Contributed by Naganarasimha G R 
(cherry picked from commit 62943b8e3a)
 2015-09-15 19:45:06 +00:00
Jason Lowe 4ea42b84b7 MAPREDUCE-5649. Reduce cannot use more than 2G memory for the final merge. Contributed by Gera Shegalov 
(cherry picked from commit 7dc3c1203d)

(cherry picked from commit 87c2d915f1)
 2015-09-10 11:00:00 -07:00
Hitesh Shah f6b7dc24f0 YARN-2890. MiniYarnCluster should turn on timeline service if configured to do so. Contributed by Mit Desai. 
(cherry picked from commit 265ed1fe80)
(cherry picked from commit 55b794e7fa)
 2015-09-10 10:55:23 -07:00
Jason Lowe d1186dbafa MAPREDUCE-6474. ShuffleHandler can possibly exhaust nodemanager file descriptors. Contributed by Kuhu Shukla 
(cherry picked from commit 8e615588d5)
 2015-09-10 16:05:48 +00:00
Tsuyoshi Ozawa e8a15495bc MAPREDUCE-6442. Stack trace is missing when error occurs in client protocol provider's constructor Contributed by Chang Li. 
(cherry picked from commit 9b685773ec)
 2015-09-05 11:24:30 +09:00
Vinod Kumar Vavilapalli f333e367e1 MAPREDUCE-6454. Fixed MapReduce to modify HADOOP_CLASSPATH to have distributed cache files so that child processes running hadoop scripts can access these files. Contributed by Junping Du. 
(cherry picked from commit 9da81a3955)
 2015-08-20 17:39:43 -07:00
Karthik Kambatla d610144d45 MAPREDUCE-6439. AM may fail instead of retrying if RM shuts down during the allocate call. (Anubhav Dhoot via kasha) 
(cherry picked from commit 8dfec7a197)
(cherry picked from commit 20734320a1)
 2015-08-15 00:56:22 -07:00