Apache
/
hadoop
1
0
Fork 0
Code Issues Pull Requests Packages Projects Releases Wiki Activity
12,441 Commits 326 Branches 371 Tags 1.5 GiB
Commit Graph

2234 Commits

Author SHA1 Message Date
Arun Suresh b3df744fb6 Fixing Job History Server Configuration parsing. (Jason Lowe via asuresh) 2017-11-09 15:25:59 -08:00
Naganarasimha 01912d18d5 MAPREDUCE-6975. Logging task counters. Contributed by Prabhu Joseph. 
(cherry picked from commit 61bbdc511e)
 2017-11-06 14:09:07 +05:30
Jason Lowe 2dec52da20 HADOOP-14919. BZip2 drops records when reading data in splits. Contributed by Jason Lowe 
(cherry picked from commit 2fae63aa60)
 2017-10-31 09:35:32 -05:00
Jason Lowe 9d3ea2c0ed YARN-7244. ShuffleHandler is not aware of disks that are added. Contributed by Kuhu Shukla 2017-10-30 16:08:14 -05:00
Junping Du 5ce84db79f Add several jdiff xml files for 2.8.2 release. 
(cherry picked from commit a25b5aa0cf)
(cherry picked from commit 1f2a2ba2be)
 2017-10-26 23:26:21 -07:00
Akira Ajisaka dade7396a2
MAPREDUCE-6966. DistSum should use Time.monotonicNow for measuring durations. Contributed by Chetna Chaudhari. 
(cherry picked from commit c71d137960)
(cherry picked from commit d80e2f2eae)
 2017-09-22 15:10:42 +09:00
Eric Payne 67ada5d485 MAPREDUCE-6960. Shuffle Handler prints disk error stack traces for every read failure. 
(cherry picked from commit 595d478408)
 2017-09-19 11:00:20 -05:00
Jason Lowe 03a380daec MAPREDUCE-6958. Shuffle audit logger should log size of shuffle transfer. Contributed by Jason Lowe 2017-09-19 10:27:04 -05:00
Daniel Templeton b1e1f7546c MAPREDUCE-6808. Log map attempts as part of shuffle handler audit log (Contributed by Gergő Pásztor via Daniel Templeton) 
(cherry picked from commit a7463b6c88)
 2017-09-19 09:08:00 -05:00
Jason Lowe e63a2d4d33 MAPREDUCE-6957. shuffle hangs after a node manager connection timeout. Contributed by Jooseong Kim 
(cherry picked from commit 4d98936eec)

Conflicts:
	hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-core/src/main/java/org/apache/hadoop/mapreduce/task/reduce/Fetcher.java
 2017-09-13 17:24:19 -05:00
Haibo Chen 2fce0fa8d7 MAPREDUCE-6937. Backport MAPREDUCE-6870 to branch-2 while preserving compatibility. (Peter Bacsko via Haibo Chen) 2017-08-31 09:21:09 -07:00
Konstantin V Shvachko 7ab03ffbf0 MAPREDUCE-6931. Remove TestDFSIO "Total Throughput" calculation. Contributed by Dennis Huo. 
(cherry picked from commit 3e0e2033cc)

Conflicts:
	hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-jobclient/src/test/java/org/apache/hadoop/fs/TestDFSIO.java
 2017-08-30 14:46:39 -07:00
Jason Lowe fe4aff7330 MAPREDUCE-6641. TestTaskAttempt fails in trunk. Contributed by Haibo Chen. 
(cherry picked from commit cd08aa72a6)
 2017-08-29 12:54:32 -05:00
Jason Lowe ea67e1f266 MAPREDUCE-6927. MR job should only set tracking url if history was successfully written. Contributed by Eric Badger 
(cherry picked from commit 735fce5bec)

Conflicts:
	hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-app/src/main/java/org/apache/hadoop/mapreduce/v2/app/MRAppMaster.java
 2017-08-08 15:03:34 -05:00
Ravi Prakash d6228fbebd MAPREDUCE-6910. MapReduceTrackingUriPlugin can not return the right URI of history server with HTTPS. Contributed by Lantao Jin 
(cherry picked from commit 43f0503286)
(cherry picked from commit 756a068143)
 2017-07-13 22:57:44 -07:00
Akira Ajisaka c180005fcf MAPREDUCE-6697. Concurrent task limits should only be applied when necessary. Contributed by Nathan Roberts. 
(cherry picked from commit a5c0476a99)
 2017-07-13 13:30:33 -07:00
Junping Du 28be69ae6d MAPREDUCE-5621. mr-jobhistory-daemon.sh doesn't have to execute mkdir and chown all the time. Contributed by Shinichi Yamashita. 
(cherry picked from commit ea3dba37cd)
 2017-07-12 12:54:35 -07:00
Junping Du 5ad710bc7a MAPREDUCE-6246. DBOutputFormat.java appending extra semicolon to query which is incompatible with DB2. Contributed by ramtin and Gergely Novák. 
(cherry picked from commit f484a6ff60)
(cherry picked from commit f823f9fd78)
 2017-07-07 13:32:21 -07:00
Jason Lowe c8e6012f62 MAPREDUCE-6911. TestMapreduceConfigFields.testCompareXmlAgainstConfigurationClass fails consistently. Contributed by Eric Badger 
(cherry picked from commit f0435db089)
 2017-07-06 09:24:01 -05:00
Akira Ajisaka 71f49406f2
MAPREDUCE-6909. LocalJobRunner fails when run on a node from multiple users. Contributed by Jason Lowe. 
(cherry picked from commit ac333155fb)
 2017-07-04 15:26:46 +09:00
Naganarasimha acc504dc58 MAPREDUCE-6905. HADOOP_JOB_HISTORY_OPTS should be HADOOP_JOB_HISTORYSERVER_OPTS in mapred-config.sh. Contributed by LiXin Ge. 
(cherry picked from commit bf1f59937d)
 2017-07-02 16:18:30 +05:30
Jason Lowe e47ec9413a Revert "MAPREDUCE-6898. TestKill.testKillTask is flaky. Contributed by Peter Bacsko" 
This reverts commit ea26ccb7dd.
 2017-06-16 14:49:56 -05:00
Jason Lowe ea26ccb7dd MAPREDUCE-6898. TestKill.testKillTask is flaky. Contributed by Peter Bacsko 
(cherry picked from commit e78542f4be)
 2017-06-16 14:26:36 -05:00
Vinod Kumar Vavilapalli (I am also known as @tshooter.) 396de0fd8f Preparing for 2.8.3 development 2017-06-07 15:38:20 -07:00
Brahma Reddy Battula c60516aee5 MAPREDUCE-6676. NNBench should Throw IOException when rename and delete fails. Contributed by Brahma Reddy Battula. 
(cherry picked from commit 24181f559f)
 2017-06-07 16:38:05 +08:00
Jason Lowe 030d521a69 MAPREDUCE-6882. Increase MapReduce test timeouts from 1 second to 10 seconds. Contributed by Eric Badger 
(cherry picked from commit 1e71fe8c42)
 2017-05-10 09:59:10 -05:00
Junping Du dfac95a3a2 Preparing for 2.8.1 development 2017-04-11 11:21:49 -07:00
Jason Lowe a351a30d9d MAPREDUCE-6850. Shuffle Handler keep-alive connections are closed from the server side. Contributed by Jonathan Eagles 
(cherry picked from commit c8bd5fc7a8)

Conflicts:
	hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-shuffle/src/main/java/org/apache/hadoop/mapred/ShuffleHandler.java
	hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-shuffle/src/test/java/org/apache/hadoop/mapred/TestShuffleHandler.java
 2017-03-30 11:15:25 -05:00
Zhe Zhang 0acf2bca9c MAPREDUCE-6873. MR Job Submission Fails if MR framework application path not on defaultFS. Contributed by Erik Krogen. 
(cherry picked from commit de7efd2687)
(cherry picked from commit 3f5dc51ec9)
 2017-03-29 22:19:50 -07:00
Junping Du 19d479168e MAPREDUCE-6868. License check for jdiff output files should be ignored. Contributed by Akira Ajisaka. 
(cherry picked from commit 945b006fe8)
(cherry picked from commit 0e8beb214d)
 2017-03-26 23:19:49 -07:00
Joe Mészáros 9909174677
MAPREDUCE-6866. Fix getNumMapTasks() documentation in JobConf. 
This closes #205

Signed-off-by: Akira Ajisaka <aajisaka@apache.org>
(cherry picked from commit e0a2eb71e3)
(cherry picked from commit 34a11b9108)
 2017-03-27 13:53:08 +09:00
Junping Du 86800b30a9 Add COMMON/MAPREDUCE/YARN jdiff of 2.8.0 release. 
(cherry picked from commit 6df029db36)
(cherry picked from commit 3ad4d37363)
 2017-03-25 14:34:55 -07:00
Akira Ajisaka dd5d0103df
Treat encrypted files as private. Contributed by Daniel Templeton. 
(cherry picked from commit f01a69f84f)
(cherry picked from commit 120f680318)
 2017-03-07 13:38:51 +09:00
Akira Ajisaka 8b5307c0e7
MAPREDUCE-6855. Specify charset when create String in CredentialsTestJob. Contributed by Kai Sasaki. 
(cherry picked from commit 14413989ca)
(cherry picked from commit aa1f944bba)
 2017-03-07 13:13:52 +09:00
Akira Ajisaka c6bf43e0be MAPREDUCE-6753. Variable in byte printed directly in mapreduce client. Contributed by Kai Sasaki. 
(cherry picked from commit 19a1fc6373)
(cherry picked from commit e2fb9538f0)
 2017-03-03 16:14:03 +09:00
Akira Ajisaka 6833b4ea30 MAPREDUCE-6841. Fix dead link in MapReduce tutorial document. Contributed by Victor Nee. 
(cherry picked from commit 4d33683882)
(cherry picked from commit 69f2ae449b28530d01b73a172f2be6c256dca800)
 2017-02-27 19:47:35 +09:00
Wangda Tan 920847edce YARN-6143. Fix incompatible issue caused by YARN-3583. (Sunil G via wangda) 
(cherry picked from commit 0566b30875dfc61dbbba4cde0d42e15b116e85b8)
 2017-02-22 11:21:57 -08:00
Jason Lowe 63275b7163 MAPREDUCE-6831. Flaky test TestJobImpl.testKilledDuringKillAbort. Contributed by Peter Bacsko 
(cherry picked from commit 78b487bde1)
 2017-01-17 16:05:00 +00:00
Junping Du 7706a63fb4 Preparing for 2.8.1 development after cutting off 2.8.0 branch. 2017-01-10 18:03:51 -08:00
Junping Du 031ad740b9 MAPREDUCE-6711. JobImpl fails to handle preemption events on state COMMITTING. Contributed by Prabhu Joseph. 
(cherry picked from commit 679478d0c6)
(cherry picked from commit bf87c523ab)
 2017-01-07 22:08:07 -08:00
Jason Lowe 8ffe5eb7bd MAPREDUCE-6682. TestMRCJCFileOutputCommitter fails intermittently Contributed by Akira Ajisaka. 
(cherry picked from commit 8f1c374bec)

Conflicts:

	hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-jobclient/src/test/java/org/apache/hadoop/mapred/TestMRTimelineEventHandling.java
 2017-01-06 18:55:48 +00:00
Rohith Sharma K S 0c1615c368 MAPREDUCE-6662. Clear ASF Warnings on test data files. Contributed by Vinayakumar B 
(cherry picked from commit 0ef8bbfd87)
(cherry picked from commit f22ce96b50)
 2017-01-05 18:03:48 +09:00
Jian He 42d1b38989 YARN-4164. Changed updateApplicationPriority API to return the updated application priority. Contributed by Rohith Sharma K S 2017-01-04 15:43:06 -08:00
Varun Saxena 2b4950b094 MAPREDUCE-6801. Fix flaky TestKill.testKillJob (Haibo Chen via Varun Saxena) 
(cherry picked from commit 7584fbf4cb)
 2017-01-04 18:29:21 +00:00
Robert Kanter d738f4b652 MAPREDUCE-6675. TestJobImpl.testUnusableNode failed (haibochen via rkanter) 
(cherry picked from commit 9d3fcdfbb3)
 2017-01-04 18:18:29 +00:00
Akira Ajisaka 58a1753373 HADOOP-12733. Remove references to obsolete io.seqfile configuration variables. Contributed by Ray Chiang. 
(cherry picked from commit 01d31fe938)
(cherry picked from commit 3e2d26a97f)
 2017-01-04 14:20:04 +09:00
Brahma Reddy Battula b8be8f0d4f HADOOP-13883. Addendum patch to correct the message and alphabetized with the earlier patch. Contributed by Yiqun Lin. 2016-12-31 16:27:17 +05:30
Junping Du 5f36b62265 Revert "MAPREDUCE-5875. Make Counter limits consistent across JobClient, MRAppMaster, and YarnChild. (Gera Shegalov via kasha)" 
This reverts commit 7bfd9e068d.

Conflicts:
	hadoop-mapreduce-project/CHANGES.txt
	hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-core/src/main/java/org/apache/hadoop/mapreduce/JobSubmitter.java
	hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-core/src/main/java/org/apache/hadoop/mapreduce/jobhistory/HistoryViewer.java
	hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-hs/src/main/java/org/apache/hadoop/mapreduce/v2/hs/CompletedJob.java
 2016-12-28 17:37:20 -08:00
Junping Du 8755a6f5ac Revert "MAPREDUCE-6199. AbstractCounters are not reset completely on deserialization (adhoot via rkanter)" 
This reverts commit f9341c1e2c.

Conflicts:
	hadoop-mapreduce-project/CHANGES.txt
 2016-12-28 17:13:04 -08:00
Junping Du e8d7801319 Revert "MAPREDUCE-6286. A typo in HistoryViewer makes some code useless, which causes counter limits are not reset correctly. Contributed by Zhihai Xu." 
This reverts commit 7c72c7f529.

Conflicts:
	hadoop-mapreduce-project/CHANGES.txt
 2016-12-28 17:05:17 -08:00