Apache
/
hadoop
1
0
Fork 0
Code Issues Pull Requests Packages Projects Releases Wiki Activity
12,431 Commits 326 Branches 371 Tags 1.5 GiB
Commit Graph

2229 Commits

Author SHA1 Message Date
Akira Ajisaka dade7396a2
MAPREDUCE-6966. DistSum should use Time.monotonicNow for measuring durations. Contributed by Chetna Chaudhari. 
(cherry picked from commit c71d137960)
(cherry picked from commit d80e2f2eae)
 2017-09-22 15:10:42 +09:00
Eric Payne 67ada5d485 MAPREDUCE-6960. Shuffle Handler prints disk error stack traces for every read failure. 
(cherry picked from commit 595d478408)
 2017-09-19 11:00:20 -05:00
Jason Lowe 03a380daec MAPREDUCE-6958. Shuffle audit logger should log size of shuffle transfer. Contributed by Jason Lowe 2017-09-19 10:27:04 -05:00
Daniel Templeton b1e1f7546c MAPREDUCE-6808. Log map attempts as part of shuffle handler audit log (Contributed by Gergő Pásztor via Daniel Templeton) 
(cherry picked from commit a7463b6c88)
 2017-09-19 09:08:00 -05:00
Jason Lowe e63a2d4d33 MAPREDUCE-6957. shuffle hangs after a node manager connection timeout. Contributed by Jooseong Kim 
(cherry picked from commit 4d98936eec)

Conflicts:
	hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-core/src/main/java/org/apache/hadoop/mapreduce/task/reduce/Fetcher.java
 2017-09-13 17:24:19 -05:00
Haibo Chen 2fce0fa8d7 MAPREDUCE-6937. Backport MAPREDUCE-6870 to branch-2 while preserving compatibility. (Peter Bacsko via Haibo Chen) 2017-08-31 09:21:09 -07:00
Konstantin V Shvachko 7ab03ffbf0 MAPREDUCE-6931. Remove TestDFSIO "Total Throughput" calculation. Contributed by Dennis Huo. 
(cherry picked from commit 3e0e2033cc)

Conflicts:
	hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-jobclient/src/test/java/org/apache/hadoop/fs/TestDFSIO.java
 2017-08-30 14:46:39 -07:00
Jason Lowe fe4aff7330 MAPREDUCE-6641. TestTaskAttempt fails in trunk. Contributed by Haibo Chen. 
(cherry picked from commit cd08aa72a6)
 2017-08-29 12:54:32 -05:00
Jason Lowe ea67e1f266 MAPREDUCE-6927. MR job should only set tracking url if history was successfully written. Contributed by Eric Badger 
(cherry picked from commit 735fce5bec)

Conflicts:
	hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-app/src/main/java/org/apache/hadoop/mapreduce/v2/app/MRAppMaster.java
 2017-08-08 15:03:34 -05:00
Ravi Prakash d6228fbebd MAPREDUCE-6910. MapReduceTrackingUriPlugin can not return the right URI of history server with HTTPS. Contributed by Lantao Jin 
(cherry picked from commit 43f0503286)
(cherry picked from commit 756a068143)
 2017-07-13 22:57:44 -07:00
Akira Ajisaka c180005fcf MAPREDUCE-6697. Concurrent task limits should only be applied when necessary. Contributed by Nathan Roberts. 
(cherry picked from commit a5c0476a99)
 2017-07-13 13:30:33 -07:00
Junping Du 28be69ae6d MAPREDUCE-5621. mr-jobhistory-daemon.sh doesn't have to execute mkdir and chown all the time. Contributed by Shinichi Yamashita. 
(cherry picked from commit ea3dba37cd)
 2017-07-12 12:54:35 -07:00
Junping Du 5ad710bc7a MAPREDUCE-6246. DBOutputFormat.java appending extra semicolon to query which is incompatible with DB2. Contributed by ramtin and Gergely Novák. 
(cherry picked from commit f484a6ff60)
(cherry picked from commit f823f9fd78)
 2017-07-07 13:32:21 -07:00
Jason Lowe c8e6012f62 MAPREDUCE-6911. TestMapreduceConfigFields.testCompareXmlAgainstConfigurationClass fails consistently. Contributed by Eric Badger 
(cherry picked from commit f0435db089)
 2017-07-06 09:24:01 -05:00
Akira Ajisaka 71f49406f2
MAPREDUCE-6909. LocalJobRunner fails when run on a node from multiple users. Contributed by Jason Lowe. 
(cherry picked from commit ac333155fb)
 2017-07-04 15:26:46 +09:00
Naganarasimha acc504dc58 MAPREDUCE-6905. HADOOP_JOB_HISTORY_OPTS should be HADOOP_JOB_HISTORYSERVER_OPTS in mapred-config.sh. Contributed by LiXin Ge. 
(cherry picked from commit bf1f59937d)
 2017-07-02 16:18:30 +05:30
Jason Lowe e47ec9413a Revert "MAPREDUCE-6898. TestKill.testKillTask is flaky. Contributed by Peter Bacsko" 
This reverts commit ea26ccb7dd.
 2017-06-16 14:49:56 -05:00
Jason Lowe ea26ccb7dd MAPREDUCE-6898. TestKill.testKillTask is flaky. Contributed by Peter Bacsko 
(cherry picked from commit e78542f4be)
 2017-06-16 14:26:36 -05:00
Vinod Kumar Vavilapalli (I am also known as @tshooter.) 396de0fd8f Preparing for 2.8.3 development 2017-06-07 15:38:20 -07:00
Brahma Reddy Battula c60516aee5 MAPREDUCE-6676. NNBench should Throw IOException when rename and delete fails. Contributed by Brahma Reddy Battula. 
(cherry picked from commit 24181f559f)
 2017-06-07 16:38:05 +08:00
Jason Lowe 030d521a69 MAPREDUCE-6882. Increase MapReduce test timeouts from 1 second to 10 seconds. Contributed by Eric Badger 
(cherry picked from commit 1e71fe8c42)
 2017-05-10 09:59:10 -05:00
Junping Du dfac95a3a2 Preparing for 2.8.1 development 2017-04-11 11:21:49 -07:00
Jason Lowe a351a30d9d MAPREDUCE-6850. Shuffle Handler keep-alive connections are closed from the server side. Contributed by Jonathan Eagles 
(cherry picked from commit c8bd5fc7a8)

Conflicts:
	hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-shuffle/src/main/java/org/apache/hadoop/mapred/ShuffleHandler.java
	hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-shuffle/src/test/java/org/apache/hadoop/mapred/TestShuffleHandler.java
 2017-03-30 11:15:25 -05:00
Zhe Zhang 0acf2bca9c MAPREDUCE-6873. MR Job Submission Fails if MR framework application path not on defaultFS. Contributed by Erik Krogen. 
(cherry picked from commit de7efd2687)
(cherry picked from commit 3f5dc51ec9)
 2017-03-29 22:19:50 -07:00
Junping Du 19d479168e MAPREDUCE-6868. License check for jdiff output files should be ignored. Contributed by Akira Ajisaka. 
(cherry picked from commit 945b006fe8)
(cherry picked from commit 0e8beb214d)
 2017-03-26 23:19:49 -07:00
Joe Mészáros 9909174677
MAPREDUCE-6866. Fix getNumMapTasks() documentation in JobConf. 
This closes #205

Signed-off-by: Akira Ajisaka <aajisaka@apache.org>
(cherry picked from commit e0a2eb71e3)
(cherry picked from commit 34a11b9108)
 2017-03-27 13:53:08 +09:00
Junping Du 86800b30a9 Add COMMON/MAPREDUCE/YARN jdiff of 2.8.0 release. 
(cherry picked from commit 6df029db36)
(cherry picked from commit 3ad4d37363)
 2017-03-25 14:34:55 -07:00
Akira Ajisaka dd5d0103df
Treat encrypted files as private. Contributed by Daniel Templeton. 
(cherry picked from commit f01a69f84f)
(cherry picked from commit 120f680318)
 2017-03-07 13:38:51 +09:00
Akira Ajisaka 8b5307c0e7
MAPREDUCE-6855. Specify charset when create String in CredentialsTestJob. Contributed by Kai Sasaki. 
(cherry picked from commit 14413989ca)
(cherry picked from commit aa1f944bba)
 2017-03-07 13:13:52 +09:00
Akira Ajisaka c6bf43e0be MAPREDUCE-6753. Variable in byte printed directly in mapreduce client. Contributed by Kai Sasaki. 
(cherry picked from commit 19a1fc6373)
(cherry picked from commit e2fb9538f0)
 2017-03-03 16:14:03 +09:00
Akira Ajisaka 6833b4ea30 MAPREDUCE-6841. Fix dead link in MapReduce tutorial document. Contributed by Victor Nee. 
(cherry picked from commit 4d33683882)
(cherry picked from commit 69f2ae449b28530d01b73a172f2be6c256dca800)
 2017-02-27 19:47:35 +09:00
Wangda Tan 920847edce YARN-6143. Fix incompatible issue caused by YARN-3583. (Sunil G via wangda) 
(cherry picked from commit 0566b30875dfc61dbbba4cde0d42e15b116e85b8)
 2017-02-22 11:21:57 -08:00
Jason Lowe 63275b7163 MAPREDUCE-6831. Flaky test TestJobImpl.testKilledDuringKillAbort. Contributed by Peter Bacsko 
(cherry picked from commit 78b487bde1)
 2017-01-17 16:05:00 +00:00
Junping Du 7706a63fb4 Preparing for 2.8.1 development after cutting off 2.8.0 branch. 2017-01-10 18:03:51 -08:00
Junping Du 031ad740b9 MAPREDUCE-6711. JobImpl fails to handle preemption events on state COMMITTING. Contributed by Prabhu Joseph. 
(cherry picked from commit 679478d0c6)
(cherry picked from commit bf87c523ab)
 2017-01-07 22:08:07 -08:00
Jason Lowe 8ffe5eb7bd MAPREDUCE-6682. TestMRCJCFileOutputCommitter fails intermittently Contributed by Akira Ajisaka. 
(cherry picked from commit 8f1c374bec)

Conflicts:

	hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-jobclient/src/test/java/org/apache/hadoop/mapred/TestMRTimelineEventHandling.java
 2017-01-06 18:55:48 +00:00
Rohith Sharma K S 0c1615c368 MAPREDUCE-6662. Clear ASF Warnings on test data files. Contributed by Vinayakumar B 
(cherry picked from commit 0ef8bbfd87)
(cherry picked from commit f22ce96b50)
 2017-01-05 18:03:48 +09:00
Jian He 42d1b38989 YARN-4164. Changed updateApplicationPriority API to return the updated application priority. Contributed by Rohith Sharma K S 2017-01-04 15:43:06 -08:00
Varun Saxena 2b4950b094 MAPREDUCE-6801. Fix flaky TestKill.testKillJob (Haibo Chen via Varun Saxena) 
(cherry picked from commit 7584fbf4cb)
 2017-01-04 18:29:21 +00:00
Robert Kanter d738f4b652 MAPREDUCE-6675. TestJobImpl.testUnusableNode failed (haibochen via rkanter) 
(cherry picked from commit 9d3fcdfbb3)
 2017-01-04 18:18:29 +00:00
Akira Ajisaka 58a1753373 HADOOP-12733. Remove references to obsolete io.seqfile configuration variables. Contributed by Ray Chiang. 
(cherry picked from commit 01d31fe938)
(cherry picked from commit 3e2d26a97f)
 2017-01-04 14:20:04 +09:00
Brahma Reddy Battula b8be8f0d4f HADOOP-13883. Addendum patch to correct the message and alphabetized with the earlier patch. Contributed by Yiqun Lin. 2016-12-31 16:27:17 +05:30
Junping Du 5f36b62265 Revert "MAPREDUCE-5875. Make Counter limits consistent across JobClient, MRAppMaster, and YarnChild. (Gera Shegalov via kasha)" 
This reverts commit 7bfd9e068d.

Conflicts:
	hadoop-mapreduce-project/CHANGES.txt
	hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-core/src/main/java/org/apache/hadoop/mapreduce/JobSubmitter.java
	hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-core/src/main/java/org/apache/hadoop/mapreduce/jobhistory/HistoryViewer.java
	hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-hs/src/main/java/org/apache/hadoop/mapreduce/v2/hs/CompletedJob.java
 2016-12-28 17:37:20 -08:00
Junping Du 8755a6f5ac Revert "MAPREDUCE-6199. AbstractCounters are not reset completely on deserialization (adhoot via rkanter)" 
This reverts commit f9341c1e2c.

Conflicts:
	hadoop-mapreduce-project/CHANGES.txt
 2016-12-28 17:13:04 -08:00
Junping Du e8d7801319 Revert "MAPREDUCE-6286. A typo in HistoryViewer makes some code useless, which causes counter limits are not reset correctly. Contributed by Zhihai Xu." 
This reverts commit 7c72c7f529.

Conflicts:
	hadoop-mapreduce-project/CHANGES.txt
 2016-12-28 17:05:17 -08:00
Akira Ajisaka 8e4479dda3 YARN-4994. Use MiniYARNCluster with try-with-resources in tests. Contributed by Andras Bokor. 
(cherry picked from commit ae401539ea)
(cherry picked from commit 522bc98026)

 Conflicts:
	hadoop-yarn-project/hadoop-yarn/hadoop-yarn-client/src/test/java/org/apache/hadoop/yarn/client/TestHedgingRequestRMFailoverProxyProvider.java
	hadoop-yarn-project/hadoop-yarn/hadoop-yarn-client/src/test/java/org/apache/hadoop/yarn/client/cli/TestYarnCLI.java
 2016-12-23 04:15:39 +09:00
Akira Ajisaka 0c58241b22 MAPREDUCE-6820. Fix dead links in Job relevant classes. Contributed by Yiqun Lin. 
(cherry picked from commit 92a8917ca5)
(cherry picked from commit 1e1ab9bc1b)
 2016-12-10 10:57:33 +09:00
Akira Ajisaka 460fe6594a MAPREDUCE-6816. Progress bars in Web UI always at 100%. Contributed by Shen Yinjie. 
(cherry picked from commit da4ecc9f82)
(cherry picked from commit 03854f0bc5)
 2016-12-07 06:44:31 +09:00
Jason Lowe 69f66424b9 MAPREDUCE-6815. Fix flaky TestKill.testKillTask(). Contributed by Haibo Chen 
(cherry picked from commit 0cfd7ad21f)
 2016-12-02 17:23:33 +00:00
Junping Du 7f4bd1af52 Revert "MAPREDUCE-6565. Configuration to use host name in delegation token service is not read from job.xml during MapReduce job execution. Contributed by Li Lu." 
This reverts commit f2284f44dd.
 2016-11-30 11:23:00 -08:00