Apache/hadoop
1
0
Fork 0
Code Issues Pull Requests Packages Projects Releases Wiki Activity
12,340 Commits 326 Branches 371 Tags
Commit Graph

1539 Commits

Author SHA1 Message Date
Jason Lowe
ea67e1f266 MAPREDUCE-6927. MR job should only set tracking url if history was successfully written. Contributed by Eric Badger 
(cherry picked from commit 735fce5bec17f4e1799daf922625c475cf588114)

Conflicts:
	hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-app/src/main/java/org/apache/hadoop/mapreduce/v2/app/MRAppMaster.java
 2017-08-08 15:03:34 -05:00
Ravi Prakash
d6228fbebd MAPREDUCE-6910. MapReduceTrackingUriPlugin can not return the right URI of history server with HTTPS. Contributed by Lantao Jin 
(cherry picked from commit 43f0503286eccbc6bb8ae77584b635bfd0c48e50)
(cherry picked from commit 756a06814355465c85b9d66f262ee875dd86dbb7)
 2017-07-13 22:57:44 -07:00
Akira Ajisaka
c180005fcf MAPREDUCE-6697. Concurrent task limits should only be applied when necessary. Contributed by Nathan Roberts. 
(cherry picked from commit a5c0476a990ec1e7eb34ce2462a45aa52cc1350d)
 2017-07-13 13:30:33 -07:00
Junping Du
5ad710bc7a MAPREDUCE-6246. DBOutputFormat.java appending extra semicolon to query which is incompatible with DB2. Contributed by ramtin and Gergely Novák. 
(cherry picked from commit f484a6ff602d48413556a1d046670e2003c71c2e)
(cherry picked from commit f823f9fd784fa4944178247a82df24fba03c2051)
 2017-07-07 13:32:21 -07:00
Jason Lowe
c8e6012f62 MAPREDUCE-6911. TestMapreduceConfigFields.testCompareXmlAgainstConfigurationClass fails consistently. Contributed by Eric Badger 
(cherry picked from commit f0435db0891fd51b54241edf5bda42125490a06c)
 2017-07-06 09:24:01 -05:00
Akira Ajisaka
71f49406f2
MAPREDUCE-6909. LocalJobRunner fails when run on a node from multiple users. Contributed by Jason Lowe. 
(cherry picked from commit ac333155fbcbbc50477869fb29c8da124e5968c0)
 2017-07-04 15:26:46 +09:00
Naganarasimha
acc504dc58 MAPREDUCE-6905. HADOOP_JOB_HISTORY_OPTS should be HADOOP_JOB_HISTORYSERVER_OPTS in mapred-config.sh. Contributed by LiXin Ge. 
(cherry picked from commit bf1f59937dd5d860c9ed7fefce203d6a9f645182)
 2017-07-02 16:18:30 +05:30
Jason Lowe
e47ec9413a Revert "MAPREDUCE-6898. TestKill.testKillTask is flaky. Contributed by Peter Bacsko" 
This reverts commit ea26ccb7dd9026f0bbc5101c97581952d15a3a74.
 2017-06-16 14:49:56 -05:00
Jason Lowe
ea26ccb7dd MAPREDUCE-6898. TestKill.testKillTask is flaky. Contributed by Peter Bacsko 
(cherry picked from commit e78542f4bea90e8545e9e06af9c07e89d2be43d9)
 2017-06-16 14:26:36 -05:00
Vinod Kumar Vavilapalli (I am also known as @tshooter.)
396de0fd8f Preparing for 2.8.3 development 2017-06-07 15:38:20 -07:00
Brahma Reddy Battula
c60516aee5 MAPREDUCE-6676. NNBench should Throw IOException when rename and delete fails. Contributed by Brahma Reddy Battula. 
(cherry picked from commit 24181f559f008fc7945702c0fad7d1028f75bfc6)
 2017-06-07 16:38:05 +08:00
Jason Lowe
030d521a69 MAPREDUCE-6882. Increase MapReduce test timeouts from 1 second to 10 seconds. Contributed by Eric Badger 
(cherry picked from commit 1e71fe8c42b7945170c47023e82a0412397ff57e)
 2017-05-10 09:59:10 -05:00
Junping Du
dfac95a3a2 Preparing for 2.8.1 development 2017-04-11 11:21:49 -07:00
Jason Lowe
a351a30d9d MAPREDUCE-6850. Shuffle Handler keep-alive connections are closed from the server side. Contributed by Jonathan Eagles 
(cherry picked from commit c8bd5fc7a86f9890ceaa37a89491ab650e7e9a64)

Conflicts:
	hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-shuffle/src/main/java/org/apache/hadoop/mapred/ShuffleHandler.java
	hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-shuffle/src/test/java/org/apache/hadoop/mapred/TestShuffleHandler.java
 2017-03-30 11:15:25 -05:00
Zhe Zhang
0acf2bca9c MAPREDUCE-6873. MR Job Submission Fails if MR framework application path not on defaultFS. Contributed by Erik Krogen. 
(cherry picked from commit de7efd2687ccb74608249abf4c83c84a9345ec0a)
(cherry picked from commit 3f5dc51ec94af5a20a319dd9bc807fc9e4db4f6a)
 2017-03-29 22:19:50 -07:00
Joe Mészáros
9909174677
MAPREDUCE-6866. Fix getNumMapTasks() documentation in JobConf. 
This closes #205

Signed-off-by: Akira Ajisaka <aajisaka@apache.org>
(cherry picked from commit e0a2eb71e3eb3efa865068089b649624132d33a1)
(cherry picked from commit 34a11b910896f9c543dab04e3a1ca88ea481b9f3)
 2017-03-27 13:53:08 +09:00
Akira Ajisaka
dd5d0103df
Treat encrypted files as private. Contributed by Daniel Templeton. 
(cherry picked from commit f01a69f84f4cc7d925d078a7ce32e5800da4e429)
(cherry picked from commit 120f680318d766f44787fd0eec88270d61172523)
 2017-03-07 13:38:51 +09:00
Akira Ajisaka
8b5307c0e7
MAPREDUCE-6855. Specify charset when create String in CredentialsTestJob. Contributed by Kai Sasaki. 
(cherry picked from commit 14413989cac9acc1fa6f8d330fac32f772613325)
(cherry picked from commit aa1f944bba02dcb9a14d9d37cfc113845d437ffa)
 2017-03-07 13:13:52 +09:00
Akira Ajisaka
c6bf43e0be MAPREDUCE-6753. Variable in byte printed directly in mapreduce client. Contributed by Kai Sasaki. 
(cherry picked from commit 19a1fc6373f0dea56e5845f19990d469428489bc)
(cherry picked from commit e2fb9538f0e2855d757bcbd586c37278824700ea)
 2017-03-03 16:14:03 +09:00
Akira Ajisaka
6833b4ea30 MAPREDUCE-6841. Fix dead link in MapReduce tutorial document. Contributed by Victor Nee. 
(cherry picked from commit 4d33683882cc41939c512db221fd8ee59ecd52a8)
(cherry picked from commit 69f2ae449b28530d01b73a172f2be6c256dca800)
 2017-02-27 19:47:35 +09:00
Wangda Tan
920847edce YARN-6143. Fix incompatible issue caused by YARN-3583. (Sunil G via wangda) 
(cherry picked from commit 0566b30875dfc61dbbba4cde0d42e15b116e85b8)
 2017-02-22 11:21:57 -08:00
Jason Lowe
63275b7163 MAPREDUCE-6831. Flaky test TestJobImpl.testKilledDuringKillAbort. Contributed by Peter Bacsko 
(cherry picked from commit 78b487bde175544ebe40e4dafab35569baa1d79e)
 2017-01-17 16:05:00 +00:00
Junping Du
7706a63fb4 Preparing for 2.8.1 development after cutting off 2.8.0 branch. 2017-01-10 18:03:51 -08:00
Junping Du
031ad740b9 MAPREDUCE-6711. JobImpl fails to handle preemption events on state COMMITTING. Contributed by Prabhu Joseph. 
(cherry picked from commit 679478d0c643defb3759a2ecd3c704315edee124)
(cherry picked from commit bf87c523ab5ffd1d5a94e74639be5de7821a6c5b)
 2017-01-07 22:08:07 -08:00
Jason Lowe
8ffe5eb7bd MAPREDUCE-6682. TestMRCJCFileOutputCommitter fails intermittently Contributed by Akira Ajisaka. 
(cherry picked from commit 8f1c374bec2451568f463ab68f7eb6db734ab14e)

Conflicts:

	hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-jobclient/src/test/java/org/apache/hadoop/mapred/TestMRTimelineEventHandling.java
 2017-01-06 18:55:48 +00:00
Rohith Sharma K S
0c1615c368 MAPREDUCE-6662. Clear ASF Warnings on test data files. Contributed by Vinayakumar B 
(cherry picked from commit 0ef8bbfd8791899c7cfed3dd9c1670182fd87575)
(cherry picked from commit f22ce96b50f8148e95a453ff41f84b99a9197bf6)
 2017-01-05 18:03:48 +09:00
Jian He
42d1b38989 YARN-4164. Changed updateApplicationPriority API to return the updated application priority. Contributed by Rohith Sharma K S 2017-01-04 15:43:06 -08:00
Varun Saxena
2b4950b094 MAPREDUCE-6801. Fix flaky TestKill.testKillJob (Haibo Chen via Varun Saxena) 
(cherry picked from commit 7584fbf4cbafd34fac4b362cefe4e06cec16a2af)
 2017-01-04 18:29:21 +00:00
Robert Kanter
d738f4b652 MAPREDUCE-6675. TestJobImpl.testUnusableNode failed (haibochen via rkanter) 
(cherry picked from commit 9d3fcdfbb314c83ba6185e4ac8de649dad51a279)
 2017-01-04 18:18:29 +00:00
Akira Ajisaka
58a1753373 HADOOP-12733. Remove references to obsolete io.seqfile configuration variables. Contributed by Ray Chiang. 
(cherry picked from commit 01d31fe9389ccdc153d7f4bf6574bf8e509867c1)
(cherry picked from commit 3e2d26a97f83461d491180bbc6aa0ba926c0681b)
 2017-01-04 14:20:04 +09:00
Brahma Reddy Battula
b8be8f0d4f HADOOP-13883. Addendum patch to correct the message and alphabetized with the earlier patch. Contributed by Yiqun Lin. 2016-12-31 16:27:17 +05:30
Junping Du
5f36b62265 Revert "MAPREDUCE-5875. Make Counter limits consistent across JobClient, MRAppMaster, and YarnChild. (Gera Shegalov via kasha)" 
This reverts commit 7bfd9e068d205f752c05d51778a4a4702329d31e.

Conflicts:
	hadoop-mapreduce-project/CHANGES.txt
	hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-core/src/main/java/org/apache/hadoop/mapreduce/JobSubmitter.java
	hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-core/src/main/java/org/apache/hadoop/mapreduce/jobhistory/HistoryViewer.java
	hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-hs/src/main/java/org/apache/hadoop/mapreduce/v2/hs/CompletedJob.java
 2016-12-28 17:37:20 -08:00
Junping Du
8755a6f5ac Revert "MAPREDUCE-6199. AbstractCounters are not reset completely on deserialization (adhoot via rkanter)" 
This reverts commit f9341c1e2cffc4e3ac37cb7da4e535ca5c6d3913.

Conflicts:
	hadoop-mapreduce-project/CHANGES.txt
 2016-12-28 17:13:04 -08:00
Junping Du
e8d7801319 Revert "MAPREDUCE-6286. A typo in HistoryViewer makes some code useless, which causes counter limits are not reset correctly. Contributed by Zhihai Xu." 
This reverts commit 7c72c7f5290f0991f65102c2bd413f965a8ca409.

Conflicts:
	hadoop-mapreduce-project/CHANGES.txt
 2016-12-28 17:05:17 -08:00
Akira Ajisaka
8e4479dda3 YARN-4994. Use MiniYARNCluster with try-with-resources in tests. Contributed by Andras Bokor. 
(cherry picked from commit ae401539eaf7745ec8690f9281726fb4cdcdbe94)
(cherry picked from commit 522bc98026c6a4a96bbf5246d9d0fbed23401eb6)

 Conflicts:
	hadoop-yarn-project/hadoop-yarn/hadoop-yarn-client/src/test/java/org/apache/hadoop/yarn/client/TestHedgingRequestRMFailoverProxyProvider.java
	hadoop-yarn-project/hadoop-yarn/hadoop-yarn-client/src/test/java/org/apache/hadoop/yarn/client/cli/TestYarnCLI.java
 2016-12-23 04:15:39 +09:00
Akira Ajisaka
0c58241b22 MAPREDUCE-6820. Fix dead links in Job relevant classes. Contributed by Yiqun Lin. 
(cherry picked from commit 92a8917ca5a25bf6bee75acbb554aae87766a7e7)
(cherry picked from commit 1e1ab9bc1bde48593eceafb7380d7501d4bbc200)
 2016-12-10 10:57:33 +09:00
Akira Ajisaka
460fe6594a MAPREDUCE-6816. Progress bars in Web UI always at 100%. Contributed by Shen Yinjie. 
(cherry picked from commit da4ecc9f8223ac82bff8483594fb29be2f8d4d34)
(cherry picked from commit 03854f0bc51c3357161dd7cefb5acd3487868546)
 2016-12-07 06:44:31 +09:00
Jason Lowe
69f66424b9 MAPREDUCE-6815. Fix flaky TestKill.testKillTask(). Contributed by Haibo Chen 
(cherry picked from commit 0cfd7ad21f4457513ed3416e5d77f3123bfe9da0)
 2016-12-02 17:23:33 +00:00
Junping Du
7f4bd1af52 Revert "MAPREDUCE-6565. Configuration to use host name in delegation token service is not read from job.xml during MapReduce job execution. Contributed by Li Lu." 
This reverts commit f2284f44ddaa086d60b1a917b0bfff461a2f9287.
 2016-11-30 11:23:00 -08:00
Junping Du
f2284f44dd MAPREDUCE-6565. Configuration to use host name in delegation token service is not read from job.xml during MapReduce job execution. Contributed by Li Lu. 
(cherry picked from commit 8f6e14399a3e77e1bdcc5034f7601e9f62163dea)
(cherry picked from commit 376a2439db9cd55ed1f2965cd876b811109990ba)
(cherry picked from commit 05a570a1981597ab8d2615ed4752e63f0a21b32b)
 2016-11-29 16:52:11 -08:00
Jason Lowe
e27fbcb50c MAPREDUCE-6804. Add timeout when starting JobHistoryServer in MiniMRYarnCluster. Contributed by Andras Bokor 
(cherry picked from commit 41f8275c50b34c919cca26510d505775d676ebb4)
 2016-11-16 15:27:32 +00:00
Zhe Zhang
efc0e640ed Addendum of HDFS-10872. Removed chan ge in MiniHadoopClusterManager.java 2016-11-14 16:43:43 -08:00
Zhe Zhang
c6e7b631bd HDFS-10872. Add MutableRate metrics for FSNamesystemLock operations. Contributed by Erik Krogen. 
(cherry picked from commit 818915aabb22653e212f4646b1b3cd0673330328)
 2016-11-14 16:39:26 -08:00
Jason Lowe
5b1b825ac8 MAPREDUCE-6797. Job history server scans can become blocked on a single, slow entry. Contributed by Prabhu Joseph 
(cherry picked from commit 99c2bbd337942e4bc7b246a88dff53f98e530651)
 2016-11-14 20:31:38 +00:00
Akira Ajisaka
61128c50de MAPREDUCE-6795. Update the document for JobConf#setNumReduceTasks. Contributed by Yiqun Lin. 
(cherry picked from commit 235f39b7c44c5907bd4de063892905577c35cd80)
(cherry picked from commit a3897f77aefaf0a9823b31a1d3e8f4d1eded8642)
 2016-11-01 20:52:52 +09:00
Akira Ajisaka
757b38ea43 MAPREDUCE-6799. Document mapreduce.jobhistory.webapp.https.address in mapred-default.xml. Contributed by Yiqun Lin. 
(cherry picked from commit f646fe34be704a22cbea322cce2826f8f201bd34)
(cherry picked from commit ef13a5ef9d7859cbb720ee46ebed8172358ad0d5)
 2016-10-31 15:19:37 +09:00
Jason Lowe
0a405c4f71 MAPREDUCE-6724. Single shuffle to memory must not exceed Integer#MAX_VALUE. (Haibo Chen via gera) 
(cherry picked from commit 6890d5b472320fa7592ed1b08b623c55a27089c6)

Conflicts:

	hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-core/src/test/java/org/apache/hadoop/mapreduce/task/reduce/TestMergeManager.java
 2016-10-28 14:40:28 +00:00
Naganarasimha
8a664eba7d MAPREDUCE-2631. Potential resource leaks in BinaryProtocol$TeeOutputStream.java. Contributed by Sunil G. 
(cherry picked from commit 28660f51af161a9fa301523d96a6f8ae4ebd6edd)
 2016-10-28 06:04:03 +05:30
Naganarasimha
27029393b7 MAPREDUCE-6541. Exclude scheduled reducer memory when calculating available mapper slots from headroom to avoid deadlock. Contributed by Varun Saxena 
(cherry picked from commit 060558c6f221ded0b014189d5b82eee4cc7b576b)
 2016-10-27 18:06:34 +05:30
Karthik Kambatla
c955a59d63 MAPREDUCE-6740. Enforce mapreduce.task.timeout to be at least mapreduce.task.progress-report.interval. (Haibo Chen via kasha) 2016-10-05 23:12:45 -07:00