Apache/hadoop
1
0
Fork 0
Code Issues Pull Requests Packages Projects Releases Wiki Activity
11,971 Commits 326 Branches 371 Tags
Commit Graph

2196 Commits

Author SHA1 Message Date
Junping Du
7706a63fb4 Preparing for 2.8.1 development after cutting off 2.8.0 branch. 2017-01-10 18:03:51 -08:00
Junping Du
031ad740b9 MAPREDUCE-6711. JobImpl fails to handle preemption events on state COMMITTING. Contributed by Prabhu Joseph. 
(cherry picked from commit 679478d0c643defb3759a2ecd3c704315edee124)
(cherry picked from commit bf87c523ab5ffd1d5a94e74639be5de7821a6c5b)
 2017-01-07 22:08:07 -08:00
Jason Lowe
8ffe5eb7bd MAPREDUCE-6682. TestMRCJCFileOutputCommitter fails intermittently Contributed by Akira Ajisaka. 
(cherry picked from commit 8f1c374bec2451568f463ab68f7eb6db734ab14e)

Conflicts:

	hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-jobclient/src/test/java/org/apache/hadoop/mapred/TestMRTimelineEventHandling.java
 2017-01-06 18:55:48 +00:00
Rohith Sharma K S
0c1615c368 MAPREDUCE-6662. Clear ASF Warnings on test data files. Contributed by Vinayakumar B 
(cherry picked from commit 0ef8bbfd8791899c7cfed3dd9c1670182fd87575)
(cherry picked from commit f22ce96b50f8148e95a453ff41f84b99a9197bf6)
 2017-01-05 18:03:48 +09:00
Jian He
42d1b38989 YARN-4164. Changed updateApplicationPriority API to return the updated application priority. Contributed by Rohith Sharma K S 2017-01-04 15:43:06 -08:00
Varun Saxena
2b4950b094 MAPREDUCE-6801. Fix flaky TestKill.testKillJob (Haibo Chen via Varun Saxena) 
(cherry picked from commit 7584fbf4cbafd34fac4b362cefe4e06cec16a2af)
 2017-01-04 18:29:21 +00:00
Robert Kanter
d738f4b652 MAPREDUCE-6675. TestJobImpl.testUnusableNode failed (haibochen via rkanter) 
(cherry picked from commit 9d3fcdfbb314c83ba6185e4ac8de649dad51a279)
 2017-01-04 18:18:29 +00:00
Akira Ajisaka
58a1753373 HADOOP-12733. Remove references to obsolete io.seqfile configuration variables. Contributed by Ray Chiang. 
(cherry picked from commit 01d31fe9389ccdc153d7f4bf6574bf8e509867c1)
(cherry picked from commit 3e2d26a97f83461d491180bbc6aa0ba926c0681b)
 2017-01-04 14:20:04 +09:00
Brahma Reddy Battula
b8be8f0d4f HADOOP-13883. Addendum patch to correct the message and alphabetized with the earlier patch. Contributed by Yiqun Lin. 2016-12-31 16:27:17 +05:30
Junping Du
5f36b62265 Revert "MAPREDUCE-5875. Make Counter limits consistent across JobClient, MRAppMaster, and YarnChild. (Gera Shegalov via kasha)" 
This reverts commit 7bfd9e068d205f752c05d51778a4a4702329d31e.

Conflicts:
	hadoop-mapreduce-project/CHANGES.txt
	hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-core/src/main/java/org/apache/hadoop/mapreduce/JobSubmitter.java
	hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-core/src/main/java/org/apache/hadoop/mapreduce/jobhistory/HistoryViewer.java
	hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-hs/src/main/java/org/apache/hadoop/mapreduce/v2/hs/CompletedJob.java
 2016-12-28 17:37:20 -08:00
Junping Du
8755a6f5ac Revert "MAPREDUCE-6199. AbstractCounters are not reset completely on deserialization (adhoot via rkanter)" 
This reverts commit f9341c1e2cffc4e3ac37cb7da4e535ca5c6d3913.

Conflicts:
	hadoop-mapreduce-project/CHANGES.txt
 2016-12-28 17:13:04 -08:00
Junping Du
e8d7801319 Revert "MAPREDUCE-6286. A typo in HistoryViewer makes some code useless, which causes counter limits are not reset correctly. Contributed by Zhihai Xu." 
This reverts commit 7c72c7f5290f0991f65102c2bd413f965a8ca409.

Conflicts:
	hadoop-mapreduce-project/CHANGES.txt
 2016-12-28 17:05:17 -08:00
Akira Ajisaka
8e4479dda3 YARN-4994. Use MiniYARNCluster with try-with-resources in tests. Contributed by Andras Bokor. 
(cherry picked from commit ae401539eaf7745ec8690f9281726fb4cdcdbe94)
(cherry picked from commit 522bc98026c6a4a96bbf5246d9d0fbed23401eb6)

 Conflicts:
	hadoop-yarn-project/hadoop-yarn/hadoop-yarn-client/src/test/java/org/apache/hadoop/yarn/client/TestHedgingRequestRMFailoverProxyProvider.java
	hadoop-yarn-project/hadoop-yarn/hadoop-yarn-client/src/test/java/org/apache/hadoop/yarn/client/cli/TestYarnCLI.java
 2016-12-23 04:15:39 +09:00
Akira Ajisaka
0c58241b22 MAPREDUCE-6820. Fix dead links in Job relevant classes. Contributed by Yiqun Lin. 
(cherry picked from commit 92a8917ca5a25bf6bee75acbb554aae87766a7e7)
(cherry picked from commit 1e1ab9bc1bde48593eceafb7380d7501d4bbc200)
 2016-12-10 10:57:33 +09:00
Akira Ajisaka
460fe6594a MAPREDUCE-6816. Progress bars in Web UI always at 100%. Contributed by Shen Yinjie. 
(cherry picked from commit da4ecc9f8223ac82bff8483594fb29be2f8d4d34)
(cherry picked from commit 03854f0bc51c3357161dd7cefb5acd3487868546)
 2016-12-07 06:44:31 +09:00
Jason Lowe
69f66424b9 MAPREDUCE-6815. Fix flaky TestKill.testKillTask(). Contributed by Haibo Chen 
(cherry picked from commit 0cfd7ad21f4457513ed3416e5d77f3123bfe9da0)
 2016-12-02 17:23:33 +00:00
Junping Du
7f4bd1af52 Revert "MAPREDUCE-6565. Configuration to use host name in delegation token service is not read from job.xml during MapReduce job execution. Contributed by Li Lu." 
This reverts commit f2284f44ddaa086d60b1a917b0bfff461a2f9287.
 2016-11-30 11:23:00 -08:00
Junping Du
f2284f44dd MAPREDUCE-6565. Configuration to use host name in delegation token service is not read from job.xml during MapReduce job execution. Contributed by Li Lu. 
(cherry picked from commit 8f6e14399a3e77e1bdcc5034f7601e9f62163dea)
(cherry picked from commit 376a2439db9cd55ed1f2965cd876b811109990ba)
(cherry picked from commit 05a570a1981597ab8d2615ed4752e63f0a21b32b)
 2016-11-29 16:52:11 -08:00
Jason Lowe
e27fbcb50c MAPREDUCE-6804. Add timeout when starting JobHistoryServer in MiniMRYarnCluster. Contributed by Andras Bokor 
(cherry picked from commit 41f8275c50b34c919cca26510d505775d676ebb4)
 2016-11-16 15:27:32 +00:00
Zhe Zhang
efc0e640ed Addendum of HDFS-10872. Removed chan ge in MiniHadoopClusterManager.java 2016-11-14 16:43:43 -08:00
Zhe Zhang
c6e7b631bd HDFS-10872. Add MutableRate metrics for FSNamesystemLock operations. Contributed by Erik Krogen. 
(cherry picked from commit 818915aabb22653e212f4646b1b3cd0673330328)
 2016-11-14 16:39:26 -08:00
Jason Lowe
5b1b825ac8 MAPREDUCE-6797. Job history server scans can become blocked on a single, slow entry. Contributed by Prabhu Joseph 
(cherry picked from commit 99c2bbd337942e4bc7b246a88dff53f98e530651)
 2016-11-14 20:31:38 +00:00
Akira Ajisaka
61128c50de MAPREDUCE-6795. Update the document for JobConf#setNumReduceTasks. Contributed by Yiqun Lin. 
(cherry picked from commit 235f39b7c44c5907bd4de063892905577c35cd80)
(cherry picked from commit a3897f77aefaf0a9823b31a1d3e8f4d1eded8642)
 2016-11-01 20:52:52 +09:00
Akira Ajisaka
757b38ea43 MAPREDUCE-6799. Document mapreduce.jobhistory.webapp.https.address in mapred-default.xml. Contributed by Yiqun Lin. 
(cherry picked from commit f646fe34be704a22cbea322cce2826f8f201bd34)
(cherry picked from commit ef13a5ef9d7859cbb720ee46ebed8172358ad0d5)
 2016-10-31 15:19:37 +09:00
Jason Lowe
0a405c4f71 MAPREDUCE-6724. Single shuffle to memory must not exceed Integer#MAX_VALUE. (Haibo Chen via gera) 
(cherry picked from commit 6890d5b472320fa7592ed1b08b623c55a27089c6)

Conflicts:

	hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-core/src/test/java/org/apache/hadoop/mapreduce/task/reduce/TestMergeManager.java
 2016-10-28 14:40:28 +00:00
Naganarasimha
8a664eba7d MAPREDUCE-2631. Potential resource leaks in BinaryProtocol$TeeOutputStream.java. Contributed by Sunil G. 
(cherry picked from commit 28660f51af161a9fa301523d96a6f8ae4ebd6edd)
 2016-10-28 06:04:03 +05:30
Naganarasimha
27029393b7 MAPREDUCE-6541. Exclude scheduled reducer memory when calculating available mapper slots from headroom to avoid deadlock. Contributed by Varun Saxena 
(cherry picked from commit 060558c6f221ded0b014189d5b82eee4cc7b576b)
 2016-10-27 18:06:34 +05:30
Karthik Kambatla
c955a59d63 MAPREDUCE-6740. Enforce mapreduce.task.timeout to be at least mapreduce.task.progress-report.interval. (Haibo Chen via kasha) 2016-10-05 23:12:45 -07:00
Jason Lowe
f1b74a3d9f MAPREDUCE-6741. Add MR support to redact job conf properties. Contributed by Haibo Chen 2016-10-05 15:24:10 +00:00
Jason Lowe
db93a64eca MAPREDUCE-6771. RMContainerAllocator sends container diagnostics event after corresponding completion event. Contributed by Haibo Chen 
(cherry picked from commit c52ad9ee86f5033caca02a7af6aeccfc5c87a99e)
 2016-09-29 15:51:36 +00:00
Wangda Tan
69f91d8c48 HADOOP-13544. JDiff reports unncessarily show unannotated APIs and cause confusion while our javadocs only show annotated and public APIs. (vinodkv via wangda) 
(cherry picked from commit 875062b5bc789158290bf93dadc71b5328ca4fee)
(cherry picked from commit 269401dc83699e4de7b99e794eeb7646a7f3fc93)
 2016-09-27 11:46:49 -07:00
Naganarasimha
fa2025316d YARN-3692. Allow REST API to set a user generated message when killing an application. Contributed by Rohith Sharma K S 2016-09-24 22:42:28 +05:30
Arun Suresh
979b29a03c YARN-5221. Expose UpdateResourceRequest API to allow AM to request for change in container properties. (asuresh) 
(cherry picked from commit d6d9cff21b7b6141ed88359652cf22e8973c0661)
(cherry picked from commit b279f42d79175bef6529dc1ac4216198a3aaee4d)
 2016-08-31 20:06:49 -07:00
Jason Lowe
9df118e5fe MAPREDUCE-4784. TestRecovery occasionally fails. Contributed by Haibo Chen 
(cherry picked from commit af508605a9edc126c170160291dbc2fe58b66dea)

Conflicts:

	hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-app/src/test/java/org/apache/hadoop/mapreduce/v2/app/TestRecovery.java
(cherry picked from commit f70f522e805b2d8dd49bae07fc65d37a78ad2fca)
 2016-08-30 14:09:45 +00:00
Jason Lowe
f6578999d1 MAPREDUCE-6768. TestRecovery.testSpeculative failed with NPE. Contributed by Haibo Chen 
(cherry picked from commit 44d0af2611cb565d8ec68f92e27937342646cca5)
 2016-08-29 19:58:23 +00:00
Wei-Chiu Chuang
8d467037cf MAPREDUCE-6764. Teragen LOG initialization bug. Contributed by Yufei Gu. 
(cherry picked from commit 1360bd2d545134b582e70f2add33a105710dc80b)
(cherry picked from commit 13cbf1677ef5ba82b16d9a5ab1661333a492226d)
 2016-08-25 09:54:57 -07:00
Jason Lowe
00c8ab91ba MAPREDUCE-6761. Regression when handling providers - invalid configuration ServiceConfiguration causes Cluster initialization failure. Contributed by Peter Vary 
(cherry picked from commit 6fa9bf4407a0b09be8ce4e52d2fde89adba34f1a)
 2016-08-24 14:42:53 +00:00
Akira Ajisaka
854af9cee1 MAPREDUCE-6587. Remove unused params in connection-related methods of Fetcher. Contributed by Yiqun Lin. 
(cherry picked from commit 8cc4a67059e37b2083cd5468b35a64a403a3e3ae)
(cherry picked from commit 41fe85f8688e6edd2a94ffd831d776e7c7a3614f)
 2016-08-23 17:12:32 +09:00
Varun Saxena
e714d82196 MAPREDUCE-6750. Fix TestHSAdminServer#testRefreshSuperUserGroups (Kihwal Lee via Varun Saxena) 2016-08-22 11:05:01 -05:00
Wangda Tan
e2bd490236 MAPREDUCE-6310. Add jdiff support to MapReduce. (Li Lu/vinodkv via wangda) 
(cherry picked from commit 3d937457ee60d617d71b36bb6a29344e771670b4)
(cherry picked from commit 65822e55fd99511ab0c39f802d3ec58290445f9d)
 2016-08-19 16:49:54 -07:00
Varun Saxena
6f1a9f1b6a MAPREDUCE-6763. Shuffle server listen queue is too small (Jason Lowe via Varun Saxena) 2016-08-20 04:08:55 +05:30
Akira Ajisaka
59693ebbd5 MAPREDUCE-6730. Use StandardCharsets instead of String overload in TextOutputFormat. Contributed by Sahil Kang. 
This closes #114

(cherry picked from commit 70c278115249898132490a89a548fd936c09f54b)
(cherry picked from commit e54de94e5ab40405d837ee7664e90921f5bc38e4)
 2016-08-04 20:18:37 +09:00
Akira Ajisaka
fa0b6ff97b MAPREDUCE-6729. Accurately compute the test execute time in DFSIO. Contributed by mingleizhang. 
This closes #112

(cherry picked from commit ce93595d7a18bd0561ae2c1983b51f6aba887179)
(cherry picked from commit 96d5607e7a2e6deebdb76858955127d09c125729)
 2016-07-30 22:16:43 +09:00
Akira Ajisaka
19062df92f MAPREDUCE-6746. Replace org.apache.commons.io.Charsets with java.nio.charset.StandardCharsets. Contributed by Vincent Poon. 
(cherry picked from commit 3f0bffea2d35083c45acee79338e10b608374d8e)
(cherry picked from commit 0fc838540ba74a12336191388661e41fc3b77928)
 2016-07-30 11:47:33 +09:00
Jason Lowe
e098a732ef MAPREDUCE-6744. Increase timeout on TestDFSIO tests. Contributed by Eric Badger 
(cherry picked from commit 703fdf86c6e42d6d555090f670ca5cbd1ef48bf9)
 2016-07-25 21:56:33 +00:00
Akira Ajisaka
3dcbd2df56 MAPREDUCE-6738. TestJobListCache.testAddExisting failed intermittently in slow VM testbed. Contributed by Junping Du. 
(cherry picked from commit ecff7d03595b45d7a5d6aed73b9931fef85a56ee)
(cherry picked from commit f4d94f2246da3a015a105cb7f55ec1ebc451e0e7)
 2016-07-21 11:38:26 -07:00
Robert Kanter
ba83b62d36 MAPREDUCE-6625. TestCLI#testGetJob fails occasionally (haibochen via rkanter) 
(cherry picked from commit ad256beb50e8851d5e6e5e743b00e819e75e3549)
 2016-07-12 16:31:24 -07:00
Jian He
23eb3c7ceb YARN-5270. Solve miscellaneous issues caused by YARN-4844. Contributed by Wangda Tan 2016-07-11 22:38:35 -07:00
Akira Ajisaka
5522a46f4c HADOOP-13320. Fix arguments check in documentation for WordCount v2.0. Contributed by niccolo becchi. 
(cherry picked from commit 9d46a49c746b9e1ef552dbb10d1e22f87db68c76)
(cherry picked from commit b3b81b4f0e1cd0b37ab2dfdf3e7301e0ef3e0676)
 2016-07-07 15:32:20 -07:00
Akira Ajisaka
a0f6109907 MAPREDUCE-6542. HistoryViewer uses SimpleDateFormat, but SimpleDateFormat is not threadsafe. Contributed by zhangyubiao. 2016-06-27 15:19:21 -07:00