Apache/hadoop
1
0
Fork 0
Code Issues Pull Requests Packages Projects Releases Wiki Activity
12,311 Commits 326 Branches 371 Tags
Commit Graph

628 Commits

Author SHA1 Message Date
Akira Ajisaka
c180005fcf MAPREDUCE-6697. Concurrent task limits should only be applied when necessary. Contributed by Nathan Roberts. 
(cherry picked from commit a5c0476a990ec1e7eb34ce2462a45aa52cc1350d)
 2017-07-13 13:30:33 -07:00
Jason Lowe
c8e6012f62 MAPREDUCE-6911. TestMapreduceConfigFields.testCompareXmlAgainstConfigurationClass fails consistently. Contributed by Eric Badger 
(cherry picked from commit f0435db0891fd51b54241edf5bda42125490a06c)
 2017-07-06 09:24:01 -05:00
Jason Lowe
e47ec9413a Revert "MAPREDUCE-6898. TestKill.testKillTask is flaky. Contributed by Peter Bacsko" 
This reverts commit ea26ccb7dd9026f0bbc5101c97581952d15a3a74.
 2017-06-16 14:49:56 -05:00
Jason Lowe
ea26ccb7dd MAPREDUCE-6898. TestKill.testKillTask is flaky. Contributed by Peter Bacsko 
(cherry picked from commit e78542f4bea90e8545e9e06af9c07e89d2be43d9)
 2017-06-16 14:26:36 -05:00
Vinod Kumar Vavilapalli (I am also known as @tshooter.)
396de0fd8f Preparing for 2.8.3 development 2017-06-07 15:38:20 -07:00
Junping Du
dfac95a3a2 Preparing for 2.8.1 development 2017-04-11 11:21:49 -07:00
Jason Lowe
63275b7163 MAPREDUCE-6831. Flaky test TestJobImpl.testKilledDuringKillAbort. Contributed by Peter Bacsko 
(cherry picked from commit 78b487bde175544ebe40e4dafab35569baa1d79e)
 2017-01-17 16:05:00 +00:00
Junping Du
7706a63fb4 Preparing for 2.8.1 development after cutting off 2.8.0 branch. 2017-01-10 18:03:51 -08:00
Junping Du
031ad740b9 MAPREDUCE-6711. JobImpl fails to handle preemption events on state COMMITTING. Contributed by Prabhu Joseph. 
(cherry picked from commit 679478d0c643defb3759a2ecd3c704315edee124)
(cherry picked from commit bf87c523ab5ffd1d5a94e74639be5de7821a6c5b)
 2017-01-07 22:08:07 -08:00
Varun Saxena
2b4950b094 MAPREDUCE-6801. Fix flaky TestKill.testKillJob (Haibo Chen via Varun Saxena) 
(cherry picked from commit 7584fbf4cbafd34fac4b362cefe4e06cec16a2af)
 2017-01-04 18:29:21 +00:00
Robert Kanter
d738f4b652 MAPREDUCE-6675. TestJobImpl.testUnusableNode failed (haibochen via rkanter) 
(cherry picked from commit 9d3fcdfbb314c83ba6185e4ac8de649dad51a279)
 2017-01-04 18:18:29 +00:00
Junping Du
5f36b62265 Revert "MAPREDUCE-5875. Make Counter limits consistent across JobClient, MRAppMaster, and YarnChild. (Gera Shegalov via kasha)" 
This reverts commit 7bfd9e068d205f752c05d51778a4a4702329d31e.

Conflicts:
	hadoop-mapreduce-project/CHANGES.txt
	hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-core/src/main/java/org/apache/hadoop/mapreduce/JobSubmitter.java
	hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-core/src/main/java/org/apache/hadoop/mapreduce/jobhistory/HistoryViewer.java
	hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-hs/src/main/java/org/apache/hadoop/mapreduce/v2/hs/CompletedJob.java
 2016-12-28 17:37:20 -08:00
Akira Ajisaka
8e4479dda3 YARN-4994. Use MiniYARNCluster with try-with-resources in tests. Contributed by Andras Bokor. 
(cherry picked from commit ae401539eaf7745ec8690f9281726fb4cdcdbe94)
(cherry picked from commit 522bc98026c6a4a96bbf5246d9d0fbed23401eb6)

 Conflicts:
	hadoop-yarn-project/hadoop-yarn/hadoop-yarn-client/src/test/java/org/apache/hadoop/yarn/client/TestHedgingRequestRMFailoverProxyProvider.java
	hadoop-yarn-project/hadoop-yarn/hadoop-yarn-client/src/test/java/org/apache/hadoop/yarn/client/cli/TestYarnCLI.java
 2016-12-23 04:15:39 +09:00
Akira Ajisaka
460fe6594a MAPREDUCE-6816. Progress bars in Web UI always at 100%. Contributed by Shen Yinjie. 
(cherry picked from commit da4ecc9f8223ac82bff8483594fb29be2f8d4d34)
(cherry picked from commit 03854f0bc51c3357161dd7cefb5acd3487868546)
 2016-12-07 06:44:31 +09:00
Jason Lowe
69f66424b9 MAPREDUCE-6815. Fix flaky TestKill.testKillTask(). Contributed by Haibo Chen 
(cherry picked from commit 0cfd7ad21f4457513ed3416e5d77f3123bfe9da0)
 2016-12-02 17:23:33 +00:00
Junping Du
7f4bd1af52 Revert "MAPREDUCE-6565. Configuration to use host name in delegation token service is not read from job.xml during MapReduce job execution. Contributed by Li Lu." 
This reverts commit f2284f44ddaa086d60b1a917b0bfff461a2f9287.
 2016-11-30 11:23:00 -08:00
Junping Du
f2284f44dd MAPREDUCE-6565. Configuration to use host name in delegation token service is not read from job.xml during MapReduce job execution. Contributed by Li Lu. 
(cherry picked from commit 8f6e14399a3e77e1bdcc5034f7601e9f62163dea)
(cherry picked from commit 376a2439db9cd55ed1f2965cd876b811109990ba)
(cherry picked from commit 05a570a1981597ab8d2615ed4752e63f0a21b32b)
 2016-11-29 16:52:11 -08:00
Naganarasimha
27029393b7 MAPREDUCE-6541. Exclude scheduled reducer memory when calculating available mapper slots from headroom to avoid deadlock. Contributed by Varun Saxena 
(cherry picked from commit 060558c6f221ded0b014189d5b82eee4cc7b576b)
 2016-10-27 18:06:34 +05:30
Karthik Kambatla
c955a59d63 MAPREDUCE-6740. Enforce mapreduce.task.timeout to be at least mapreduce.task.progress-report.interval. (Haibo Chen via kasha) 2016-10-05 23:12:45 -07:00
Jason Lowe
f1b74a3d9f MAPREDUCE-6741. Add MR support to redact job conf properties. Contributed by Haibo Chen 2016-10-05 15:24:10 +00:00
Jason Lowe
db93a64eca MAPREDUCE-6771. RMContainerAllocator sends container diagnostics event after corresponding completion event. Contributed by Haibo Chen 
(cherry picked from commit c52ad9ee86f5033caca02a7af6aeccfc5c87a99e)
 2016-09-29 15:51:36 +00:00
Arun Suresh
979b29a03c YARN-5221. Expose UpdateResourceRequest API to allow AM to request for change in container properties. (asuresh) 
(cherry picked from commit d6d9cff21b7b6141ed88359652cf22e8973c0661)
(cherry picked from commit b279f42d79175bef6529dc1ac4216198a3aaee4d)
 2016-08-31 20:06:49 -07:00
Jason Lowe
9df118e5fe MAPREDUCE-4784. TestRecovery occasionally fails. Contributed by Haibo Chen 
(cherry picked from commit af508605a9edc126c170160291dbc2fe58b66dea)

Conflicts:

	hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-app/src/test/java/org/apache/hadoop/mapreduce/v2/app/TestRecovery.java
(cherry picked from commit f70f522e805b2d8dd49bae07fc65d37a78ad2fca)
 2016-08-30 14:09:45 +00:00
Jason Lowe
f6578999d1 MAPREDUCE-6768. TestRecovery.testSpeculative failed with NPE. Contributed by Haibo Chen 
(cherry picked from commit 44d0af2611cb565d8ec68f92e27937342646cca5)
 2016-08-29 19:58:23 +00:00
Jian He
23eb3c7ceb YARN-5270. Solve miscellaneous issues caused by YARN-4844. Contributed by Wangda Tan 2016-07-11 22:38:35 -07:00
Wangda Tan
d838c6443d YARN-1942. Deprecate toString/fromString methods from ConverterUtils and move them to records classes like ContainerId/ApplicationId, etc. (wangda) 2016-06-14 15:21:41 -07:00
Wangda Tan
19e578870d YARN-4844. Add getMemorySize/getVirtualCoresSize to o.a.h.y.api.records.Resource. (wangda) 2016-06-07 12:41:50 -07:00
Ming Ma
ec4f9a14f9 MAPREDUCE-5044. Have AM trigger jstack on task attempts that timeout before killing them. (Eric Payne and Gera Shegalov via mingma) 
(cherry picked from commit 4a1cedc010d3fa1d8ef3f2773ca12acadfee5ba5)
(cherry picked from commit 74e2b5efa26f27027fed212b4b2108f0e95587fb)
 2016-06-06 14:49:43 -07:00
Wangda Tan
2f3e1d965d Revert "YARn-4844. Add getMemoryLong/getVirtualCoreLong to o.a.h.y.api.records.Resource. Contributed by Wangda Tan." 
This reverts commit 457884737f75c796413ce860b1859a31cc5292ca.
 2016-05-31 22:16:53 -07:00
Varun Vasudev
457884737f YARn-4844. Add getMemoryLong/getVirtualCoreLong to o.a.h.y.api.records.Resource. Contributed by Wangda Tan. 2016-05-29 20:57:56 +05:30
Akira Ajisaka
4422beab72 MAPREDUCE-6607. Enable regex pattern matching when mapreduce.task.files.preserve.filepattern is set. Contributed by Kai Sasaki. 
(cherry picked from commit a59f30272dd2a8901a41c5c4300ce9d196e62606)
 2016-05-23 07:26:14 +09:00
Rohith Sharma K S
726c1f14b8 YARN-3840. Resource Manager web ui issue when sorting application by id (with application having id > 9999). Contributed by Mohammad Shahid Khan and Varun Saxena 2016-05-19 10:50:32 +05:30
Jason Lowe
7fc44311d0 MAPREDUCE-6689. MapReduce job can infinitely increase number of reducer resource requests. Contributed by Wangda Tan 
(cherry picked from commit c9bb96fa81fc925e33ccc0b02c98cc2d929df120)
 2016-05-06 22:28:03 +00:00
Vinod Kumar Vavilapalli
921562720e MAPREDUCE-6514. Fixed MapReduce ApplicationMaster to properly updated resources ask after ramping down of all reducers avoiding job hangs. Contributed by Varun Saxena and Wangda Tan. 
(cherry picked from commit 8d48266720dcf0e71cfd87fef18b60a53aa1bef9)
 2016-05-05 20:43:55 -07:00
Xuan
84ab382939 YARN-4965. Distributed shell AM failed due to ClientHandlerException thrown by jersey. Contributed by Junping Du 
(cherry picked from commit e6c0742012ffeacad2bcaf712d86a7e5d1420b26)
(cherry picked from commit 94e4d349b018496ab25afc164e2f25c780010cfd)
 2016-04-16 19:40:46 -07:00
Wangda Tan
c202ad6721 MAPREDUCE-6513. MR job got hanged forever when one NM unstable for some time. (Varun Saxena via wangda) 2016-04-14 17:19:26 -07:00
Robert Kanter
8da0a490c6 MAPREDUCE-4785. TestMRApp occasionally fails (haibochen via rkanter) 
(cherry picked from commit ff0ee84d77d9438f0954ae4e1497d63997bb7347)
(cherry picked from commit f9f57265cf0ad2685d0d96ec59b8b78ac8bed175)

Conflicts:
	hadoop-mapreduce-project/CHANGES.txt
 2016-04-14 14:54:42 -07:00
Akira Ajisaka
321da0604a Revert "HDFS-9686. Remove useless boxing/unboxing code. Contributed by Kousuke Saruta." 
This reverts commit 3f8a30c956d464b86f91be3f0136ed7e15604f02.
 2016-02-10 23:45:44 +09:00
Akira Ajisaka
f943bf0e22 MAPREDUCE-6626. Reuse ObjectMapper instance in MapReduce. Contributed by Lin Yiqun. 
(cherry picked from commit 10393d9a666419a5d8dc0cfb48da96e9c690c11e)
(cherry picked from commit 8cbe65a07dd0deb150e91b13e3a1cec13e3716ee)
 2016-02-10 03:05:23 +09:00
Akira Ajisaka
3f8a30c956 HDFS-9686. Remove useless boxing/unboxing code. Contributed by Kousuke Saruta. 
(cherry picked from commit fe124da5ffc16e4795c3dd5542accd58361e1b08)
(cherry picked from commit e4c01b8b1ce511de24db00afacef1e737845a4f7)
 2016-02-06 19:33:11 +09:00
Jian He
b6fe3c20c5 MAPREDUCE-6610. JobHistoryEventHandler should not swallow timeline response. Contributed by Li Lu 2016-01-25 23:01:52 -08:00
Akira Ajisaka
8ffa844d48 MAPREDUCE-6614. Remove unnecessary code in TestMapreduceConfigFields. Contributed by Kai Sasaki. 
(cherry picked from commit a9386cae6ef80d10f6427950b8ca9b77d5f86ca3)
 2016-01-25 11:40:08 +09:00
Akira Ajisaka
b85b8d90d1 MAPREDUCE-6605. Fix typos mapreduce.map.skip.proc.count.autoincr and mapreduce.reduce.skip.proc.count.autoincr in mapred-default.xml. Contributed by Kai Sasaki. 
(cherry picked from commit 7f215ffdd5219fc585ec484073b317a940895a60)
 2016-01-22 18:38:08 +09:00
Wangda Tan
00fca06b43 YARN-4582. Label-related invalid resource request exception should be able to properly handled by application. (Bibin A Chundatt via wangda) 
(cherry picked from commit 9e792da01419998c2ebfafd7161070150d85d3ac)
 2016-01-12 12:56:12 +08:00
Akira Ajisaka
5daa07481e MAPREDUCE-6593. TestJobHistoryEventHandler.testTimelineEventHandling fails on trunk because of NPE. Contributed by Naganarasimha G R. 
(cherry picked from commit 7dafee11d865c7c121c7886ac66aa9d088ea13f7)
(cherry picked from commit 6d235f0b91b9ba3041f5465f480278c33f01bd36)
 2016-01-04 16:31:42 +09:00
Wangda Tan
aa85ce73de MAPREDUCE-6574. MR AM should print host of failed tasks. (Mohammad Shahid Khan via wangda) 
(cherry picked from commit 32b3dc11eba8f2d8f22ca502bf30d6aa8839460e)
(cherry picked from commit ca575e6addb801accafa7e3a6dad08d75cf79bdc)
 2015-12-28 11:08:31 -08:00
Junping Du
628c782870 Tests in mapreduce-client-app are writing outside of target. Contributed by Akira AJISAKA. 
(cherry picked from commit 15d577bfbb3f18fc95251d22378b53aa4210115f)
 2015-11-25 09:16:22 -08:00
Akira Ajisaka
c6cf7910a4 MAPREDUCE-6553. Replace '\u2b05' with '<-' in rendering job configuration. Contributed by Gabor Liptak. 
(cherry picked from commit 5ec44fc35c129d266edaebbc6fa17754a89b2ad6)
 2015-11-25 16:33:18 +09:00
Jason Lowe
2f18218508 MAPREDUCE-5870. Support for passing Job priority through Application Submission Context in Mapreduce Side. Contributed by Sunil G 
(cherry picked from commit f634505d48d97e4d461980d68a0cbdf87223646d)

Conflicts:

	hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-core/src/test/java/org/apache/hadoop/mapred/TestJobConf.java
 2015-11-24 22:15:37 +00:00
Jian He
b339a4b8d6 YARN-4349. Support CallerContext in YARN. Contributed by Wangda Tan 
(cherry picked from commit 8676a118a12165ae5a8b80a2a4596c133471ebc1)
 2015-11-23 17:20:24 -08:00