Apache
/
hadoop
1
0
Fork 0
Code Issues Pull Requests Packages Projects Releases Wiki Activity
13,625 Commits 326 Branches 371 Tags 1.5 GiB
Commit Graph

1645 Commits

Author SHA1 Message Date
Eric Payne d2386725ff MAPREDUCE-6960. Shuffle Handler prints disk error stack traces for every read failure. 
(cherry picked from commit 595d478408)
 2017-09-19 10:46:05 -05:00
Jason Lowe 41c2a73c52 MAPREDUCE-6958. Shuffle audit logger should log size of shuffle transfer. Contributed by Jason Lowe 
(cherry picked from commit 3a20debdde)
 2017-09-19 09:14:41 -05:00
Daniel Templeton 1944b0b286 MAPREDUCE-6808. Log map attempts as part of shuffle handler audit log (Contributed by Gergő Pásztor via Daniel Templeton) 
(cherry picked from commit a7463b6c88)
 2017-09-19 09:05:39 -05:00
Jason Lowe f525dfc5ec MAPREDUCE-6957. shuffle hangs after a node manager connection timeout. Contributed by Jooseong Kim 
(cherry picked from commit 4d98936eec)
 2017-09-13 17:22:58 -05:00
Robert Kanter 72823b2738 MAPREDUCE-6953. Skip the testcase testJobWithChangePriority if FairScheduler is used (pbacsko via rkanter) 
(cherry picked from commit a323f73bae)
(cherry picked from commit f5f343a665)
 2017-09-08 13:18:59 -07:00
Haibo Chen 59f1ca7258 MAPREDUCE-6937. Backport MAPREDUCE-6870 to branch-2 while preserving compatibility. (Peter Bacsko via Haibo Chen) 2017-08-31 09:28:19 -07:00
Konstantin V Shvachko 9a0aac4170 MAPREDUCE-6931. Remove TestDFSIO "Total Throughput" calculation. Contributed by Dennis Huo. 
(cherry picked from commit 3e0e2033cc)

Conflicts:
	hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-jobclient/src/test/java/org/apache/hadoop/fs/TestDFSIO.java
 2017-08-30 14:42:36 -07:00
Jason Lowe cd08aa72a6 MAPREDUCE-6641. TestTaskAttempt fails in trunk. Contributed by Haibo Chen. 2017-08-29 12:24:42 -05:00
Robert Kanter 9de332d502 MAPREDUCE-6936. Remove unnecessary dependency of hadoop-yarn-server-common from hadoop-mapreduce-client-common (haibochen via rkanter) 
(cherry picked from commit ab051bd42e)
 2017-08-16 16:15:48 -07:00
Jason Lowe 107e1722d4 MAPREDUCE-6940. Copy-paste error in the TaskAttemptUnsuccessfulCompletionEvent constructor. Contributed by Oleg Danilov 2017-08-16 16:35:41 -05:00
Ravi Prakash c556cdaaf4 MAPREDUCE-6923. Optimize MapReduce Shuffle I/O for small partitions. Contributed by Robert Schmidtke. 
(cherry picked from commit ac7d0604bc)
 2017-08-09 15:40:57 -07:00
Jason Lowe 50dfef7275 MAPREDUCE-6927. MR job should only set tracking url if history was successfully written. Contributed by Eric Badger 
(cherry picked from commit 735fce5bec)

Conflicts:
	hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-app/src/main/java/org/apache/hadoop/mapreduce/v2/app/MRAppMaster.java
 2017-08-08 14:55:55 -05:00
Arun Suresh 88d6adf419 YARN-5977. ContainerManagementProtocol changes to support change of container ExecutionType. (Kartheek Muthyala via asuresh) 
(cherry picked from commit 35dc782923)
 2017-08-03 21:18:43 -07:00
Yufei Gu 3ef7adc748 MAPREDUCE-6914. Tests use assertTrue(....equals(...)) instead of assertEquals()). (Daniel Templeton via Yufei Gu) 
(cherry picked from commit b8e8241854)
 2017-08-03 11:47:09 -07:00
Akira Ajisaka 338d84ab71
MAPREDUCE-6921. TestUmbilicalProtocolWithJobToken#testJobTokenRpc fails. Contributed by Sonia Garudi. 
(cherry picked from commit ceacadc51e)
 2017-08-01 15:02:20 +09:00
Jason Lowe 67d31a8e7e Revert "MAPREDUCE-6829. Add peak memory usage counter for each task. (Miklos Szegedi via kasha)" 
This reverts commit a91cc63d37.
 2017-07-31 16:44:13 -05:00
Junping Du cae33098e6 Revert "MAPREDUCE-5875. Make Counter limits consistent across JobClient, MRAppMaster, and YarnChild. (Gera Shegalov via kasha)" 
This reverts commit e8a31f2e1c.

(cherry picked from commit fbb7d6bcbb)
 2017-07-31 14:15:06 -07:00
Junping Du bf360da63f Revert "MAPREDUCE-6199. AbstractCounters are not reset completely on deserialization (adhoot via rkanter)" 
This reverts commit f9341c1e2c.
 2017-07-31 14:13:26 -07:00
Junping Du c0f8fd6432 Revert "MAPREDUCE-6286. A typo in HistoryViewer makes some code useless, which causes counter limits are not reset correctly. Contributed by Zhihai Xu." 
This reverts commit 7c72c7f529.
 2017-07-31 14:11:37 -07:00
Varun Saxena 0b342a644a YARN-5548. Use MockRMMemoryStateStore to reduce test failures (Bibin A Chundatt via Varun Saxena) 
(cherry picked from commit f66fd11e51)

Conflicts:
	hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/test/java/org/apache/hadoop/yarn/server/resourcemanager/MockRM.java
	hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/test/java/org/apache/hadoop/yarn/server/resourcemanager/TestRMRestart.java
 2017-07-28 00:33:37 +05:30
Akira Ajisaka 22f237c5d9
HADOOP-14681. Remove MockitoMaker class. Contributed by Andras Bokor. 
(cherry picked from commit cca51e916b)
 2017-07-25 15:25:36 +09:00
Sunil G 958835a3dc MAPREDUCE-6889. Add Job#close API to shutdown MR client services. Contributed by Rohith Sharma K S. 
(cherry picked from commit fb3b5d33ff)
 2017-07-17 13:36:00 +05:30
Ravi Prakash 756a068143 MAPREDUCE-6910. MapReduceTrackingUriPlugin can not return the right URI of history server with HTTPS. Contributed by Lantao Jin 
(cherry picked from commit 43f0503286)
 2017-07-13 16:19:52 -07:00
Junping Du f823f9fd78 MAPREDUCE-6246. DBOutputFormat.java appending extra semicolon to query which is incompatible with DB2. Contributed by ramtin and Gergely Novák. 
(cherry picked from commit f484a6ff60)
 2017-07-07 13:27:56 -07:00
Jason Lowe f0435db089 MAPREDUCE-6911. TestMapreduceConfigFields.testCompareXmlAgainstConfigurationClass fails consistently. Contributed by Eric Badger 2017-07-06 09:22:55 -05:00
Akira Ajisaka ac333155fb
MAPREDUCE-6909. LocalJobRunner fails when run on a node from multiple users. Contributed by Jason Lowe. 2017-07-04 15:25:33 +09:00
Naganarasimha 4fd6aad010 MAPREDUCE-6905. HADOOP_JOB_HISTORY_OPTS should be HADOOP_JOB_HISTORYSERVER_OPTS in mapred-config.sh. Contributed by LiXin Ge. 
(cherry picked from commit bf1f59937d)
 2017-07-02 16:01:02 +05:30
Akira Ajisaka 2a5a313539
MAPREDUCE-6697. Concurrent task limits should only be applied when necessary. Contributed by Nathan Roberts. 
(cherry picked from commit a5c0476a99)
 2017-06-28 10:51:23 +09:00
Ravi Prakash 182e98c703 MAPREDUCE-6897. Add Unit Test to ensure Job end notification gets sent even when appMaster stop gets YarnRuntimeException. Contributed by Gergely Novák 
(cherry picked from commit 82bbcbf37f)
 2017-06-16 14:27:06 -07:00
Jason Lowe 43fb793174 Revert "MAPREDUCE-6898. TestKill.testKillTask is flaky. Contributed by Peter Bacsko" 
This reverts commit 576cc258f9.
 2017-06-16 14:49:26 -05:00
Jason Lowe 576cc258f9 MAPREDUCE-6898. TestKill.testKillTask is flaky. Contributed by Peter Bacsko 
(cherry picked from commit e78542f4be)
 2017-06-16 14:26:15 -05:00
Xiao Chen 2afe9722af HADOOP-14524. Make CryptoCodec Closeable so it can be cleaned up proactively. 2017-06-16 09:49:41 -07:00
Junping Du 03bd121ba1 MAPREDUCE-6895. Job end notification not send due to YarnRuntimeException. Contributed by yunjiong zhao. 
(cherry picked from commit 6ed54f3439)
 2017-06-13 21:45:21 -07:00
Brahma Reddy Battula a5a5bcdb60 MAPREDUCE-6676. NNBench should Throw IOException when rename and delete fails. Contributed by Brahma Reddy Battula. 
(cherry picked from commit 24181f559f)
 2017-06-07 16:37:11 +08:00
Sunil G ec346d6377 YARN-6584. Correct license headers in hadoop-common, hdfs, yarn and mapreduce. Contributed by Yeliang Cang. 2017-05-23 07:57:36 +05:30
Jason Lowe 48b5faa705 MAPREDUCE-6882. Increase MapReduce test timeouts from 1 second to 10 seconds. Contributed by Eric Badger 
(cherry picked from commit 1e71fe8c42)
 2017-05-10 09:58:51 -05:00
Robert Kanter d686560b35 MAPREDUCE-6871. Allow users to specify racks and nodes for strict locality for AMs (rkanter) 
(cherry picked from commit 3721cfe1fb)
 2017-04-21 16:12:45 -07:00
Haibo Chen d4f553d42f MAPREDUCE-6673. Add a test example job that grows in memory usage over time (Karthik Kambatla via Haibo Chen) 
Change-Id: Iccfc8c67c38c526cc61726d87bfcbcf69ac36fea
(cherry picked from commit 25ac44709b4bbed78b607ea48021237b64e01b9f)
 2017-04-14 17:37:37 -07:00
Daniel Templeton 2f32accaaf MAPREDUCE-6846. Fragments specified for libjar paths are not handled correctly 
(Contributed by Chris Trezzo via Daniel Templeton)

(cherry picked from commit a92bc8a6c1c50d34d8229a67dc9d197823001551)
 2017-04-05 17:26:06 -07:00
Akira Ajisaka 438901f6c8
MAPREDUCE-6824. TaskAttemptImpl#createCommonContainerLaunchContext is longer than 150 lines. Contributed by Chris Trezzo. 2017-04-05 16:45:38 +09:00
Jason Lowe 3ae529ba54 YARN-6411. Clean up the overwrite of createDispatcher() in subclass of MockRM. Contributed by Yufei Gu 
(cherry picked from commit 4d1fac5df2)

Conflicts:
	hadoop-yarn-project/hadoop-yarn/hadoop-yarn-client/src/test/java/org/apache/hadoop/yarn/client/api/impl/TestAMRMClientOnRMRestart.java
	hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/test/java/org/apache/hadoop/yarn/server/resourcemanager/security/TestClientToAMTokens.java
 2017-03-31 10:21:44 -05:00
Jason Lowe a52b30f7e8 MAPREDUCE-6836. exception thrown when accessing the job configuration web UI. Contributed by Haibo Chen 
(cherry picked from commit c9b7ce9273)
 2017-03-30 12:56:04 -05:00
Jason Lowe 526499533e MAPREDUCE-6850. Shuffle Handler keep-alive connections are closed from the server side. Contributed by Jonathan Eagles 
(cherry picked from commit c8bd5fc7a8)

Conflicts:
	hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-shuffle/src/main/java/org/apache/hadoop/mapred/ShuffleHandler.java
 2017-03-30 11:10:20 -05:00
Zhe Zhang 3f5dc51ec9 MAPREDUCE-6873. MR Job Submission Fails if MR framework application path not on defaultFS. Contributed by Erik Krogen. 
(cherry picked from commit de7efd2687)
 2017-03-29 22:11:22 -07:00
Ming Ma 2ae9ae1864 MAPREDUCE-6862. Fragments are not handled correctly by resource limit checking. (Chris Trezzo via mingma) 
(cherry picked from commit ceab00ac62)
 2017-03-29 17:42:40 -07:00
Robert Kanter a65011a128 YARN-6050. AMs can't be scheduled on racks or nodes (rkanter) 2017-03-28 14:33:08 -07:00
Joe Mészáros 34a11b9108
MAPREDUCE-6866. Fix getNumMapTasks() documentation in JobConf. 
This closes #205

Signed-off-by: Akira Ajisaka <aajisaka@apache.org>
(cherry picked from commit e0a2eb71e3)
 2017-03-27 13:52:26 +09:00
Naganarasimha 0cfa6ec6c8 MAPREDUCE-6865. Fix typo in javadoc for DistributedCache. Contributed by Attila Sasvari. 
(cherry picked from commit 34a931c0fa)
 2017-03-20 08:26:05 +05:30
Robert Kanter 7953ac4da6 MAPREDUCE-6839. TestRecovery.testCrashed failed (pairg via rkanter) 
(cherry picked from commit 38d75dfd3a)
 2017-03-07 13:39:40 -08:00
Akira Ajisaka 120f680318
Treat encrypted files as private. Contributed by Daniel Templeton. 
(cherry picked from commit f01a69f84f)
 2017-03-07 13:38:20 +09:00