Apache
/
hadoop
1
0
Fork 0
Code Issues Pull Requests Packages Projects Releases Wiki Activity
14,134 Commits 326 Branches 371 Tags 1.5 GiB
Commit Graph

2353 Commits

Author SHA1 Message Date
Arun Suresh 7af9b8ad1e Fixing Job History Server Configuration parsing. (Jason Lowe via asuresh) 2017-11-09 15:10:21 -08:00
Naganarasimha 45e6d36e5c MAPREDUCE-6975. Logging task counters. Contributed by Prabhu Joseph. 
(cherry picked from commit 61bbdc511e)
 2017-11-06 13:10:41 +05:30
Arun Suresh 71a55bebfb Preparing for 2.10.0 development 2017-10-31 12:51:14 -07:00
Jason Lowe 347997c959 HADOOP-14919. BZip2 drops records when reading data in splits. Contributed by Jason Lowe 
(cherry picked from commit 2fae63aa60)
 2017-10-31 09:32:45 -05:00
Jason Lowe 9093ad6168 YARN-7244. ShuffleHandler is not aware of disks that are added. Contributed by Kuhu Shukla 
(cherry picked from commit 665bb147aa)
 2017-10-27 17:09:29 -05:00
Junping Du 1f2a2ba2be Add several jdiff xml files for 2.8.2 release. 
(cherry picked from commit a25b5aa0cf)
 2017-10-26 23:20:24 -07:00
Varun Saxena 56e0873e22 Fix javac, javadoc and checkstyle issues before branch-2 merge. 2017-10-17 23:36:41 +05:30
Jian He cbd5e89b24 MAPREDUCE-6838. [ATSv2 Security] Add timeline delegation token received in allocate response to UGI. Contributed by Varun Saxena 
(cherry picked from commit c6ce422a3423305ac665f80f039f61294adaa32c)

Conflicts:
	hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-app/src/main/java/org/apache/hadoop/mapreduce/v2/app/rm/RMContainerAllocator.java
 2017-10-17 23:36:40 +05:30
Rohith Sharma K S 5a7c272e0a YARN-6130. [ATSv2 Security] Generate a delegation token for AM when app collector is created and pass it to AM via NM and RM. Contributed by Varun Saxena. 2017-10-17 23:36:39 +05:30
Sangjin Lee bb100d35fd YARN-4675. Reorganize TimelineClient and TimelineClientImpl into separate classes for ATSv1.x and ATSv2. Contributed by Naganarasimha G R. 
(cherry picked from commit 73235ab30361b41293846189f3c5fef321ae7cac)
 2017-10-17 23:36:36 +05:30
Sangjin Lee c8a8e680bd MAPREDUCE-6818. Remove direct reference to TimelineClientImpl. Contributed by Li Lu. 
(cherry picked from commit 6217b87f4a056cf704cef2e073b386b7803415de)
 2017-10-17 23:36:35 +05:30
Sangjin Lee b8cfb4fcb3 YARN-5792. Adopt the id prefix for YARN, MR, and DS entities. Contributed by Varun Saxena. 
(cherry picked from commit f734977b27a514ce0561638c0a6a17b1ef093026)
 2017-10-17 23:36:35 +05:30
Varun Saxena 0820df611d MAPREDUCE-6731. TestMRTimelineEventHandling#testMRNewTimelineServiceEventHandling fails randomly for concurrent tests. (Sangjin Lee via Varun Saxena). 
(cherry picked from commit d14e729a55)
 2017-10-17 23:36:33 +05:30
Sangjin Lee e212e562ec YARN-5355. Backported YARN-2928 into our branch-2 feature branch. 2017-10-17 23:36:33 +05:30
Chris Trezzo 3e7c06af41 MAPREDUCE-5951. Add support for the YARN Shared Cache. 
(cherry picked from commit e46d5bb962)
 2017-10-12 11:30:38 -07:00
Rohith Sharma K S c2b8d4b9ab MAPREDUCE-6951. Improve exception message when mapreduce.jobhistory.webapp.address is in wrong format. Contributed by Prabhu Joseph. 
(cherry picked from commit 639f98cc9d)
 2017-10-11 14:38:00 +05:30
Jason Lowe b94acb6efb MAPREDUCE-6965. QuasiMonteCarlo should use Time.monotonicNow for measuring durations. Contributed by Chetna Chaudhari 
(cherry picked from commit 908d8e914e)
 2017-09-22 09:37:09 -05:00
Akira Ajisaka d80e2f2eae
MAPREDUCE-6966. DistSum should use Time.monotonicNow for measuring durations. Contributed by Chetna Chaudhari. 
(cherry picked from commit c71d137960)
 2017-09-22 15:10:19 +09:00
Akira Ajisaka 9d4de9c556
MAPREDUCE-6947. Moving logging APIs over to slf4j in hadoop-mapreduce-examples. Contributed by Gergely Novák. 
(cherry picked from commit 53be075241)

Conflicts:
	hadoop-mapreduce-project/hadoop-mapreduce-examples/src/main/java/org/apache/hadoop/examples/terasort/TeraOutputFormat.java
 2017-09-22 13:29:17 +09:00
Jason Lowe cf2615961a MAPREDUCE-6964. BaileyBorweinPlouffe should use Time.monotonicNow for measuring durations. Contributed by Chetna Chaudhari 
(cherry picked from commit b9db0afa1e)
 2017-09-21 09:39:46 -05:00
Akira Ajisaka 7d747df52d
Revert "MAPREDUCE-6947. Moving logging APIs over to slf4j in hadoop-mapreduce-examples. Contributed by Gergery Novák." 
This reverts commit e61baf9496.
 2017-09-21 11:16:48 +09:00
Eric Payne d2386725ff MAPREDUCE-6960. Shuffle Handler prints disk error stack traces for every read failure. 
(cherry picked from commit 595d478408)
 2017-09-19 10:46:05 -05:00
Jason Lowe 41c2a73c52 MAPREDUCE-6958. Shuffle audit logger should log size of shuffle transfer. Contributed by Jason Lowe 
(cherry picked from commit 3a20debdde)
 2017-09-19 09:14:41 -05:00
Daniel Templeton 1944b0b286 MAPREDUCE-6808. Log map attempts as part of shuffle handler audit log (Contributed by Gergő Pásztor via Daniel Templeton) 
(cherry picked from commit a7463b6c88)
 2017-09-19 09:05:39 -05:00
Akira Ajisaka e61baf9496
MAPREDUCE-6947. Moving logging APIs over to slf4j in hadoop-mapreduce-examples. Contributed by Gergery Novák. 
(cherry picked from commit 2018538fdb)

Conflicts:
	hadoop-mapreduce-project/hadoop-mapreduce-examples/src/main/java/org/apache/hadoop/examples/terasort/TeraOutputFormat.java
 2017-09-19 11:25:29 +09:00
Jason Lowe f525dfc5ec MAPREDUCE-6957. shuffle hangs after a node manager connection timeout. Contributed by Jooseong Kim 
(cherry picked from commit 4d98936eec)
 2017-09-13 17:22:58 -05:00
Robert Kanter 72823b2738 MAPREDUCE-6953. Skip the testcase testJobWithChangePriority if FairScheduler is used (pbacsko via rkanter) 
(cherry picked from commit a323f73bae)
(cherry picked from commit f5f343a665)
 2017-09-08 13:18:59 -07:00
Haibo Chen 59f1ca7258 MAPREDUCE-6937. Backport MAPREDUCE-6870 to branch-2 while preserving compatibility. (Peter Bacsko via Haibo Chen) 2017-08-31 09:28:19 -07:00
Konstantin V Shvachko 9a0aac4170 MAPREDUCE-6931. Remove TestDFSIO "Total Throughput" calculation. Contributed by Dennis Huo. 
(cherry picked from commit 3e0e2033cc)

Conflicts:
	hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-jobclient/src/test/java/org/apache/hadoop/fs/TestDFSIO.java
 2017-08-30 14:42:36 -07:00
Jason Lowe cd08aa72a6 MAPREDUCE-6641. TestTaskAttempt fails in trunk. Contributed by Haibo Chen. 2017-08-29 12:24:42 -05:00
Robert Kanter 9de332d502 MAPREDUCE-6936. Remove unnecessary dependency of hadoop-yarn-server-common from hadoop-mapreduce-client-common (haibochen via rkanter) 
(cherry picked from commit ab051bd42e)
 2017-08-16 16:15:48 -07:00
Jason Lowe 107e1722d4 MAPREDUCE-6940. Copy-paste error in the TaskAttemptUnsuccessfulCompletionEvent constructor. Contributed by Oleg Danilov 2017-08-16 16:35:41 -05:00
Ravi Prakash c556cdaaf4 MAPREDUCE-6923. Optimize MapReduce Shuffle I/O for small partitions. Contributed by Robert Schmidtke. 
(cherry picked from commit ac7d0604bc)
 2017-08-09 15:40:57 -07:00
Jason Lowe 50dfef7275 MAPREDUCE-6927. MR job should only set tracking url if history was successfully written. Contributed by Eric Badger 
(cherry picked from commit 735fce5bec)

Conflicts:
	hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-app/src/main/java/org/apache/hadoop/mapreduce/v2/app/MRAppMaster.java
 2017-08-08 14:55:55 -05:00
Arun Suresh 88d6adf419 YARN-5977. ContainerManagementProtocol changes to support change of container ExecutionType. (Kartheek Muthyala via asuresh) 
(cherry picked from commit 35dc782923)
 2017-08-03 21:18:43 -07:00
Yufei Gu 3ef7adc748 MAPREDUCE-6914. Tests use assertTrue(....equals(...)) instead of assertEquals()). (Daniel Templeton via Yufei Gu) 
(cherry picked from commit b8e8241854)
 2017-08-03 11:47:09 -07:00
Akira Ajisaka 338d84ab71
MAPREDUCE-6921. TestUmbilicalProtocolWithJobToken#testJobTokenRpc fails. Contributed by Sonia Garudi. 
(cherry picked from commit ceacadc51e)
 2017-08-01 15:02:20 +09:00
Jason Lowe 67d31a8e7e Revert "MAPREDUCE-6829. Add peak memory usage counter for each task. (Miklos Szegedi via kasha)" 
This reverts commit a91cc63d37.
 2017-07-31 16:44:13 -05:00
Junping Du cae33098e6 Revert "MAPREDUCE-5875. Make Counter limits consistent across JobClient, MRAppMaster, and YarnChild. (Gera Shegalov via kasha)" 
This reverts commit e8a31f2e1c.

(cherry picked from commit fbb7d6bcbb)
 2017-07-31 14:15:06 -07:00
Junping Du bf360da63f Revert "MAPREDUCE-6199. AbstractCounters are not reset completely on deserialization (adhoot via rkanter)" 
This reverts commit f9341c1e2c.
 2017-07-31 14:13:26 -07:00
Junping Du c0f8fd6432 Revert "MAPREDUCE-6286. A typo in HistoryViewer makes some code useless, which causes counter limits are not reset correctly. Contributed by Zhihai Xu." 
This reverts commit 7c72c7f529.
 2017-07-31 14:11:37 -07:00
Varun Saxena 0b342a644a YARN-5548. Use MockRMMemoryStateStore to reduce test failures (Bibin A Chundatt via Varun Saxena) 
(cherry picked from commit f66fd11e51)

Conflicts:
	hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/test/java/org/apache/hadoop/yarn/server/resourcemanager/MockRM.java
	hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/test/java/org/apache/hadoop/yarn/server/resourcemanager/TestRMRestart.java
 2017-07-28 00:33:37 +05:30
Akira Ajisaka 22f237c5d9
HADOOP-14681. Remove MockitoMaker class. Contributed by Andras Bokor. 
(cherry picked from commit cca51e916b)
 2017-07-25 15:25:36 +09:00
Sunil G 958835a3dc MAPREDUCE-6889. Add Job#close API to shutdown MR client services. Contributed by Rohith Sharma K S. 
(cherry picked from commit fb3b5d33ff)
 2017-07-17 13:36:00 +05:30
Ravi Prakash 756a068143 MAPREDUCE-6910. MapReduceTrackingUriPlugin can not return the right URI of history server with HTTPS. Contributed by Lantao Jin 
(cherry picked from commit 43f0503286)
 2017-07-13 16:19:52 -07:00
Junping Du ea3dba37cd MAPREDUCE-5621. mr-jobhistory-daemon.sh doesn't have to execute mkdir and chown all the time. Contributed by Shinichi Yamashita. 2017-07-12 11:54:26 -07:00
Junping Du f823f9fd78 MAPREDUCE-6246. DBOutputFormat.java appending extra semicolon to query which is incompatible with DB2. Contributed by ramtin and Gergely Novák. 
(cherry picked from commit f484a6ff60)
 2017-07-07 13:27:56 -07:00
Jason Lowe f0435db089 MAPREDUCE-6911. TestMapreduceConfigFields.testCompareXmlAgainstConfigurationClass fails consistently. Contributed by Eric Badger 2017-07-06 09:22:55 -05:00
Akira Ajisaka ac333155fb
MAPREDUCE-6909. LocalJobRunner fails when run on a node from multiple users. Contributed by Jason Lowe. 2017-07-04 15:25:33 +09:00
Naganarasimha 4fd6aad010 MAPREDUCE-6905. HADOOP_JOB_HISTORY_OPTS should be HADOOP_JOB_HISTORYSERVER_OPTS in mapred-config.sh. Contributed by LiXin Ge. 
(cherry picked from commit bf1f59937d)
 2017-07-02 16:01:02 +05:30