2687 Commits

Author SHA1 Message Date
Akira Ajisaka
835717210a
Revert "MAPREDUCE-6947. Moving logging APIs over to slf4j in hadoop-mapreduce-examples. Contributed by Gergery Novák."
This reverts commit 9c90400c9677c25f3e11ee4366da093bfa6c89fe.
2017-09-21 11:17:35 +09:00
Andrew Wang
5c8331c230 HDFS-12445. Correct spellings of choosen to chosen. Contributed by hu xiaodong.
(cherry picked from commit 51edaacd09d86419f99ca96545a1393db1f43f73)
2017-09-19 13:48:25 -07:00
Eric Payne
839d5b124d MAPREDUCE-6960. Shuffle Handler prints disk error stack traces for every read failure.
(cherry picked from commit 595d478408104bdfe1f08efd79930e18862fafbb)
2017-09-19 10:40:41 -05:00
Jason Lowe
731a622627 MAPREDUCE-6958. Shuffle audit logger should log size of shuffle transfer. Contributed by Jason Lowe
(cherry picked from commit 3a20debddeac69596ceb5b36f8413529ea8570e6)
2017-09-19 09:14:12 -05:00
Jason Lowe
db88b6179d Revert "MAPREDUCE-6958. Shuffle audit logger should log size of shuffle transfer. Contributed by Jason Lowe"
This reverts commit b3d61304f2fa4a99526f7a60ccaac9f262083079.

(cherry picked from commit ea845ba58c585647c4be8d30d9b814f098e34a12)
2017-09-19 08:50:20 -05:00
Akira Ajisaka
9c90400c96
MAPREDUCE-6947. Moving logging APIs over to slf4j in hadoop-mapreduce-examples. Contributed by Gergery Novák.
(cherry picked from commit 2018538fdba1a95a6556187569e872fce7f9e1c3)
2017-09-19 11:07:08 +09:00
Andrew Wang
531290c2c8 HADOOP-14835. mvn site build throws SAX errors. Contributed by Andrew Wang and Sean Mackrory.
(cherry picked from commit 3cf3540f19b5fd1a174690db9f1b7be2977d96ba)
2017-09-18 15:13:45 -07:00
Jason Lowe
8894654fde MAPREDUCE-6958. Shuffle audit logger should log size of shuffle transfer. Contributed by Jason Lowe
(cherry picked from commit b3d61304f2fa4a99526f7a60ccaac9f262083079)
2017-09-18 17:06:02 -05:00
Robert Kanter
917e875e51 MAPREDUCE-6954. Disable erasure coding for files that are uploaded to the MR staging area (pbacsko via rkanter)
(cherry picked from commit 0adc0471d0c06f66a31060f270dcb50a7b4ffafa)
2017-09-18 10:42:01 -07:00
Robert Kanter
3847b8c861 Revert "YARN-7162. Remove XML excludes file format (rkanter)" - wrong commit message
This reverts commit 293ab432e116b93ada6ee56e518ae07aa853ab41.
2017-09-18 10:41:46 -07:00
Robert Kanter
293ab432e1 YARN-7162. Remove XML excludes file format (rkanter)
(cherry picked from commit 3a8d57a0a2e047b34be82f602a2b6cf5593d2125)
2017-09-15 12:01:41 -07:00
Steve Loughran
b5e9982355
MAPREDUCE-6956 FileOutputCommitter to gain abstract superclass PathOutputCommitter.
Contributed by Steve Loughran

(cherry picked from commit 11390c2d111910b01d9c4d3e39dee49babae272f)
2017-09-15 17:00:16 +01:00
Andrew Wang
ad0fe570d9 HADOOP-14089. Automated checking for malformed client. Contributed by Sean Busbey.
(cherry picked from commit c3f35c422bbb7fe9c8e6509063896de549b127d1)
2017-09-13 16:57:53 -07:00
Jason Lowe
12e9558ed1 MAPREDUCE-6957. shuffle hangs after a node manager connection timeout. Contributed by Jooseong Kim
(cherry picked from commit 4d98936eec1b5d196053426c70d455cf8f83f84f)
2017-09-13 17:22:29 -05:00
Ray Chiang
656c6d1acc HADOOP-14799. Update nimbus-jose-jwt to 4.41.1. (rchiang)
(cherry picked from commit 556812c179aa094c21acf610439a8d69fe6420ab)
2017-09-12 10:19:58 -07:00
Robert Kanter
f5f343a665 MAPREDUCE-6953. Skip the testcase testJobWithChangePriority if FairScheduler is used (pbacsko via rkanter)
(cherry picked from commit a323f73bae65483652de290ef819268850803a17)
2017-09-08 13:14:52 -07:00
Allen Wittenauer
190410085b
HADOOP-14364. refresh changelog/release notes with newer Apache Yetus build
Signed-off-by: Andrew Wang <wang@apache.org>
2017-08-31 19:06:49 -07:00
Allen Wittenauer
71bbb86d69
HADOOP-14670. Increase minimum cmake version for all platforms
Signed-off-by: Chris Douglas <cdouglas@apache.org>
2017-08-30 21:47:24 -07:00
Konstantin V Shvachko
3e0e2033cc MAPREDUCE-6931. Remove TestDFSIO "Total Throughput" calculation. Contributed by Dennis Huo. 2017-08-30 14:07:23 -07:00
Haibo Chen
d04f85f387 MAPREDUCE-6892. Issues with the count of failed/killed tasks in the jhist file. (Peter Bacsko via Haibo Chen) 2017-08-30 10:07:48 -07:00
Varun Saxena
3d00c8f394 Made fixes for whitespace errors and checstyle warnings before merge. 2017-08-30 11:29:54 +05:30
Jian He
08f40bcc7f MAPREDUCE-6838. [ATSv2 Security] Add timeline delegation token received in allocate response to UGI. Contributed by Varun Saxena 2017-08-30 11:29:54 +05:30
Rohith Sharma K S
7594d1de7b YARN-6130. [ATSv2 Security] Generate a delegation token for AM when app collector is created and pass it to AM via NM and RM. Contributed by Varun Saxena. 2017-08-30 11:29:53 +05:30
Sangjin Lee
684de1a902 YARN-4675. Reorganize TimelineClient and TimelineClientImpl into separate classes for ATSv1.x and ATSv2. Contributed by Naganarasimha G R. 2017-08-30 11:29:52 +05:30
Sangjin Lee
8499299740 MAPREDUCE-6818. Remove direct reference to TimelineClientImpl. Contributed by Li Lu. 2017-08-30 11:29:51 +05:30
Sangjin Lee
092fead5d9 YARN-5792. Adopt the id prefix for YARN, MR, and DS entities. Contributed by Varun Saxena. 2017-08-30 11:29:51 +05:30
Mingliang Liu
ae7abad3d9 MAPREDUCE-6945. TestMapFileOutputFormat missing @after annotation. Contributed by Ajay Kumar 2017-08-27 15:19:55 -07:00
Arpit Agarwal
8b7cbe3840 HADOOP-14729. Upgrade JUnit 3 test cases to JUnit 4. Contributed by Ajay Kumar. 2017-08-25 09:55:46 -07:00
Andrew Wang
dd7916d3cd HDFS-12250. Reduce usage of FsPermissionExtension in unit tests. Contributed by Chris Douglas. 2017-08-17 09:35:36 -07:00
Robert Kanter
ab051bd42e MAPREDUCE-6936. Remove unnecessary dependency of hadoop-yarn-server-common from hadoop-mapreduce-client-common (haibochen via rkanter) 2017-08-16 16:14:04 -07:00
Jason Lowe
0acc5e0036 MAPREDUCE-6940. Copy-paste error in the TaskAttemptUnsuccessfulCompletionEvent constructor. Contributed by Oleg Danilov 2017-08-16 16:34:06 -05:00
Chris Douglas
645a8f2a4d HADOOP-14726. Mark FileStatus::isDir as final 2017-08-14 21:57:20 -07:00
Lei Xu
ce797a1706 HDFS-12221. Replace xcerces in XmlEditsVisitor. (Ajay Kumar via lei) 2017-08-14 10:27:47 -07:00
Allen Wittenauer
4222c97108
HADOOP-10392. Use FileSystem#makeQualified(Path) instead of Path#makeQualified(FileSystem) (ajisakaa via aw) 2017-08-11 09:25:56 -07:00
Haibo Chen
a32e0138fb MAPREDUCE-6870. Add configuration for MR job to finish when all reducers are complete. (Peter Bacsko via Haibo Chen) 2017-08-10 15:17:36 -07:00
Ravi Prakash
ac7d0604bc MAPREDUCE-6923. Optimize MapReduce Shuffle I/O for small partitions. Contributed by Robert Schmidtke. 2017-08-09 15:39:52 -07:00
Jason Lowe
735fce5bec MAPREDUCE-6927. MR job should only set tracking url if history was successfully written. Contributed by Eric Badger 2017-08-08 14:46:47 -05:00
Arun Suresh
35dc782923 YARN-5977. ContainerManagementProtocol changes to support change of container ExecutionType. (Kartheek Muthyala via asuresh) 2017-08-03 21:15:40 -07:00
Yufei Gu
b8e8241854 MAPREDUCE-6914. Tests use assertTrue(....equals(...)) instead of assertEquals()). (Daniel Templeton via Yufei Gu) 2017-08-03 11:44:34 -07:00
Sean Mackrory
1a1bf6b7d0 HADOOP-13595. Rework hadoop_usage to be broken up by clients/daemons/etc. Contributed by Allen Wittenauer. 2017-08-02 12:25:05 -06:00
Akira Ajisaka
ceacadc51e
MAPREDUCE-6921. TestUmbilicalProtocolWithJobToken#testJobTokenRpc fails. 2017-08-01 14:56:42 +09:00
Junping Du
fbb7d6bcbb Revert "MAPREDUCE-5875. Make Counter limits consistent across JobClient, MRAppMaster, and YarnChild. (Gera Shegalov via kasha)"
This reverts commit e8a31f2e1c34514fba2f480e8db652f6e2ed65d8.
2017-07-31 14:09:16 -07:00
Junping Du
713349a9af Revert "MAPREDUCE-6199. AbstractCounters are not reset completely on deserialization (adhoot via rkanter)"
This reverts commit 390a7c12f543b2c94a74f08d6d2a28410472043a.
2017-07-28 14:21:04 -07:00
Junping Du
e15f928295 Revert "MAPREDUCE-6286. A typo in HistoryViewer makes some code useless, which causes counter limits are not reset correctly. Contributed by Zhihai Xu."
This reverts commit 433542904aba5ddebf9bd9d299378647351eb13a.
2017-07-28 14:06:59 -07:00
Akira Ajisaka
38c6fa5c7a
HADOOP-11875. [JDK9] Adding a second copy of Hamlet without _ as a one-character identifier. 2017-07-28 09:10:52 +09:00
Varun Saxena
f66fd11e51 YARN-5548. Use MockRMMemoryStateStore to reduce test failures (Bibin A Chundatt via Varun Saxena) 2017-07-27 23:14:50 +05:30
Akira Ajisaka
cca51e916b
HADOOP-14681. Remove MockitoMaker class. Contributed by Andras Bokor. 2017-07-25 15:24:56 +09:00
Sunil G
fb3b5d33ff MAPREDUCE-6889. Add Job#close API to shutdown MR client services. Contributed by Rohith Sharma K S. 2017-07-17 13:35:15 +05:30
Ravi Prakash
43f0503286 MAPREDUCE-6910. MapReduceTrackingUriPlugin can not return the right URI of history server with HTTPS. Contributed by Lantao Jin 2017-07-13 16:16:45 -07:00
Junping Du
f484a6ff60 MAPREDUCE-6246. DBOutputFormat.java appending extra semicolon to query which is incompatible with DB2. Contributed by ramtin and Gergely Novák. 2017-07-07 13:26:16 -07:00