Apache/hadoop
1
0
Fork 0
Code Issues Pull Requests Packages Projects Releases Wiki Activity
16,846 Commits 326 Branches 371 Tags
Commit Graph

16846 Commits

This Branch
This Branch
All Branches
Author SHA1 Message Date
Sangjin Lee
f919bcadb7 YARN-6771. Use classloader inside configuration class to make new 
classes. Contributed by Jongyoul Lee.

(cherry picked from commit 8b336632acad10e45d029596c5e3196e1857d891)
 2017-09-20 22:54:30 -07:00
Steve Loughran
4767d1bb1f Revert "HADOOP-14771. hadoop-client does not include hadoop-yarn-client. (Ajay Kumar via Haibo Chen)" 
HADOOP-14879 Build failure due to failing hadoop-client-check-invariants
This reverts commit 1ee25278c891e95ba2ab142e5b78aebd752ea163.

(cherry picked from commit aa6e8d2dff533c3d0c86776567c860548723c21c)
 2017-09-20 21:49:34 -07:00
Akira Ajisaka
835717210a
Revert "MAPREDUCE-6947. Moving logging APIs over to slf4j in hadoop-mapreduce-examples. Contributed by Gergery Novák." 
This reverts commit 9c90400c9677c25f3e11ee4366da093bfa6c89fe.
 2017-09-21 11:17:35 +09:00
Andrew Wang
1792093bae HDFS-12447. Rename AddECPolicyResponse to AddErasureCodingPolicyResponse. Contributed by SammiChen. 
(cherry picked from commit a12f09ba3c4a3aa4c4558090c5e1b7bcaebe3b94)
 2017-09-20 11:51:21 -07:00
Ming Ma
0006ee681a HDFS-11035. Better documentation for maintenace mode and upgrade domain. 
(cherry picked from commit ce943eb17a4218d8ac1f5293c6726122371d8442)
 2017-09-20 09:38:15 -07:00
Ming Ma
816933722a HDFS-12473. Change hosts JSON file format. 
(cherry picked from commit 230b85d5865b7e08fb7aaeab45295b5b966011ef)
 2017-09-20 09:05:56 -07:00
Akira Ajisaka
5c158f2f51
HADOOP-7308. Remove unused TaskLogAppender configurations from log4j.properties. Contributed by Todd Lipcon and J.Andreina. 
(cherry picked from commit 7e58b2478ce10f54b9b9a647f22a69dd528a81e6)
 2017-09-20 21:08:35 +09:00
Junping Du
974605c025 YARN-7186. Fix finicky TestContainerManager tests. Contributed by Arun Suresh. 
(cherry picked from commit 647b7527a9cdf4717e7dcbbb660e5812b67a17f1)
 2017-09-19 18:31:54 -07:00
Andrew Wang
f4082bbb23 HDFS-12437. Fix test setup in TestLeaseRecoveryStriped. 
(cherry picked from commit 12d9d7bc509bca82b8f40301e3dc5ca764be45eb)
 2017-09-19 16:42:23 -07:00
Andrew Wang
5c8331c230 HDFS-12445. Correct spellings of choosen to chosen. Contributed by hu xiaodong. 
(cherry picked from commit 51edaacd09d86419f99ca96545a1393db1f43f73)
 2017-09-19 13:48:25 -07:00
Andrew Wang
fdfbeb0243 HDFS-12444. Reduce runtime of TestWriteReadStripedFile. Contributed by Huafeng Wang and Andrew Wang. 
(cherry picked from commit 59830ca772dfb5dcc8b3e5281ca482dea5a5fa3e)
 2017-09-19 13:44:53 -07:00
Lei Xu
7d9ca2e21b HDFS-12449. TestReconstructStripedFile.testNNSendsErasureCodingTasks randomly cannot finish in 60s. (SammiChen via lei) 2017-09-19 11:51:44 -07:00
Brahma Reddy Battula
1ce365e91e HDFS-11799. Introduce a config to allow setting up write pipeline with fewer nodes than replication factor. Contributed by Brahma Reddy Battula 
(cherry picked from commit fda1221c55101d97ac62e1ee4e3ddf9a915d5363)
 2017-09-19 11:27:41 +05:30
Brahma Reddy Battula
6ee79d573a HDFS-12480. TestNameNodeMetrics#testTransactionAndCheckpointMetrics Fails in trunk. Contributed by Hanisha Koneru 
(cherry picked from commit 31b58406ac369716ef1665b7d60a3409117bdf9d)
 2017-09-19 10:46:01 +05:30
Eric Payne
839d5b124d MAPREDUCE-6960. Shuffle Handler prints disk error stack traces for every read failure. 
(cherry picked from commit 595d478408104bdfe1f08efd79930e18862fafbb)
 2017-09-19 10:40:41 -05:00
Jason Lowe
731a622627 MAPREDUCE-6958. Shuffle audit logger should log size of shuffle transfer. Contributed by Jason Lowe 
(cherry picked from commit 3a20debddeac69596ceb5b36f8413529ea8570e6)
 2017-09-19 09:14:12 -05:00
Jason Lowe
db88b6179d Revert "MAPREDUCE-6958. Shuffle audit logger should log size of shuffle transfer. Contributed by Jason Lowe" 
This reverts commit b3d61304f2fa4a99526f7a60ccaac9f262083079.

(cherry picked from commit ea845ba58c585647c4be8d30d9b814f098e34a12)
 2017-09-19 08:50:20 -05:00
Kai Zheng
16a75b4ed3 HDFS-12479. Some misuses of lock in DFSStripedOutputStream. Contributed by Huafeng Wang 2017-09-19 17:46:39 +08:00
Akira Ajisaka
9c90400c96
MAPREDUCE-6947. Moving logging APIs over to slf4j in hadoop-mapreduce-examples. Contributed by Gergery Novák. 
(cherry picked from commit 2018538fdba1a95a6556187569e872fce7f9e1c3)
 2017-09-19 11:07:08 +09:00
Arun Suresh
2ccfe92a4e YARN-7203. Add container ExecutionType into ContainerReport. (Botong Huang via asuresh) 
(cherry picked from commit 56ef5279c1db93d03b2f1e04badbfe804f548918)
 2017-09-18 15:50:39 -07:00
Andrew Wang
531290c2c8 HADOOP-14835. mvn site build throws SAX errors. Contributed by Andrew Wang and Sean Mackrory. 
(cherry picked from commit 3cf3540f19b5fd1a174690db9f1b7be2977d96ba)
 2017-09-18 15:13:45 -07:00
Jason Lowe
8894654fde MAPREDUCE-6958. Shuffle audit logger should log size of shuffle transfer. Contributed by Jason Lowe 
(cherry picked from commit b3d61304f2fa4a99526f7a60ccaac9f262083079)
 2017-09-18 17:06:02 -05:00
Haibo Chen
5aa2c6baa7 HADOOP-14771. hadoop-client does not include hadoop-yarn-client. (Ajay Kumar via Haibo Chen) 
(cherry picked from commit 1ee25278c891e95ba2ab142e5b78aebd752ea163)
 2017-09-18 14:27:02 -07:00
Xuan
e91447ccc9 YARN-6570. No logs were found for running application, running 
container. Contributed by Junping Du

(cherry picked from commit 7c732924a889cd280e972882619a1827877fbafa)
 2017-09-18 14:05:02 -07:00
Erik Krogen
a57f76a3c3 HDFS-12323. NameNode terminates after full GC thinking QJM unresponsive if full GC is much longer than timeout. Contributed by Erik Krogen. 
(cherry picked from commit 90894c7262df0243e795b675f3ac9f7b322ccd11)
 2017-09-18 13:35:28 -07:00
Robert Kanter
917e875e51 MAPREDUCE-6954. Disable erasure coding for files that are uploaded to the MR staging area (pbacsko via rkanter) 
(cherry picked from commit 0adc0471d0c06f66a31060f270dcb50a7b4ffafa)
 2017-09-18 10:42:01 -07:00
Robert Kanter
3847b8c861 Revert "YARN-7162. Remove XML excludes file format (rkanter)" - wrong commit message 
This reverts commit 293ab432e116b93ada6ee56e518ae07aa853ab41.
 2017-09-18 10:41:46 -07:00
Jason Lowe
7e72be9456 YARN-7192. Add a pluggable StateMachine Listener that is notified of NM Container State changes. Contributed by Arun Suresh 
(cherry picked from commit a4f9c7c9247801dd37beec6fc195622af1b884ad)
 2017-09-18 10:23:22 -05:00
Rohith Sharma K S
508e93e48c YARN-7157. Add admin configuration to filter per-user's apps in secure cluster. Contributed by Sunil G. 2017-09-18 22:56:52 +10:00
Kai Zheng
4f8240bd88 HDFS-12460. Make addErasureCodingPolicy an idempotent operation. Contributed by Sammi Chen 2017-09-18 18:06:18 +08:00
Wangda Tan
e3a0104980 YARN-7149. Cross-queue preemption sometimes starves an underserved queue. (Eric Payne via wangda) 
Change-Id: Ib269991dbebce160378e8372ee6d24849c4a5ed6
(cherry picked from commit 3dfa937a1fadfc62947755872515f549b3b15e6a)
 2017-09-17 21:19:20 -07:00
Wangda Tan
03b6e95fa5 YARN-6977. Node information is not provided for non am containers in RM logs. (Suma Shivaprasad via wangda) 
Change-Id: I0c44d09a560446dee2ba68c2b9ae69fce0ec1d3e
(cherry picked from commit 8a42e922fad613f3cf1cc6cb0f3fa72546a9cc56)
 2017-09-17 21:19:20 -07:00
Arpit Agarwal
31cdd260f8 HDFS-12472. Add JUNIT timeout to TestBlockStatsMXBean. Contributed by Bharat Viswanadham. 2017-09-16 10:09:36 -07:00
Andrew Wang
7ea10e9f96 HDFS-10701. TestDFSStripedOutputStreamWithFailure#testBlockTokenExpired occasionally fails. Contributed by SammiChen. 
(cherry picked from commit ef8cd5dc565f901b4954befe784675e130e84c3c)
 2017-09-15 16:20:39 -07:00
Junping Du
21f6aa77ab YARN-7174. Add retry logic in LogsCLI when fetch running application logs. Contributed by Xuan Gong. 
(cherry picked from commit 1a84c24b0cf6674fa755403971fa57d8e412b320)
 2017-09-15 15:34:30 -07:00
Robert Kanter
b27ce4d599 HADOOP-14853. hadoop-mapreduce-client-app is not a client module (haibochen via rkanter) 
(cherry picked from commit b9b607daa74322a2928f7671a985a60388b9b9c2)
 2017-09-15 13:53:27 -07:00
Lei Xu
2787766b41 HDFS-12349. Improve log message when it could not alloc enough blocks for EC. (Lei (Eddy) Xu) 
(cherry picked from commit fbe06b58805aac4861fb27dfa273914b69e8bdc6)
 2017-09-15 12:15:49 -07:00
Robert Kanter
293ab432e1 YARN-7162. Remove XML excludes file format (rkanter) 
(cherry picked from commit 3a8d57a0a2e047b34be82f602a2b6cf5593d2125)
 2017-09-15 12:01:41 -07:00
Nathan Roberts
944b315dd1 HDFS-12441. Supress UnresolvedPathException in namenode log. Contributed by Kihwal Lee. 
(cherry picked from commit de197fc1562c4d1c39d24738c7cf8a8869dcec9c)
 2017-09-15 13:29:14 -05:00
Yufei Gu
6362a9503d YARN-6799. Remove the duplicated code in CGroupsHandlerImp.java. (Contributed by Weiyuan via Yufei Gu) 
(cherry picked from commit 5ab21dfe9c8e14585db71e02bcb021a526507f98)
 2017-09-15 09:51:50 -07:00
Robert Kanter
d3d32d3580 YARN-7162. Remove XML excludes file format (rkanter) 2017-09-15 09:48:59 -07:00
Steve Loughran
9f6b08f840
HADOOP-14553. Add (parallelized) integration tests to hadoop-azure 
Contributed by Steve Loughran

(cherry picked from commit 2d2d97fa7d4224369b3c13bc4a45e8cc9e29afb1)
 2017-09-15 17:04:43 +01:00
Steve Loughran
b5e9982355
MAPREDUCE-6956 FileOutputCommitter to gain abstract superclass PathOutputCommitter. 
Contributed by Steve Loughran

(cherry picked from commit 11390c2d111910b01d9c4d3e39dee49babae272f)
 2017-09-15 17:00:16 +01:00
Kihwal Lee
ba26097ea7 HDFS-12424. Datatable sorting on the Datanode Information page in the Namenode UI is broken. Contributed by Shawna Martell. 
(cherry picked from commit 78bdf10ae40300a5f2f3b95491d280802edc1df2)
 2017-09-15 09:55:04 -05:00
Kai Zheng
8e23929ad8 HADOOP-14869. Upgrade Apache Kerby version to v1.0.1. Contributed by Wei Zhou 2017-09-15 19:51:05 +08:00
Arpit Agarwal
a3d6ce25a3 HDFS-12456. TestNamenodeMetrics.testSyncAndBlockReportMetric fails. Contributed by Bharat Viswanadham. 2017-09-14 22:23:49 -07:00
Kai Zheng
754b27f17b HDFS-12395. Support erasure coding policy operations in namenode edit log. Contributed by Sammi Chen 2017-09-15 09:42:26 +08:00
Kai Zheng
02c8807312 HDFS-7859. Erasure Coding: Persist erasure coding policies in NameNode. Contributed by Sammi Chen 2017-09-15 09:09:31 +08:00
Lei Xu
e140489147 HDFS-12378. TestClientProtocolForPipelineRecovery#testZeroByteBlockRecovery fails on trunk. (Lei (Eddy) Xu) 2017-09-14 17:04:29 -07:00
Aaron Fabbri
62e8a5ceed
HADOOP-14738 Remove S3N and obsolete bits of S3A; rework docs. Contributed by Steve Loughran. 2017-09-14 16:12:29 -07:00