Apache/hadoop
1
0
Fork 0
Code Issues Pull Requests Packages Projects Releases Wiki Activity
12,435 Commits 326 Branches 371 Tags
Commit Graph

12435 Commits

This Branch
This Branch
All Branches
Author SHA1 Message Date
Jason Lowe
9d3ea2c0ed YARN-7244. ShuffleHandler is not aware of disks that are added. Contributed by Kuhu Shukla 2017-10-30 16:08:14 -05:00
Arpit Agarwal
2d4629f386 HDFS-9914. Fix configurable WebhDFS connect/read timeout. Contributed by Xiaoyu Yao. 2017-10-27 07:45:27 -07:00
Junping Du
5ce84db79f Add several jdiff xml files for 2.8.2 release. 
(cherry picked from commit a25b5aa0cf5189247fa38e7b0a188d568eba1b6c)
(cherry picked from commit 1f2a2ba2be50e4361b3093578a98e09d6f82af5c)
 2017-10-26 23:26:21 -07:00
Junping Du
ae7ff3da09 Set jdiff stable version to 2.8.2. 
(cherry picked from commit 06b56f649376020cde382bde25f90dc0547c4277)
(cherry picked from commit 4b75ebb68eb87e9f18676915652085c30ea78f7e)
 2017-10-26 23:26:07 -07:00
Junping Du
13b042b576 Add CHANGES.md and RELEASENOTES for 2.8.2 release. 
(cherry picked from commit 3793d2b9cfdae97a534a6eb6e38e84e2f00c2b83)
 2017-10-26 23:25:49 -07:00
Zhe Zhang
239dadbd46 Revert "HDFS-12502. nntop should support a category based on FilesInGetListingOps." 
This reverts commit f1ff21e00c21ba2514b81779389141db150882cc.
 2017-10-24 15:04:37 -07:00
Eric Payne
6b0dba318d YARN-4163: Audit getQueueInfo and getApplications calls. Contributed by Chang Li. 2017-10-23 17:37:50 -05:00
Naganarasimha
5ee67c1202 HADOOP-14966. Handle JDK-8071638 for hadoop-common. Contributed by Bibin A Chundatt. 
(cherry picked from commit 6b8122458ede9ececb138bed99d368580e5439a1)
 2017-10-23 23:48:43 +05:30
Wei-Chiu Chuang
a83f87e170 HDFS-12619. Do not catch and throw unchecked exceptions if IBRs fail to process. Contributed by Wei-Chiu Chuang. 
(cherry picked from commit 4ab0c8f96a41c573cc1f1e71c18871d243f952b9)
(cherry picked from commit ce93a2b84badacb6d0b19bbb149afbcf8f38c665)
 2017-10-19 06:19:11 -07:00
Wei-Chiu Chuang
3b47f6bdfd HADOOP-14880. [KMS] Document&test missing KMS client side configs. Contributed by Gabor Bota. 
(cherry picked from commit 97c70c7ac6881f87eee1575bcbdd28b31ecac231)

Conflicts:
	hadoop-common-project/hadoop-kms/src/test/java/org/apache/hadoop/crypto/key/kms/server/TestKMS.java
(cherry picked from commit 21b9e4fb979ecbc8f254b76f6ed86e6ef63d74f3)
 2017-10-19 06:05:53 -07:00
Zhe Zhang
f1ff21e00c HDFS-12502. nntop should support a category based on FilesInGetListingOps. 
(cherry picked from commit 60bfee270ed3a653c44c0bc92396167b5022df6e)
(cherry picked from commit aecf6c50d351e7aed2008a682677a1edb104585d)
(cherry picked from commit 87f440739604f97261da9624afb36920a6a4c28f)
 2017-10-19 00:09:33 -07:00
Junping Du
4935334b0f HADOOP-14958. Fix source-level compatibility after HADOOP-11252. Contributed by Junping Du. 
(cherry picked from commit b016f08f67830ed3ca741bc6a10c3f5164781be5)
(cherry picked from commit 9433f9eb09af0ca61f01a2eb42ff3ffe31a94d5f)
(cherry picked from commit e7e2b8158d9aaf314698784d44f4dd9313df3b50)
 2017-10-18 15:11:21 -07:00
Junping Du
4ea3ae36cb YARN-7230. Document DockerContainerRuntime for branch-2.8 with proper scope and claim as an experimental feature. Contributed by Shane Kumpf. 2017-10-18 14:12:37 -07:00
Arpit Agarwal
dcdb3f250c HDFS-12420. Add an option to disallow 'namenode format -force'. Contributed by Ajay Kumar. 
(cherry picked from commit b6942cbe9b8c9469e8c2b64c3268d671f5a43e75)
(cherry picked from commit 5897095d539be086ed37df011f024e37eb37b0cd)
(cherry picked from commit 0aa1b62b396f6fa43f15900c26dd28b95c4e40ce)
 2017-10-17 23:46:14 -07:00
Xiao Chen
a1d29717a2 Revert "HADOOP-14948. Document missing config key hadoop.treat.subject.external. Contributed by Ajay Kumar." 
This reverts commit 19db2837cbdb883b7c38adf01c2b7f9cad8d49d4.
 2017-10-17 10:11:45 -07:00
Wei-Chiu Chuang
19db2837cb HADOOP-14948. Document missing config key hadoop.treat.subject.external. Contributed by Ajay Kumar. 
(cherry picked from commit e906108fc98a011630d12a43e557b81d7ef7ea5d)
(cherry picked from commit 1452ba139ea845ad9e82a5902d239b59a789bff3)

Conflicts:
	hadoop-common-project/hadoop-common/src/main/resources/core-default.xml
 2017-10-16 16:46:31 -07:00
Nathan Roberts
602a884d55 YARN-7333. container-executor fails to remove entries from a directory that is not writable or executable. Contributed by Jason Lowe. 
(cherry picked from commit 8620140a6a3ec0117675ede06d92d830da3da551)
 2017-10-16 16:45:17 -05:00
Wei-Chiu Chuang
843b8b7b96 HDFS-12485. expunge may fail to remove trash from encryption zone. Contributed by Wei-Chiu Chuang. 
(cherry picked from commit 8dbc8909c92d502d10a7f94d1de3171878a43b04)
(cherry picked from commit a596c6f5eb553cf38123d61b844696ddbec99be0)

Conflicts:
	hadoop-hdfs-project/hadoop-hdfs-client/src/main/java/org/apache/hadoop/hdfs/DistributedFileSystem.java
 2017-10-16 13:04:36 -07:00
Xiao Chen
6a55d0e671 HDFS-12642. Log block and datanode details in BlockRecoveryWorker. 
(cherry picked from commit 971b3ca1acc15d9a1f8e774291813dafd16ef704)
 2017-10-16 10:34:31 -07:00
Xiao Chen
db726c4483 HDFS-12603. Enable async edit logging by default. Contributed by Andrew Wang. 
(cherry picked from commit d0311dfc5fe92a0e0f3fb8ae92e412edce609740)
(cherry picked from commit f6e768104b0e9a087584c25e28bf8cd29802c719)

Conflicts:
	hadoop-hdfs-project/hadoop-hdfs/src/main/resources/hdfs-default.xml
 2017-10-16 09:51:34 -07:00
Xiao Chen
27badf6df9 HDFS-12659. Update TestDeadDatanode#testNonDFSUsedONDeadNodeReReg to increase heartbeat recheck interval. Contributed by Ajay Kumar. 
(cherry picked from commit 11a7729c1474aa379a62c0440c8c1a11be261a75)
 2017-10-15 19:28:45 -07:00
Jason Lowe
b41c5e41bc YARN-7246. Fix the default docker binary path. Contributed by Shane Kumpf 2017-10-13 15:32:35 -05:00
Jason Lowe
ce76ed675e YARN-7325. Remove unused container variable in DockerLinuxContainerRuntime. Contributed by Shane Kumpf 2017-10-13 14:54:06 -05:00
Jason Lowe
5c4f269b26 YARN-6930. Admins should be able to explicitly enable specific LinuxContainerRuntime in the NodeManager. Contributed by Shane Kumpf 2017-10-11 09:57:49 -05:00
Jason Lowe
108c2394c3 HADOOP-14912. FairCallQueue may defer servicing calls. Contributed by Daryn Sharp 
(cherry picked from commit 1123f8f0b62292197f5433cd40e66d8620044608)
 2017-10-10 14:51:19 -05:00
Sunil G
b41a702751 YARN-7245. Max AM Resource column in Active Users Info section of Capacity Scheduler UI page should be updated per-user. Contributed by Eric Payne. 
(cherry picked from commit 8137254a1f4367974bf1883f1982e31ab2865107)
 2017-10-09 14:58:42 +05:30
Xiao Chen
99b7ced3bf HADOOP-14521. KMS client needs retry logic. Contributed by Rushabh S Shah. 
(cherry picked from commit 212342d6db14673f8f5652a5209e1c4fcc0f0ede)

Conflicts:
	hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/crypto/key/kms/LoadBalancingKMSClientProvider.java
 2017-10-05 19:47:26 -07:00
Naganarasimha
d4cd8ccb5d YARN-7285. ContainerExecutor always launches with priorities due to yarn-default property. Contributed by Jason Lowe. 
(cherry picked from commit d0046bb5c6919c9620ffa5330eb37b39484151ad)
 2017-10-06 00:31:24 +05:30
Sidharta S
935ef1c24e YARN-7226. Whitelisted variables do not support delayed variable expansion. Contributed by Jason Lowe 2017-10-03 18:44:36 -07:00
Jason Lowe
3629c95c51 YARN-5444. Fix failing unit tests in TestLinuxContainerExecutorWithMocks. Contributed by Yufei Gu. 
(cherry picked from commit 95694b70cd38dfc591dc4f0e9413809cf3c91dbe)
 2017-10-03 14:44:04 -05:00
Eric Payne
4540cbb3a0 YARN-7084. TestSchedulingMonitor#testRMStarts fails sporadically. Contributed by Jason Lowe. 
(cherry picked from commit 7f6118f918e0fbe81cb64e8b9f6959b8d66679f6)
 2017-09-29 10:56:56 -05:00
Jason Lowe
949525b81a HADOOP-14902. LoadGenerator#genFile write close timing is incorrectly calculated. Contributed by Hanisha Koneru 
(cherry picked from commit 6f789fe05766a61b12ca10df3f26ee354eac84aa)

Conflicts:
	hadoop-common-project/hadoop-common/src/test/java/org/apache/hadoop/fs/loadGenerator/LoadGenerator.java
 2017-09-28 16:45:02 -05:00
Jason Lowe
ba5656d5e0 HADOOP-14881. LoadGenerator should use Time.monotonicNow() to measure durations. Contributed by Bharat Viswanadham 
(cherry picked from commit ac05a51bbb2a3fad4e85f9334a3408571967900a)
 2017-09-28 16:27:08 -05:00
Yiqun Lin
fb5de93c46 HDFS-12495. TestPendingInvalidateBlock#testPendingDeleteUnknownBlocks fails intermittently. Contributed by Eric Badger. 
(cherry picked from commit e9b790db9de7c3fbe451bcdc3fd928a27a0d8b4e)
 2017-09-27 10:51:00 +08:00
Daryn Sharp
438291787a [HDFS-12386] Add fsserver defaults call to WebhdfsFileSystem. (Rushabh Shah via daryn) 2017-09-26 15:25:12 -05:00
Jason Lowe
86e092029d HADOOP-14891. Remove references to Guava Objects.toStringHelper. Contributed by Jonathan Eagles 
(cherry picked from commit 0aac5af6e1a2b265cc8faa99b386276ed14faeda)
 2017-09-26 10:13:24 -05:00
Eric Payne
f794adde3b YARN-7249. Fix CapacityScheduler NPE issue when a container preempted while the node is being removed. Contributed by Wangda Tan. 2017-09-26 09:07:04 -05:00
Akira Ajisaka
75aed26511
HADOOP-14893. WritableRpcEngine should use Time.monotonicNow. Contributed by Chetna Chaudhari. 
(cherry picked from commit d08b8c801a908b4242e7b21a54f3b1e4072f1eae)
(cherry picked from commit b017af23911c08d1f7e928007ebd42dbdbd82507)
 2017-09-26 16:48:28 +09:00
Akira Ajisaka
ea7e655897
HADOOP-14892. MetricsSystemImpl should use Time.monotonicNow for measuring durations. Contributed by Chetna Chaudhari. 
(cherry picked from commit 3a10367a17b99e2c3035ec54048f94f334d93227)
(cherry picked from commit 6004fb46761810ef87367c5207fafcdbb6507c39)
 2017-09-26 01:48:54 +09:00
Jason Lowe
44b51b0047 Revert "YARN-6570. No logs were found for running application, running" 
This reverts commit e103acc8f495135e52ec1e700dc33747e735967e.
 2017-09-25 11:13:50 -05:00
Anu Engineer
3ecc1431fc
HDFS-12531. Fix conflict in the javadoc of UnderReplicatedBlocks.java in branch-2. Contributed by Bharat Viswanadham. 
(cherry picked from commit de5744c9035308654c7096bf5bd04e769a3a19e3)
 2017-09-25 13:12:27 +09:00
Xuan
e103acc8f4 YARN-6570. No logs were found for running application, running 
container. (Contributed by Junping Du)
 2017-09-22 14:48:12 -07:00
Sunil G
1132f48253 YARN-7116. CapacityScheduler Web UI: Queue's AM usage is always show on per-user's AM usage. Contributed by Wangda Tan. 
(cherry picked from commit f9e0cc8cdca387bf2811c07beccae4af07362952)
 2017-09-22 11:52:30 -05:00
Akira Ajisaka
dade7396a2
MAPREDUCE-6966. DistSum should use Time.monotonicNow for measuring durations. Contributed by Chetna Chaudhari. 
(cherry picked from commit c71d13796039815350d3c73c51f9fe48531af0fa)
(cherry picked from commit d80e2f2eae1d584b99da305d715a3f5e958114e9)
 2017-09-22 15:10:42 +09:00
Junping Du
1b3503e4b9 YARN-7034. DefaultLinuxContainerRuntime and DockerLinuxContainerRuntime sends client environment variables to container-executor. Contributed by Miklos Szegedi. 2017-09-21 14:18:50 -07:00
Jason Lowe
b0b10fc94a YARN-5195. RM intermittently crashed with NPE while handling APP_ATTEMPT_REMOVED event when async-scheduling enabled in CapacityScheduler. Contributed by sandflee and Jonathan Hung 2017-09-21 15:17:03 -05:00
Kihwal Lee
c0bb242360 HDFS-12371. BlockVerificationFailures and BlocksVerified show up as 0 in Datanode JMX. Contributed by Hanisha Koneru. 
(cherry picked from commit 6bf921a5c3152a307b5c0903056d73ce07775a08)
 2017-09-21 09:00:26 -05:00
Akira Ajisaka
4ac3fc10c2
HDFS-12526. FSDirectory should use Time.monotonicNow for durations. Contributed by Bharat Viswanadham. 
(cherry picked from commit 50849ec9eb9bad90586a95d2b2380ee6e8724d6b)
(cherry picked from commit 3527327f349264ae62f3673ab368505a16cc0a34)
 2017-09-21 19:29:18 +09:00
Sangjin Lee
6d6758f751 YARN-6771. Use classloader inside configuration class to make new 
classes. Contributed by Jongyoul Lee.

(cherry picked from commit 8b336632acad10e45d029596c5e3196e1857d891)
 2017-09-20 22:50:55 -07:00
Ming Ma
c54310a638 HDFS-12473. Change hosts JSON file format. 2017-09-20 09:21:32 -07:00