Apache/hadoop
1
0
Fork 0
Code Issues Pull Requests Packages Projects Releases Wiki Activity
12,448 Commits 326 Branches 371 Tags
Commit Graph

12448 Commits

This Branch
This Branch
All Branches
Author SHA1 Message Date
Wangda Tan
dd47142565 YARN-7496. CS Intra-queue preemption user-limit calculations are not in line with LeafQueue user-limit calculations. (Eric Payne via wangda) 
Change-Id: I4dada78a227408a1f2d9bc18099041aad81d67d7
 2017-11-23 20:18:22 -08:00
Akira Ajisaka
a9f0536ced
HADOOP-15047. Python is required for -Preleasedoc but not documented in branch-2.8. Contributed by Bharat Viswanadham. 2017-11-22 11:13:40 +09:00
bibinchundatt
deb21e8c3a YARN-7525. Incorrect query parameters in cluster nodes REST API document. Contributed by Tao Yang. 2017-11-21 00:34:53 +05:30
bibinchundatt
7e46b770f1 YARN-7489. ConcurrentModificationException in RMAppImpl#getRMAppMetrics. Contributed by Tao Yang. 
(cherry picked from commit 5c2986e58dfc86fada8554a3f48cd417a4458abc)
 2017-11-20 21:37:33 +05:30
Sunil G
b53d244b79 YARN-7469. Capacity Scheduler Intra-queue preemption: User can starve if newest app is exactly at user limit. Contributed by Eric Payne. 
(cherry picked from commit 61ace174cdcbca9d22abce7aa0aa71148f37ad55)
 2017-11-16 23:01:29 +05:30
Jason Lowe
d262815564 YARN-7361. Improve the docker container runtime documentation. Contributed by Shane Kumpf 
(cherry picked from commit fac72eef23bb0a74a34f289dd6ef50ffa4303aa4)

Conflicts:
	hadoop-yarn-project/hadoop-yarn/hadoop-yarn-site/src/site/markdown/DockerContainers.md
 2017-11-15 12:39:55 -06:00
Junping Du
d9b006ff34 Preparing for 2.8.4 development. 2017-11-13 16:49:15 -08:00
Arun Suresh
b3df744fb6 Fixing Job History Server Configuration parsing. (Jason Lowe via asuresh) 2017-11-09 15:25:59 -08:00
John Zhuge
90c1fac6e3 HDFS-12318. Fix IOException condition for openInfo in DFSInputStream. Contributed by legend. 
(cherry picked from commit 3123895db0518552eb357093fddf0e8ff1a6cadf)
 2017-11-09 09:22:30 -08:00
Naganarasimha
01912d18d5 MAPREDUCE-6975. Logging task counters. Contributed by Prabhu Joseph. 
(cherry picked from commit 61bbdc511e4e98c11a05d7c3bc01f3e971adbe13)
 2017-11-06 14:09:07 +05:30
Eric Payne
85130d80fe YARN-7370: Preemption properties should be refreshable. Contrubted by Gergely Novák. 
(cherry picked from commit e6ec02001fc4eed9eb51c8653d8f931135e49eda)
 2017-11-02 13:38:36 -05:00
Jason Lowe
2dec52da20 HADOOP-14919. BZip2 drops records when reading data in splits. Contributed by Jason Lowe 
(cherry picked from commit 2fae63aa60c43b62bd908a9499562fe528603185)
 2017-10-31 09:35:32 -05:00
Junping Du
d80e37ec94 HADOOP-14990. Clean up jdiff xml files added for 2.8.2 release. 
(cherry picked from commit a8083aaa6f4bcc7287f2b65018a946a840123c50)
 2017-10-30 15:17:55 -07:00
Jason Lowe
9d3ea2c0ed YARN-7244. ShuffleHandler is not aware of disks that are added. Contributed by Kuhu Shukla 2017-10-30 16:08:14 -05:00
Arpit Agarwal
2d4629f386 HDFS-9914. Fix configurable WebhDFS connect/read timeout. Contributed by Xiaoyu Yao. 2017-10-27 07:45:27 -07:00
Junping Du
5ce84db79f Add several jdiff xml files for 2.8.2 release. 
(cherry picked from commit a25b5aa0cf5189247fa38e7b0a188d568eba1b6c)
(cherry picked from commit 1f2a2ba2be50e4361b3093578a98e09d6f82af5c)
 2017-10-26 23:26:21 -07:00
Junping Du
ae7ff3da09 Set jdiff stable version to 2.8.2. 
(cherry picked from commit 06b56f649376020cde382bde25f90dc0547c4277)
(cherry picked from commit 4b75ebb68eb87e9f18676915652085c30ea78f7e)
 2017-10-26 23:26:07 -07:00
Junping Du
13b042b576 Add CHANGES.md and RELEASENOTES for 2.8.2 release. 
(cherry picked from commit 3793d2b9cfdae97a534a6eb6e38e84e2f00c2b83)
 2017-10-26 23:25:49 -07:00
Zhe Zhang
239dadbd46 Revert "HDFS-12502. nntop should support a category based on FilesInGetListingOps." 
This reverts commit f1ff21e00c21ba2514b81779389141db150882cc.
 2017-10-24 15:04:37 -07:00
Eric Payne
6b0dba318d YARN-4163: Audit getQueueInfo and getApplications calls. Contributed by Chang Li. 2017-10-23 17:37:50 -05:00
Naganarasimha
5ee67c1202 HADOOP-14966. Handle JDK-8071638 for hadoop-common. Contributed by Bibin A Chundatt. 
(cherry picked from commit 6b8122458ede9ececb138bed99d368580e5439a1)
 2017-10-23 23:48:43 +05:30
Wei-Chiu Chuang
a83f87e170 HDFS-12619. Do not catch and throw unchecked exceptions if IBRs fail to process. Contributed by Wei-Chiu Chuang. 
(cherry picked from commit 4ab0c8f96a41c573cc1f1e71c18871d243f952b9)
(cherry picked from commit ce93a2b84badacb6d0b19bbb149afbcf8f38c665)
 2017-10-19 06:19:11 -07:00
Wei-Chiu Chuang
3b47f6bdfd HADOOP-14880. [KMS] Document&test missing KMS client side configs. Contributed by Gabor Bota. 
(cherry picked from commit 97c70c7ac6881f87eee1575bcbdd28b31ecac231)

Conflicts:
	hadoop-common-project/hadoop-kms/src/test/java/org/apache/hadoop/crypto/key/kms/server/TestKMS.java
(cherry picked from commit 21b9e4fb979ecbc8f254b76f6ed86e6ef63d74f3)
 2017-10-19 06:05:53 -07:00
Zhe Zhang
f1ff21e00c HDFS-12502. nntop should support a category based on FilesInGetListingOps. 
(cherry picked from commit 60bfee270ed3a653c44c0bc92396167b5022df6e)
(cherry picked from commit aecf6c50d351e7aed2008a682677a1edb104585d)
(cherry picked from commit 87f440739604f97261da9624afb36920a6a4c28f)
 2017-10-19 00:09:33 -07:00
Junping Du
4935334b0f HADOOP-14958. Fix source-level compatibility after HADOOP-11252. Contributed by Junping Du. 
(cherry picked from commit b016f08f67830ed3ca741bc6a10c3f5164781be5)
(cherry picked from commit 9433f9eb09af0ca61f01a2eb42ff3ffe31a94d5f)
(cherry picked from commit e7e2b8158d9aaf314698784d44f4dd9313df3b50)
 2017-10-18 15:11:21 -07:00
Junping Du
4ea3ae36cb YARN-7230. Document DockerContainerRuntime for branch-2.8 with proper scope and claim as an experimental feature. Contributed by Shane Kumpf. 2017-10-18 14:12:37 -07:00
Arpit Agarwal
dcdb3f250c HDFS-12420. Add an option to disallow 'namenode format -force'. Contributed by Ajay Kumar. 
(cherry picked from commit b6942cbe9b8c9469e8c2b64c3268d671f5a43e75)
(cherry picked from commit 5897095d539be086ed37df011f024e37eb37b0cd)
(cherry picked from commit 0aa1b62b396f6fa43f15900c26dd28b95c4e40ce)
 2017-10-17 23:46:14 -07:00
Xiao Chen
a1d29717a2 Revert "HADOOP-14948. Document missing config key hadoop.treat.subject.external. Contributed by Ajay Kumar." 
This reverts commit 19db2837cbdb883b7c38adf01c2b7f9cad8d49d4.
 2017-10-17 10:11:45 -07:00
Wei-Chiu Chuang
19db2837cb HADOOP-14948. Document missing config key hadoop.treat.subject.external. Contributed by Ajay Kumar. 
(cherry picked from commit e906108fc98a011630d12a43e557b81d7ef7ea5d)
(cherry picked from commit 1452ba139ea845ad9e82a5902d239b59a789bff3)

Conflicts:
	hadoop-common-project/hadoop-common/src/main/resources/core-default.xml
 2017-10-16 16:46:31 -07:00
Nathan Roberts
602a884d55 YARN-7333. container-executor fails to remove entries from a directory that is not writable or executable. Contributed by Jason Lowe. 
(cherry picked from commit 8620140a6a3ec0117675ede06d92d830da3da551)
 2017-10-16 16:45:17 -05:00
Wei-Chiu Chuang
843b8b7b96 HDFS-12485. expunge may fail to remove trash from encryption zone. Contributed by Wei-Chiu Chuang. 
(cherry picked from commit 8dbc8909c92d502d10a7f94d1de3171878a43b04)
(cherry picked from commit a596c6f5eb553cf38123d61b844696ddbec99be0)

Conflicts:
	hadoop-hdfs-project/hadoop-hdfs-client/src/main/java/org/apache/hadoop/hdfs/DistributedFileSystem.java
 2017-10-16 13:04:36 -07:00
Xiao Chen
6a55d0e671 HDFS-12642. Log block and datanode details in BlockRecoveryWorker. 
(cherry picked from commit 971b3ca1acc15d9a1f8e774291813dafd16ef704)
 2017-10-16 10:34:31 -07:00
Xiao Chen
db726c4483 HDFS-12603. Enable async edit logging by default. Contributed by Andrew Wang. 
(cherry picked from commit d0311dfc5fe92a0e0f3fb8ae92e412edce609740)
(cherry picked from commit f6e768104b0e9a087584c25e28bf8cd29802c719)

Conflicts:
	hadoop-hdfs-project/hadoop-hdfs/src/main/resources/hdfs-default.xml
 2017-10-16 09:51:34 -07:00
Xiao Chen
27badf6df9 HDFS-12659. Update TestDeadDatanode#testNonDFSUsedONDeadNodeReReg to increase heartbeat recheck interval. Contributed by Ajay Kumar. 
(cherry picked from commit 11a7729c1474aa379a62c0440c8c1a11be261a75)
 2017-10-15 19:28:45 -07:00
Jason Lowe
b41c5e41bc YARN-7246. Fix the default docker binary path. Contributed by Shane Kumpf 2017-10-13 15:32:35 -05:00
Jason Lowe
ce76ed675e YARN-7325. Remove unused container variable in DockerLinuxContainerRuntime. Contributed by Shane Kumpf 2017-10-13 14:54:06 -05:00
Jason Lowe
5c4f269b26 YARN-6930. Admins should be able to explicitly enable specific LinuxContainerRuntime in the NodeManager. Contributed by Shane Kumpf 2017-10-11 09:57:49 -05:00
Jason Lowe
108c2394c3 HADOOP-14912. FairCallQueue may defer servicing calls. Contributed by Daryn Sharp 
(cherry picked from commit 1123f8f0b62292197f5433cd40e66d8620044608)
 2017-10-10 14:51:19 -05:00
Sunil G
b41a702751 YARN-7245. Max AM Resource column in Active Users Info section of Capacity Scheduler UI page should be updated per-user. Contributed by Eric Payne. 
(cherry picked from commit 8137254a1f4367974bf1883f1982e31ab2865107)
 2017-10-09 14:58:42 +05:30
Xiao Chen
99b7ced3bf HADOOP-14521. KMS client needs retry logic. Contributed by Rushabh S Shah. 
(cherry picked from commit 212342d6db14673f8f5652a5209e1c4fcc0f0ede)

Conflicts:
	hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/crypto/key/kms/LoadBalancingKMSClientProvider.java
 2017-10-05 19:47:26 -07:00
Naganarasimha
d4cd8ccb5d YARN-7285. ContainerExecutor always launches with priorities due to yarn-default property. Contributed by Jason Lowe. 
(cherry picked from commit d0046bb5c6919c9620ffa5330eb37b39484151ad)
 2017-10-06 00:31:24 +05:30
Sidharta S
935ef1c24e YARN-7226. Whitelisted variables do not support delayed variable expansion. Contributed by Jason Lowe 2017-10-03 18:44:36 -07:00
Jason Lowe
3629c95c51 YARN-5444. Fix failing unit tests in TestLinuxContainerExecutorWithMocks. Contributed by Yufei Gu. 
(cherry picked from commit 95694b70cd38dfc591dc4f0e9413809cf3c91dbe)
 2017-10-03 14:44:04 -05:00
Eric Payne
4540cbb3a0 YARN-7084. TestSchedulingMonitor#testRMStarts fails sporadically. Contributed by Jason Lowe. 
(cherry picked from commit 7f6118f918e0fbe81cb64e8b9f6959b8d66679f6)
 2017-09-29 10:56:56 -05:00
Jason Lowe
949525b81a HADOOP-14902. LoadGenerator#genFile write close timing is incorrectly calculated. Contributed by Hanisha Koneru 
(cherry picked from commit 6f789fe05766a61b12ca10df3f26ee354eac84aa)

Conflicts:
	hadoop-common-project/hadoop-common/src/test/java/org/apache/hadoop/fs/loadGenerator/LoadGenerator.java
 2017-09-28 16:45:02 -05:00
Jason Lowe
ba5656d5e0 HADOOP-14881. LoadGenerator should use Time.monotonicNow() to measure durations. Contributed by Bharat Viswanadham 
(cherry picked from commit ac05a51bbb2a3fad4e85f9334a3408571967900a)
 2017-09-28 16:27:08 -05:00
Yiqun Lin
fb5de93c46 HDFS-12495. TestPendingInvalidateBlock#testPendingDeleteUnknownBlocks fails intermittently. Contributed by Eric Badger. 
(cherry picked from commit e9b790db9de7c3fbe451bcdc3fd928a27a0d8b4e)
 2017-09-27 10:51:00 +08:00
Daryn Sharp
438291787a [HDFS-12386] Add fsserver defaults call to WebhdfsFileSystem. (Rushabh Shah via daryn) 2017-09-26 15:25:12 -05:00
Jason Lowe
86e092029d HADOOP-14891. Remove references to Guava Objects.toStringHelper. Contributed by Jonathan Eagles 
(cherry picked from commit 0aac5af6e1a2b265cc8faa99b386276ed14faeda)
 2017-09-26 10:13:24 -05:00
Eric Payne
f794adde3b YARN-7249. Fix CapacityScheduler NPE issue when a container preempted while the node is being removed. Contributed by Wangda Tan. 2017-09-26 09:07:04 -05:00