Apache
/
hadoop
1
0
Fork 0
Code Issues Pull Requests Packages Projects Releases Wiki Activity
14,660 Commits 326 Branches 371 Tags 1.5 GiB
Commit Graph

14660 Commits

Author SHA1 Message Date
Vrushali C 7ed627af6b YARN-3879 [Storage implementation] Create HDFS backing storage implementation for ATS reads. Contributed by Abhishek Modi. 2018-10-11 21:23:34 -07:00
Wangda Tan 91f18a5148 YARN-7274. Ability to disable elasticity at leaf queue level. (Zian Chen via wangda) 
Change-Id: Ic8d43e297f0f5de788b562f7eff8106c5c35e8d2
(cherry picked from commit 74665e3a7d)
 2018-10-11 16:58:07 +00:00
Steve Loughran 5da49631b6
HADOOP-15679. ShutdownHookManager shutdown time needs to be configurable & extended. 2018-10-11 12:43:09 +01:00
Vrushali C 7a5d27dde4 YARN-6989 Ensure timeline service v2 codebase gets UGI from HttpServletRequest in a consistent way. Contributed by Abhishek Modi 2018-10-10 15:25:58 -07:00
Weiwei Yang cc1bf7f894 YARN-8858. CapacityScheduler should respect maximum node resource when per-queue maximum-allocation is being used. Contributed by Wangda Tan. 
(cherry picked from commit edce866489)
 2018-10-10 10:17:18 +08:00
Weiwei Yang d412785175 YARN-8720. CapacityScheduler does not enforce max resource allocation check at queue level. Contributed by Tarun Parimi. 
(cherry picked from commit f1a893fdbc)
 2018-10-10 10:16:16 +08:00
Inigo Goiri b0900ad310 YARN-7652. Handle AM register requests asynchronously in FederationInterceptor. Contributed by Botong Huang. 
(cherry picked from commit c3d22d3b45)
 2018-10-09 10:30:42 -07:00
Vidura Mudalige fa4a111037
HADOOP-15818. Fix deprecated maven-surefire-plugin configuration in hadoop-kms module 
This closes #425

Signed-off-by: Akira Ajisaka <aajisaka@apache.org>
(cherry picked from commit a30b1d1824)
 2018-10-09 12:57:42 +09:00
Yiqun Lin c632c6e6e9 HDFS-13768. Adding replicas to volume map makes DataNode start slowly. Contributed by Surendra Singh Lilhore. 2018-10-09 10:20:37 +08:00
Inigo Goiri 5539dd97d2 HDFS-13964. RBF: TestRouterWebHDFSContractAppend fails with No Active Namenode under nameservice. Contributed by Ayush Saxena. 
(cherry picked from commit 3bfff0f79e)
 2018-10-06 10:43:36 -07:00
Jason Lowe d7e9ea37bf HADOOP-15820. ZStandardDecompressor native code sets an integer field as a long. Contributed by Jason Lowe 
(cherry picked from commit f13e231025)
 2018-10-05 09:30:50 -05:00
Yiqun Lin 7b88a57c37 HADOOP-15817. Reuse Object Mapper in KMSJSONReader. Contributed by Jonathan Eagles. 
(cherry picked from commit 81f635f47f)
 2018-10-04 10:47:22 +08:00
Akira Ajisaka 65aaa10173
YARN-8840. Add missing cleanupSSLConfig() call for TestTimelineClient test. Contributed by Aki Tanaka. 
(cherry picked from commit 39b35036ba)
 2018-10-04 10:15:37 +09:00
Daniel Templeton 1505058440
MAPREDUCE-6896. Document wrong spelling in usage of MapredTestDriver tools. 
(Contributed by LiXin Ge via Daniel Templeton)

(cherry picked from commit 036a24bab3)
 2018-10-04 10:12:57 +09:00
Giovanni Matteo Fumarola 57caab6f9b YARN-8760. [AMRMProxy] Fix concurrent re-register due to YarnRM failover in AMRMClientRelayer. Contributed by Botong Huang. 2018-10-01 14:27:01 -07:00
Eric E Payne d7d0e55e0a YARN-8774. Memory leak when CapacityScheduler allocates from reserved container with non-default label. Contributed by Tao Yang. 2018-09-28 16:10:31 +00:00
Jason Lowe 1b0a110501 YARN-8804. resourceLimits may be wrongly calculated when leaf-queue is blocked in cluster with 3+ level queues. Contributed by Tao Yang 
(cherry picked from commit 6b988d821e)
 2018-09-26 17:06:07 -07:00
Brahma Reddy Battula 3d67080a45 HDFS-13790. RBF: Move ClientProtocol APIs to its own module Contributed by Chao Sun. 2018-09-26 22:46:43 +05:30
Arun Suresh 4d69741a61 YARN-7974. Allow updating application tracking url after registration. (Jonathan Hung via asuresh) 2018-09-26 00:08:10 -07:00
Sammi Chen c617dba497 HADOOP-15671. AliyunOSS: Support Assume Roles in AliyunOSS. Contributed by Jinhu Wu. 
(cherry picked from commit 2b635125fb)
(cherry picked from commit 5da3e83597)
(cherry picked from commit 85e00477b8)
 2018-09-25 19:55:13 +08:00
Giovanni Matteo Fumarola 6937925838 YARN-8696. [AMRMProxy] FederationInterceptor upgrade: home sub-cluster heartbeat async. Contributed by Botong Huang. 2018-09-24 11:40:07 -07:00
Giovanni Matteo Fumarola 60565976e1 YARN-8658. [AMRMProxy] Metrics for AMRMClientRelayer inside FederationInterceptor. Contributed by Young Chen. 2018-09-21 10:36:36 -07:00
Chen Liang 3a6ad9cd3d HADOOP-15726. Create utility to limit frequency of log statements. Contributed by Erik Krogen. 2018-09-20 10:00:13 -07:00
Steve Loughran 5b77991548
HADOOP-15748. S3 listing inconsistency can raise NPE in globber. 
Contributed by Steve Loughran.

(cherry picked from commit 646874c326)
 2018-09-20 13:07:07 +01:00
Arpit Agarwal 0dd686114f HADOOP-15772. Remove the 'Path ... should be specified as a URI' warnings on startup. Contributed by Ayush Saxena. 
(cherry picked from commit 2eb597b151)
 2018-09-19 13:49:16 -07:00
Inigo Goiri 3e476dcae8 HDFS-13908. TestDataNodeMultipleRegistrations is flaky. Contributed by Ayush Saxena. 
(cherry picked from commit 56e0d635e0)
 2018-09-19 09:32:35 -07:00
Jason Lowe 726aee360a MAPREDUCE-7138. ThrottledContainerAllocator in MRAppBenchmark should implement RMHeartbeatHandler. Contributed by Oleksandr Shevchenko 
(cherry picked from commit 8382b860d4)

Conflicts:
	hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-app/src/test/java/org/apache/hadoop/mapreduce/v2/app/MRAppBenchmark.java
 2018-09-18 17:19:21 -05:00
Jason Lowe f5cc9f71e6 MAPREDUCE-7137. MRAppBenchmark.benchmark1() fails with NullPointerException. Contributed by Oleksandr Shevchenko 
(cherry picked from commit 34b2237e42)
 2018-09-18 17:15:58 -05:00
Jason Lowe 88e2ca405a HADOOP-15755. StringUtils#createStartupShutdownMessage throws NPE when args is null. Contributed by Lokesh Jain and Dinesh Chitlangia 
(cherry picked from commit e71f61ecb8)

Conflicts:
	hadoop-common-project/hadoop-common/src/test/java/org/apache/hadoop/util/TestStringUtils.java
 2018-09-18 16:05:47 -05:00
Eric E Payne 97a13eb046 MAPREDUCE-3801: org.apache.hadoop.mapreduce.v2.app.TestRuntimeEstimators.testExponentialEstimator fails intermittently. Contributed by Jason Lowe 
(cherry picked from commit 2cf8927683)
 2018-09-18 20:54:07 +00:00
Sammi Chen f6ab07e28c HADOOP-15759. AliyunOSS: Update oss-sdk version to 3.0.0. Contributed by Jinhu Wu. 
(cherry picked from commit e4fca6aae4)
(cherry picked from commit 53528d5ec5)
(cherry picked from commit 61847f4ea0)
 2018-09-18 20:18:15 +08:00
Inigo Goiri 9bd2616138 HDFS-13844. Fix the fmt_bytes function in the dfs-dust.js. Contributed by yanghuafeng. 
(cherry picked from commit a9c1c55cafc0338bb35d2991f217d4157463f1d3)
 2018-09-17 14:40:00 -07:00
Jason Lowe 1e61f389e9 MAPREDUCE-7133. History Server task attempts REST API returns invalid data. Contributed by Oleksandr Shevchenko 
(cherry picked from commit 2886024ac3)
 2018-09-13 14:51:19 -05:00
Inigo Goiri bbd09877d0 HDFS-13914. Fix DN UI logs link broken when https is enabled after HDFS-13902. Contributed by Jianfei Jiang. 
(cherry picked from commit e1b242a984)
 2018-09-13 09:18:40 -07:00
Brahma Reddy Battula 9befa5ed95 HDFS-13902. Add JMX, conf and stacks menus to the datanode page. Contributed by fengchuang. 
(cherry picked from commit b2432d254c)
 2018-09-12 06:37:25 +05:30
Xiao Chen a7b9e7d858 HDFS-13051. Fix dead lock during async editlog rolling if edit queue is full. Contributed by Daryn Sharp. 
(cherry picked from commit 8e54da1511)

 Conflicts:
	hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/namenode/TestEditLogRace.java

(cherry picked from commit 2dd27c999b)

 Conflicts:
	hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/namenode/TestEditLogRace.java

(cherry picked from commit a5da4df73c)
 2018-09-10 23:13:14 -07:00
Eric E Payne 121cefd472 YARN-8709: CS preemption monitor always fails since one under-served queue was deleted. Contributed by Tao Yang. 
(cherry picked from commit 987d8191ad)
 2018-09-10 20:32:20 +00:00
Brahma Reddy Battula e56f183696 HDFS-12716. 'dfs.datanode.failed.volumes.tolerated' to support minimum number of volumes to be available. Contributed by Ranith Sardar 2018-09-10 07:11:26 +05:30
Giovanni Matteo Fumarola 6c8fc8f786 HADOOP-15731. TestDistributedShell fails on Windows. Contributed by Botong Huang. 2018-09-07 14:20:28 -07:00
Inigo Goiri 1b12925d69 HDFS-13836. RBF: Handle mount table znode with null value. Contributed by yanghuafeng. 
(cherry picked from commit 527288ef89)
 2018-09-06 16:50:09 -07:00
Jason Lowe 3eb70d4dcb MAPREDUCE-7131. Job History Server has race condition where it moves files from intermediate to finished but thinks file is in intermediate. Contributed by Anthony Hsu 
(cherry picked from commit eb0b5a844f)
 2018-09-06 15:27:45 -05:00
Vrushali C f6e4b13d5f HADOOP-15657 Registering MutableQuantiles via Metric annotation. Contributed by Sushil Ks 2018-09-05 21:09:16 -07:00
Hrishikesh Gadre 0468b6e736 HDFS-13812. Fix the inconsistent default refresh interval on Caching documentation. Contributed by Hrishikesh Gadre. 
Signed-off-by: Xiao Chen <xiao@apache.org>
(cherry picked from commit 6ccb809c2d)
 2018-09-04 21:55:09 -07:00
Inigo Goiri 2e744bd31d HDFS-13857. RBF: Choose to enable the default nameservice to read/write files. Contributed by yanghuafeng. 
(cherry picked from commit 54f2044595)
 2018-09-04 12:18:38 -07:00
Steve Loughran 5902c0658e HADOOP-10219. ipc.Client.setupIOstreams() needs to check for ClientCache.stopClient requested shutdowns. 
Contributed by Kihwal Lee and Lukas Majercak.

(cherry picked from commit 9e96ac666d)
 2018-09-04 11:58:56 -07:00
Steve Loughran 4e342f603f
HADOOP-15547/ WASB: improve listStatus performance. 
Contributed by Thomas Marquardt.
 2018-09-04 16:32:22 +01:00
Eric E Payne 21aa7f1d82 YARN-8051: TestRMEmbeddedElector#testCallbackSynchronization is flakey. Contributed by Robert Kanter and Jason Lowe. 2018-08-29 21:30:38 +00:00
Yiqun Lin 8137b473aa HDFS-13634. RBF: Configurable value in xml for async connection request queue size. Contributed by CR Hota. 
(cherry picked from commit a0ebb6b39f)
 2018-08-29 16:23:17 +08:00
Brahma Reddy Battula 2a8f4aefd4 HDFS-13854. RBF: The ProcessingAvgTime and ProxyAvgTime should display by JMX with ms unit. Contributed by yanghuafeng. 
(cherry picked from commit 64ad0298d4)
 2018-08-29 08:32:36 +05:30
Giovanni Matteo Fumarola bb0e75f72f YARN-8697. LocalityMulticastAMRMProxyPolicy should fallback to random sub-cluster when cannot resolve resource. Contributed by Botong Huang. 2018-08-28 16:07:31 -07:00