Apache/hadoop
1
0
Fork 0
Code Issues Pull Requests Packages Projects Releases Wiki Activity
18,833 Commits 326 Branches 371 Tags
Commit Graph

18833 Commits

This Branch
This Branch
All Branches
Author SHA1 Message Date
Eric Yang
0b97dc5869 YARN-8751. Reduce conditions that mark node manager as unhealthy. 
Contributed by Craig Condit

(cherry picked from commit 7d623343879ce9a8f8e64601024d018efc02794c)
 2018-09-07 20:32:11 -04:00
Giovanni Matteo Fumarola
7fed1a07ab HADOOP-15731. TestDistributedShell fails on Windows. Contributed by Botong Huang. 2018-09-07 16:37:03 -07:00
Inigo Goiri
a400ed3aab HDFS-13836. RBF: Handle mount table znode with null value. Contributed by yanghuafeng. 
(cherry picked from commit 527288ef891dc26019d003bd85ddfd50eb4f3b7b)
 2018-09-06 16:48:44 -07:00
Jason Lowe
4a55092c6d MAPREDUCE-7131. Job History Server has race condition where it moves files from intermediate to finished but thinks file is in intermediate. Contributed by Anthony Hsu 
(cherry picked from commit eb0b5a844f960017f6f48d746174d0f5826f0e5f)
 2018-09-06 14:29:51 -05:00
Wei-Chiu Chuang
d1ea7df43d HADOOP-15696. KMS performance regression due to too many open file descriptors after Jetty migration. Contributed by Wei-Chiu Chuang. 
(cherry picked from commit e780556ae9229fe7a90817eb4e5449d7eed35dd8)

Conflicts:
	hadoop-common-project/hadoop-common/src/test/java/org/apache/hadoop/http/TestHttpServer.java
 2018-09-05 10:17:58 -07:00
Yiqun Lin
c898757f55 HDFS-13815. RBF: Add check to order command. Contributed by Ranith Sardar. 
(cherry picked from commit 9315db5f5da09c2ef86be168465c16932afa2d85)
 2018-09-05 23:35:30 +08:00
Shane Kumpf
2d68708a1d YARN-8638. Allow linux container runtimes to be pluggable. Contributed by Craig Condit 
(cherry picked from commit dffb7bfe6cd2292162f08ec0bded736bc5194c3f)
 2018-09-05 06:55:25 -06:00
Hrishikesh Gadre
fde3b5ac22 HDFS-13812. Fix the inconsistent default refresh interval on Caching documentation. Contributed by Hrishikesh Gadre. 
Signed-off-by: Xiao Chen <xiao@apache.org>
(cherry picked from commit 6ccb809c2d38a45e716153ba16e135cb76167b2b)
 2018-09-04 21:54:43 -07:00
Inigo Goiri
a26565960a HDFS-13857. RBF: Choose to enable the default nameservice to read/write files. Contributed by yanghuafeng. 
(cherry picked from commit 54f2044595206455484284b43e5976c8a1982aaf)
 2018-09-04 12:17:49 -07:00
Steve Loughran
142d878c90 HADOOP-10219. ipc.Client.setupIOstreams() needs to check for ClientCache.stopClient requested shutdowns. 
Contributed by Kihwal Lee and Lukas Majercak.

(cherry picked from commit 9e96ac666d783376a8cdea9c3cc84098c5bdcb56)
 2018-09-04 12:00:31 -07:00
Vinayakumar B
9cf35d99b9 HDFS-13774. EC: 'hdfs ec -getPolicy' is not retrieving policy details when the special REPLICATION policy set on the directory. Contributed by Ayush Saxena. 
(cherry picked from commit 3801436e49822c724c8f4e393e6e5abbd0d2573a)
 2018-09-03 14:39:11 +05:30
Vinayakumar B
6776db5aec HDFS-13867. RBF: Add validation for max arguments for Router admin ls, clrQuota, setQuota, rm and nameservice commands. Contributed by Ayush Saxena. 
(cherry picked from commit 780df9034f265a8e602856b34cc21d9be02f5c48)
 2018-09-03 14:29:28 +05:30
bibinchundatt
e2e0fc26a2 YARN-8535. Fix DistributedShell unit tests. Contributed by Abhishek Modi. 
(cherry picked from commit eed8415dc18fa7415ebd105350bd0532b3b1b6bb)
 2018-09-02 13:37:38 +05:30
Wei-Chiu Chuang
9f0cd2ade2 Revert "HDFS-13838. WebHdfsFileSystem.getFileStatus() won't return correct "snapshot enabled" status. Contributed by Siyao Meng." 
This reverts commit 41732e7b28bb70f7ba56cf6131ac10c062f93fc3.
 2018-08-30 11:58:42 -07:00
Vinayakumar B
65977e5d81 HDFS-13027. Handle possible NPEs due to deleted blocks in race condition. Contributed by Vinayakumar B. 2018-08-30 22:14:47 +05:30
Daniel Templeton
68301ad7d3 HADOOP-15706. Typo in compatibility doc: SHOUD -> SHOULD 
(Contributed by Laszlo Kollar via Daniel Templeton)

Change-Id: I6e2459d0700df7f3bad4eac8297a11690191c3ba
(cherry picked from commit f2c2a68ec208f640e778fc41f95f0284fcc44729)
 2018-08-30 09:14:29 -07:00
Steve Loughran
a0766bf66a
HADOOP-15107. Stabilize/tune S3A committers; review correctness & docs. 
Contributed by Steve Loughran.

(cherry picked from commit 5a0babf76550f63dad4c17173c4da2bf335c6532)
 2018-08-30 15:23:08 +01:00
Steve Loughran
6f939d4294
HADOOP-15680. ITestNativeAzureFileSystemConcurrencyLive times out. 
Contributed by Andras Bokor.

(cherry picked from commit e8d138ca7c1b695688515d816ac693437c87df62)
 2018-08-30 14:37:07 +01:00
Kitti Nanasi
12eb9cc3bb HADOOP-15698. KMS log4j is not initialized properly at startup. Contributed by Kitti Nanasi. 
(cherry picked from commit 781437c219dc3422797a32dc7ba72cd4f5ee38e2)
 2018-08-29 22:09:05 -07:00
Yiqun Lin
8a3be0d5a4 HDFS-13863. FsDatasetImpl should log DiskOutOfSpaceException. Contributed by Fei Hui. 
(cherry picked from commit 582cb10ec74ed5666946a3769002ceb80ba660cb)
 2018-08-30 11:22:57 +08:00
Daniel Templeton
90c8cca783 HADOOP-15705. Typo in the definition of "stable" in the interface classification 
Change-Id: I3eae2143400a534903db4f186400561fc8d2bd56
(cherry picked from commit d53a10b0a552155de700e396fd7f450a4c5f9c22)
 2018-08-29 14:00:42 -07:00
Shane Kumpf
b8618556ee YARN-8642. Add support for tmpfs mounts with the Docker runtime. Contributed by Craig Condit 
(cherry picked from commit 73625168c0f29aa646d7a715c9fb15e43d6c7e05)
 2018-08-29 07:11:38 -06:00
Yiqun Lin
b9b2b97e9d HDFS-13634. RBF: Configurable value in xml for async connection request queue size. Contributed by CR Hota. 
(cherry picked from commit a0ebb6b39f2932d3ea2fb5e287f52b841e108428)
 2018-08-29 16:18:15 +08:00
Weiwei Yang
f164568b47 YARN-8723. Fix a typo in CS init error message when resource calculator is not correctly set. Contributed by Abhishek Modi. 
(cherry picked from commit 3fa46394214181ed1cc7f06b886282bbdf67a10f)
 2018-08-29 11:15:02 +08:00
Brahma Reddy Battula
c01c240f6f HDFS-13854. RBF: The ProcessingAvgTime and ProxyAvgTime should display by JMX with ms unit. Contributed by yanghuafeng. 
(cherry picked from commit 64ad0298d441559951bc9589a40f8aab17c93a5f)
 2018-08-29 08:31:19 +05:30
Zsolt Venczel
5cbb9b1ca9 HDFS-13731. ReencryptionUpdater fails with ConcurrentModificationException during processCheckpoints. Contributed by Zsolt Venczel. 
(cherry picked from commit 3e18b957ebdf20925224ab9c28e6c2f4b6bbdb24)
 2018-08-28 15:14:13 -07:00
Brahma Reddy Battula
450ba6790d HDFS-13861. RBF: Illegal Router Admin command leads to printing usage for all commands. Contributed by Ayush Saxena. 
(cherry picked from commit cb9d371ae2cda1624fc83316ddc09de37d8d0bd3)
 2018-08-29 00:31:16 +05:30
Yiqun Lin
2cbc3c7d43 HDFS-13831. Make block increment deletion number configurable. Contributed by Ryan Wu. 
(cherry picked from commit b9b964d25335943fb15cdfcf369d123bbd7e454a)
 2018-08-28 09:36:02 -07:00
Vinayakumar B
9265d1a61a HDFS-13858. RBF: Add check to have single valid argument to safemode command. Contributed by Ayush Saxena. 
(cherry picked from commit 75691ad600473d4d315434b0876d6d10d3050a6b)
 2018-08-28 09:21:42 +05:30
Wei-Chiu Chuang
41732e7b28 HDFS-13838. WebHdfsFileSystem.getFileStatus() won't return correct "snapshot enabled" status. Contributed by Siyao Meng. 
(cherry picked from commit 26c2a97c566969f50eb8e8432009724c51152a98)
 2018-08-27 16:03:52 -07:00
John Zhuge
84d761d49c
HADOOP-15633. fs.TrashPolicyDefault: Can't create trash directory. Contributed by Fei Hui. 
(cherry picked from commit f9c6fd94711458b77ecf3fa425aad7fda5089376)
 2018-08-27 15:21:39 -07:00
Billie Rinaldi
eefd780918 YARN-8675. Remove default hostname for docker containers when net=host. Contributed by Suma Shivaprasad 
(cherry picked from commit 05b2bbeb357d4fa03e71f2bfd5d8eeb0ea6c3f60)
 2018-08-27 11:42:09 -07:00
Weiwei Yang
271ed86ac0 YARN-8719. Typo correction for yarn configuration in OpportunisticContainers(federation) docs. Contributed by Y. SREENIVASULU REDDY. 
(cherry picked from commit e8b063f63049d781f4bd67e2ac928c03fd7b7941)
 2018-08-28 01:07:05 +08:00
Konstantin V Shvachko
60fd202a59 HDFS-13848. Refactor NameNode failover proxy providers. Contributed by Konstantin Shvachko. 
(cherry picked from commit a4121c71c29d13866a605d9c0d013e5de9c147c3)
 2018-08-24 18:44:04 -07:00
Haibo Chen
e4282c077b YARN-8051. TestRMEmbeddedElector#testCallbackSynchronization is flaky. (Robert Kanter via Haibo Chen) 
(cherry picked from commit 93d47a0ed504ee81d4b74d340c1815bdbb3c9b14)

Conflicts:
	hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/test/java/org/apache/hadoop/yarn/server/resourcemanager/TestRMEmbeddedElector.java
 2018-08-24 13:24:08 -05:00
Inigo Goiri
246086984e YARN-7773. YARN Federation used Mysql as state store throw exception, Unknown column 'homeSubCluster' in 'field list'. Contributed by Yiran Wu. 
(cherry picked from commit 60e5c1b516647bf99ef3e9501c0ce04deea13ce9)
 2018-08-24 07:10:02 +05:30
Jason Lowe
84654451fa YARN-8649. NPE in localizer hearbeat processing if a container is killed while localizing. Contributed by lujie 
(cherry picked from commit 585ebd873a55bedd2a364d256837f08ada8ba032)
 2018-08-23 09:37:43 -05:00
Takanobu Asanuma
4fa97013a9 HADOOP-14314. The OpenSolaris taxonomy link is dead in InterfaceClassification.md. Contributed by Rui Gao. 
(cherry picked from commit bed8cb6979e0460141ed77e3b15d4f18db098a8e)
 2018-08-23 21:55:40 +09:00
Yiqun Lin
0d73497aa7 HDFS-13821. RBF: Add dfs.federation.router.mount-table.cache.enable so that users can disable cache. Contributed by Fei Hui. 
(cherry picked from commit 81847392badcd58d934333e7c3b5bf14b4fa1f3f)
 2018-08-22 13:05:23 +08:00
Rohith Sharma K S
63d5214332 YARN-8129. Improve error message for invalid value in fields attribute. Contributed by Abhishek Modi. 
(cherry picked from commit d3fef7a5c5b83d27e87b5e49928254a7d1b935e5)
 2018-08-21 12:11:12 +05:30
Vinayakumar B
8df2eb8119 HDFS-13772. Erasure coding: Unnecessary NameNode Logs displaying for Enabling/Disabling Erasure coding policies which are already enabled/disabled. Contributed by Ayush Saxena 2018-08-21 09:34:57 +05:30
Steve Loughran
95fcdc0435
HADOOP-15679. ShutdownHookManager shutdown time needs to be configurable & extended. Contributed by Steve Loughran. 
Change-Id: Ifd4a6e3f796b4dc88e97f63066289e2534c77a29
 2018-08-20 18:48:49 -07:00
Xiao Chen
3712b79b38 HADOOP-9214. Create a new touch command to allow modifying atime and mtime. Contributed by Hrishikesh Gadre. 2018-08-20 14:59:14 -07:00
Xiao Chen
a630a27c53 HADOOP-15655. Enhance KMS client retry behavior. Contributed by Kitti Nanasi. 2018-08-20 14:57:51 -07:00
Wei-Chiu Chuang
865650052b HDFS-10240. Race between close/recoverLease leads to missing block. Contributed by Jinglun, zhouyingchao and Wei-Chiu Chuang. 2018-08-20 14:57:40 -07:00
Xiao Chen
0424715207 HDFS-13732. ECAdmin should print the policy name when an EC policy is set. Contributed by Zsolt Venczel. 2018-08-20 14:57:07 -07:00
Xiao Chen
81fbfe500e HDFS-13217. Audit log all EC policy names during addErasureCodingPolicies. Contributed by liaoyuxiangqin. 2018-08-20 14:56:47 -07:00
Daniel Templeton
c15853f87a HDFS-13819. TestDirectoryScanner#testDirectoryScannerInFederatedCluster is flaky 
Change-Id: I1cea6e67fcec72702ad202775dee3373261ac5cd
 2018-08-20 14:54:58 -07:00
drankye
975d60685e HDFS-13668. FSPermissionChecker may throws AIOOE when check inode permission. Contributed by He Xiaoqiao. 2018-08-20 14:53:32 -07:00
Xiao Chen
c0ac0a5337 HDFS-13658. Expose HighestPriorityLowRedundancy blocks statistics. Contributed by Kitti Nanasi. 2018-08-20 14:52:28 -07:00