Apache/hadoop
1
0
Fork 0
Code Issues Pull Requests Packages Projects Releases Wiki Activity
17,759 Commits 326 Branches 371 Tags
Commit Graph

17759 Commits

This Branch
This Branch
All Branches
Author SHA1 Message Date
Sammi Chen
85e00477b8 HADOOP-15671. AliyunOSS: Support Assume Roles in AliyunOSS. Contributed by Jinhu Wu. 
(cherry picked from commit 2b635125fb059fc204ed35bc0e264c42dd3a9fe9)
(cherry picked from commit 5da3e8359757c0c1afaccc1d3a0f2bdc453e0311)
 2018-09-25 19:53:03 +08:00
Siyao Meng
2449795b8e HDFS-13830. Backport HDFS-13141 to branch-3.0: WebHDFS: Add support for getting snasphottable directory list. Contributed by Siyao Meng, Lokesh Jain. 
Signed-off-by: Wei-Chiu Chuang <weichiu@apache.org>
 2018-09-21 12:41:03 -07:00
Steve Loughran
fc6d851f70
HADOOP-15748. S3 listing inconsistency can raise NPE in globber. 
Contributed by Steve Loughran.

(cherry picked from commit 646874c326139457b79cf8cfa547b3c91a78c7b4)
 2018-09-20 13:06:20 +01:00
Vinayakumar B
d2655ea60c HADOOP-15736. Trash : Negative Value For Deletion Interval Leads To Abnormal Behaviour. Contributed by Ayush Saxena. 
(cherry picked from commit 7ad27e97f05b13b33fdcef9cb63ace9c1728bfb5)
(cherry picked from commit 2aa385463305beaae8770c8deb22faed6943ef00)
 2018-09-20 09:34:37 +05:30
Chen Liang
78e0c0a1bb HADOOP-15726. Create utility to limit frequency of log statements. Contributed by Erik Krogen. 2018-09-19 14:51:23 -07:00
Arpit Agarwal
ffef819ee8 HADOOP-15772. Remove the 'Path ... should be specified as a URI' warnings on startup. Contributed by Ayush Saxena. 
(cherry picked from commit 2eb597b1511f8f46866abe4eeec820f4191cc295)
 2018-09-19 13:49:57 -07:00
Inigo Goiri
3fc3a5c01c HADOOP-15684. triggerActiveLogRoll stuck on dead name node, when ConnectTimeoutException happens. Contributed by Rong Tang. 
(cherry picked from commit 7f9a89e1b54a9712af50ffef70bed7cfb91ed34e)
 2018-09-19 12:59:42 -07:00
Inigo Goiri
b93aeb43a4 HDFS-13908. TestDataNodeMultipleRegistrations is flaky. Contributed by Ayush Saxena. 
(cherry picked from commit 56e0d635e0f49772d001c1fdb385110c705a622a)
 2018-09-19 09:31:58 -07:00
Weiwei Yang
4edcde055b Revert "YARN-8771. CapacityScheduler fails to unreserve when cluster resource contains empty resource type. Contributed by Tao Yang." 
This reverts commit f56e36935da69b5b12086d80bfe1cdb446213882.
 2018-09-19 19:47:06 +08:00
Weiwei Yang
f56e36935d YARN-8771. CapacityScheduler fails to unreserve when cluster resource contains empty resource type. Contributed by Tao Yang. 
(cherry picked from commit 0712537e799bc03855d548d1f4bd690dd478b871)
 2018-09-19 19:41:07 +08:00
Wei-Chiu Chuang
823b1fceee HDFS-13868. WebHDFS: GETSNAPSHOTDIFF API NPE when param "snapshotname" is given but "oldsnapshotname" is not. Contributed by Pranay Singh. 
(cherry picked from commit 28ceb34a725cd06d28fb51361c49bb45464f5368)
(cherry picked from commit 7442b7941dc57ebf4bd0ded4d46cfbf5af6fea81)

Conflicts:
	hadoop-hdfs-project/hadoop-hdfs-client/src/main/java/org/apache/hadoop/hdfs/DFSClient.java
 2018-09-19 03:20:26 -07:00
Wei-Chiu Chuang
3a32e84d3e HDFS-13886. HttpFSFileSystem.getFileStatus() doesn't return "snapshot enabled" bit. Contributed by Siyao Meng. 
(cherry picked from commit 44857476fa993fbf9c97f979b91e19d27632c10a)
(cherry picked from commit 595ce945704f37a4df5cb8c9b2c509b8d6a14ec4)
 2018-09-18 15:48:05 -07:00
Wei-Chiu Chuang
2ef3911f7c HDFS-13838. WebHdfsFileSystem.getFileStatus() won't return correct "snapshot enabled" status. Contributed by Siyao Meng. 
(cherry picked from commit a1de8cbac5fb9af403db2a02814575f0940d5f39)
(cherry picked from commit 829399a9f38db40b5bb247cb9c0d5f930c094e37)
 2018-09-18 15:48:04 -07:00
Jason Lowe
f5ebb2ccf5 MAPREDUCE-7138. ThrottledContainerAllocator in MRAppBenchmark should implement RMHeartbeatHandler. Contributed by Oleksandr Shevchenko 
(cherry picked from commit 8382b860d4ef4f20d000537ded42a88e98bd2190)
 2018-09-18 17:15:17 -05:00
Jason Lowe
4a2a7d37dd MAPREDUCE-7137. MRAppBenchmark.benchmark1() fails with NullPointerException. Contributed by Oleksandr Shevchenko 
(cherry picked from commit 34b2237e420cfbe3a97ddd44968de8bbe1ed30ab)
 2018-09-18 17:13:00 -05:00
Jason Lowe
f2eeba5c5b HADOOP-15755. StringUtils#createStartupShutdownMessage throws NPE when args is null. Contributed by Lokesh Jain and Dinesh Chitlangia 
(cherry picked from commit e71f61ecb87e04727a5a76e578a75714c9db6706)

Conflicts:
	hadoop-common-project/hadoop-common/src/test/java/org/apache/hadoop/util/TestStringUtils.java
 2018-09-18 16:03:42 -05:00
Eric E Payne
373c7a0ceb MAPREDUCE-3801: org.apache.hadoop.mapreduce.v2.app.TestRuntimeEstimators.testExponentialEstimator fails intermittently. Contributed by Jason Lowe 
(cherry picked from commit 2cf8927683d49f4cdc3cba038480128c473c5461)
 2018-09-18 20:42:35 +00:00
Sammi Chen
61847f4ea0 HADOOP-15759. AliyunOSS: Update oss-sdk version to 3.0.0. Contributed by Jinhu Wu. 
(cherry picked from commit e4fca6aae46a3c04fc56897986a4ab4e5aa98503)
(cherry picked from commit 53528d5ec5ab812484e2d5a6622d58ba8c86a5b0)
 2018-09-18 18:45:49 +08:00
Inigo Goiri
f0537b5161 HDFS-13844. Fix the fmt_bytes function in the dfs-dust.js. Contributed by yanghuafeng. 
(cherry picked from commit a9c1c55cafc0338bb35d2991f217d4157463f1d3)
 2018-09-17 14:39:31 -07:00
Weiwei Yang
14b9e64d80 YARN-8782. Fix exception message in Resource.throwExceptionWhenArrayOutOfBound. Contributed by Gergely Pollak. 
(cherry picked from commit 9a265fa673ef1b8774cfd69c76cdd29bf344e79d)
 2018-09-17 22:27:59 +08:00
Weiwei Yang
ef17728c58 YARN-8720. CapacityScheduler does not enforce max resource allocation check at queue level. Contributed by Tarun Parimi. 
(cherry picked from commit f1a893fdbc2dbe949cae786f08bdb2651b88d673)
 2018-09-14 16:43:28 +08:00
Jason Lowe
f918e4d894 MAPREDUCE-7133. History Server task attempts REST API returns invalid data. Contributed by Oleksandr Shevchenko 
(cherry picked from commit 2886024ac3a8613ecc27f1595b278ce6fc2d03ba)
 2018-09-13 14:47:37 -05:00
Inigo Goiri
1314dc2397 HDFS-13914. Fix DN UI logs link broken when https is enabled after HDFS-13902. Contributed by Jianfei Jiang. 
(cherry picked from commit e1b242a9844d6438c93cdc2ab2443d042b4d5aea)
 2018-09-13 09:18:11 -07:00
Weiwei Yang
4e105a4b9b YARN-8729. Node status updater thread could be lost after it is restarted. Contributed by Tao Yang. 
(cherry picked from commit 39c1ea1ed454b6c61f0985fc951f20913ed964fb)
 2018-09-13 23:18:16 +08:00
Brahma Reddy Battula
0c8aab690c HDFS-13902. Add JMX, conf and stacks menus to the datanode page. Contributed by fengchuang. 
(cherry picked from commit b2432d254c486d0e360b76b7b094a71a011678ab)
 2018-09-12 06:35:29 +05:30
Xiao Chen
a5da4df73c HDFS-13051. Fix dead lock during async editlog rolling if edit queue is full. Contributed by Daryn Sharp. 
(cherry picked from commit 8e54da1511e78477c1d4655d5ff0a69d0330869f)

 Conflicts:
	hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/namenode/TestEditLogRace.java

(cherry picked from commit 2dd27c999b22c550058de0e6eca7209b346cd143)

 Conflicts:
	hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/namenode/TestEditLogRace.java
 2018-09-10 23:00:41 -07:00
Eric E Payne
48dc8de28b YARN-8709: CS preemption monitor always fails since one under-served queue was deleted. Contributed by Tao Yang. 
(cherry picked from commit 987d8191ad409298570f7ef981e9bc8fb72ff16c)
 2018-09-10 20:18:14 +00:00
Giovanni Matteo Fumarola
37032a3aca HADOOP-15731. TestDistributedShell fails on Windows. Contributed by Botong Huang. 2018-09-07 16:37:15 -07:00
Inigo Goiri
a9392bdb17 HDFS-13836. RBF: Handle mount table znode with null value. Contributed by yanghuafeng. 
(cherry picked from commit 527288ef891dc26019d003bd85ddfd50eb4f3b7b)
 2018-09-06 16:49:14 -07:00
Jason Lowe
09e1851b0d MAPREDUCE-7131. Job History Server has race condition where it moves files from intermediate to finished but thinks file is in intermediate. Contributed by Anthony Hsu 
(cherry picked from commit eb0b5a844f960017f6f48d746174d0f5826f0e5f)
 2018-09-06 15:14:38 -05:00
Wei-Chiu Chuang
e99ed05444 HADOOP-15696. KMS performance regression due to too many open file descriptors after Jetty migration. Contributed by Wei-Chiu Chuang. 
(cherry picked from commit e780556ae9229fe7a90817eb4e5449d7eed35dd8)

Conflicts:
	hadoop-common-project/hadoop-common/src/test/java/org/apache/hadoop/http/TestHttpServer.java

(cherry picked from commit d1ea7df43d6a5623b139acbbc9992831d5c7e484)
 2018-09-05 10:18:56 -07:00
Steve Loughran
9ee9d68e4b HADOOP-10219. ipc.Client.setupIOstreams() needs to check for ClientCache.stopClient requested shutdowns. 
Contributed by Kihwal Lee and Lukas Majercak.

(cherry picked from commit 9e96ac666d783376a8cdea9c3cc84098c5bdcb56)
(cherry picked from commit 142d878c902da6bc02d0c21cdd5ca8674da7f5c8)
 2018-09-04 22:25:24 -07:00
Hrishikesh Gadre
25337c045a HDFS-13812. Fix the inconsistent default refresh interval on Caching documentation. Contributed by Hrishikesh Gadre. 
Signed-off-by: Xiao Chen <xiao@apache.org>
(cherry picked from commit 6ccb809c2d38a45e716153ba16e135cb76167b2b)
 2018-09-04 21:54:56 -07:00
Inigo Goiri
54d3189cfa HDFS-13857. RBF: Choose to enable the default nameservice to read/write files. Contributed by yanghuafeng. 
(cherry picked from commit 54f2044595206455484284b43e5976c8a1982aaf)
 2018-09-04 12:18:08 -07:00
bibinchundatt
5514f02a73 YARN-8535. Fix DistributedShell unit tests. Contributed by Abhishek Modi. 
(cherry picked from commit eed8415dc18fa7415ebd105350bd0532b3b1b6bb)
 2018-09-02 13:39:26 +05:30
Wei-Chiu Chuang
d06717ef94 Revert "HDFS-13838. WebHdfsFileSystem.getFileStatus() won't return correct "snapshot enabled" status. Contributed by Siyao Meng." 
This reverts commit d89a485b2545c759a0f62810ff9757ede6d622f2.
 2018-08-30 11:59:39 -07:00
Vinayakumar B
67a5491962 HDFS-13027. Handle possible NPEs due to deleted blocks in race condition. Contributed by Vinayakumar B. 
(cherry picked from commit 65977e5d8124be2bc208af25beed934933f170b3)
 2018-08-30 22:17:00 +05:30
Daniel Templeton
9a02c4ff65 HADOOP-15706. Typo in compatibility doc: SHOUD -> SHOULD 
(Contributed by Laszlo Kollar via Daniel Templeton)

Change-Id: I6e2459d0700df7f3bad4eac8297a11690191c3ba
(cherry picked from commit f2c2a68ec208f640e778fc41f95f0284fcc44729)
(cherry picked from commit 68301ad7d36d516e265d34dd47c344239e3c6c7d)
 2018-08-30 09:15:25 -07:00
Kitti Nanasi
fa32269cee HADOOP-15698. KMS log4j is not initialized properly at startup. Contributed by Kitti Nanasi. 
(cherry picked from commit 781437c219dc3422797a32dc7ba72cd4f5ee38e2)
 2018-08-29 22:09:13 -07:00
Yiqun Lin
6a547856ef HDFS-13863. FsDatasetImpl should log DiskOutOfSpaceException. Contributed by Fei Hui. 
(cherry picked from commit 582cb10ec74ed5666946a3769002ceb80ba660cb)
 2018-08-30 11:24:25 +08:00
Daniel Templeton
fbaa11ef4c HADOOP-15705. Typo in the definition of "stable" in the interface classification 
Change-Id: I3eae2143400a534903db4f186400561fc8d2bd56
(cherry picked from commit d53a10b0a552155de700e396fd7f450a4c5f9c22)
(cherry picked from commit 90c8cca78389f534429ebf81576ce7ebba9356f7)
 2018-08-29 14:01:19 -07:00
Brahma Reddy Battula
3b01279612 HDFS-13854. RBF: The ProcessingAvgTime and ProxyAvgTime should display by JMX with ms unit. Contributed by yanghuafeng. 
(cherry picked from commit 64ad0298d441559951bc9589a40f8aab17c93a5f)
 2018-08-29 08:32:05 +05:30
Wei-Chiu Chuang
72e1e1904b HDFS-13831. Make block increment deletion number configurable. Contributed by Ryan Wu. 2018-08-28 16:36:30 -07:00
Zsolt Venczel
2d1f819979 HDFS-13731. ReencryptionUpdater fails with ConcurrentModificationException during processCheckpoints. Contributed by Zsolt Venczel. 
(cherry picked from commit 3e18b957ebdf20925224ab9c28e6c2f4b6bbdb24)
 2018-08-28 15:14:57 -07:00
Wei-Chiu Chuang
d89a485b25 HDFS-13838. WebHdfsFileSystem.getFileStatus() won't return correct "snapshot enabled" status. Contributed by Siyao Meng. 
(cherry picked from commit 26c2a97c566969f50eb8e8432009724c51152a98)
(cherry picked from commit 41732e7b28bb70f7ba56cf6131ac10c062f93fc3)
 2018-08-27 16:04:16 -07:00
John Zhuge
2260718e24
HADOOP-15633. fs.TrashPolicyDefault: Can't create trash directory. Contributed by Fei Hui. 
(cherry picked from commit f9c6fd94711458b77ecf3fa425aad7fda5089376)
 2018-08-27 15:22:19 -07:00
Weiwei Yang
fdecbd94c1 YARN-8719. Typo correction for yarn configuration in OpportunisticContainers(federation) docs. Contributed by Y. SREENIVASULU REDDY. 
(cherry picked from commit e8b063f63049d781f4bd67e2ac928c03fd7b7941)
 2018-08-28 01:10:47 +08:00
Konstantin V Shvachko
27abdb390b HDFS-13848. Refactor NameNode failover proxy providers. Contributed by Konstantin Shvachko. 
(cherry picked from commit a4121c71c29d13866a605d9c0d013e5de9c147c3)
 2018-08-24 18:51:42 -07:00
Haibo Chen
49590e7c6b YARN-8051. TestRMEmbeddedElector#testCallbackSynchronization is flaky. (Robert Kanter via Haibo Chen) 
(cherry picked from commit 93d47a0ed504ee81d4b74d340c1815bdbb3c9b14)
 2018-08-24 13:27:54 -05:00
Inigo Goiri
c3ba06deeb YARN-7773. YARN Federation used Mysql as state store throw exception, Unknown column 'homeSubCluster' in 'field list'. Contributed by Yiran Wu. 
(cherry picked from commit 60e5c1b516647bf99ef3e9501c0ce04deea13ce9)
 2018-08-24 14:26:52 +05:30