Apache/hadoop
1
0
Fork 0
Code Issues Pull Requests Packages Projects Releases Wiki Activity
14,655 Commits 326 Branches 371 Tags
Commit Graph

4942 Commits

Author SHA1 Message Date
Yiqun Lin
c632c6e6e9 HDFS-13768. Adding replicas to volume map makes DataNode start slowly. Contributed by Surendra Singh Lilhore. 2018-10-09 10:20:37 +08:00
Inigo Goiri
5539dd97d2 HDFS-13964. RBF: TestRouterWebHDFSContractAppend fails with No Active Namenode under nameservice. Contributed by Ayush Saxena. 
(cherry picked from commit 3bfff0f79e7246cbb2989c720c14cfa2566a9d1a)
 2018-10-06 10:43:36 -07:00
Brahma Reddy Battula
3d67080a45 HDFS-13790. RBF: Move ClientProtocol APIs to its own module Contributed by Chao Sun. 2018-09-26 22:46:43 +05:30
Chen Liang
3a6ad9cd3d HADOOP-15726. Create utility to limit frequency of log statements. Contributed by Erik Krogen. 2018-09-20 10:00:13 -07:00
Arpit Agarwal
0dd686114f HADOOP-15772. Remove the 'Path ... should be specified as a URI' warnings on startup. Contributed by Ayush Saxena. 
(cherry picked from commit 2eb597b1511f8f46866abe4eeec820f4191cc295)
 2018-09-19 13:49:16 -07:00
Inigo Goiri
3e476dcae8 HDFS-13908. TestDataNodeMultipleRegistrations is flaky. Contributed by Ayush Saxena. 
(cherry picked from commit 56e0d635e0f49772d001c1fdb385110c705a622a)
 2018-09-19 09:32:35 -07:00
Inigo Goiri
9bd2616138 HDFS-13844. Fix the fmt_bytes function in the dfs-dust.js. Contributed by yanghuafeng. 
(cherry picked from commit a9c1c55cafc0338bb35d2991f217d4157463f1d3)
 2018-09-17 14:40:00 -07:00
Inigo Goiri
bbd09877d0 HDFS-13914. Fix DN UI logs link broken when https is enabled after HDFS-13902. Contributed by Jianfei Jiang. 
(cherry picked from commit e1b242a9844d6438c93cdc2ab2443d042b4d5aea)
 2018-09-13 09:18:40 -07:00
Brahma Reddy Battula
9befa5ed95 HDFS-13902. Add JMX, conf and stacks menus to the datanode page. Contributed by fengchuang. 
(cherry picked from commit b2432d254c486d0e360b76b7b094a71a011678ab)
 2018-09-12 06:37:25 +05:30
Xiao Chen
a7b9e7d858 HDFS-13051. Fix dead lock during async editlog rolling if edit queue is full. Contributed by Daryn Sharp. 
(cherry picked from commit 8e54da1511e78477c1d4655d5ff0a69d0330869f)

 Conflicts:
	hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/namenode/TestEditLogRace.java

(cherry picked from commit 2dd27c999b22c550058de0e6eca7209b346cd143)

 Conflicts:
	hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/namenode/TestEditLogRace.java

(cherry picked from commit a5da4df73c525d68c76487509660a6b13f7fe99e)
 2018-09-10 23:13:14 -07:00
Brahma Reddy Battula
e56f183696 HDFS-12716. 'dfs.datanode.failed.volumes.tolerated' to support minimum number of volumes to be available. Contributed by Ranith Sardar 2018-09-10 07:11:26 +05:30
Inigo Goiri
1b12925d69 HDFS-13836. RBF: Handle mount table znode with null value. Contributed by yanghuafeng. 
(cherry picked from commit 527288ef891dc26019d003bd85ddfd50eb4f3b7b)
 2018-09-06 16:50:09 -07:00
Hrishikesh Gadre
0468b6e736 HDFS-13812. Fix the inconsistent default refresh interval on Caching documentation. Contributed by Hrishikesh Gadre. 
Signed-off-by: Xiao Chen <xiao@apache.org>
(cherry picked from commit 6ccb809c2d38a45e716153ba16e135cb76167b2b)
 2018-09-04 21:55:09 -07:00
Inigo Goiri
2e744bd31d HDFS-13857. RBF: Choose to enable the default nameservice to read/write files. Contributed by yanghuafeng. 
(cherry picked from commit 54f2044595206455484284b43e5976c8a1982aaf)
 2018-09-04 12:18:38 -07:00
Yiqun Lin
8137b473aa HDFS-13634. RBF: Configurable value in xml for async connection request queue size. Contributed by CR Hota. 
(cherry picked from commit a0ebb6b39f2932d3ea2fb5e287f52b841e108428)
 2018-08-29 16:23:17 +08:00
Brahma Reddy Battula
2a8f4aefd4 HDFS-13854. RBF: The ProcessingAvgTime and ProxyAvgTime should display by JMX with ms unit. Contributed by yanghuafeng. 
(cherry picked from commit 64ad0298d441559951bc9589a40f8aab17c93a5f)
 2018-08-29 08:32:36 +05:30
Yiqun Lin
361d0e45aa HDFS-13831. Make block increment deletion number configurable. Contributed by Ryan Wu. 
(cherry picked from commit b9b964d25335943fb15cdfcf369d123bbd7e454a)
 2018-08-27 15:03:33 +08:00
Konstantin V Shvachko
64681a362d HDFS-13848. Refactor NameNode failover proxy providers. Contributed by Konstantin Shvachko. 
(cherry picked from commit a4121c71c29d13866a605d9c0d013e5de9c147c3)
 2018-08-24 19:00:21 -07:00
Yiqun Lin
8e09ff918e HDFS-13821. RBF: Add dfs.federation.router.mount-table.cache.enable so that users can disable cache. Contributed by Fei Hui. 
(cherry picked from commit 81847392badcd58d934333e7c3b5bf14b4fa1f3f)
 2018-08-22 11:48:34 +08:00
Yiqun Lin
18ebe18305 HDFS-13750. RBF: Router ID in RouterRpcClient is always null. Contributed by Takanobu Asanuma. 2018-08-20 23:08:26 +08:00
Wei-Chiu Chuang
04c8b5dbb6 HDFS-13758. DatanodeManager should throw exception if it has BlockRecoveryCommand but the block is not under construction. Contributed by chencan. 2018-08-14 11:57:30 -07:00
Wei-Chiu Chuang
ad5c3300ff HDFS-13813. Exit NameNode if dangling child inode is detected when saving FsImage. Contributed by Siyao Meng. 
(cherry picked from commit 23854443efa62aa70a1c30c32c3816750e5d7a5b)
(cherry picked from commit b38649c59a70b3112811443464b3b3180f4b9873)
(cherry picked from commit 4b606eb9fd5dde5629e791d3338886a70b5aa57c)
 2018-08-13 16:15:36 -07:00
Chao Sun
305b9a652a Remove super user privilege requirement for NameNode.getServiceStatus. Contributed by Chao Sun. 
(cherry picked from commit a2a8c486998b81d2c73804a07cc74f5269bfd904)
 2018-08-10 16:17:04 -07:00
Wei-Chiu Chuang
5ef8fd5119 HDFS-11060. make DEFAULT_MAX_CORRUPT_FILEBLOCKS_RETURNED configurable. Contributed by Lantao Jin. 
(cherry picked from commit e95c5e9f62452ee848875ec2f8642eab4992cd23)
(cherry picked from commit 964f3454d1237fa25aa828504612b4a11d8e76ff)
(cherry picked from commit 0af04ad89f2e3d867ed4011d519a53b764e3be6e)

Conflicts:
	hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/DFSConfigKeys.java
 2018-07-25 11:26:19 -07:00
Yiqun Lin
e6fcaccf5a HDFS-13583. RBF: Router admin clrQuota is not synchronized with nameservice. Contributed by Dibyendu Karmakar. 
(cherry picked from commit 17a87977f29ced49724f561a68565217c8cb4e94)
 2018-07-24 11:21:37 +08:00
Yiqun Lin
8a68a1590f HDFS-13743. RBF: Router throws NullPointerException due to the invalid initialization of MountTableResolver. Contributed by Takanobu Asanuma. 
(cherry picked from commit 7b25fb949bf6f02df997beeca7df46c9e84c8d96)
 2018-07-20 17:33:31 +08:00
Yiqun Lin
4c2ab22130 HDFS-13733. RBF: Add Web UI configurations and descriptions to RBF document. Contributed by Takanobu Asanuma. 
(cherry picked from commit 1af87df242c4286474961078d306a5692f85debc)
 2018-07-17 10:49:50 +08:00
Konstantin V Shvachko
fccb3c02f0 Fix potential FSImage corruption. Contributed by Ekanth Sethuramalingam & Arpit Agarwal. 
(cherry picked from commit 0a1e922f3d8eca4e852be57124ec1bcafaadb289)
 2018-07-16 18:44:01 -07:00
Inigo Goiri
436fceebf9 HDFS-13475. RBF: Admin cannot enforce Router enter SafeMode. Contributed by Chao Sun. 
(cherry picked from commit 359ea4e18147af5677c6d88265e26de6b6c72999)
 2018-07-16 09:47:57 -07:00
Akira Ajisaka
ff46db2ea6
HDFS-13729. Fix broken links to RBF documentation. Contributed by Gabor Bota. 2018-07-12 13:53:08 -04:00
Yiqun Lin
fbe7192815 HDFS-13726. RBF: Fix RBF configuration links. Contributed by Takanobu Asanuma. 
(cherry picked from commit 2ae13d41dcd4f49e6b4ebc099e5f8bb8280b9872)
 2018-07-11 22:17:43 +08:00
Andrew Wang
913417bbea HDFS-13719. Docs around dfs.image.transfer.timeout are misleading. Contributed by Kitti Nansi. 2018-07-09 15:17:26 +02:00
Yiqun Lin
512d20e9c5 HDFS-13710. RBF: setQuota and getQuotaUsage should check the dfs.federation.router.quota.enable. Contributed by yanghuafeng. 
(cherry picked from commit 43f7fe8aae0eca89cce4d67bfc4965fe8ce63e38)
 2018-07-09 15:08:39 +08:00
Yiqun Lin
1a96293c63 HDFS-13528. RBF: If a directory exceeds quota limit then quota usage is not refreshed for other mount entries. Contributed by Dibyendu Karmakar. 
(cherry picked from commit 3b637155a47d2aa93284969a96208347a647083d)
 2018-07-04 15:07:47 +08:00
Inigo Goiri
3e177a1d62 HDFS-13714. Fix TestNameNodePrunesMissingStorages test failures on Windows. Contributed by Lukas Majercak. 
(cherry picked from commit 7296b644f7c44400f03995668573a4cfd2ee552c)
 2018-07-02 16:38:13 -07:00
Inigo Goiri
9e148aa0bc HDFS-13681. Fix TestStartup.testNNFailToStartOnReadOnlyNNDir test failure on Windows. Contributed by Xiao Liang. 
(cherry picked from commit 8762e9cf10fa100dd5f7fd695f5e52b75a94c5d4)
 2018-06-15 16:51:03 -07:00
Inigo Goiri
b52316d796 HDFS-13686. Add overall metrics for FSNamesystemLock. Contributed by Lukas Majercak. 
(cherry picked from commit d31a3ce767d3bb68bdbb4f36d45600eab9f4f8b7)
 2018-06-15 16:00:45 -07:00
Sean Mackrory
b122961bc0 HDFS-13582. Improve backward compatibility for HDFS-13176 (WebHdfs file path gets truncated when having semicolon (;) inside). Contributed by Zsolt Venczel. 2018-06-15 16:28:36 -06:00
Inigo Goiri
fe831cb5b2 HDFS-13676. TestEditLogRace fails on Windows. Contributed by Zuoming Zhang. 2018-06-15 13:15:20 -07:00
Inigo Goiri
55b50eecc4 HDFS-13673. TestNameNodeMetrics fails on Windows. Contributed by Zuoming Zhang. 
(cherry picked from commit 43d994e4a6dfd1c24eafb909d6f8a0663b20769a)
 2018-06-15 10:35:05 -07:00
Inigo Goiri
fcd7ae2745 HDFS-13675. Speed up TestDFSAdminWithHA. Contributed by Lukas Majercak. 2018-06-14 14:12:54 -07:00
Inigo Goiri
06c87edf51 HDFS-13563. TestDFSAdminWithHA times out on Windows. Contributed by Lukas Majercak. 
(cherry picked from commit 8d4926f38bf53b32453cd2bc7322c8818f752f85)
 2018-06-14 10:00:26 -07:00
Rohith Sharma K S
0af3bea05d YARN-8405. RM zk-state-store.parent-path ACLs has been changed since HADOOP-14773. Contributed by Íñigo Goiri. 
(cherry picked from commit 2df73dace06cfd2b3193a14cd455297f8f989617)
 2018-06-12 17:30:02 +05:30
Inigo Goiri
8059390798 HDFS-13653. Make dfs.client.failover.random.order a per nameservice configuration. Contributed by Ekanth Sethuramalingam. 2018-06-11 13:47:08 -07:00
Chao Sun
4049fa37af HDFS-13664. Refactor ConfiguredFailoverProxyProvider to make inheritance easier. Contributed by Chao Sun. 
(cherry picked from commit fba1c42adc1c8ae57951e1865ec2ab05c8707bdf)
 2018-06-08 16:44:36 -07:00
Yiqun Lin
1465830f7e HDFS-13644. Backport HDFS-10376 to branch-2. Contributed by Zsolt Venczel. 2018-06-07 18:10:52 +08:00
Inigo Goiri
65d6943b4f HDFS-13652. Randomize baseDir for MiniDFSCluster in TestBlockScanner. Contributed by Anbang Hu. 
(cherry picked from commit a28848104f2a067669f9b1706cd0da05ba71bcb7)
 2018-06-04 18:14:34 -07:00
Rushabh Shah
0caf40efe5 HDFS-13281 Namenode#createFile should be /.reserved/raw/ aware.. Contributed by Rushabh S Shah 
(cherry picked from commit e2289c8d1496a5eff88e6bcb8776a11d45371ffc)
(cherry picked from commit 4ecb7aaf77d3f63e2ee006b32f5e25e95fde1831)
 2018-06-04 10:07:24 -05:00
Inigo Goiri
c8f183edd3 HDFS-13637. RBF: Router fails when threadIndex (in ConnectionPool) wraps around Integer.MIN_VALUE. Contributed by CR Hota. 
(cherry picked from commit e11d67404945d80db2eb8a99453606419dbdc938)
 2018-06-01 17:01:02 -07:00
Chao Sun
87e805ba4d HDFS-13602. Add checkOperation(WRITE) checks in FSNamesystem. Contributed by Chao Sun. 
(cherry picked from commit ff013d2c952272f3176dcf624251b05d610503b5)
 2018-05-31 17:44:39 -07:00