Apache/hadoop
1
0
Fork 0
Code Issues Pull Requests Packages Projects Releases Wiki Activity
15,123 Commits 326 Branches 371 Tags
Commit Graph

5122 Commits

Author SHA1 Message Date
Wei-Chiu Chuang
04c8b5dbb6 HDFS-13758. DatanodeManager should throw exception if it has BlockRecoveryCommand but the block is not under construction. Contributed by chencan. 2018-08-14 11:57:30 -07:00
Wei-Chiu Chuang
ad5c3300ff HDFS-13813. Exit NameNode if dangling child inode is detected when saving FsImage. Contributed by Siyao Meng. 
(cherry picked from commit 23854443efa62aa70a1c30c32c3816750e5d7a5b)
(cherry picked from commit b38649c59a70b3112811443464b3b3180f4b9873)
(cherry picked from commit 4b606eb9fd5dde5629e791d3338886a70b5aa57c)
 2018-08-13 16:15:36 -07:00
Chao Sun
305b9a652a Remove super user privilege requirement for NameNode.getServiceStatus. Contributed by Chao Sun. 
(cherry picked from commit a2a8c486998b81d2c73804a07cc74f5269bfd904)
 2018-08-10 16:17:04 -07:00
Wei-Chiu Chuang
5ef8fd5119 HDFS-11060. make DEFAULT_MAX_CORRUPT_FILEBLOCKS_RETURNED configurable. Contributed by Lantao Jin. 
(cherry picked from commit e95c5e9f62452ee848875ec2f8642eab4992cd23)
(cherry picked from commit 964f3454d1237fa25aa828504612b4a11d8e76ff)
(cherry picked from commit 0af04ad89f2e3d867ed4011d519a53b764e3be6e)

Conflicts:
	hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/DFSConfigKeys.java
 2018-07-25 11:26:19 -07:00
Yiqun Lin
e6fcaccf5a HDFS-13583. RBF: Router admin clrQuota is not synchronized with nameservice. Contributed by Dibyendu Karmakar. 
(cherry picked from commit 17a87977f29ced49724f561a68565217c8cb4e94)
 2018-07-24 11:21:37 +08:00
Yiqun Lin
8a68a1590f HDFS-13743. RBF: Router throws NullPointerException due to the invalid initialization of MountTableResolver. Contributed by Takanobu Asanuma. 
(cherry picked from commit 7b25fb949bf6f02df997beeca7df46c9e84c8d96)
 2018-07-20 17:33:31 +08:00
Yiqun Lin
4c2ab22130 HDFS-13733. RBF: Add Web UI configurations and descriptions to RBF document. Contributed by Takanobu Asanuma. 
(cherry picked from commit 1af87df242c4286474961078d306a5692f85debc)
 2018-07-17 10:49:50 +08:00
Konstantin V Shvachko
fccb3c02f0 Fix potential FSImage corruption. Contributed by Ekanth Sethuramalingam & Arpit Agarwal. 
(cherry picked from commit 0a1e922f3d8eca4e852be57124ec1bcafaadb289)
 2018-07-16 18:44:01 -07:00
Inigo Goiri
436fceebf9 HDFS-13475. RBF: Admin cannot enforce Router enter SafeMode. Contributed by Chao Sun. 
(cherry picked from commit 359ea4e18147af5677c6d88265e26de6b6c72999)
 2018-07-16 09:47:57 -07:00
Akira Ajisaka
ff46db2ea6
HDFS-13729. Fix broken links to RBF documentation. Contributed by Gabor Bota. 2018-07-12 13:53:08 -04:00
Yiqun Lin
fbe7192815 HDFS-13726. RBF: Fix RBF configuration links. Contributed by Takanobu Asanuma. 
(cherry picked from commit 2ae13d41dcd4f49e6b4ebc099e5f8bb8280b9872)
 2018-07-11 22:17:43 +08:00
Andrew Wang
913417bbea HDFS-13719. Docs around dfs.image.transfer.timeout are misleading. Contributed by Kitti Nansi. 2018-07-09 15:17:26 +02:00
Yiqun Lin
512d20e9c5 HDFS-13710. RBF: setQuota and getQuotaUsage should check the dfs.federation.router.quota.enable. Contributed by yanghuafeng. 
(cherry picked from commit 43f7fe8aae0eca89cce4d67bfc4965fe8ce63e38)
 2018-07-09 15:08:39 +08:00
Yiqun Lin
1a96293c63 HDFS-13528. RBF: If a directory exceeds quota limit then quota usage is not refreshed for other mount entries. Contributed by Dibyendu Karmakar. 
(cherry picked from commit 3b637155a47d2aa93284969a96208347a647083d)
 2018-07-04 15:07:47 +08:00
Inigo Goiri
3e177a1d62 HDFS-13714. Fix TestNameNodePrunesMissingStorages test failures on Windows. Contributed by Lukas Majercak. 
(cherry picked from commit 7296b644f7c44400f03995668573a4cfd2ee552c)
 2018-07-02 16:38:13 -07:00
Inigo Goiri
9e148aa0bc HDFS-13681. Fix TestStartup.testNNFailToStartOnReadOnlyNNDir test failure on Windows. Contributed by Xiao Liang. 
(cherry picked from commit 8762e9cf10fa100dd5f7fd695f5e52b75a94c5d4)
 2018-06-15 16:51:03 -07:00
Inigo Goiri
b52316d796 HDFS-13686. Add overall metrics for FSNamesystemLock. Contributed by Lukas Majercak. 
(cherry picked from commit d31a3ce767d3bb68bdbb4f36d45600eab9f4f8b7)
 2018-06-15 16:00:45 -07:00
Sean Mackrory
b122961bc0 HDFS-13582. Improve backward compatibility for HDFS-13176 (WebHdfs file path gets truncated when having semicolon (;) inside). Contributed by Zsolt Venczel. 2018-06-15 16:28:36 -06:00
Inigo Goiri
fe831cb5b2 HDFS-13676. TestEditLogRace fails on Windows. Contributed by Zuoming Zhang. 2018-06-15 13:15:20 -07:00
Inigo Goiri
55b50eecc4 HDFS-13673. TestNameNodeMetrics fails on Windows. Contributed by Zuoming Zhang. 
(cherry picked from commit 43d994e4a6dfd1c24eafb909d6f8a0663b20769a)
 2018-06-15 10:35:05 -07:00
Inigo Goiri
fcd7ae2745 HDFS-13675. Speed up TestDFSAdminWithHA. Contributed by Lukas Majercak. 2018-06-14 14:12:54 -07:00
Inigo Goiri
06c87edf51 HDFS-13563. TestDFSAdminWithHA times out on Windows. Contributed by Lukas Majercak. 
(cherry picked from commit 8d4926f38bf53b32453cd2bc7322c8818f752f85)
 2018-06-14 10:00:26 -07:00
Rohith Sharma K S
0af3bea05d YARN-8405. RM zk-state-store.parent-path ACLs has been changed since HADOOP-14773. Contributed by Íñigo Goiri. 
(cherry picked from commit 2df73dace06cfd2b3193a14cd455297f8f989617)
 2018-06-12 17:30:02 +05:30
Inigo Goiri
8059390798 HDFS-13653. Make dfs.client.failover.random.order a per nameservice configuration. Contributed by Ekanth Sethuramalingam. 2018-06-11 13:47:08 -07:00
Chao Sun
4049fa37af HDFS-13664. Refactor ConfiguredFailoverProxyProvider to make inheritance easier. Contributed by Chao Sun. 
(cherry picked from commit fba1c42adc1c8ae57951e1865ec2ab05c8707bdf)
 2018-06-08 16:44:36 -07:00
Yiqun Lin
1465830f7e HDFS-13644. Backport HDFS-10376 to branch-2. Contributed by Zsolt Venczel. 2018-06-07 18:10:52 +08:00
Inigo Goiri
65d6943b4f HDFS-13652. Randomize baseDir for MiniDFSCluster in TestBlockScanner. Contributed by Anbang Hu. 
(cherry picked from commit a28848104f2a067669f9b1706cd0da05ba71bcb7)
 2018-06-04 18:14:34 -07:00
Rushabh Shah
0caf40efe5 HDFS-13281 Namenode#createFile should be /.reserved/raw/ aware.. Contributed by Rushabh S Shah 
(cherry picked from commit e2289c8d1496a5eff88e6bcb8776a11d45371ffc)
(cherry picked from commit 4ecb7aaf77d3f63e2ee006b32f5e25e95fde1831)
 2018-06-04 10:07:24 -05:00
Inigo Goiri
c8f183edd3 HDFS-13637. RBF: Router fails when threadIndex (in ConnectionPool) wraps around Integer.MIN_VALUE. Contributed by CR Hota. 
(cherry picked from commit e11d67404945d80db2eb8a99453606419dbdc938)
 2018-06-01 17:01:02 -07:00
Chao Sun
87e805ba4d HDFS-13602. Add checkOperation(WRITE) checks in FSNamesystem. Contributed by Chao Sun. 
(cherry picked from commit ff013d2c952272f3176dcf624251b05d610503b5)
 2018-05-31 17:44:39 -07:00
Inigo Goiri
6b9ef2d5b7 HDFS-13632. Randomize baseDir for MiniJournalCluster in MiniQJMHACluster for TestDFSAdminWithHA. Contributed by Anbang Hu. 2018-05-30 10:10:25 -07:00
Yiqun Lin
cb962139ee HDFS-13626. Fix incorrect username when deny the setOwner operation. Contributed by Zsolt Venczel. 2018-05-30 16:55:39 +08:00
Inigo Goiri
86736cb279 HDFS-13591. TestDFSShell#testSetrepLow fails on Windows. Contributed by Anbang Hu. 
(cherry picked from commit 9dbf4f01665d5480a70395a24519cbab5d4db0c5)
 2018-05-28 16:36:06 -07:00
Inigo Goiri
27091d2d16 HDFS-13620. Randomize the test directory path for TestHDFSFileSystemContract. Contributed by Anbang Hu. 2018-05-25 19:48:16 -07:00
Inigo Goiri
1542484161 HDFS-13618. Fix TestDataNodeFaultInjector test failures on Windows. Contributed by Xiao Liang. 
(cherry picked from commit a8e9ee3fdf3aa1b889dfdd3bade91ba754ed9ee1)
 2018-05-25 09:12:44 -07:00
Andrew Wang
097347c857 HDFS-13611. Unsafe use of Text as a ConcurrentHashMap key in PBHelperClient. 
(cherry picked from commit c9b63deb533274ca8ef4939f6cd13f728a067f7b)
 2018-05-24 09:56:54 -07:00
Inigo Goiri
c5ff9553fb HDFS-13587. TestQuorumJournalManager fails on Windows. Contributed by Anbang Hu. 
(cherry picked from commit c13dea87d9de7a9872fc8b0c939b41b1666a61e5)
 2018-05-23 11:37:55 -07:00
Inigo Goiri
411a2f609b HDFS-13588. Fix TestFsDatasetImpl test failures on Windows. Contributed by Xiao Liang. 2018-05-23 09:48:35 -07:00
Andrew Wang
0f0d29a8d1 HDFS-13601. Optimize ByteString conversions in PBHelper. 2018-05-22 23:58:11 -07:00
Inigo Goiri
8f45d119a1 HDFS-13590. Backport HDFS-12378 to branch-2. Contributed by Lukas Majercak. 2018-05-18 11:07:12 -07:00
Inigo Goiri
1af41dadc2 HDFS-13593. TestBlockReaderLocalLegacy#testBlockReaderLocalLegacyWithAppend fails on Windows. Contributed by Anbang Hu. 
(cherry picked from commit 9775ecb2355d7bed3514fcd54bf69e8351c4ab99)
 2018-05-18 09:47:50 -07:00
Inigo Goiri
b0e0ec1648 HDFS-13592. TestNameNodePrunesMissingStorages#testNameNodePrunesUnreportedStorages does not shut down cluster properly. Contributed by Anbang Hu. 
(cherry picked from commit 57b893de3d36d20f65ee81b5cc3cfef12594b75b)
 2018-05-18 09:40:10 -07:00
Inigo Goiri
a5ec44c655 HDFS-13560. Insufficient system resources exist to complete the requested service for some tests on Windows. Contributed by Anbang Hu. 
(cherry picked from commit 53b807a6a8486cefe0b036f7893de9f619bd44a1)
 2018-05-17 17:04:59 -07:00
Inigo Goiri
6df8fa1bc9 HDFS-13556. TestNestedEncryptionZones does not shut down cluster. Contributed by Anbang Hu. 
(cherry picked from commit a97a2042f210e9db97646baad6f56064d672f447)
 2018-05-17 16:55:01 -07:00
Inigo Goiri
315ffd1ada HDFS-13554. TestDatanodeRegistration#testForcedRegistration does not shut down cluster. Contributed by Anbang Hu. 
(cherry picked from commit 65476458fa05656010809be632356e4015b59a17)
 2018-05-17 14:49:37 -07:00
Inigo Goiri
b87cd6354b HDFS-13558. TestDatanodeHttpXFrame does not shut down cluster. Contributed by Anbang Hu. 
(cherry picked from commit 26f1e22fc9ee326e9c76503d347552faeb6c2d3b)
 2018-05-17 13:36:53 -07:00
Inigo Goiri
33dfbed8d2 HDFS-13570. TestQuotaByStorageType,TestQuota,TestDFSOutputStream fail on Windows. Contributed by Anbang Hu. 
(cherry picked from commit c0ec061e280c0d3e5299c0735c8b8697e0f31216)
 2018-05-17 09:23:11 -07:00
Kihwal Lee
033b25cbe9 HDFS-13581. DN UI logs link is broken when https is enabled. Contributed by Shashikant Banerjee. 
(cherry picked from commit 0ce6290de6a08beab01cb5184d366d97c9abd811)
 2018-05-17 09:29:08 -05:00
Inigo Goiri
d2a3bd1430 HDFS-13559. TestBlockScanner does not close TestContext properly. Contributed by Anbang Hu. 
(cherry picked from commit 454de3b543c8b00a9ab566c7d1c64d7e4cffee0f)
 2018-05-16 19:24:36 -07:00
Inigo Goiri
9c96822096 HDFS-13550.TestDebugAdmin#testComputeMetaCommand fails on Windows. Contributed by Anbang Hu. 
(cherry picked from commit 55d5549087183cb9c8d805e33267acabe9537667)
 2018-05-16 10:10:37 -07:00