Apache/hadoop
1
0
Fork 0
Code Issues Pull Requests Packages Projects Releases Wiki Activity
18,737 Commits 326 Branches 371 Tags
Commit Graph

7155 Commits

Author SHA1 Message Date
Daniel Templeton
fbedf89377 HDFS-13746. Still occasional "Should be different group" failure in TestRefreshUserMappings#testGroupMappingRefresh 
(Contributed by Siyao Meng via Daniel Templeton)

Change-Id: I9fad1537ace38367a463d9fe67aaa28d3178fc69
(cherry picked from commit 8512e1a91be3e340d919c7cdc9c09dfb762a6a4e)
 2018-08-16 15:01:34 -07:00
Xiao Chen
83d0d82a69 HDFS-13788. Update EC documentation about rack fault tolerance. Contributed by Kitti Nanasi. 
(cherry picked from commit cede33997f7ab09fc046017508b680e282289ce3)
 2018-08-14 11:57:50 -07:00
Wei-Chiu Chuang
7a115e8de4 HDFS-13758. DatanodeManager should throw exception if it has BlockRecoveryCommand but the block is not under construction. Contributed by chencan. 
(cherry picked from commit 61a9b4f58b639e71c564d84b529ac66aaae7f8ef)
 2018-08-14 11:52:24 -07:00
Wei-Chiu Chuang
74d4fd975a HDFS-13738. fsck -list-corruptfileblocks has infinite loop if user is not privileged. Contributed by Yuen-Kuei Hsueh. 
(cherry picked from commit 4023eeba05aefe48384e870da3c95bb3af474514)
 2018-08-13 17:37:12 -07:00
Wei-Chiu Chuang
b38649c59a HDFS-13813. Exit NameNode if dangling child inode is detected when saving FsImage. Contributed by Siyao Meng. 
(cherry picked from commit 23854443efa62aa70a1c30c32c3816750e5d7a5b)
 2018-08-13 16:13:36 -07:00
Vinayakumar B
b3f29e3504 HDFS-13786. EC: Display erasure coding policy for sub-directories is not working. Contributed by Ayush Saxena. 
(cherry picked from commit 2b0f9772417d205e8df16bac6921c2bb8bdcf740)
 2018-08-13 13:58:27 -07:00
Chen Liang
e05b38f132 HDFS-13735. Make QJM HTTP URL connection timeout configurable. Contributed by Chao Sun. 2018-08-13 12:48:12 -07:00
Arpit Agarwal
4e197a4b46 HDFS-13823. NameNode UI : "Utilities -> Browse the file system -> open a file -> Head the file" is not working. Contributed by Nanda kumar. 2018-08-13 11:36:15 -07:00
Chao Sun
baca64cc68 Remove super user privilege requirement for NameNode.getServiceStatus. Contributed by Chao Sun. 
(cherry picked from commit a2a8c486998b81d2c73804a07cc74f5269bfd904)
 2018-08-10 16:07:02 -07:00
Xiao Chen
7d1e2e1fbb HDFS-13721. NPE in DataNode due to uninitialized DiskBalancer. 2018-08-10 13:53:34 -07:00
Wangda Tan
781620a44d Make 3.1.1 awared by other branches 
(cherry picked from commit 49c687608b65b772faeed614700ece8e526432e8)
 2018-08-08 13:04:05 -07:00
Xiao Chen
bf03b25f4b HDFS-13728. Disk Balancer should not fail if volume usage is greater than capacity. Contributed by Stephen O'Donnell. 
(cherry picked from commit 6677717c689cc94a15f14c3466242e23652d473b)
 2018-08-07 22:05:51 -07:00
Wangda Tan
5583711419 Preparing for 3.1.2 release 
Change-Id: If2793e2ed2b5b349a9e1f98f78df43f309dcfcbd
 2018-07-31 13:08:55 -07:00
Brahma Reddy Battula
6f6d9ee9bf HDFS-12716. 'dfs.datanode.failed.volumes.tolerated' to support minimum number of volumes to be available. Contributed by Ranith Sardar and usharani 
(cherry picked from commit 3108d27edde941d153a58f71fb1096cce2995531)
 2018-07-30 15:57:58 +05:30
Arpit Agarwal
0eb187f72b HDFS-13765. Fix javadoc for FSDirMkdirOp#createParentDirectories. Contributed by Lokesh Jain. 2018-07-27 10:14:20 -07:00
Anu Engineer
2ebd553072 HDFS-13727. Log full stack trace if DiskBalancer exits with an unhandled exception. 
Contributed by Gabor Bota.

(cherry picked from commit 64e739e344ac474046d4f4ecf0865dd92be13762)
 2018-07-27 09:43:44 -07:00
Wei-Chiu Chuang
964f3454d1 HDFS-11060. make DEFAULT_MAX_CORRUPT_FILEBLOCKS_RETURNED configurable. Contributed by Lantao Jin. 
(cherry picked from commit e95c5e9f62452ee848875ec2f8642eab4992cd23)
 2018-07-25 11:05:18 -07:00
Daniel Templeton
00c476abd8 HDFS-13448. HDFS Block Placement - Ignore Locality for First Block Replica 
(Contributed by BELUGA BEHR via Daniel Templeton)

Change-Id: I965d1cfa642ad24296038b83e3d5c9983545267d
(cherry picked from commit 849c45db187224095b13fe297a4d7377fbb9d2cd)
 2018-07-24 16:12:43 -07:00
Yiqun Lin
5aca0588ea HDFS-13583. RBF: Router admin clrQuota is not synchronized with nameservice. Contributed by Dibyendu Karmakar. 
(cherry picked from commit 17a87977f29ced49724f561a68565217c8cb4e94)
 2018-07-24 11:19:21 +08:00
Yiqun Lin
823d576a66 HDFS-13743. RBF: Router throws NullPointerException due to the invalid initialization of MountTableResolver. Contributed by Takanobu Asanuma. 
(cherry picked from commit 7b25fb949bf6f02df997beeca7df46c9e84c8d96)
 2018-07-20 17:31:13 +08:00
Yiqun Lin
228508a0ee HDFS-13733. RBF: Add Web UI configurations and descriptions to RBF document. Contributed by Takanobu Asanuma. 
(cherry picked from commit 1af87df242c4286474961078d306a5692f85debc)
 2018-07-17 10:47:07 +08:00
Konstantin V Shvachko
53c7d82d53 Fix potential FSImage corruption. Contributed by Ekanth Sethuramalingam & Arpit Agarwal. 
(cherry picked from commit 0a1e922f3d8eca4e852be57124ec1bcafaadb289)
 2018-07-16 18:29:43 -07:00
Wei-Chiu Chuang
ac9155d6cb HDFS-13524. Occasional "All datanodes are bad" error in TestLargeBlock#testLargeBlockSize. Contributed by Siyao Meng. 
(cherry picked from commit 88b2794244d19b6432253eb649a375e5bcdcf964)
 2018-07-16 10:52:56 -07:00
Inigo Goiri
4898edf4f7 HDFS-13475. RBF: Admin cannot enforce Router enter SafeMode. Contributed by Chao Sun. 
(cherry picked from commit 359ea4e18147af5677c6d88265e26de6b6c72999)
 2018-07-16 09:47:00 -07:00
Xiao Chen
d5d987550a HDFS-12837. Intermittent failure in TestReencryptionWithKMS. 
(cherry picked from commit b37074be5ab35c238e18bb9c3b89db6d7f8d0986)
 2018-07-11 21:03:45 -07:00
Akira Ajisaka
fe256a98ff
HDFS-13729. Fix broken links to RBF documentation. Contributed by Gabor Bota. 
(cherry picked from commit 418cc7f3aeabedc57c94aa9d4c4248c1476ac90e)
 2018-07-11 14:47:21 -04:00
Wei-Chiu Chuang
f9fa3cb157 HDFS-13723. Occasional "Should be different group" error in TestRefreshUserMappings#testGroupMappingRefresh. Contributed by Siyao Meng. 
(cherry picked from commit 162228e8db937d4bdb9cf61d15ed555f1c96368f)
 2018-07-11 10:04:07 -07:00
Yiqun Lin
99e98bf19a HDFS-13726. RBF: Fix RBF configuration links. Contributed by Takanobu Asanuma. 
(cherry picked from commit 2ae13d41dcd4f49e6b4ebc099e5f8bb8280b9872)
 2018-07-11 22:15:07 +08:00
Andrew Wang
242b5acdb3 HDFS-13719. Docs around dfs.image.transfer.timeout are misleading. Contributed by Kitti Nansi. 
(cherry picked from commit eecb5baaaaa54599aeae758abd4007e55e5b531f)
 2018-07-09 15:17:34 +02:00
Yiqun Lin
e994c4f8aa HDFS-13710. RBF: setQuota and getQuotaUsage should check the dfs.federation.router.quota.enable. Contributed by yanghuafeng. 
(cherry picked from commit 43f7fe8aae0eca89cce4d67bfc4965fe8ce63e38)
 2018-07-09 15:07:47 +08:00
Wei-Chiu Chuang
f5f4d0b7e7 HDFS-13121. NPE when request file descriptors when SC read. Contributed by Zsolt Venczel. 
(cherry picked from commit 0247cb6318507afe06816e337a19f396afc53efa)
 2018-07-06 15:01:10 -07:00
Yiqun Lin
48addcdbbe HDFS-13528. RBF: If a directory exceeds quota limit then quota usage is not refreshed for other mount entries. Contributed by Dibyendu Karmakar. 
(cherry picked from commit 3b637155a47d2aa93284969a96208347a647083d)
 2018-07-04 15:30:17 +08:00
Andrew Wang
3b266abdad HDFS-13712. BlockReaderRemote.read() logging improvement. Contributed by Gergo Repas. 
(cherry picked from commit 344f324710522ffb27852c1a673c4f7d3d6eac4b)
 2018-07-03 11:07:57 +02:00
Bharat Viswanadham
0d7b811ee6 HDFS-13715:diskbalancer does not work if one of the blockpools are empty on a Federated cluster. Contributed by Bharat Viswanadham 
(cherry picked from commit 59a3038bc3d7913dca3de971026bc32cef536a2d)
 2018-07-02 21:48:09 -07:00
Inigo Goiri
177ccbc43f HDFS-13714. Fix TestNameNodePrunesMissingStorages test failures on Windows. Contributed by Lukas Majercak. 
(cherry picked from commit 7296b644f7c44400f03995668573a4cfd2ee552c)
 2018-07-02 16:37:12 -07:00
Wei-Chiu Chuang
2ce892fcf5 HDFS-13635. Incorrect message when block is not found. Contributed by Gabor Bota. 
(cherry picked from commit fef20a446f7bf9f29e0f0ee690987fb6fc78a031)
 2018-07-02 09:43:44 -07:00
Andrew Wang
2d2639beaa HDFS-13702. Remove HTrace hooks from DFSClient to reduce CPU usage. Contributed by Todd Lipcon. 
(cherry picked from commit 5d748bd056a32f2c6922514cd0c5b31d866a9919)
 2018-07-02 12:11:11 +02:00
Andrew Wang
0268b954e0 HDFS-13703. Avoid allocation of CorruptedBlocks hashmap when no corrupted blocks are hit. Contributed by Todd Lipcon. 
(cherry picked from commit 6ba99741086170b83c38d3e7e715d9e8046a1e00)
 2018-07-02 12:02:48 +02:00
Sunil G
aa8db00fc3 HADOOP-15483. Upgrade jquery to version 3.3.1. Contributed by Lokesh Jain, Mukul Kumar Singh and Sunil Govindan. 2018-06-26 15:02:49 -07:00
Yiqun Lin
6b5b58a495 HDFS-13692. StorageInfoDefragmenter floods log when compacting StorageInfo TreeSet. Contributed by Bharat Viswanadham. 
(cherry picked from commit 30728aced4a6b05394b3fc8c613f39fade9cf3c2)
 2018-06-22 10:53:33 +08:00
Xiao Chen
e8655b8c72 HDFS-13682. Cannot create encryption zone after KMS auth token expires. 
(cherry picked from commit 32f867a6a907c05a312657139d295a92756d98ef)
 2018-06-20 15:59:23 -07:00
Inigo Goiri
88e8a3e5bc HDFS-13681. Fix TestStartup.testNNFailToStartOnReadOnlyNNDir test failure on Windows. Contributed by Xiao Liang. 
(cherry picked from commit 8762e9cf10fa100dd5f7fd695f5e52b75a94c5d4)
 2018-06-15 16:49:51 -07:00
Inigo Goiri
7c06be3912 HDFS-13686. Add overall metrics for FSNamesystemLock. Contributed by Lukas Majercak. 
(cherry picked from commit d31a3ce767d3bb68bdbb4f36d45600eab9f4f8b7)
 2018-06-15 15:59:49 -07:00
Wei-Chiu Chuang
975d4b3d60 HDFS-13174. hdfs mover -p /path times out after 20 min. Contributed by Istvan Fajth. 
(cherry picked from commit c966a3837af1c1a1c4a441f491b0d76d5c9e5d78)
 2018-06-15 13:37:19 -07:00
Inigo Goiri
271ea7a3d9 HDFS-13676. TestEditLogRace fails on Windows. Contributed by Zuoming Zhang. 
(cherry picked from commit eebeb6033fd09791fcbff626f128a98e393f0a88)
 2018-06-15 13:08:24 -07:00
Inigo Goiri
4add4c1be0 HDFS-13673. TestNameNodeMetrics fails on Windows. Contributed by Zuoming Zhang. 
(cherry picked from commit 43d994e4a6dfd1c24eafb909d6f8a0663b20769a)
 2018-06-15 10:34:08 -07:00
Inigo Goiri
fd7cd596b7 HDFS-13675. Speed up TestDFSAdminWithHA. Contributed by Lukas Majercak. 
(cherry picked from commit 5d7449d2b8bcd0963d172fc30df784279671176f)
 2018-06-14 13:43:51 -07:00
Inigo Goiri
e45541be51 HDFS-13563. TestDFSAdminWithHA times out on Windows. Contributed by Lukas Majercak. 
(cherry picked from commit 8d4926f38bf53b32453cd2bc7322c8818f752f85)
 2018-06-14 09:59:28 -07:00
Yiqun Lin
5c1248b2da HDFS-13641. Add metrics for edit log tailing. Contributed by Chao Sun. 
(cherry picked from commit 8e7548d33be9c4874daab18b2e774bdc2ed216d3)
 2018-06-13 20:10:46 +08:00
Xiao Chen
3b646a9836 HDFS-13511. Provide specialized exception when block length cannot be obtained. Contributed by Gabor Bota. 
(cherry picked from commit 774c1f199e11d886d0c0a1069325f0284da35deb)
 2018-06-12 11:17:54 -07:00