Apache/hadoop
1
0
Fork 0
Code Issues Pull Requests Packages Projects Releases Wiki Activity
17,704 Commits 326 Branches 371 Tags
Commit Graph

6737 Commits

Author SHA1 Message Date
Xiao Chen
185c8f2abc HDFS-13747. Statistic for list_located_status is incremented incorrectly by listStatusIterator. Contributed by Antal Mihalyi. 
(cherry picked from commit c67b0650ea10896c6289703595faef0d262c00b3)
 2018-08-16 23:15:58 -07:00
Daniel Templeton
27508086de HDFS-13746. Still occasional "Should be different group" failure in TestRefreshUserMappings#testGroupMappingRefresh 
(Contributed by Siyao Meng via Daniel Templeton)

Change-Id: I9fad1537ace38367a463d9fe67aaa28d3178fc69
(cherry picked from commit 8512e1a91be3e340d919c7cdc9c09dfb762a6a4e)
(cherry picked from commit fbedf89377e540fb10239a880fc2e01ef7021b93)
 2018-08-16 15:02:18 -07:00
Xiao Chen
5d16301ad9 HDFS-13788. Update EC documentation about rack fault tolerance. Contributed by Kitti Nanasi. 
(cherry picked from commit cede33997f7ab09fc046017508b680e282289ce3)
 2018-08-14 11:59:05 -07:00
Wei-Chiu Chuang
6f2a46578f HDFS-13758. DatanodeManager should throw exception if it has BlockRecoveryCommand but the block is not under construction. Contributed by chencan. 
(cherry picked from commit 61a9b4f58b639e71c564d84b529ac66aaae7f8ef)
(cherry picked from commit 7a115e8de4732d6e96acc21f21e2abad76754e18)
 2018-08-14 11:52:55 -07:00
Wei-Chiu Chuang
d502abe947 HDFS-13738. fsck -list-corruptfileblocks has infinite loop if user is not privileged. Contributed by Yuen-Kuei Hsueh. 
(cherry picked from commit 4023eeba05aefe48384e870da3c95bb3af474514)
(cherry picked from commit 74d4fd975a1f67ae3e1b67554319e1f5e525ed6f)
 2018-08-13 17:37:36 -07:00
Wei-Chiu Chuang
4b606eb9fd HDFS-13813. Exit NameNode if dangling child inode is detected when saving FsImage. Contributed by Siyao Meng. 
(cherry picked from commit 23854443efa62aa70a1c30c32c3816750e5d7a5b)
(cherry picked from commit b38649c59a70b3112811443464b3b3180f4b9873)
 2018-08-13 16:14:01 -07:00
Vinayakumar B
eecfbaff3b HDFS-13786. EC: Display erasure coding policy for sub-directories is not working. Contributed by Ayush Saxena. 
(cherry picked from commit 2b0f9772417d205e8df16bac6921c2bb8bdcf740)
 2018-08-13 13:58:37 -07:00
Chen Liang
2d985b6c81 HDFS-13735. Make QJM HTTP URL connection timeout configurable. Contributed by Chao Sun. 2018-08-13 12:47:17 -07:00
Chao Sun
07054c5994 Remove super user privilege requirement for NameNode.getServiceStatus. Contributed by Chao Sun. 
(cherry picked from commit a2a8c486998b81d2c73804a07cc74f5269bfd904)
 2018-08-10 16:12:05 -07:00
Wangda Tan
8b4392769c Make 3.1.1 awared by other branches 
(cherry picked from commit 49c687608b65b772faeed614700ece8e526432e8)
(cherry picked from commit 781620a44db1eea511406f6c300f5b12b3a24daf)
 2018-08-08 13:07:09 -07:00
Xiao Chen
8e5081569f HDFS-13728. Disk Balancer should not fail if volume usage is greater than capacity. Contributed by Stephen O'Donnell. 
(cherry picked from commit 6677717c689cc94a15f14c3466242e23652d473b)
 2018-08-07 22:05:59 -07:00
Brahma Reddy Battula
abc32c863e HDFS-12716. 'dfs.datanode.failed.volumes.tolerated' to support minimum number of volumes to be available. Contributed by Ranith Sardar and usharani 
(cherry picked from commit 3108d27edde941d153a58f71fb1096cce2995531)
 2018-07-30 16:08:49 +05:30
Arpit Agarwal
eb40d4fab4 HDFS-13765. Fix javadoc for FSDirMkdirOp#createParentDirectories. Contributed by Lokesh Jain. 2018-07-27 10:14:11 -07:00
Anu Engineer
faaf72dcb3 HDFS-13727. Log full stack trace if DiskBalancer exits with an unhandled exception. 
Contributed by Gabor Bota.

(cherry picked from commit 64e739e344ac474046d4f4ecf0865dd92be13762)
 2018-07-27 09:35:02 -07:00
Wei-Chiu Chuang
0af04ad89f HDFS-11060. make DEFAULT_MAX_CORRUPT_FILEBLOCKS_RETURNED configurable. Contributed by Lantao Jin. 
(cherry picked from commit e95c5e9f62452ee848875ec2f8642eab4992cd23)
(cherry picked from commit 964f3454d1237fa25aa828504612b4a11d8e76ff)
 2018-07-25 11:05:36 -07:00
Daniel Templeton
3210b3d8aa HDFS-13448. HDFS Block Placement - Ignore Locality for First Block Replica 
(Contributed by BELUGA BEHR via Daniel Templeton)

Change-Id: I965d1cfa642ad24296038b83e3d5c9983545267d
(cherry picked from commit 849c45db187224095b13fe297a4d7377fbb9d2cd)
(cherry picked from commit 00c476abd8f1d34414b646219856859477558458)
 2018-07-24 16:13:30 -07:00
Konstantin V Shvachko
475c282939 Fix potential FSImage corruption. Contributed by Ekanth Sethuramalingam & Arpit Agarwal. 
(cherry picked from commit 0a1e922f3d8eca4e852be57124ec1bcafaadb289)
 2018-07-16 18:33:36 -07:00
Wei-Chiu Chuang
0d2b983dc2 HDFS-13524. Occasional "All datanodes are bad" error in TestLargeBlock#testLargeBlockSize. Contributed by Siyao Meng. 
(cherry picked from commit 88b2794244d19b6432253eb649a375e5bcdcf964)
(cherry picked from commit ac9155d6cb174ba19a8cf64b7d0142e147248e04)
 2018-07-16 10:53:30 -07:00
Xiao Chen
883df537d6 HDFS-12837. Intermittent failure in TestReencryptionWithKMS. 
(cherry picked from commit b37074be5ab35c238e18bb9c3b89db6d7f8d0986)
 2018-07-11 21:03:53 -07:00
Akira Ajisaka
4d5473c89d
HDFS-13729. Fix broken links to RBF documentation. Contributed by Gabor Bota. 
(cherry picked from commit 418cc7f3aeabedc57c94aa9d4c4248c1476ac90e)
 2018-07-11 14:48:12 -04:00
Wei-Chiu Chuang
74aa4be92d HDFS-13723. Occasional "Should be different group" error in TestRefreshUserMappings#testGroupMappingRefresh. Contributed by Siyao Meng. 
(cherry picked from commit 162228e8db937d4bdb9cf61d15ed555f1c96368f)
(cherry picked from commit f9fa3cb1570af6497fdb51f47a457268fbea6bd5)
 2018-07-11 10:04:38 -07:00
Andrew Wang
0e57e81b0c HDFS-13719. Docs around dfs.image.transfer.timeout are misleading. Contributed by Kitti Nansi. 
(cherry picked from commit eecb5baaaaa54599aeae758abd4007e55e5b531f)
 2018-07-09 15:17:48 +02:00
Wei-Chiu Chuang
68290606ea HDFS-13121. NPE when request file descriptors when SC read. Contributed by Zsolt Venczel. 
(cherry picked from commit 0247cb6318507afe06816e337a19f396afc53efa)
(cherry picked from commit f5f4d0b7e72eaf2ea9c9e791f71d0ea95f5b4b20)
 2018-07-06 15:01:35 -07:00
Inigo Goiri
98f2444a7d HDFS-13714. Fix TestNameNodePrunesMissingStorages test failures on Windows. Contributed by Lukas Majercak. 
(cherry picked from commit 7296b644f7c44400f03995668573a4cfd2ee552c)
 2018-07-02 16:37:36 -07:00
Andrew Wang
d34260f122 HDFS-13702. Remove HTrace hooks from DFSClient to reduce CPU usage. Contributed by Todd Lipcon. 
(cherry picked from commit 5d748bd056a32f2c6922514cd0c5b31d866a9919)
 2018-07-02 12:11:14 +02:00
Andrew Wang
7cd769b31c HDFS-13703. Avoid allocation of CorruptedBlocks hashmap when no corrupted blocks are hit. Contributed by Todd Lipcon. 
(cherry picked from commit 6ba99741086170b83c38d3e7e715d9e8046a1e00)
 2018-07-02 12:02:27 +02:00
Yiqun Lin
db1f37a711 HDFS-13692. StorageInfoDefragmenter floods log when compacting StorageInfo TreeSet. Contributed by Bharat Viswanadham. 
(cherry picked from commit 30728aced4a6b05394b3fc8c613f39fade9cf3c2)
 2018-06-22 10:55:09 +08:00
Xiao Chen
aca66f6155 HDFS-13682. Cannot create encryption zone after KMS auth token expires. 
(cherry picked from commit 32f867a6a907c05a312657139d295a92756d98ef)
 2018-06-20 15:59:45 -07:00
Surendra Singh Lilhore
5667582bc4 HDFS-12857. StoragePolicyAdmin should support schema based path. Contributed by Surendra Singh Lilhore. 
(cherry picked from commit 30941d99c9014431981eeb09ab24e90bef629fee)
 2018-06-19 10:09:05 -07:00
Inigo Goiri
11d3db2dba HDFS-13681. Fix TestStartup.testNNFailToStartOnReadOnlyNNDir test failure on Windows. Contributed by Xiao Liang. 
(cherry picked from commit 8762e9cf10fa100dd5f7fd695f5e52b75a94c5d4)
 2018-06-15 16:50:20 -07:00
Inigo Goiri
69abd1fc5c HDFS-13686. Add overall metrics for FSNamesystemLock. Contributed by Lukas Majercak. 
(cherry picked from commit d31a3ce767d3bb68bdbb4f36d45600eab9f4f8b7)
 2018-06-15 16:00:16 -07:00
Wei-Chiu Chuang
73badce597 HDFS-13174. hdfs mover -p /path times out after 20 min. Contributed by Istvan Fajth. 
(cherry picked from commit c966a3837af1c1a1c4a441f491b0d76d5c9e5d78)
(cherry picked from commit 975d4b3d603632a5edacb138cf4a1ce92ebed02e)
 2018-06-15 13:38:11 -07:00
Inigo Goiri
fe1926a50f HDFS-13676. TestEditLogRace fails on Windows. Contributed by Zuoming Zhang. 
(cherry picked from commit eebeb6033fd09791fcbff626f128a98e393f0a88)
 2018-06-15 13:08:47 -07:00
Inigo Goiri
94e0b9839f HDFS-13673. TestNameNodeMetrics fails on Windows. Contributed by Zuoming Zhang. 
(cherry picked from commit 43d994e4a6dfd1c24eafb909d6f8a0663b20769a)
 2018-06-15 10:34:33 -07:00
Inigo Goiri
a3c0c03899 HDFS-13563. TestDFSAdminWithHA times out on Windows. Contributed by Lukas Majercak. 
(cherry picked from commit 8d4926f38bf53b32453cd2bc7322c8818f752f85)
 2018-06-14 09:59:50 -07:00
Yiqun Lin
b94b071b10 HDFS-13641. Add metrics for edit log tailing. Contributed by Chao Sun. 
(cherry picked from commit 8e7548d33be9c4874daab18b2e774bdc2ed216d3)
 2018-06-13 20:24:12 +08:00
Yongjun Zhang
8202c334fe Update CHANGES, RELEASENOTES, and jdiff for 3.0.3 release. 
(cherry picked from commit 5a82f10e3254bc0745c2dda6bcef888d3ff0d1c3)
 2018-06-11 14:28:44 -07:00
Inigo Goiri
6eea6d28de HDFS-13653. Make dfs.client.failover.random.order a per nameservice configuration. Contributed by Ekanth Sethuramalingam. 
(cherry picked from commit 784dbbd2d1fc0c361fee851d3f9b3aa50be5481d)
 2018-06-11 11:18:04 -07:00
Bharat Viswanadham
7c144e4ae4 HDFS-13667:Typo: Marking all datandoes as stale. Contributed by Nanda Kumar 
(cherry picked from commit ccfb816d39878abf4172933327d788c59b9eb082)
 2018-06-09 16:44:47 -07:00
Arpit Agarwal
ea05171ceb HDFS-12670. can't renew HDFS tokens with only the hdfs client jar. Contributed by Arpit Agarwal. 2018-06-08 17:58:02 -07:00
Brahma Reddy Battula
a45e9812d6 HDFS-12950. [oiv] ls will fail in secure cluster. Contributed by Wei-Chiu Chuang. 
(cherry picked from commit 9654dd1f472052c4bb4a48a7412149c2e4859a10)
(cherry picked from commit 630f43faf7a1d4aaaefd860e3dcf25c288ca1d2f)
 2018-06-07 09:18:20 -07:00
Wei-Chiu Chuang
8e423a7779 HDFS-13659. Add more test coverage for contentSummary for snapshottable path. Contributed by Wei-Chiu Chuang. 
(cherry picked from commit e39b113db0f2e4bcf93f873801be770e472601da)
(cherry picked from commit b2e919dc1a8688bb6499f7334ad67c49dcc1bc93)
 2018-06-07 08:32:53 -07:00
Akira Ajisaka
cc735146dc
HDFS-13545. "guarded" is misspelled as "gaurded" in FSPermissionChecker.java. Contributed by Jianchao Jia. 
(cherry picked from commit 0e3c31579d1c733decfb9b00721bdc7b86be60f2)
 2018-06-05 17:55:33 +09:00
Inigo Goiri
e794ff2dcb HDFS-13650. Randomize baseDir for MiniDFSCluster in TestDFSStripedInputStream and TestDFSStripedInputStreamWithRandomECPolicy. Contributed by Anbang Hu. 
(cherry picked from commit 8d31ddcfeb5c2e97692dd386e5bd62f433b44f8e)
 2018-06-04 18:29:12 -07:00
Inigo Goiri
076dc95c76 HDFS-13649. Randomize baseDir for MiniDFSCluster in TestReconstructStripedFile and TestReconstructStripedFileWithRandomECPolicy. Contributed by Anbang Hu. 
(cherry picked from commit 16316b60112842cf8a328b79c89297f6a8b30e7b)
 2018-06-04 18:22:35 -07:00
Inigo Goiri
2e5f849ea5 HDFS-13652. Randomize baseDir for MiniDFSCluster in TestBlockScanner. Contributed by Anbang Hu. 
(cherry picked from commit a28848104f2a067669f9b1706cd0da05ba71bcb7)
 2018-06-04 18:13:50 -07:00
Rushabh Shah
4ecb7aaf77 HDFS-13281 Namenode#createFile should be /.reserved/raw/ aware.. Contributed by Rushabh S Shah 
(cherry picked from commit e2289c8d1496a5eff88e6bcb8776a11d45371ffc)
 2018-06-04 09:58:38 -05:00
Wei-Chiu Chuang
63fc8f82cb HDFS-13155. BlockPlacementPolicyDefault.chooseTargetInOrder Not Checking Return Value for NULL. Contributed by Zsolt Venczel. 
(cherry picked from commit bccdfeee0aaef9cb98d09ee39909b63fdcbeeafc)
(cherry picked from commit 302c36ffdce4cba247f084737a539022d56d8f6d)
 2018-06-04 07:04:32 -07:00
Xiao Chen
3ee5fd16f4 HDFS-13339. Volume reference can't be released and may lead to deadlock when DataXceiver does a check volume. Contributed by liaoyuxiangqin and Zsolt Venczel. 
(cherry picked from commit 9efb4b7db00d79aded52997ec89a1be94ecdd268)
 2018-06-03 22:37:36 -07:00
Inigo Goiri
a274d21ea3 HDFS-13651. TestReencryptionHandler fails on Windows. Contributed by Anbang Hu. 
(cherry picked from commit e308ac88d399e9bed3581ff4b1450aa2070dcedf)
 2018-06-02 20:13:55 -07:00