Apache/hadoop
1
0
Fork 0
Code Issues Pull Requests Packages Projects Releases Wiki Activity
11,911 Commits 326 Branches 371 Tags
Commit Graph

4171 Commits

Author SHA1 Message Date
Wei-Chiu Chuang
fa3b2bee61 HDFS-11229. HDFS-11056 failed to close meta file. Contributed by Wei-Chiu Chuang. 
(cherry picked from commit 292bd78b4494ba1a1073b32b3f89d95e72f3204d)
 2016-12-09 16:12:09 -08:00
Andrew Wang
e8b0ef0618 HDFS-8630. WebHDFS : Support get/set/unset StoragePolicy. Contributed by Surendra Singh Lilhore. 2016-12-09 14:56:49 -08:00
Andrew Wang
bced9e2e5b HDFS-11217. Annotate NameNode and DataNode MXBean interfaces as Private/Stable. Contributed by Jagadesh Kiran N. 
(cherry picked from commit 5bd7dece920b20da95755cf7fbba9b3f39c7da69)
(cherry picked from commit 85adc6677ba28bf106361be595e709941cb36496)
 2016-12-09 12:05:50 -08:00
Vinayakumar B
b1c08737d8 HDFS-11224. Lifeline message should be ignored for dead nodes (Contributed by Vinayakumar B) 
(cherry picked from commit d1d4aba71b21871140b162583a4b94ce118e1fb3)
(cherry picked from commit 199d8b559434c296c22ce7cfe5fb8de0dead0d12)
 2016-12-09 14:58:28 +05:30
Xiao Chen
892ee3f191 HDFS-11197. Listing encryption zones fails when deleting a EZ that is on a snapshotted directory. Contributed by Wellington Chevreuil. 
(cherry picked from commit 401c7318723d8d62c7fc29728f7f4e8d336b4d2f)
 2016-12-08 12:41:31 -08:00
Akira Ajisaka
59b0857c6b HDFS-11140. Directory Scanner should log startup message time correctly. Contributed by Yiqun Lin. 
(cherry picked from commit 9ef89ede2f18c76c601fd585cb9d47511f5fc3ed)
(cherry picked from commit e00a124cb5c43e34674faa8b450254074b7fd3a6)
 2016-12-08 13:42:02 +09:00
Vinayakumar B
7dc22dbe0c HDFS-9057. allow/disallow snapshots via webhdfs (Contributed by Brahma Reddy Battula) 
(cherry picked from commit 6d2332ae375e26d024358c6e75fdb3c68a781a66)
(cherry picked from commit 4b21b47f5607027672403fce211165abe03b9a77)
 2016-12-07 15:58:29 -08:00
Kihwal Lee
ac9801fec8 HDFS-11198. NN UI should link DN web address using hostnames. Contributed by Weiwei Yang. 2016-12-06 08:08:28 -06:00
Andrew Wang
1a7b2ab9bb Revert "HDFS-11156. Add new op GETFILEBLOCKLOCATIONS to WebHDFS REST API. Contributed by Weiwei Yang" 
This reverts commit 3b80424d4f2ff073310c9628f4645aab6cfb6d4d.
 2016-12-05 23:09:30 -08:00
Andrew Wang
79f830ed22 HDFS-10581. Hide redundant table on NameNode WebUI when no nodes are decomissioning. Contributed by Weiwei Yang. 
(cherry picked from commit 8e63fa98eabac55bdb2254306584ad1e759c79eb)
(cherry picked from commit 2fdae7324b1c98a2262d0de8e73c0c9099ca6ff0)
 2016-12-05 18:14:09 -08:00
Mingliang Liu
3b80424d4f HDFS-11156. Add new op GETFILEBLOCKLOCATIONS to WebHDFS REST API. Contributed by Weiwei Yang 
(cherry picked from commit c7ff34f8dcca3a2024230c5383abd9299daa1b20)
(cherry picked from commit be969e591883aa6cdd69bb62cea4e8904ece65f1)
 2016-12-02 11:38:36 -08:00
Akira Ajisaka
5025a898e1 HDFS-11180. Intermittent deadlock in NameNode when failover happens. 
(cherry picked from commit 5a7941a4fc193259ab4a306f7fe4f68bf101d0e0)
 2016-12-02 11:35:22 +09:00
Kihwal Lee
dd4acebb41 HDFS-8674. Improve performance of postponed block scans. Contributed by Daryn Sharp. 
(cherry picked from commit 0d8a35bd6de5d2a5a9b816ca98f31975e94bd7c6)

Conflicts:
	hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/blockmanagement/BlockManager.java
 2016-12-01 12:20:30 -06:00
Robert Kanter
92ee5e2077 HADOOP-13838. KMSTokenRenewer should close providers (xiaochen via rkanter) 
(cherry picked from commit 47ca9e26fba4a639e43bee5bfc001ffc4b42330d)
(cherry picked from commit 285edf691db42a3e17138d960f8560c62d8d0dde)
 2016-11-28 18:23:23 -08:00
Akira Ajisaka
011d478aab HDFS-11175. Document uppercase key names are not supported in TransparentEncryption.md. Contributed by Yiqun Lin. 
(cherry picked from commit 636f7547b7067c315cf5448d77e49a8fe55651b3)
(cherry picked from commit 0df7f5cdf673d9b0f97f85099c52a0a77342836e)
 2016-11-25 17:50:43 +09:00
Brahma Reddy Battula
9be7b353dd HADOOP-13646. Remove outdated overview.html. Contributed By Brahma Reddy Battula. 
(cherry picked from commit afcf8d38e750f935c06629e641a1321b79c4cace)
 2016-11-22 20:36:32 +05:30
Kihwal Lee
f5410f0814 HDFS-10966. Enhance Dispatcher logic on deciding when to give up a source DataNode. Contributed by Mark Wagner and Zhe Zhang. 
(cherry picked from commit 774b0cd845a67bc0e2ef99b415ce1e5432769551)

Conflicts:
	hadoop-hdfs-project/hadoop-hdfs/src/main/resources/hdfs-default.xml
 2016-11-21 10:52:42 -06:00
Brahma Reddy Battula
ce4cd98784 HDFS-11101. TestDFSShell#testMoveWithTargetPortEmpty fails intermittently. Contributed By Brahma Reddy Battula. 
(cherry picked from commit f92206774899b84e8abaecdf97399b65e0995ed8)
 2016-11-21 19:36:57 +05:30
Brahma Reddy Battula
23ddb98e1b HDFS-11144. TestFileCreationDelete#testFileCreationDeleteParent fails wind bind exception. Contributed By Brahma Reddy Battula. 
(cherry picked from commit c68dad18ab5cdf01f3dea1bb5988f896609956b4)
 2016-11-21 11:00:15 +05:30
Brahma Reddy Battula
f0e2271149 HDFS-11134. Fix bind exception threw in TestRenameWhileOpen. Contributed By Yiqun Lin. 
(cherry picked from commit b2d4b7b1b902aa830d44889d7747a4e1b816ab9d)
 2016-11-17 10:26:50 +05:30
Mingliang Liu
a8dadd8c53 HDFS-11105. TestRBWBlockInvalidation#testRWRInvalidation fails intermittently. Contributed by Yiqun Lin 
(cherry picked from commit c90891e7b3afaa66dbca967b83a9997e80e893d8)
 2016-11-16 13:13:19 -08:00
Kihwal Lee
15f08e9298 HDFS-8870. Lease is leaked on write failure. Contributed by Kuhu Shukla. 
(cherry picked from commit 4fcea8a0c8019d6d9a5e6f315c83659938b93a40)
 2016-11-15 14:57:04 -06:00
Zhe Zhang
c6e7b631bd HDFS-10872. Add MutableRate metrics for FSNamesystemLock operations. Contributed by Erik Krogen. 
(cherry picked from commit 818915aabb22653e212f4646b1b3cd0673330328)
 2016-11-14 16:39:26 -08:00
Brahma Reddy Battula
4dc40fe28e HDFS-11135. The tests in TestBalancer run fails due to NPE. Contributed By Yiqun Lin 
(cherry picked from commit 6efb8c9c6018ec688eeba8c61c220009ec350a44)
 2016-11-14 20:42:50 +05:30
Erik Krogen
3d52459712 HDFS-11087. NamenodeFsck should check if the output writer is still writable. Contributed by Erik Krogen. 
(cherry picked from commit 98d9e9c4c823d865ddfc017e71c82d12c8328e53)
 2016-11-13 14:54:49 -08:00
Mingliang Liu
4bcef1c616 HADOOP-13813. TestDelegationTokenFetcher#testDelegationTokenWithoutRenewer is failing. Contributed by Mingliang Liu 
(cherry picked from commit fb3a5085e8057c79e0785e7e7bf7da031c3f3c12)
 2016-11-12 10:37:46 -08:00
Mingliang Liu
fcedb6812b HDFS-11129. TestAppendSnapshotTruncate fails with bind exception. Contributed by Brahma Reddy Battula 
(cherry picked from commit 2ee18fc15e60dff9a42a80eb0c30e8bd8cedc26a)
 2016-11-11 22:19:39 -08:00
Xiaoyu Yao
c443ecb267 HDFS-10941. Improve BlockManager#processMisReplicatesAsync log. Contributed by Chen Liang. 
(cherry picked from commit 4484b48498b2ab2a40a404c487c7a4e875df10dc)
 2016-11-11 14:32:11 -08:00
Mingliang Liu
ae7a20f524 HDFS-11122. TestDFSAdmin#testReportCommand fails due to timed out. Contributed by Yiqun Lin 
(cherry picked from commit aa6010ccca3045ce9f0bb819fb2cb7ff65e1822b)
 2016-11-11 12:35:21 -08:00
Brahma Reddy Battula
121b56de14 HDFS-11128. CreateEditsLog throws NullPointerException. Contributed by Hanisha Koneru. 
(cherry picked from commit 1ae57f0f75695178dc135b0c259d066f7da31c9d)
 2016-11-11 20:11:13 +05:30
Xiao Chen
d81706cd99 HDFS-11120. TestEncryptionZones should waitActive. Contributed by John Zhuge. 
(cherry picked from commit 907b1431c0ead934e26cf7dbc51bfab5d7cc5ddc)
 2016-11-09 17:16:36 -08:00
Wei-Chiu Chuang
1d75da8e26 HDFS-11056. Concurrent append and read operations lead to checksum error. Contributed by Wei-Chiu Chuang. 
(cherry picked from commit 16430c64b788bce6997f3a9e4401a2914f37b959)
 2016-11-09 13:32:35 -08:00
Mingliang Liu
ad992e05a6 HDFS-11083. Add unit test for DFSAdmin -report command. Contributed by Xiaobing Zhou 
(cherry picked from commit 62d8c17dfda75a6a6de06aedad2f22699a1cbad6)
 2016-11-08 17:10:59 -08:00
Mingliang Liu
00625004b7 HDFS-11085. Add unit test for NameNode failing to start when name dir is unwritable. Contributed by Xiaobing Zhou 
(cherry picked from commit 0c0ab102ab392ba07ed2aa8d8a67eef4c20cad9b)
 2016-11-04 14:45:43 -07:00
Mingliang Liu
b082d4b746 HDFS-11076. Add unit test for extended Acls. Contributed by Chen Liang 
(cherry picked from commit 7534aee09af47c6961100588312da8d133be1b27)
 2016-11-03 13:55:43 -07:00
Arpit Agarwal
3f2bfdba3e HDFS-11097. Fix warnings for deprecated StorageReceivedDeletedBlocks constructor. Contributed by Yiqun Lin. 2016-11-03 10:33:28 -07:00
Brahma Reddy Battula
77142827e8 HDFS-11031. Add additional unit test for DataNode startup behavior when volumes fail. Contributed by Mingliang Liu. 
(cherry picked from commit 35e6081020d34be966724f44df678f1f5ee6f6d8)
 2016-11-02 10:40:22 +05:30
Mingliang Liu
10eb21e52e HDFS-11065. Add space quota tests for heterogenous storages. Contributed by Xiaobing Zhou 
(cherry picked from commit fa1512a740b2ed2661743d6b5483ef3eb49e5634)
 2016-11-01 14:50:16 -07:00
Kihwal Lee
db6dfeca1a HDFS-10997. Reduce number of path resolving methods. Contributed by Daryn Sharp. 
(cherry picked from commit 7c1a1834e49cf88c64837d92b78bbc07ea9e9efc)
 2016-11-01 08:26:25 -05:00
Brahma Reddy Battula
c895f08936 HDFS-10455. Logging the username when deny the setOwner operation. Contributed by Rakesh R. 
(cherry picked from commit 1c5708f6ded6ba9fc007eed93c1f65ba44451241)
 2016-11-01 10:53:37 +05:30
Kihwal Lee
91640a0170 Addendum of HDFS-10722 to include HDFS-7964 logic. 
(cherry picked from commit 4a8e6dc02fb1d0dc89c3981b444575494511444a)
(cherry picked from commit 3b2554f3b0f686a66e3df20b7c381babb9bc74d3)
(cherry picked from commit c2bb9bf00c4a32644f96c673078a812087ef3c46)
 2016-10-31 16:37:00 -07:00
Zhe Zhang
7900b1e803 Revert "HDFS-10722. Fix race condition in TestEditLog#testBatchedSyncWithClosedLogs. Contributed by Daryn Sharp." 
This reverts commit c2bb9bf00c4a32644f96c673078a812087ef3c46.
 2016-10-31 16:35:05 -07:00
Kihwal Lee
c2bb9bf00c HDFS-10722. Fix race condition in TestEditLog#testBatchedSyncWithClosedLogs. Contributed by Daryn Sharp. 
(cherry picked from commit 4a8e6dc02fb1d0dc89c3981b444575494511444a)
(cherry picked from commit 3b2554f3b0f686a66e3df20b7c381babb9bc74d3)
 2016-10-31 16:30:39 -07:00
Akira Ajisaka
a17fc1dca3 HDFS-10485. Fix findbugs warning in FSEditLog.java. (aajisaka) 
(cherry picked from commit e620530301fd3e62537d4b7bc3d8ed296bda1ffc)
(cherry picked from commit b8c1db6f63f9d01f44f8f43921aadfe999eefd76)
 2016-10-31 16:24:35 -07:00
Jing Zhao
e327325a37 HDFS-7964. Add support for async edit logging. Contributed by Daryn Sharp. 
(cherry picked from commit 2151716832ad14932dd65b1a4e47e64d8d6cd767)
(cherry picked from commit 3cb7ae11a839c01b8be629774874c1873f51b747)
 2016-10-31 16:24:35 -07:00
Mingliang Liu
fece3c881b HDFS-11030. TestDataNodeVolumeFailure#testVolumeFailure is flaky (though passing). Contributed by Mingliang Liu 
(cherry picked from commit 6bb12ecc3f63afb2070296c2db009c6f9a2d884b)
 2016-10-31 16:06:41 -07:00
Arpit Agarwal
f0a6d17e3b HDFS-11074. Remove unused method FsDatasetSpi#getFinalizedBlocksOnPersistentStorage. Contributed by Hanisha Koneru. 2016-10-31 12:40:03 -07:00
Mingliang Liu
616c1194fe HDFS-11047. Remove deep copies of FinalizedReplica to alleviate heap consumption on DataNode. Contributed by Xiaobing Zhou 
(cherry picked from commit 2848a27700e76fc2c2d3ab87adba54ca8e93ba39)
 2016-10-28 11:18:26 -07:00
Mingliang Liu
a8a99f3423 HDFS-11048. Audit Log should escape control characters. Contributed by Eric Badger 
(cherry picked from commit 8a9388e5f6d622152798aaaa256064919e4fec37)
 2016-10-28 11:01:09 -07:00
Erik Krogen
2ec90a1a4b HDFS-9500. Fix software version counts for DataNodes during rolling upgrade. Contributed by Erik Krogen. 
(cherry picked from commit f3ac1f41b8fa82a0ac87a207d7afa2061d90a9bd)
 2016-10-27 16:04:58 -07:00