4036 Commits

Author SHA1 Message Date
Jing Zhao
6a7fe835f1 HDFS-10653. Optimize conversion from path string to components. Contributed by Daryn Sharp.
(cherry picked from commit bd3dcf46e263b6e6aa3fca6a5d9936cc49e3280f)
2016-07-21 11:29:27 -07:00
Akira Ajisaka
2f14d585c4 HDFS-10287. MiniDFSCluster should implement AutoCloseable. Contributed by Andras Bokor.
(cherry picked from commit fcde6940e0cbdedb1105007e4857137ecdfa1284)
(cherry picked from commit 1222889f1a3b45df30c48a80d8c2bec7ad45ee43)
2016-07-21 11:28:46 -07:00
Akira Ajisaka
91f47ab5f9 HDFS-10620. StringBuilder created and appended even if logging is disabled. Contributed by Staffan Friberg.
(cherry picked from commit f7dabe3addf3f6eb32ea9b8ec1354fb442ce4222)
(cherry picked from commit 2685e16f381532d53aea413ce1c0a14181ee7c89)
2016-07-19 15:19:02 -07:00
Akira Ajisaka
d8181a8cb8 HDFS-10603. Fix flaky tests in org.apache.hadoop.hdfs.server.namenode.snapshot.TestOpenFilesWithSnapshot. Contributed by Yiqun Lin.
(cherry picked from commit fc570b55b9d1d1d57329eca223f443e2c86f34b7)
(cherry picked from commit c92601e9814a041800e466fabdcc3c44309364c4)
2016-07-19 13:52:17 -07:00
Jitendra Pandey
a363277be5 HADOOP-13368. DFSOpsCountStatistics$OpType#fromSymbol and s3a.Statistic#fromSymbol should be O(1) operation. Contributed by Mingliang Liu. 2016-07-15 14:32:20 -07:00
Akira Ajisaka
83126ac90d HDFS-10628. Log HDFS Balancer exit message to its own log. Contributed by Jiayi Zhou.
(cherry picked from commit c48e9d608c1daf7826f2e738c0a67d8eab79d8de)
(cherry picked from commit 617149837531062e4e1ee9eeb50a324766b14c60)
2016-07-15 11:43:54 -07:00
Akira Ajisaka
6a04d9ab63 HADOOP-13351. TestDFSClientSocketSize buffer size tests are flaky. Contributed by Aaron Fabbri and Mingliang Liu.
(cherry picked from commit 5537c6b23430285ebee33c6d9b69d3ec1e9b17b1)
(cherry picked from commit c9f5348bbfca664c6ab4c13f40cae0a2c5754da1)
2016-07-14 14:42:55 -07:00
Zhe Zhang
af94dfaa53 HDFS-10544. Balancer doesn't work with IPFailoverProxyProvider.
(cherry picked from commit 087290e6b1cb1082646d966b65494082712ebe3e)
(cherry picked from commit 5b4e916b3cae2540267d48716fd9240dfc055288)
2016-07-12 23:28:16 -07:00
Jitendra Pandey
35108025e7 HDFS-10579. HDFS web interfaces lack configs for X-FRAME-OPTIONS protection. Contributed by Anu Engineer.
Conflicts:
	hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/NameNodeHttpServer.java
2016-07-12 14:38:34 -07:00
Yongjun Zhang
96e68e722f HDFS-10512. VolumeScanner may terminate due to NPE in DataNode.reportBadBlocks. Contributed by Wei-Chiu Chuang and Yiqun Lin.
(cherry picked from commit da6f1b88dd47e22b24d44f6fc8bbee73e85746f7)
2016-07-08 22:20:27 -07:00
Jitendra Pandey
9c7ce389c7 HADOOP-13283. Support reset operation for new global storage statistics and per FS storage stats. Contributed by Mingliang Liu. 2016-07-07 15:04:47 -07:00
Uma Maheswara Rao G
097022e172 HDFS-10592:Fix intermittent test failure of TestNameNodeResourceChecker#testCheckThatNameNodeResourceMonitorIsRunning. Contributed by Rakesh R
(cherry picked from commit ab092c56c251f27a9426e3c69a784417fdf9a26e)
2016-07-07 14:02:25 -07:00
Andrew Wang
f826d92909 HDFS-10488. Update WebHDFS documentation regarding CREATE and MKDIR default permissions. Contributed by Wellington Chevreuil.
(cherry picked from commit 647b084738b26dcf0c0c4afa4ec3039e6aca2d61)
2016-07-07 10:37:16 -07:00
Akira Ajisaka
4fba09d7e8 HDFS-10582. Change deprecated configuration fs.checkpoint.dir to dfs.namenode.checkpoint.dir in HDFS Commands Doc. Contributed by Pan Yuxuan.
(cherry picked from commit a3f93beaa19d9acb7b1db2958102d0004b0eee46)
(cherry picked from commit 8a9440298db2c3943cbe399c68e247adeefa28b0)
2016-07-06 17:54:44 -07:00
Akira Ajisaka
4b263b30c2 HDFS-10589. Javadoc for HAState#HAState and HAState#setStateInternal contains non-existent params. Contributed by shenyinjie.
(cherry picked from commit 8b4b5259d5088360476b4582c4cf7b0e7973eff3)
(cherry picked from commit 0401adcaca5f2e9680bc2dc3602af8696d204f21)
2016-07-04 07:15:54 -07:00
Jitendra Pandey
bc7fd76a1f HADOOP-13305. Define common statistics names across schemes. Contributed by Mingliang Liu. 2016-07-01 15:37:32 -07:00
Vinayakumar B
f44d7782f6 HDFS-10440. Improve DataNode web UI (Contributed by Weiwei Yang)
(cherry picked from commit 2a0082c51da7cbe2770eddb5f72cd7f8d72fa5f6)
(cherry picked from commit 1e347631817d882353bfb91d68f109cb8232e8c4)
2016-06-28 16:55:57 +05:30
Akira Ajisaka
700dacf8c4 HDFS-7597. DelegationTokenIdentifier should cache the TokenIdentifier to UGI mapping. Contributed by Daryn Sharp, Bob Hansen, and Xiao Chen.
(cherry picked from commit d433b16ce6d74f1a44bc29446c74b1cb5f8a10fa)
(cherry picked from commit ad50a64ded1ba0c32d7bc89e5100f3b4d86a790c)
2016-06-22 11:02:48 +09:00
Brahma Reddy Battula
aa8f4cc48c HDFS-9530. ReservedSpace is not cleared for abandoned Blocks (Contributed by Brahma Reddy Battula)
(cherry picked from commit f2ac132d6a21c215093b7f87acf2843ac8123716)
(cherry picked from commit ee0f389ecd0c479aa38dae7ee6fd02ed66a64e4e)
2016-06-21 15:55:57 +05:30
Colin Patrick Mccabe
9b51420ea5 HDFS-10448. CacheManager#addInternal tracks bytesNeeded incorrectly when dealing with replication factors other than 1 (Yiqun Lin via cmccabe)
(cherry picked from commit 46f1602e896273b308fbd5df6c75f6c142828227)
(cherry picked from commit 4e11f33ccc849355eaeb0694dbee702e34340a74)
2016-06-20 18:32:00 -07:00
Jitendra Pandey
29c3ee9744 HDFS-10538. Remove AsyncDistributedFileSystem. Contributed by Xiaobing Zhou and Jitendra Pandey. 2016-06-20 18:05:32 -07:00
Jitendra Pandey
2f95d22b17 HADOOP-13291. Probing stats in DFSOpsCountStatistics/S3AStorageStatistics should be correctly implemented. Contributed by Mingliang Liu. 2016-06-20 16:31:54 -07:00
Ravi Prakash
588ea6d431 HDFS-10220. A large number of expired leases can make namenode unresponsive and cause failover (Nicolas Fraison via raviprak)
(cherry picked from commit ae047655f4355288406cd5396fb4e3ea7c307b14)
(cherry picked from commit 9c5f7f290eb05808fe89835a15bee0947b91d1a0)
2016-06-20 14:38:05 -07:00
Brahma Reddy Battula
9d27530c36 HDFS-10474. hftp copy fails when file name with Chinese+special char in branch-2 (Contributed by Brahma Reddy Battula)
(cherry picked from commit 3f27f405039926bbee4b36a60d6cecdb6322fff5)
2016-06-20 12:40:11 +05:30
Andrew Wang
3f2d2cd0a1 Fix hadoop-hdfs-native-client compilation on Windows. Contributed by Brahma Reddy Battula.
(cherry picked from commit 3ff0510ffdc20e44f06aec591782f8875b686327)
(cherry picked from commit 94759470b8a922d410797b8b76e055fd2fc6dd69)
2016-06-17 15:55:43 -07:00
Kihwal Lee
edd5373095 HDFS-10493. Add links to datanode web UI in namenode datanodes page. Contributed by Weiwei Yang.
(cherry picked from commit 280069510b71d81a150b2b28d9fa987891d82774)
2016-06-17 11:13:12 -05:00
Akira Ajisaka
242263c56d HADOOP-12943. Add -w -r options in dfs -test command. Contributed by Weiwei Yang.
(cherry picked from commit 09e82acaf9a6d7663bc51bbca0cdeca4b582b535)
(cherry picked from commit a36aa920f437f9e21772e8067e5db86fbffe7895)
2016-06-17 16:21:31 +09:00
Colin Patrick Mccabe
cc53ca8d69 HDFS-9466. TestShortCircuitCache#testDataXceiverCleansUpSlotsOnFailure is flaky (Wei-Chiu Chuang via cmccabe)
(cherry picked from commit c7921c9bddb79c9db5059b6c3f7a3a586a3cd95b)
(cherry picked from commit 49c0d9b325a21fc0a05da58bb9d4c76d82181a5d)
2016-06-16 13:53:55 -07:00
Arpit Agarwal
394dae17ac HDFS-10532. Typo in RollingUpgrade docs. (Contributed by Yiqun Lin) 2016-06-16 10:11:05 -07:00
Colin Patrick Mccabe
ea2e7321d6 HDFS-10525. Fix NPE in CacheReplicationMonitor#rescanCachedBlockMap (Xiao Chen via cmccabe)
(cherry picked from commit 2ca73445f5c2929d9c2ff4232dca58a63a0570a0)
(cherry picked from commit 8280e3bcc21adbbbb21e091bd4ec786bf3beb72b)
2016-06-15 22:51:27 -07:00
Colin Patrick Mccabe
ef08eb9031 HDFS-10505. OIV's ReverseXML processor should support ACLs (Surendra Singh Lilhore via cmccabe)
(cherry picked from commit 2449db507d84b1c4fac70a800fb2ad8905cf3db7)
(cherry picked from commit 17301ab3958d0b36fc6179efe1090a3b12b34772)
2016-06-15 22:44:41 -07:00
Akira Ajisaka
b35c7cdd14 HADOOP-12892. fix/rewrite create-release. Contributed by Allen Wittenauer. 2016-06-16 14:06:08 +09:00
Ming Ma
03ed3f9b66 Revert "HDFS-9016. Display upgrade domain information in fsck. (mingma)"
This reverts commit 68ab0aa9d0b9bd03768dcd7c422ef201ad3f58f6.
2016-06-14 20:14:00 -07:00
Ming Ma
68ab0aa9d0 HDFS-9016. Display upgrade domain information in fsck. (mingma)
(cherry picked from commit b7436f44682442e4d431ec3c97e4753e03435f3e)
2016-06-14 20:08:29 -07:00
Andrew Wang
56e29d2711 HDFS-10516. Fix bug when warming up EDEK cache of more than one encryption zone. Contributed by Xiao Chen.
(cherry picked from commit 709a814fe0153e86a37806796ea27c8252d9c6d1)
(cherry picked from commit ff4466fe15c338e56bab2e4afd61c46b634843b3)
2016-06-13 11:50:27 -07:00
Andrew Wang
f583fa5f4e Remove redundant TestMiniDFSCluster.testDualClusters. Contributed by Jiayi Zhou.
(cherry picked from commit 1ee9ea002609971ad58082bf525d57fca8a37035)
(cherry picked from commit 9319665461201ca84c7fa007c09f087f5753c8cc)
2016-06-08 12:59:08 -07:00
Zhe Zhang
72addd8d5c HDFS-10458. getFileEncryptionInfo should return quickly for non-encrypted cluster.
(cherry picked from commit 6de9213df111a9a4ed875db995d67af72d08a798)
(cherry picked from commit 06e38c835d0ad9619d4bea662f2dd7d0f62007a9)
(cherry picked from commit 0499907466e78a5b674ac72a5503af892adc348f)
2016-06-07 09:09:18 -07:00
Jason Lowe
738b67e9d3 HDFS-9580. TestComputeInvalidateWork#testDatanodeReRegistration failed due to unexpected number of invalidate blocks. Contributed by Wei-Chiu Chuang.
Change-Id: I9411dfb1dd4605cbcc515ca3361218d35d7c5fe3
(cherry picked from commit 0087734cc1e18e029ea39f06ca25c8a7d60af004)

Conflicts:

	hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt
2016-06-07 15:36:53 +00:00
Wangda Tan
e91eba2f81 Revert "HDFS-10458. getFileEncryptionInfo should return quickly for non-encrypted cluster."
This reverts commit 0274636529cfac4d64582a7c21631aebbf4deb1b.
2016-06-06 22:13:02 -07:00
Zhe Zhang
0274636529 HDFS-10458. getFileEncryptionInfo should return quickly for non-encrypted cluster.
(cherry picked from commit 6de9213df111a9a4ed875db995d67af72d08a798)
(cherry picked from commit 06e38c835d0ad9619d4bea662f2dd7d0f62007a9)
2016-06-06 15:54:36 -07:00
Tsz-Wo Nicholas Sze
49535c0683 Revert "Revert "HADOOP-13226 Support async call retry and failover.""
This reverts commit 886e2396062ef076b3ea4aa93395cc90287efd4f.
2016-06-06 16:54:02 +08:00
Tsz-Wo Nicholas Sze
359a316736 Revert "Revert "HDFS-10430. Reuse FileSystem#access in TestAsyncDFS. Contributed by Xiaobing Zhou.""
This reverts commit f176311366eab0364458d055f248fb5796bdec23.
2016-06-06 16:53:56 +08:00
Tsz-Wo Nicholas Sze
025f62afcc Revert "Revert "HDFS-10431 Refactor and speedup TestAsyncDFSRename. Contributed by Xiaobing Zhou""
This reverts commit 0af02cfd37a564b4a0701dbc6d562fe237565bd5.
2016-06-06 16:53:51 +08:00
Tsz-Wo Nicholas Sze
79beac49ec Revert "Revert "HDFS-10390. Implement asynchronous setAcl/getAclStatus for DistributedFileSystem. Contributed by Xiaobing Zhou""
This reverts commit 1de712f22abbe313692cca6bece7f265da96aee5.
2016-06-06 16:53:46 +08:00
Tsz-Wo Nicholas Sze
c949001eee Revert "Revert "HADOOP-13168. Support Future.get with timeout in ipc async calls.""
This reverts commit 308d28640d8c797f07aa41a8adc5dcd9420f1d0a.
2016-06-06 16:53:41 +08:00
Tsz-Wo Nicholas Sze
5b054b1bfc Revert "Revert "HDFS-10346. Implement asynchronous setPermission/setOwner for DistributedFileSystem. Contributed by Xiaobing Zhou""
This reverts commit ffa85f28a2ec612a3d63c890806a17cfd3a2e7e2.
2016-06-06 16:53:36 +08:00
Tsz-Wo Nicholas Sze
8ccc8aeb6c Revert "Revert "HADOOP-12957. Limit the number of outstanding async calls. Contributed by Xiaobing Zhou""
This reverts commit 8a07026e7792e43e692409c3ec998a306b7733ff.
2016-06-06 16:53:31 +08:00
Tsz-Wo Nicholas Sze
2cf663bb59 Revert "Revert "HDFS-10224. Implement asynchronous rename for DistributedFileSystem. Contributed by Xiaobing Zhou""
This reverts commit d740a90260d9d26a67688b004e5fcda60cc30723.
2016-06-06 16:53:24 +08:00
Andrew Wang
d740a90260 Revert "HDFS-10224. Implement asynchronous rename for DistributedFileSystem. Contributed by Xiaobing Zhou"
This reverts commit bdc45bef646cafdc04a59c19e23dcba3bb16b20c.
2016-06-03 18:17:33 -07:00
Andrew Wang
8a07026e77 Revert "HADOOP-12957. Limit the number of outstanding async calls. Contributed by Xiaobing Zhou"
This reverts commit a1ba6eee38420ae9436f9e9ecc3db83702fbe762.
2016-06-03 18:17:33 -07:00