Apache/hadoop
1
0
Fork 0
Code Issues Pull Requests Packages Projects Releases Wiki Activity
12,127 Commits 326 Branches 371 Tags
Commit Graph

4299 Commits

Author SHA1 Message Date
Brahma Reddy Battula
5dc29ab0f2 HDFS-10668. Fix intermittently failing UT TestDataNodeMXBean#testDataNodeMXBeanBlockCount. Contributed by Mingliang Liu. 
(cherry picked from commit 096d3fa25cbbdda4fa2ad30ea5a76f33e2631c7d)
 2016-07-27 10:31:29 +05:30
Arpit Agarwal
45a6ee97b2 HDFS-10642. TestLazyPersistReplicaRecovery#testDnRestartWithSavedReplicas fails intermittently. (Contributed by Mingliang Liu) 2016-07-26 12:45:06 -07:00
Wei-Chiu Chuang
53bf632702 HDFS-9937. Update dfsadmin command line help and HdfsQuotaAdminGuide. Contributed by Kai Sasaki. 
(cherry picked from commit 25f0a9b3ab136d44f70eb2805598e2df8325612a)
 2016-07-26 08:31:00 -07:00
Steve Loughran
7921ad949f Revert "HDFS-10668. Fix intermittently failing UT TestDataNodeMXBean#testDataNodeMXBeanBlockCount. Contributed by Mingliang Liu." 
This reverts commit 32468b1ba5fe5e00e92656a8bf1591114383d7b3.
 2016-07-26 14:31:02 +01:00
Masatake Iwasaki
6b5d546014 HDFS-10671. Fix typo in HdfsRollingUpgrade.md. Contributed by Yiqun Lin. 
(cherry picked from commit 59466b8c180716dda7aa670728580a88e54eb4d2)
 2016-07-26 19:52:23 +09:00
Brahma Reddy Battula
32468b1ba5 HDFS-10668. Fix intermittently failing UT TestDataNodeMXBean#testDataNodeMXBeanBlockCount. Contributed by Mingliang Liu. 
(cherry picked from commit 7cac7655fd84ac394250705b31e3927fe548e34a)
 2016-07-26 11:00:11 +05:30
Akira Ajisaka
e377e94f64 Revert "HDFS-10620. StringBuilder created and appended even if logging is disabled. Contributed by Staffan Friberg." 
This reverts commit 91f47ab5f97989872339a934ea351bf68a077553.
 2016-07-26 11:06:10 +09:00
Vinitha Reddy Gankidi
9ab153d7ae HDFS-10301. Interleaving processing of storages from repeated block reports causes false zombie storage detection, removes valid blocks. Contributed by Vinitha Gankidi. 2016-07-25 18:59:42 -07:00
Jing Zhao
62800dc8b6 HDFS-10688. BPServiceActor may run into a tight loop for sending block report when hitting IOException. Contributed by Chen Liang. 
(cherry picked from commit 0cde9e12a7175e4d8bc4ccd5c36055b280d1fbd6)
(cherry picked from commit 30c772fc595b71d3b26a13bd0473570a84b5e807)
 2016-07-25 18:44:06 -07:00
Arpit Agarwal
00cbd3136e HDFS-10660. Expose storage policy apis via HDFSAdmin interface. (Contributed by Rakesh R) 2016-07-25 09:51:59 -07:00
Jing Zhao
54be36a6d6 HDFS-9894. Add unsetStoragePolicy API to FileContext/AbstractFileSystem and derivatives. Contributed by Xiaobing Zhou. 
(cherry picked from commit 7149cdb3c2d9dd390cd8668883cbe5db94090e0a)
 2016-07-25 09:51:59 -07:00
Arpit Agarwal
bbf711c030 HDFS-9534. Add CLI command to clear storage policy from a path. (Contributed by Xiaobing Zhou) 2016-07-25 09:51:59 -07:00
Xiao Chen
8559442ed1 HDFS-10225. DataNode hot swap drives should disallow storage type changes. Contributed by Lei (Eddy) Xu. 
(cherry picked from commit 132deb4cacc413a85a6af2f390ec79a76c91961c)
 2016-07-22 15:04:21 -07:00
Jing Zhao
6a7fe835f1 HDFS-10653. Optimize conversion from path string to components. Contributed by Daryn Sharp. 
(cherry picked from commit bd3dcf46e263b6e6aa3fca6a5d9936cc49e3280f)
 2016-07-21 11:29:27 -07:00
Akira Ajisaka
2f14d585c4 HDFS-10287. MiniDFSCluster should implement AutoCloseable. Contributed by Andras Bokor. 
(cherry picked from commit fcde6940e0cbdedb1105007e4857137ecdfa1284)
(cherry picked from commit 1222889f1a3b45df30c48a80d8c2bec7ad45ee43)
 2016-07-21 11:28:46 -07:00
Akira Ajisaka
91f47ab5f9 HDFS-10620. StringBuilder created and appended even if logging is disabled. Contributed by Staffan Friberg. 
(cherry picked from commit f7dabe3addf3f6eb32ea9b8ec1354fb442ce4222)
(cherry picked from commit 2685e16f381532d53aea413ce1c0a14181ee7c89)
 2016-07-19 15:19:02 -07:00
Akira Ajisaka
d8181a8cb8 HDFS-10603. Fix flaky tests in org.apache.hadoop.hdfs.server.namenode.snapshot.TestOpenFilesWithSnapshot. Contributed by Yiqun Lin. 
(cherry picked from commit fc570b55b9d1d1d57329eca223f443e2c86f34b7)
(cherry picked from commit c92601e9814a041800e466fabdcc3c44309364c4)
 2016-07-19 13:52:17 -07:00
Jitendra Pandey
a363277be5 HADOOP-13368. DFSOpsCountStatistics$OpType#fromSymbol and s3a.Statistic#fromSymbol should be O(1) operation. Contributed by Mingliang Liu. 2016-07-15 14:32:20 -07:00
Akira Ajisaka
83126ac90d HDFS-10628. Log HDFS Balancer exit message to its own log. Contributed by Jiayi Zhou. 
(cherry picked from commit c48e9d608c1daf7826f2e738c0a67d8eab79d8de)
(cherry picked from commit 617149837531062e4e1ee9eeb50a324766b14c60)
 2016-07-15 11:43:54 -07:00
Akira Ajisaka
6a04d9ab63 HADOOP-13351. TestDFSClientSocketSize buffer size tests are flaky. Contributed by Aaron Fabbri and Mingliang Liu. 
(cherry picked from commit 5537c6b23430285ebee33c6d9b69d3ec1e9b17b1)
(cherry picked from commit c9f5348bbfca664c6ab4c13f40cae0a2c5754da1)
 2016-07-14 14:42:55 -07:00
Zhe Zhang
af94dfaa53 HDFS-10544. Balancer doesn't work with IPFailoverProxyProvider. 
(cherry picked from commit 087290e6b1cb1082646d966b65494082712ebe3e)
(cherry picked from commit 5b4e916b3cae2540267d48716fd9240dfc055288)
 2016-07-12 23:28:16 -07:00
Jitendra Pandey
35108025e7 HDFS-10579. HDFS web interfaces lack configs for X-FRAME-OPTIONS protection. Contributed by Anu Engineer. 
Conflicts:
	hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/NameNodeHttpServer.java
 2016-07-12 14:38:34 -07:00
Yongjun Zhang
96e68e722f HDFS-10512. VolumeScanner may terminate due to NPE in DataNode.reportBadBlocks. Contributed by Wei-Chiu Chuang and Yiqun Lin. 
(cherry picked from commit da6f1b88dd47e22b24d44f6fc8bbee73e85746f7)
 2016-07-08 22:20:27 -07:00
Jitendra Pandey
9c7ce389c7 HADOOP-13283. Support reset operation for new global storage statistics and per FS storage stats. Contributed by Mingliang Liu. 2016-07-07 15:04:47 -07:00
Uma Maheswara Rao G
097022e172 HDFS-10592:Fix intermittent test failure of TestNameNodeResourceChecker#testCheckThatNameNodeResourceMonitorIsRunning. Contributed by Rakesh R 
(cherry picked from commit ab092c56c251f27a9426e3c69a784417fdf9a26e)
 2016-07-07 14:02:25 -07:00
Andrew Wang
f826d92909 HDFS-10488. Update WebHDFS documentation regarding CREATE and MKDIR default permissions. Contributed by Wellington Chevreuil. 
(cherry picked from commit 647b084738b26dcf0c0c4afa4ec3039e6aca2d61)
 2016-07-07 10:37:16 -07:00
Akira Ajisaka
4fba09d7e8 HDFS-10582. Change deprecated configuration fs.checkpoint.dir to dfs.namenode.checkpoint.dir in HDFS Commands Doc. Contributed by Pan Yuxuan. 
(cherry picked from commit a3f93beaa19d9acb7b1db2958102d0004b0eee46)
(cherry picked from commit 8a9440298db2c3943cbe399c68e247adeefa28b0)
 2016-07-06 17:54:44 -07:00
Akira Ajisaka
4b263b30c2 HDFS-10589. Javadoc for HAState#HAState and HAState#setStateInternal contains non-existent params. Contributed by shenyinjie. 
(cherry picked from commit 8b4b5259d5088360476b4582c4cf7b0e7973eff3)
(cherry picked from commit 0401adcaca5f2e9680bc2dc3602af8696d204f21)
 2016-07-04 07:15:54 -07:00
Jitendra Pandey
bc7fd76a1f HADOOP-13305. Define common statistics names across schemes. Contributed by Mingliang Liu. 2016-07-01 15:37:32 -07:00
Vinayakumar B
f44d7782f6 HDFS-10440. Improve DataNode web UI (Contributed by Weiwei Yang) 
(cherry picked from commit 2a0082c51da7cbe2770eddb5f72cd7f8d72fa5f6)
(cherry picked from commit 1e347631817d882353bfb91d68f109cb8232e8c4)
 2016-06-28 16:55:57 +05:30
Akira Ajisaka
700dacf8c4 HDFS-7597. DelegationTokenIdentifier should cache the TokenIdentifier to UGI mapping. Contributed by Daryn Sharp, Bob Hansen, and Xiao Chen. 
(cherry picked from commit d433b16ce6d74f1a44bc29446c74b1cb5f8a10fa)
(cherry picked from commit ad50a64ded1ba0c32d7bc89e5100f3b4d86a790c)
 2016-06-22 11:02:48 +09:00
Brahma Reddy Battula
aa8f4cc48c HDFS-9530. ReservedSpace is not cleared for abandoned Blocks (Contributed by Brahma Reddy Battula) 
(cherry picked from commit f2ac132d6a21c215093b7f87acf2843ac8123716)
(cherry picked from commit ee0f389ecd0c479aa38dae7ee6fd02ed66a64e4e)
 2016-06-21 15:55:57 +05:30
Colin Patrick Mccabe
9b51420ea5 HDFS-10448. CacheManager#addInternal tracks bytesNeeded incorrectly when dealing with replication factors other than 1 (Yiqun Lin via cmccabe) 
(cherry picked from commit 46f1602e896273b308fbd5df6c75f6c142828227)
(cherry picked from commit 4e11f33ccc849355eaeb0694dbee702e34340a74)
 2016-06-20 18:32:00 -07:00
Jitendra Pandey
29c3ee9744 HDFS-10538. Remove AsyncDistributedFileSystem. Contributed by Xiaobing Zhou and Jitendra Pandey. 2016-06-20 18:05:32 -07:00
Jitendra Pandey
2f95d22b17 HADOOP-13291. Probing stats in DFSOpsCountStatistics/S3AStorageStatistics should be correctly implemented. Contributed by Mingliang Liu. 2016-06-20 16:31:54 -07:00
Ravi Prakash
588ea6d431 HDFS-10220. A large number of expired leases can make namenode unresponsive and cause failover (Nicolas Fraison via raviprak) 
(cherry picked from commit ae047655f4355288406cd5396fb4e3ea7c307b14)
(cherry picked from commit 9c5f7f290eb05808fe89835a15bee0947b91d1a0)
 2016-06-20 14:38:05 -07:00
Brahma Reddy Battula
9d27530c36 HDFS-10474. hftp copy fails when file name with Chinese+special char in branch-2 (Contributed by Brahma Reddy Battula) 
(cherry picked from commit 3f27f405039926bbee4b36a60d6cecdb6322fff5)
 2016-06-20 12:40:11 +05:30
Andrew Wang
3f2d2cd0a1 Fix hadoop-hdfs-native-client compilation on Windows. Contributed by Brahma Reddy Battula. 
(cherry picked from commit 3ff0510ffdc20e44f06aec591782f8875b686327)
(cherry picked from commit 94759470b8a922d410797b8b76e055fd2fc6dd69)
 2016-06-17 15:55:43 -07:00
Kihwal Lee
edd5373095 HDFS-10493. Add links to datanode web UI in namenode datanodes page. Contributed by Weiwei Yang. 
(cherry picked from commit 280069510b71d81a150b2b28d9fa987891d82774)
 2016-06-17 11:13:12 -05:00
Akira Ajisaka
242263c56d HADOOP-12943. Add -w -r options in dfs -test command. Contributed by Weiwei Yang. 
(cherry picked from commit 09e82acaf9a6d7663bc51bbca0cdeca4b582b535)
(cherry picked from commit a36aa920f437f9e21772e8067e5db86fbffe7895)
 2016-06-17 16:21:31 +09:00
Colin Patrick Mccabe
cc53ca8d69 HDFS-9466. TestShortCircuitCache#testDataXceiverCleansUpSlotsOnFailure is flaky (Wei-Chiu Chuang via cmccabe) 
(cherry picked from commit c7921c9bddb79c9db5059b6c3f7a3a586a3cd95b)
(cherry picked from commit 49c0d9b325a21fc0a05da58bb9d4c76d82181a5d)
 2016-06-16 13:53:55 -07:00
Arpit Agarwal
394dae17ac HDFS-10532. Typo in RollingUpgrade docs. (Contributed by Yiqun Lin) 2016-06-16 10:11:05 -07:00
Colin Patrick Mccabe
ea2e7321d6 HDFS-10525. Fix NPE in CacheReplicationMonitor#rescanCachedBlockMap (Xiao Chen via cmccabe) 
(cherry picked from commit 2ca73445f5c2929d9c2ff4232dca58a63a0570a0)
(cherry picked from commit 8280e3bcc21adbbbb21e091bd4ec786bf3beb72b)
 2016-06-15 22:51:27 -07:00
Colin Patrick Mccabe
ef08eb9031 HDFS-10505. OIV's ReverseXML processor should support ACLs (Surendra Singh Lilhore via cmccabe) 
(cherry picked from commit 2449db507d84b1c4fac70a800fb2ad8905cf3db7)
(cherry picked from commit 17301ab3958d0b36fc6179efe1090a3b12b34772)
 2016-06-15 22:44:41 -07:00
Akira Ajisaka
b35c7cdd14 HADOOP-12892. fix/rewrite create-release. Contributed by Allen Wittenauer. 2016-06-16 14:06:08 +09:00
Ming Ma
03ed3f9b66 Revert "HDFS-9016. Display upgrade domain information in fsck. (mingma)" 
This reverts commit 68ab0aa9d0b9bd03768dcd7c422ef201ad3f58f6.
 2016-06-14 20:14:00 -07:00
Ming Ma
68ab0aa9d0 HDFS-9016. Display upgrade domain information in fsck. (mingma) 
(cherry picked from commit b7436f44682442e4d431ec3c97e4753e03435f3e)
 2016-06-14 20:08:29 -07:00
Andrew Wang
56e29d2711 HDFS-10516. Fix bug when warming up EDEK cache of more than one encryption zone. Contributed by Xiao Chen. 
(cherry picked from commit 709a814fe0153e86a37806796ea27c8252d9c6d1)
(cherry picked from commit ff4466fe15c338e56bab2e4afd61c46b634843b3)
 2016-06-13 11:50:27 -07:00
Andrew Wang
f583fa5f4e Remove redundant TestMiniDFSCluster.testDualClusters. Contributed by Jiayi Zhou. 
(cherry picked from commit 1ee9ea002609971ad58082bf525d57fca8a37035)
(cherry picked from commit 9319665461201ca84c7fa007c09f087f5753c8cc)
 2016-06-08 12:59:08 -07:00
Zhe Zhang
72addd8d5c HDFS-10458. getFileEncryptionInfo should return quickly for non-encrypted cluster. 
(cherry picked from commit 6de9213df111a9a4ed875db995d67af72d08a798)
(cherry picked from commit 06e38c835d0ad9619d4bea662f2dd7d0f62007a9)
(cherry picked from commit 0499907466e78a5b674ac72a5503af892adc348f)
 2016-06-07 09:09:18 -07:00