Apache
/
hadoop
1
0
Fork 0
Code Issues Pull Requests Packages Projects Releases Wiki Activity
11,444 Commits 326 Branches 371 Tags 1.5 GiB
Commit Graph

4020 Commits

Author SHA1 Message Date
Konstantin V Shvachko 6b3114a2a5 HDFS-10693. metaSave should print blocks, not LightWeightHashSet. Contributed by Yuanbo Liu. 
(cherry picked from commit 4d3af47f27)
 2016-08-07 14:36:24 -07:00
Kihwal Lee 5b3deac2e3 HDFS-9958. BlockManager#createLocatedBlocks can throw NPE for corruptBlocks on failed storages. Contributed by Kuhu Shukla. 
(cherry picked from commit f715f14185)
 2016-08-05 11:43:43 -05:00
Akira Ajisaka 5758026833 HDFS-10715. NPE when applying AvailableSpaceBlockPlacementPolicy. Contributed by Guangbin Zhu. 
(cherry picked from commit ef432579a7)
(cherry picked from commit 6863866127)
 2016-08-05 17:33:07 +09:00
Kihwal Lee e8cd1439d3 HDFS-10343. BlockManager#createLocatedBlocks may return blocks on failed storages. Contributed by Kuhu Shukla. 
(cherry picked from commit 57369a678c)

Conflicts:
	hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/blockmanagement/BlockManager.java

(cherry picked from commit 690ec789f0)
 2016-08-04 15:27:51 -05:00
Kihwal Lee eb9af6ca34 HDFS-10717. Fix findbugs warnings of hadoop-hdfs-client in branch-2. Contributed by Akira Ajisaka. 
(cherry picked from commit 6ead6f3bb8)
 2016-08-04 12:36:51 -05:00
Arpit Agarwal f3424d662a HDFS-10722. Fix build break. 2016-08-04 10:28:43 -07:00
Kihwal Lee 3a85aea42d HDFS-10722. Fix race condition in TestEditLog#testBatchedSyncWithClosedLogs. Contributed by Daryn Sharp. 
(cherry picked from commit 4a8e6dc02f)
 2016-08-04 11:56:26 -05:00
Tsz-Wo Nicholas Sze cefa21e98a In Balancer, the target task should be removed when its size < 0. Contributed by Yiqun Lin 2016-08-04 09:54:15 -07:00
Kihwal Lee 58df27b873 HDFS-10662. Optimize UTF8 string/byte conversions. Contributed by Daryn Sharp. 
(cherry picked from commit 6ae39199da)

Conflicts:
	hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/FSDirWriteFileOp.java

(cherry picked from commit 77b61d1f4e)
 2016-08-04 09:45:03 -05:00
Kihwal Lee 804db5b277 HDFS-10707. Replace org.apache.commons.io.Charsets with java.nio.charset.StandardCharsets. Contributed by Vincent Poon. 
(cherry picked from commit c0166b0743)

Conflicts:
	hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/datanode/web/webhdfs/WebHdfsHandler.java
 2016-08-04 09:35:16 -05:00
Kihwal Lee ceac628378 HDFS-10569. A bug causes OutOfIndex error in BlockListAsLongs. Contributed by Weiwei Yang. 
(cherry picked from commit 6f63566694)

Conflicts:
	hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/protocol/TestBlockListAsLongs.java

(cherry picked from commit 1ca2ab4013)
 2016-08-03 15:28:13 -05:00
Jing Zhao 5bdee64bde HDFS-10710. In BlockManager#rescanPostponedMisreplicatedBlocks(), postponed misreplicated block counts should be retrieved within the NN lock protection.Contributed by GAO Rui. 
(cherry picked from commit f4ba5ff1d7)
(cherry picked from commit 180a26d9fb)
 2016-08-03 12:44:49 -07:00
Kihwal Lee 33a87ffe10 HDFS-742. A down DataNode makes Balancer to hang on repeatingly asking NameNode its partial block list. Contributed by Mit Desai. 
(cherry picked from commit 58db263e93)
 2016-08-03 12:19:05 -05:00
Kihwal Lee 288f9ccde2 HDFS-5805. TestCheckpoint.testCheckpoint fails intermittently on branch-2. Contributed by Eric Badger. 
(cherry picked from commit 5e5b8793fb)
 2016-08-02 15:43:11 -05:00
Konstantin V Shvachko 9b3b7703b1 Revert "HDFS-10301. Interleaving processing of storages from repeated block reports causes false zombie storage detection, removes valid blocks. Contributed by Vinitha Gankidi." 
This reverts commit 9ab153d7ae.
 2016-08-01 22:48:09 -07:00
Daryn Sharp ecb143ed21 HDFS-10655. Fix path related byte array conversion bugs. (daryn) 
(cherry picked from commit 9f473cf903)
 2016-08-01 10:18:45 -05:00
Yongjun Zhang cc066de22c HDFS-10703. HA NameNode Web UI should show last checkpoint time. (John Zhuge via Yongjun Zhang) 
(cherry picked from commit 8d32bd884d)
 2016-07-29 23:47:46 -07:00
Akira Ajisaka e467f838bb HDFS-10691. FileDistribution fails in hdfs oiv command due to ArrayIndexOutOfBoundsException. Contributed by Yiqun Lin. 
(cherry picked from commit 204a2055b1)
(cherry picked from commit 0cff416c35)
 2016-07-29 15:41:17 +09:00
Xiaoyu Yao ac510e5cd0 HDFS-10676. Add namenode metric to measure time spent in generating EDEKs. Contributed by Hanisha Koneru. 2016-07-28 16:16:44 -07:00
Akira Ajisaka 960caf624d HDFS-10696. TestHDFSCLI fails. Contributed by Kai Sasaki. 
(cherry picked from commit 414fbfab41)
(cherry picked from commit ab49c2b399)
 2016-07-28 16:59:49 +09:00
Yongjun Zhang 71c49b1786 HDFS-10667. Report more accurate info about data corruption location. (Yuanbo Liu via Yongjun Zhang) 
(cherry picked from commit eb7ff0c992)
 2016-07-27 15:49:56 -07:00
Brahma Reddy Battula 5dc29ab0f2 HDFS-10668. Fix intermittently failing UT TestDataNodeMXBean#testDataNodeMXBeanBlockCount. Contributed by Mingliang Liu. 
(cherry picked from commit 096d3fa25c)
 2016-07-27 10:31:29 +05:30
Arpit Agarwal 45a6ee97b2 HDFS-10642. TestLazyPersistReplicaRecovery#testDnRestartWithSavedReplicas fails intermittently. (Contributed by Mingliang Liu) 2016-07-26 12:45:06 -07:00
Wei-Chiu Chuang 53bf632702 HDFS-9937. Update dfsadmin command line help and HdfsQuotaAdminGuide. Contributed by Kai Sasaki. 
(cherry picked from commit 25f0a9b3ab)
 2016-07-26 08:31:00 -07:00
Steve Loughran 7921ad949f Revert "HDFS-10668. Fix intermittently failing UT TestDataNodeMXBean#testDataNodeMXBeanBlockCount. Contributed by Mingliang Liu." 
This reverts commit 32468b1ba5.
 2016-07-26 14:31:02 +01:00
Masatake Iwasaki 6b5d546014 HDFS-10671. Fix typo in HdfsRollingUpgrade.md. Contributed by Yiqun Lin. 
(cherry picked from commit 59466b8c18)
 2016-07-26 19:52:23 +09:00
Brahma Reddy Battula 32468b1ba5 HDFS-10668. Fix intermittently failing UT TestDataNodeMXBean#testDataNodeMXBeanBlockCount. Contributed by Mingliang Liu. 
(cherry picked from commit 7cac7655fd)
 2016-07-26 11:00:11 +05:30
Akira Ajisaka e377e94f64 Revert "HDFS-10620. StringBuilder created and appended even if logging is disabled. Contributed by Staffan Friberg." 
This reverts commit 91f47ab5f9.
 2016-07-26 11:06:10 +09:00
Vinitha Reddy Gankidi 9ab153d7ae HDFS-10301. Interleaving processing of storages from repeated block reports causes false zombie storage detection, removes valid blocks. Contributed by Vinitha Gankidi. 2016-07-25 18:59:42 -07:00
Jing Zhao 62800dc8b6 HDFS-10688. BPServiceActor may run into a tight loop for sending block report when hitting IOException. Contributed by Chen Liang. 
(cherry picked from commit 0cde9e12a7)
(cherry picked from commit 30c772fc59)
 2016-07-25 18:44:06 -07:00
Arpit Agarwal 00cbd3136e HDFS-10660. Expose storage policy apis via HDFSAdmin interface. (Contributed by Rakesh R) 2016-07-25 09:51:59 -07:00
Jing Zhao 54be36a6d6 HDFS-9894. Add unsetStoragePolicy API to FileContext/AbstractFileSystem and derivatives. Contributed by Xiaobing Zhou. 
(cherry picked from commit 7149cdb3c2d9dd390cd8668883cbe5db94090e0a)
 2016-07-25 09:51:59 -07:00
Arpit Agarwal bbf711c030 HDFS-9534. Add CLI command to clear storage policy from a path. (Contributed by Xiaobing Zhou) 2016-07-25 09:51:59 -07:00
Xiao Chen 8559442ed1 HDFS-10225. DataNode hot swap drives should disallow storage type changes. Contributed by Lei (Eddy) Xu. 
(cherry picked from commit 132deb4cac)
 2016-07-22 15:04:21 -07:00
Jing Zhao 6a7fe835f1 HDFS-10653. Optimize conversion from path string to components. Contributed by Daryn Sharp. 
(cherry picked from commit bd3dcf46e2)
 2016-07-21 11:29:27 -07:00
Akira Ajisaka 2f14d585c4 HDFS-10287. MiniDFSCluster should implement AutoCloseable. Contributed by Andras Bokor. 
(cherry picked from commit fcde6940e0)
(cherry picked from commit 1222889f1a)
 2016-07-21 11:28:46 -07:00
Akira Ajisaka 91f47ab5f9 HDFS-10620. StringBuilder created and appended even if logging is disabled. Contributed by Staffan Friberg. 
(cherry picked from commit f7dabe3add)
(cherry picked from commit 2685e16f38)
 2016-07-19 15:19:02 -07:00
Akira Ajisaka d8181a8cb8 HDFS-10603. Fix flaky tests in org.apache.hadoop.hdfs.server.namenode.snapshot.TestOpenFilesWithSnapshot. Contributed by Yiqun Lin. 
(cherry picked from commit fc570b55b9)
(cherry picked from commit c92601e981)
 2016-07-19 13:52:17 -07:00
Jitendra Pandey a363277be5 HADOOP-13368. DFSOpsCountStatistics$OpType#fromSymbol and s3a.Statistic#fromSymbol should be O(1) operation. Contributed by Mingliang Liu. 2016-07-15 14:32:20 -07:00
Akira Ajisaka 83126ac90d HDFS-10628. Log HDFS Balancer exit message to its own log. Contributed by Jiayi Zhou. 
(cherry picked from commit c48e9d608c)
(cherry picked from commit 6171498375)
 2016-07-15 11:43:54 -07:00
Akira Ajisaka 6a04d9ab63 HADOOP-13351. TestDFSClientSocketSize buffer size tests are flaky. Contributed by Aaron Fabbri and Mingliang Liu. 
(cherry picked from commit 5537c6b234)
(cherry picked from commit c9f5348bbf)
 2016-07-14 14:42:55 -07:00
Zhe Zhang af94dfaa53 HDFS-10544. Balancer doesn't work with IPFailoverProxyProvider. 
(cherry picked from commit 087290e6b1)
(cherry picked from commit 5b4e916b3c)
 2016-07-12 23:28:16 -07:00
Jitendra Pandey 35108025e7 HDFS-10579. HDFS web interfaces lack configs for X-FRAME-OPTIONS protection. Contributed by Anu Engineer. 
Conflicts:
	hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/NameNodeHttpServer.java
 2016-07-12 14:38:34 -07:00
Yongjun Zhang 96e68e722f HDFS-10512. VolumeScanner may terminate due to NPE in DataNode.reportBadBlocks. Contributed by Wei-Chiu Chuang and Yiqun Lin. 
(cherry picked from commit da6f1b88dd)
 2016-07-08 22:20:27 -07:00
Jitendra Pandey 9c7ce389c7 HADOOP-13283. Support reset operation for new global storage statistics and per FS storage stats. Contributed by Mingliang Liu. 2016-07-07 15:04:47 -07:00
Uma Maheswara Rao G 097022e172 HDFS-10592:Fix intermittent test failure of TestNameNodeResourceChecker#testCheckThatNameNodeResourceMonitorIsRunning. Contributed by Rakesh R 
(cherry picked from commit ab092c56c2)
 2016-07-07 14:02:25 -07:00
Andrew Wang f826d92909 HDFS-10488. Update WebHDFS documentation regarding CREATE and MKDIR default permissions. Contributed by Wellington Chevreuil. 
(cherry picked from commit 647b084738b26dcf0c0c4afa4ec3039e6aca2d61)
 2016-07-07 10:37:16 -07:00
Akira Ajisaka 4fba09d7e8 HDFS-10582. Change deprecated configuration fs.checkpoint.dir to dfs.namenode.checkpoint.dir in HDFS Commands Doc. Contributed by Pan Yuxuan. 
(cherry picked from commit a3f93beaa1)
(cherry picked from commit 8a9440298d)
 2016-07-06 17:54:44 -07:00
Akira Ajisaka 4b263b30c2 HDFS-10589. Javadoc for HAState#HAState and HAState#setStateInternal contains non-existent params. Contributed by shenyinjie. 
(cherry picked from commit 8b4b5259d5)
(cherry picked from commit 0401adcaca)
 2016-07-04 07:15:54 -07:00
Jitendra Pandey bc7fd76a1f HADOOP-13305. Define common statistics names across schemes. Contributed by Mingliang Liu. 2016-07-01 15:37:32 -07:00