Apache
/
hadoop
1
0
Fork 0
Code Issues Pull Requests Packages Projects Releases Wiki Activity
11,465 Commits 326 Branches 371 Tags 1.5 GiB
Commit Graph

4030 Commits

Author SHA1 Message Date
Wei-Chiu Chuang 53f097a474 HDFS-10731. FSDirectory#verifyMaxDirItems does not log path name. Contributed by Wei-Chiu Chuang. 
(cherry picked from commit 9019606b69)
(cherry picked from commit 20a8ff0a8d)
 2016-08-12 06:11:22 -07:00
Zhe Zhang 9c1c48c0d4 HDFS-9888. Allow reseting KerberosName in unit tests. Contributed by Xiao Chen. 
(cherry picked from commit 3e8099a45a)
(cherry picked from commit 00ff3d737c)
 2016-08-11 12:42:17 -07:00
Akira Ajisaka f29d0739d7 HDFS-10336. TestBalancer failing intermittently because of not reseting UserGroupInformation completely. Contributed by Yiqun Lin. 
(cherry picked from commit 932aed64d7)
(cherry picked from commit 0529da7239)
 2016-08-11 12:42:17 -07:00
Zhe Zhang b8b4ea67d8 HDFS-9804. Allow long-running Balancer to login with keytab. Contributed by Xiao Chen. 
(cherry picked from commit ccff6035f5)
(cherry picked from commit 7d402692b4)
 2016-08-11 12:42:17 -07:00
Wei-Chiu Chuang 65e3657149 HDFS-8897. Balancer should handle fs.defaultFS trailing slash in HA. Contributed by John Zhuge. 
(cherry picked from commit f1d5a95cf3)
 2016-08-10 22:30:06 -07:00
Xiaoyu Yao 7a1a3f7754 HDFS-10643. Namenode should use loginUser(hdfs) to generateEncryptedKey. Contributed by Xiaoyu Yao. 
(cherry picked from commit ec289bbece)
 2016-08-10 15:56:56 -07:00
Xiaoyu Yao 8808df09c6 Revert "Namenode should use loginUser(hdfs) to generateEncryptedKey. Contributed by Xiaoyu Yao." 
This reverts commit 9f816d7ca7.

(cherry picked from commit fdafd0cb4f)
 2016-08-10 15:56:43 -07:00
Xiaoyu Yao c0e331935d Namenode should use loginUser(hdfs) to generateEncryptedKey. Contributed by Xiaoyu Yao. 
(cherry picked from commit 9f816d7ca7)
 2016-08-10 15:19:10 -07:00
Wei-Chiu Chuang 753edc4931 HDFS-8224. Schedule a block for scanning if its metadata file is corrupt. Contributed by Rushabh S Shah. 
(cherry picked from commit 8efd4959f3)
 2016-08-10 11:39:12 -07:00
Yuanbo Liu d46f5ca7e2 HDFS-10694. processReport() should print blockReportId in each log message. Contributed by Yuanbo Liu. 
(cherry picked from commit 10e84c6a6e)
 2016-08-10 11:12:21 -07:00
Konstantin V Shvachko 6b3114a2a5 HDFS-10693. metaSave should print blocks, not LightWeightHashSet. Contributed by Yuanbo Liu. 
(cherry picked from commit 4d3af47f27)
 2016-08-07 14:36:24 -07:00
Kihwal Lee 5b3deac2e3 HDFS-9958. BlockManager#createLocatedBlocks can throw NPE for corruptBlocks on failed storages. Contributed by Kuhu Shukla. 
(cherry picked from commit f715f14185)
 2016-08-05 11:43:43 -05:00
Akira Ajisaka 5758026833 HDFS-10715. NPE when applying AvailableSpaceBlockPlacementPolicy. Contributed by Guangbin Zhu. 
(cherry picked from commit ef432579a7)
(cherry picked from commit 6863866127)
 2016-08-05 17:33:07 +09:00
Kihwal Lee e8cd1439d3 HDFS-10343. BlockManager#createLocatedBlocks may return blocks on failed storages. Contributed by Kuhu Shukla. 
(cherry picked from commit 57369a678c)

Conflicts:
	hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/blockmanagement/BlockManager.java

(cherry picked from commit 690ec789f0)
 2016-08-04 15:27:51 -05:00
Kihwal Lee eb9af6ca34 HDFS-10717. Fix findbugs warnings of hadoop-hdfs-client in branch-2. Contributed by Akira Ajisaka. 
(cherry picked from commit 6ead6f3bb8)
 2016-08-04 12:36:51 -05:00
Arpit Agarwal f3424d662a HDFS-10722. Fix build break. 2016-08-04 10:28:43 -07:00
Kihwal Lee 3a85aea42d HDFS-10722. Fix race condition in TestEditLog#testBatchedSyncWithClosedLogs. Contributed by Daryn Sharp. 
(cherry picked from commit 4a8e6dc02f)
 2016-08-04 11:56:26 -05:00
Tsz-Wo Nicholas Sze cefa21e98a In Balancer, the target task should be removed when its size < 0. Contributed by Yiqun Lin 2016-08-04 09:54:15 -07:00
Kihwal Lee 58df27b873 HDFS-10662. Optimize UTF8 string/byte conversions. Contributed by Daryn Sharp. 
(cherry picked from commit 6ae39199da)

Conflicts:
	hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/FSDirWriteFileOp.java

(cherry picked from commit 77b61d1f4e)
 2016-08-04 09:45:03 -05:00
Kihwal Lee 804db5b277 HDFS-10707. Replace org.apache.commons.io.Charsets with java.nio.charset.StandardCharsets. Contributed by Vincent Poon. 
(cherry picked from commit c0166b0743)

Conflicts:
	hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/datanode/web/webhdfs/WebHdfsHandler.java
 2016-08-04 09:35:16 -05:00
Kihwal Lee ceac628378 HDFS-10569. A bug causes OutOfIndex error in BlockListAsLongs. Contributed by Weiwei Yang. 
(cherry picked from commit 6f63566694)

Conflicts:
	hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/protocol/TestBlockListAsLongs.java

(cherry picked from commit 1ca2ab4013)
 2016-08-03 15:28:13 -05:00
Jing Zhao 5bdee64bde HDFS-10710. In BlockManager#rescanPostponedMisreplicatedBlocks(), postponed misreplicated block counts should be retrieved within the NN lock protection.Contributed by GAO Rui. 
(cherry picked from commit f4ba5ff1d7)
(cherry picked from commit 180a26d9fb)
 2016-08-03 12:44:49 -07:00
Kihwal Lee 33a87ffe10 HDFS-742. A down DataNode makes Balancer to hang on repeatingly asking NameNode its partial block list. Contributed by Mit Desai. 
(cherry picked from commit 58db263e93)
 2016-08-03 12:19:05 -05:00
Kihwal Lee 288f9ccde2 HDFS-5805. TestCheckpoint.testCheckpoint fails intermittently on branch-2. Contributed by Eric Badger. 
(cherry picked from commit 5e5b8793fb)
 2016-08-02 15:43:11 -05:00
Konstantin V Shvachko 9b3b7703b1 Revert "HDFS-10301. Interleaving processing of storages from repeated block reports causes false zombie storage detection, removes valid blocks. Contributed by Vinitha Gankidi." 
This reverts commit 9ab153d7ae.
 2016-08-01 22:48:09 -07:00
Daryn Sharp ecb143ed21 HDFS-10655. Fix path related byte array conversion bugs. (daryn) 
(cherry picked from commit 9f473cf903)
 2016-08-01 10:18:45 -05:00
Yongjun Zhang cc066de22c HDFS-10703. HA NameNode Web UI should show last checkpoint time. (John Zhuge via Yongjun Zhang) 
(cherry picked from commit 8d32bd884d)
 2016-07-29 23:47:46 -07:00
Akira Ajisaka e467f838bb HDFS-10691. FileDistribution fails in hdfs oiv command due to ArrayIndexOutOfBoundsException. Contributed by Yiqun Lin. 
(cherry picked from commit 204a2055b1)
(cherry picked from commit 0cff416c35)
 2016-07-29 15:41:17 +09:00
Xiaoyu Yao ac510e5cd0 HDFS-10676. Add namenode metric to measure time spent in generating EDEKs. Contributed by Hanisha Koneru. 2016-07-28 16:16:44 -07:00
Akira Ajisaka 960caf624d HDFS-10696. TestHDFSCLI fails. Contributed by Kai Sasaki. 
(cherry picked from commit 414fbfab41)
(cherry picked from commit ab49c2b399)
 2016-07-28 16:59:49 +09:00
Yongjun Zhang 71c49b1786 HDFS-10667. Report more accurate info about data corruption location. (Yuanbo Liu via Yongjun Zhang) 
(cherry picked from commit eb7ff0c992)
 2016-07-27 15:49:56 -07:00
Brahma Reddy Battula 5dc29ab0f2 HDFS-10668. Fix intermittently failing UT TestDataNodeMXBean#testDataNodeMXBeanBlockCount. Contributed by Mingliang Liu. 
(cherry picked from commit 096d3fa25c)
 2016-07-27 10:31:29 +05:30
Arpit Agarwal 45a6ee97b2 HDFS-10642. TestLazyPersistReplicaRecovery#testDnRestartWithSavedReplicas fails intermittently. (Contributed by Mingliang Liu) 2016-07-26 12:45:06 -07:00
Wei-Chiu Chuang 53bf632702 HDFS-9937. Update dfsadmin command line help and HdfsQuotaAdminGuide. Contributed by Kai Sasaki. 
(cherry picked from commit 25f0a9b3ab)
 2016-07-26 08:31:00 -07:00
Steve Loughran 7921ad949f Revert "HDFS-10668. Fix intermittently failing UT TestDataNodeMXBean#testDataNodeMXBeanBlockCount. Contributed by Mingliang Liu." 
This reverts commit 32468b1ba5.
 2016-07-26 14:31:02 +01:00
Masatake Iwasaki 6b5d546014 HDFS-10671. Fix typo in HdfsRollingUpgrade.md. Contributed by Yiqun Lin. 
(cherry picked from commit 59466b8c18)
 2016-07-26 19:52:23 +09:00
Brahma Reddy Battula 32468b1ba5 HDFS-10668. Fix intermittently failing UT TestDataNodeMXBean#testDataNodeMXBeanBlockCount. Contributed by Mingliang Liu. 
(cherry picked from commit 7cac7655fd)
 2016-07-26 11:00:11 +05:30
Akira Ajisaka e377e94f64 Revert "HDFS-10620. StringBuilder created and appended even if logging is disabled. Contributed by Staffan Friberg." 
This reverts commit 91f47ab5f9.
 2016-07-26 11:06:10 +09:00
Vinitha Reddy Gankidi 9ab153d7ae HDFS-10301. Interleaving processing of storages from repeated block reports causes false zombie storage detection, removes valid blocks. Contributed by Vinitha Gankidi. 2016-07-25 18:59:42 -07:00
Jing Zhao 62800dc8b6 HDFS-10688. BPServiceActor may run into a tight loop for sending block report when hitting IOException. Contributed by Chen Liang. 
(cherry picked from commit 0cde9e12a7)
(cherry picked from commit 30c772fc59)
 2016-07-25 18:44:06 -07:00
Arpit Agarwal 00cbd3136e HDFS-10660. Expose storage policy apis via HDFSAdmin interface. (Contributed by Rakesh R) 2016-07-25 09:51:59 -07:00
Jing Zhao 54be36a6d6 HDFS-9894. Add unsetStoragePolicy API to FileContext/AbstractFileSystem and derivatives. Contributed by Xiaobing Zhou. 
(cherry picked from commit 7149cdb3c2d9dd390cd8668883cbe5db94090e0a)
 2016-07-25 09:51:59 -07:00
Arpit Agarwal bbf711c030 HDFS-9534. Add CLI command to clear storage policy from a path. (Contributed by Xiaobing Zhou) 2016-07-25 09:51:59 -07:00
Xiao Chen 8559442ed1 HDFS-10225. DataNode hot swap drives should disallow storage type changes. Contributed by Lei (Eddy) Xu. 
(cherry picked from commit 132deb4cac)
 2016-07-22 15:04:21 -07:00
Jing Zhao 6a7fe835f1 HDFS-10653. Optimize conversion from path string to components. Contributed by Daryn Sharp. 
(cherry picked from commit bd3dcf46e2)
 2016-07-21 11:29:27 -07:00
Akira Ajisaka 2f14d585c4 HDFS-10287. MiniDFSCluster should implement AutoCloseable. Contributed by Andras Bokor. 
(cherry picked from commit fcde6940e0)
(cherry picked from commit 1222889f1a)
 2016-07-21 11:28:46 -07:00
Akira Ajisaka 91f47ab5f9 HDFS-10620. StringBuilder created and appended even if logging is disabled. Contributed by Staffan Friberg. 
(cherry picked from commit f7dabe3add)
(cherry picked from commit 2685e16f38)
 2016-07-19 15:19:02 -07:00
Akira Ajisaka d8181a8cb8 HDFS-10603. Fix flaky tests in org.apache.hadoop.hdfs.server.namenode.snapshot.TestOpenFilesWithSnapshot. Contributed by Yiqun Lin. 
(cherry picked from commit fc570b55b9)
(cherry picked from commit c92601e981)
 2016-07-19 13:52:17 -07:00
Jitendra Pandey a363277be5 HADOOP-13368. DFSOpsCountStatistics$OpType#fromSymbol and s3a.Statistic#fromSymbol should be O(1) operation. Contributed by Mingliang Liu. 2016-07-15 14:32:20 -07:00
Akira Ajisaka 83126ac90d HDFS-10628. Log HDFS Balancer exit message to its own log. Contributed by Jiayi Zhou. 
(cherry picked from commit c48e9d608c)
(cherry picked from commit 6171498375)
 2016-07-15 11:43:54 -07:00