Apache
/
hadoop
1
0
Fork 0
Code Issues Pull Requests Packages Projects Releases Wiki Activity
11,951 Commits 326 Branches 371 Tags 1.5 GiB
Commit Graph

4088 Commits

Author SHA1 Message Date
Kihwal Lee 2a6497cc7c HDFS-8915. TestFSNamesystem.testFSLockGetWaiterCount fails intermittently. Contributed by Masatake Iwasaki. 
(cherry picked from commit 13fb1b50e6)
 2016-08-26 10:00:36 -05:00
Xiaoyu Yao fc67501393 HDFS-10748. TestFileTruncate#testTruncateWithDataNodesRestart runs sometimes timeout. Contributed by Yiqun Lin. 
(cherry picked from commit 4da5000dd3)
 2016-08-25 09:22:02 -07:00
Kihwal Lee ba3257baf5 HDFS-10772. Reduce byte/string conversions for get listing. Contributed by Daryn Sharp. 
(cherry picked from commit a1f3293762)
 2016-08-24 15:25:33 -05:00
Wei-Chiu Chuang d19e29bd68 HDFS-8986. Add option to -du to calculate directory space usage excluding snapshots. Contributed by Xiao Chen. 2016-08-24 08:49:54 -07:00
Daryn Sharp a30f6a68fa HDFS-10762. Pass IIP for file status related methods 
(cherry picked from commit ec252ce0fc)
 2016-08-24 08:51:05 -05:00
Jason Lowe 8cd3996246 HDFS-9745. TestSecureNNWithQJM#testSecureMode sometimes fails with timeouts. Contributed by Xiao Chen 
(cherry picked from commit 126d165efd)
 2016-08-23 14:48:35 +00:00
Akira Ajisaka 94a844b4f6 HDFS-10783. The option '-maxSize' and '-step' fail in OfflineImageViewer. Contributed by Yiqun Lin. 
(cherry picked from commit e90f3359de)
 2016-08-23 19:58:13 +09:00
Eric Yang 9faacbec2c HDFS-8312. Added permission check for moving file to Trash. (Weiwei Yang via Eric Yang) 2016-08-22 18:32:06 -07:00
Vinod Kumar Vavilapalli (I am also known as @tshooter.) 8a6a7c82f2 HDFS-10692. Update JDiff report's base version for HDFS from 2.6.0 to 2.7.2. Contributed by Wangda Tan. 
(cherry picked from commit dc7a1c54f4)
 2016-08-22 11:36:38 -07:00
Akira Ajisaka 62c7d09efa HDFS-10645. Make block report size as a metric and add this metric to datanode web ui. Contributed by Yuanbo Liu. 2016-08-22 14:22:37 +09:00
Jing Zhao fff0418458 HDFS-10764. Fix INodeFile#getBlocks to not return null. Contributed by Arpit Agarwal. 
(cherry picked from commit 0faee62a0c)
 2016-08-19 22:23:55 -07:00
Kihwal Lee 43d1279df0 HDFS-10711. Optimize FSPermissionChecker group membership check. Contributed by Daryn Sharp. 
(cherry picked from commit 2550371f66)
 2016-08-19 09:46:30 -05:00
Kihwal Lee 922d045e11 HDFS-10745. Directly resolve paths into INodesInPath. Contributed by Daryn Sharp. 2016-08-18 12:37:57 -05:00
Xiao Chen a20b943cf9 HDFS-10549. Correctly revoke file leases when closing files. Contributed by Yiqun Lin. 
(cherry picked from commit 85aacaadb5a3f8c78b191867c0bde09b3c4b3c3c)

Conflicts:
	hadoop-hdfs-project/hadoop-hdfs-client/src/main/java/org/apache/hadoop/hdfs/DFSStripedOutputStream.java
 2016-08-17 15:50:13 -07:00
Vinod Kumar Vavilapalli (I am also known as @tshooter.) 2486c4c63a Revert "HDFS-9395. Make HDFS audit logging consistant. Contributed by Kuhu Shukla." 
Reverting this on branch-2.* as it's an incompatible change.

This reverts commit 83f7f62be3.
 2016-08-17 13:33:09 -07:00
Kihwal Lee 65a41696b2 HDFS-10342. BlockManager#createLocatedBlocks should not check corrupt replicas if none are corrupt. Contributed by Kuhu Shukla. 
(cherry picked from commit b10c936020)

Conflicts:
	hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/blockmanagement/BlockManager.java
 2016-08-16 09:29:42 -05:00
Mingliang Liu 2f4660ce98 HDFS-10641. TestBlockManager#testBlockReportQueueing fails intermittently. (Contributed by Daryn Sharp) 2016-08-15 20:39:12 -07:00
Mingliang Liu 82623ea9c9 HDFS-10747. o.a.h.hdfs.tools.DebugAdmin usage message is misleading. (Contributed by Mingliang Liu) 2016-08-15 20:38:17 -07:00
Mingliang Liu f9a7e59066 HDFS-10678. Documenting NNThroughputBenchmark tool. (Contributed by Mingliang Liu) 2016-08-15 20:37:55 -07:00
Kihwal Lee e78db7d2a4 HDFS-10763. Open files can leak permanently due to inconsistent lease update. Contributed by Kihwal Lee. 
(cherry picked from commit 864f878d59)
 2016-08-15 17:33:16 -05:00
Kihwal Lee 1ef8d7a638 HDFS-10744. Internally optimize path component resolution. Contributed by Daryn Sharp. 
(cherry picked from commit 03dea65e0b)

Conflicts:
	hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/FSDirWriteFileOp.java
 2016-08-15 17:01:40 -05:00
Kihwal Lee 4766a3153d HDFS-9696. Garbage snapshot records linger forever. Contributed by Kihwal Lee 
(cherry picked from commit 83e57e083f)

Conflicts:
	hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/namenode/TestSaveNamespace.java
 2016-08-15 13:09:50 -05:00
Wei-Chiu Chuang 20a8ff0a8d HDFS-10731. FSDirectory#verifyMaxDirItems does not log path name. Contributed by Wei-Chiu Chuang. 
(cherry picked from commit 9019606b69)
 2016-08-12 06:02:47 -07:00
Zhe Zhang 00ff3d737c HDFS-9888. Allow reseting KerberosName in unit tests. Contributed by Xiao Chen. 
(cherry picked from commit 3e8099a45a)
 2016-08-11 11:32:54 -07:00
Akira Ajisaka 0529da7239 HDFS-10336. TestBalancer failing intermittently because of not reseting UserGroupInformation completely. Contributed by Yiqun Lin. 
(cherry picked from commit 932aed64d7)
 2016-08-11 10:55:53 -07:00
Zhe Zhang 7d402692b4 HDFS-9804. Allow long-running Balancer to login with keytab. Contributed by Xiao Chen. 
(cherry picked from commit ccff6035f5)
 2016-08-11 10:53:16 -07:00
Wei-Chiu Chuang f1d5a95cf3 HDFS-8897. Balancer should handle fs.defaultFS trailing slash in HA. Contributed by John Zhuge. 2016-08-10 22:24:21 -07:00
Xiaoyu Yao de6eafc696 HDFS-10643. Namenode should use loginUser(hdfs) to generateEncryptedKey. Contributed by Xiaoyu Yao. 
(cherry picked from commit ec289bbece)
 2016-08-10 15:55:12 -07:00
Xiaoyu Yao c341280129 Revert "Namenode should use loginUser(hdfs) to generateEncryptedKey. Contributed by Xiaoyu Yao." 
This reverts commit 9f816d7ca7.

(cherry picked from commit fdafd0cb4f)
 2016-08-10 15:54:59 -07:00
Xiaoyu Yao d583a357f9 Namenode should use loginUser(hdfs) to generateEncryptedKey. Contributed by Xiaoyu Yao. 
(cherry picked from commit 9f816d7ca7)
 2016-08-10 15:15:51 -07:00
Wei-Chiu Chuang 8efd4959f3 HDFS-8224. Schedule a block for scanning if its metadata file is corrupt. Contributed by Rushabh S Shah. 2016-08-10 11:34:59 -07:00
Yuanbo Liu 44162e555a HDFS-10694. processReport() should print blockReportId in each log message. Contributed by Yuanbo Liu. 
(cherry picked from commit 10e84c6a6e)
 2016-08-10 11:10:00 -07:00
Kihwal Lee 7f1879abe6 HDFS-10738. Fix TestRefreshUserMappings.testRefreshSuperUserGroupsConfiguration test failure. Contributed by Rakesh R. 
(cherry picked from commit 0f701f433d)
 2016-08-09 14:55:46 -05:00
Lei Xu e7c701586d HDFS-4176. EditLogTailer should call rollEdits with a timeout. (lei) 2016-08-08 16:32:01 -07:00
Lei Xu 9cb6d291ea HDFS-10457. DataNode should not auto-format block pool directory if VERSION is missing. (Wei-Chiu Chuang via lei) 
(cherry picked from commit bb3bcb9397593fc8a2fa63a48eba126609f72c42)
 2016-08-08 16:19:58 -07:00
Konstantin V Shvachko ab130f97ca HDFS-10693. metaSave should print blocks, not LightWeightHashSet. Contributed by Yuanbo Liu. 
(cherry picked from commit 4d3af47f27)
 2016-08-07 14:35:35 -07:00
Akira Ajisaka 6863866127 HDFS-10715. NPE when applying AvailableSpaceBlockPlacementPolicy. Contributed by Guangbin Zhu. 
(cherry picked from commit ef432579a7)
 2016-08-05 17:32:23 +09:00
Kihwal Lee 09b06a6e9e HDFS-10673. Optimize FSPermissionChecker's internal path usage. Contributed by Daryn Sharp. 
(cherry picked from commit 438a9f047e)
 2016-08-04 16:17:41 -05:00
Kihwal Lee 690ec789f0 HDFS-10343. BlockManager#createLocatedBlocks may return blocks on failed storages. Contributed by Kuhu Shukla. 
(cherry picked from commit 57369a678c)

Conflicts:
	hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/blockmanagement/BlockManager.java
 2016-08-04 15:12:21 -05:00
Kihwal Lee 3b2554f3b0 HDFS-10722. Fix race condition in TestEditLog#testBatchedSyncWithClosedLogs. Contributed by Daryn Sharp. 
(cherry picked from commit 4a8e6dc02f)
 2016-08-04 11:55:52 -05:00
Tsz-Wo Nicholas Sze 5004b0571e In Balancer, the target task should be removed when its size < 0. Contributed by Yiqun Lin 2016-08-04 09:53:15 -07:00
Kihwal Lee 77b61d1f4e HDFS-10662. Optimize UTF8 string/byte conversions. Contributed by Daryn Sharp. 
(cherry picked from commit 6ae39199da)

Conflicts:
	hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/FSDirWriteFileOp.java
 2016-08-04 09:43:56 -05:00
Kihwal Lee c0166b0743 HDFS-10707. Replace org.apache.commons.io.Charsets with java.nio.charset.StandardCharsets. Contributed by Vincent Poon. 2016-08-04 09:28:30 -05:00
Kihwal Lee 1ca2ab4013 HDFS-10569. A bug causes OutOfIndex error in BlockListAsLongs. Contributed by Weiwei Yang. 
(cherry picked from commit 6f63566694)

Conflicts:
	hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/protocol/TestBlockListAsLongs.java
 2016-08-03 15:27:03 -05:00
Jing Zhao 180a26d9fb HDFS-10710. In BlockManager#rescanPostponedMisreplicatedBlocks(), postponed misreplicated block counts should be retrieved within the NN lock protection.Contributed by GAO Rui. 
(cherry picked from commit f4ba5ff1d7)
 2016-08-03 12:42:44 -07:00
Kihwal Lee a5d12d9c1f HDFS-10674. Optimize creating a full path from an inode. Contributed by Daryn Sharp. 
(cherry picked from commit 22ef5286bc)
 2016-08-03 13:15:28 -05:00
Kihwal Lee c15959fa60 HDFS-742. A down DataNode makes Balancer to hang on repeatingly asking NameNode its partial block list. Contributed by Mit Desai. 
(cherry picked from commit 58db263e93)
 2016-08-03 12:18:15 -05:00
Kihwal Lee ab7072838f HDFS-10656. Optimize conversion of byte arrays back to path string. Contributed by Daryn Sharp. 
(cherry picked from commit bebf10d245)
 2016-08-03 11:54:51 -05:00
Kihwal Lee dc0a17e828 HDFS-5805. TestCheckpoint.testCheckpoint fails intermittently on branch-2. Contributed by Eric Badger. 
(cherry picked from commit 5e5b8793fb)
 2016-08-02 15:39:03 -05:00
Konstantin V Shvachko 24574f0e16 Revert "HDFS-10301. Interleaving processing of storages from repeated block reports causes false zombie storage detection, removes valid blocks. Contributed by Vinitha Gankidi." 
This reverts commit 0dde4035c9.
 2016-08-01 22:44:32 -07:00