Apache/hadoop
1
0
Fork 0
Code Issues Pull Requests Packages Projects Releases Wiki Activity
11,434 Commits 326 Branches 371 Tags
Commit Graph

3497 Commits

Author SHA1 Message Date
Yongjun Zhang
0664782025 HDFS-9439. Include status of closeAck into exception message in DataNode#run. (Xiao Chen via Yongjun Zhang) 
(cherry picked from commit 31dc5c650e3fcfd5b581cbc1132b895d3eadaf49)
 2015-11-19 14:22:44 -08:00
Ming Ma
5a3db21563 HDFS-8056. Decommissioned dead nodes should continue to be counted as dead after NN restart. (mingma) 
(cherry picked from commit 1c4951a7a09433fbbcfe26f243d6c2d8043c71bb)
 2015-11-19 10:04:40 -08:00
Akira Ajisaka
c74e42b4a2 HDFS-9431. DistributedFileSystem#concat fails if the target path is relative. Contributed by Kazuho Fujii. 
(cherry picked from commit ac1aa6c8197268ea83f44747199d560be9e299ae)
 2015-11-19 20:55:36 +09:00
Colin Patrick Mccabe
0f9f9ba3cb HDFS-9252. Change TestFileTruncate to use FsDatasetTestUtils to get block file size and genstamp. (Lei (Eddy) Xu via cmccabe) 
(cherry picked from commit dfbde3fc511495ac998f07d674a87355de75fc04)
 2015-11-17 10:54:32 -08:00
Arpit Agarwal
efcaf9c28f HDFS-9397. Fix typo for readChecksum() LOG.warn in BlockSender.java. (Contributed by Enrique Flores) 2015-11-16 21:02:06 -08:00
Walter Su
bff9d08a43 HDFS-9358. TestNodeCount#testNodeCount timed out. Contributed by Masatake Iwasaki. 
(cherry picked from commit 621cbb4f69072bde259f213629f84494416ae12f)
 2015-11-17 10:53:50 +08:00
Xiaoyu Yao
dceed6b0a8 HDFS-9421. NNThroughputBenchmark replication test NPE with -namenode option. Contributed by Mingliang Liu. 
(cherry picked from commit a4f62a2d58d4e00cda3632411c6abda6eaa12a0e)
 2015-11-16 16:42:57 -08:00
Xiaoyu Yao
5ea3f03ecc HDFS-9387. Fix namenodeUri parameter parsing in NNThroughputBenchmark. Contributed by Mingliang Liu. 
(cherry picked from commit 2701f2d2558f3ade879539f3f7bedf749709f2f1)
 2015-11-16 16:31:06 -08:00
Ming Ma
42b55ff23e HDFS-9413. getContentSummary() on standby should throw StandbyException. (Brahma Reddy Battula via mingma) 
(cherry picked from commit 02653add98f34deedc27f4da2254d25e83e55b58)
 2015-11-16 09:34:03 -08:00
Walter Su
0f5aae5fd8 HDFS-9410. Some tests should always reset sysout and syserr. Contributed by Xiao Chen. 
(cherry picked from commit cccf88480b0df71f15fb36e9693d492c9e16c685)
 2015-11-13 14:29:25 +08:00
Kihwal Lee
8726a5e020 HDFS-9396. Total files and directories on jmx and web UI on standby is uninitialized. Contributed by Kihwal Lee. 
(cherry picked from commit a2a104f3d30182953db6690304146f0a582ed241)
 2015-11-11 15:37:11 -06:00
Xiaoyu Yao
48a494cdc3 HDFS-9234. WebHdfs: getContentSummary() should give quota for storage types. Contributed by Surendra Singh Lilhore. 
(cherry picked from commit 41d3f8899d8b96568f56331eaf598bb356ecdae0)
 2015-11-10 16:48:06 -08:00
Zhe Zhang
edd4b2d458 HDFS-9364. Unnecessary DNS resolution attempts when creating NameNodeProxies. Contributed by Xiao Chen. 
Change-Id: I9e42f724f27924cf73891425a832de37ce014a1e
 2015-11-10 09:55:39 -08:00
Walter Su
e5420feb09 HDFS-9401. Fix findbugs warnings in BlockRecoveryWorker. Contributed by Brahma Reddy Battula. 
(cherry picked from commit 2fda45b9dc9c0bf9bb1380134c80836e89d50471)
 2015-11-10 14:54:04 +08:00
Haohui Mai
c91bd3aabb HDFS-2261. AOP unit tests are not getting compiled or run. Contributed by Haohui Mai. 2015-11-09 15:26:01 -08:00
Yongjun Zhang
6bca317157 HDFS-9249. NPE is thrown if an IOException is thrown in NameNode constructor. (Wei-Chiu Chuang via Yongjun Zhang) 
(cherry picked from commit 2741a2109b98d0febb463cb318018ecbd3995102)
 2015-11-09 14:12:20 -08:00
cnauroth
385dcc4064 HDFS-9394. branch-2 hadoop-hdfs-client fails during FileSystem ServiceLoader initialization, because HftpFileSystem is missing. Contributed by Mingliang Liu. 
(cherry picked from commit 1ee2db5c41d2e608eb3102d0e13e132448c5ed8f)
 2015-11-07 07:39:45 -08:00
Arpit Agarwal
1e0746e756 HDFS-9379. Make NNThroughputBenchmark support more than 10 datanodes. (Contributed by Mingliang Liu) 2015-11-06 18:59:03 -08:00
Kihwal Lee
481e7248de HDFS-9318. considerLoad factor can be improved. Contributed by Kuhu Shukla. 
(cherry picked from commit bf6aa30a156b3c5cac5469014a5989e0dfdc7256)
 2015-11-06 14:08:10 -06:00
Yongjun Zhang
41d0d9a32c HDFS-9236. Missing sanity check for block size during block recovery. (Tony Wu via Yongjun Zhang) 
(cherry picked from commit b64242c0d2cabd225a8fb7d25fed449d252e4fa1)
 2015-11-06 11:26:41 -08:00
Arpit Agarwal
33908c6f52 HDFS-6481. DatanodeManager#getDatanodeStorageInfos() should check the length of storageIDs. (Contributed by szetszwo) 2015-11-06 10:19:00 -08:00
Yongjun Zhang
1bb89cbac7 HDFS-9377. Fix findbugs warnings in FSDirSnapshotOp. (Mingliang Liu via Yongjun Zhang) 
(cherry picked from commit 286cc6483fe458894cdd22ff1d8822c304e0d94f)
 2015-11-05 14:54:29 -08:00
Lei Xu
489c920c9a HDFS-9282. Make data directory count and storage raw capacity related tests FsDataset-agnostic. (Tony Wu via lei) 
(cherry picked from commit 5a1a54c9f5d9a8f7585000fe33ef106e62dbb43a)
 2015-11-05 10:46:46 -08:00
Lei Xu
7a2a17cb5f HDFS-9363. Add fetchReplica to FsDatasetTestUtils to return FsDataset-agnostic replica. (Tony Wu via lei) 
(cherry picked from commit 4976a02841592b61f2d2b40d2aadd5146571322c)
 2015-11-05 10:46:46 -08:00
Haohui Mai
aa03030e4a HDFS-9372. Remove dead code in DataStorage.recoverTransitionRead. Contributed by Duo Zhang. 2015-11-05 10:00:18 -08:00
Kihwal Lee
37abd0e33b HDFS-4937. ReplicationMonitor can infinite-loop in BlockPlacementPolicyDefault#chooseRandom(). Contributed by Kihwal Lee. 
(cherry picked from commit ff47f35deed14ba6463cba76f0e6a6c15abb3eca)
 2015-11-05 09:26:53 -06:00
Xiaoyu Yao
891be39993 HDFS-9360. Storage type usage isn't updated properly after file deletion. Contributed by Ming Ma. 
(cherry picked from commit ea5bb483269b51a349c358b71f84904c76693a66)
 2015-11-05 01:40:05 -08:00
Lei Xu
8ed1fd2171 HDFS-9331. Modify TestNameNodeMXBean#testNameNodeMXBeanInfo() to account for filesystem entirely allocated for DFS use. (Tony Wu via lei) 
(cherry picked from commit e2a5441b062fd0758138079d24a2740fc5e5e350)
 2015-11-04 10:28:00 -08:00
Haohui Mai
6d92778191 Revert "HDFS-8855. Webhdfs client leaks active NameNode connections. Contributed by Xiaobing Zhou." 
This reverts commit e7203b69548a7926a345e2b18f0a227f7f8ca52c.
 2015-11-04 10:22:21 -08:00
Lei Xu
3f2be7b18f HDFS-9007. Fix HDFS Balancer to honor upgrade domain policy. (Ming Ma via lei) 2015-11-04 10:21:58 -08:00
Haohui Mai
b61aa716e8 HDFS-9357. NN UI renders icons of decommissioned DN incorrectly. Contributed by Surendra Singh Lilhore. 2015-11-04 09:17:01 -08:00
Yongjun Zhang
0b61ccaeca HDFS-9351. checkNNStartup() need to be called when fsck calls FSNamesystem.getSnapshottableDirs(). (Xiao Chen via Yongjun Zhang) 
(cherry picked from commit 194251c85250fcbe80a6ffee88b2cd4689334be3)
 2015-11-03 17:27:14 -08:00
Zhe Zhang
4f29f5d192 HDFS-9289. Make DataStreamer#block thread safe and verify genStamp in commitBlock. Contributed by Chang Li. 
Change-Id: I0591b1f9c6a0a8b368408b195151de2e0aa440c4
 2015-11-03 14:23:02 -08:00
cnauroth
246ab44b5e HDFS-9362. TestAuditLogger#testAuditLoggerWithCallContext assumes Unix line endings, fails on Windows. Contributed by Chris Nauroth. 
(cherry picked from commit 7e2829662b4c4bf33ebaf2fa09312d0bed3d6f92)
 2015-11-03 11:55:07 -08:00
cnauroth
9d11d2a8f2 HDFS-9354. Fix TestBalancer#testBalancerWithZeroThreadsForMove on Windows. Contributed by Xiaoyu Yao. 
(cherry picked from commit 095ac834022df6136b42961c507ec745c6cf8f97)

Conflicts:
	hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/balancer/TestBalancer.java
 2015-11-03 11:41:50 -08:00
Ming Ma
fd9a7f84bc HDFS-9313. Possible NullPointerException in BlockManager if no excess replica can be chosen. (mingma) 
(cherry picked from commit d565480da2f646b40c3180e1ccb2935c9863dfef)
 2015-11-02 19:37:38 -08:00
Lei Xu
c6b9c8d698 HDFS-9308. Add truncateMeta() and deleteMeta() to MiniDFSCluster. (Tony Wu via lei) 
(cherry picked from commit 8e05dbf2bddce95d5f5a5bae5df61acabf0ba7c5)
 2015-11-02 18:06:34 -08:00
Lei Xu
41faf438e1 HDFS-9312. Fix TestReplication to be FsDataset-agnostic. (lei) 
(cherry picked from commit 7632409482aaf06ecc6fe370a9f519afb969ad30)
 2015-11-02 17:10:16 -08:00
Zhe Zhang
bd2867cd39 HDFS-9339. Extend full test of KMS ACLs. Contributed by Daniel Templeton. 
Change-Id: I61eeee8fa5e85250eabc1eef3d8c11f32700d6fb
 2015-11-02 13:52:21 -08:00
Zhe Zhang
a47c4e7819 HDFS-9329. TestBootstrapStandby#testRateThrottling is flaky because fsimage size is smaller than IO buffer size. Contributed by Zhe Zhang. 
Conflicts:
	hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/namenode/ha/TestBootstrapStandby.java

Change-Id: Iffa62547483c1d9dc82a196d5e8d0856b397217d
 2015-11-02 10:32:28 -08:00
Arpit Agarwal
bb2ee9f8eb HDFS-9343. Empty caller context considered invalid. (Contributed by Mingliang Liu) 2015-11-01 15:35:11 -08:00
yliu
caa5da06f2 Revert "HDFS-4937. ReplicationMonitor can infinite-loop in BlockPlacementPolicyDefault#chooseRandom(). Contributed by Kihwal Lee." 
This reverts commit 0c7fd26302109273a07b6087a9b92c9dff080816.
 2015-10-31 16:23:26 +08:00
Kihwal Lee
0c7fd26302 HDFS-4937. ReplicationMonitor can infinite-loop in BlockPlacementPolicyDefault#chooseRandom(). Contributed by Kihwal Lee. 
(cherry picked from commit 43539b5ff4ac0874a8a454dc93a2a782b0e0ea8f)
 2015-10-30 09:30:11 -05:00
Andrew Wang
5e5e693a0a HDFS-9332. Fix Precondition failures from NameNodeEditLogRoller while saving namespace. 
(cherry picked from commit 888c6245e20ba6bdaa57d16b5c62b4a9eda2cdaf)
 2015-10-29 11:30:52 -07:00
Zhe Zhang
30791cb0d0 HDFS-9229. Expose size of NameNode directory as a metric. Contributed by Surendra Singh Lilhore. 
Conflicts:
	hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/ha/EditLogTailer.java

Change-Id: I5f5895bac4f7d7e66e95788765a514b80b9b6766
 2015-10-29 11:21:57 -07:00
Vinayakumar B
e3f1a02f50 HDFS-9044. Give Priority to FavouredNodes , before selecting nodes from FavouredNode's Node Group (Contributed by J.Andreina) 
(cherry picked from commit 588baab160e7c328dca1c45cf3541e79218406e8)
 2015-10-29 11:44:52 +05:30
Vinayakumar B
dea66a8392 HDFS-8545. Refactor FS#getUsed() to use ContentSummary and add an API to fetch the total file length from a specific path (Contributed by J.Andreina) 
(cherry picked from commit 7d2d16f4ee87ae56dc20016a91c109dd5130f7d4)
 2015-10-29 11:30:59 +05:30
Haohui Mai
9e911a2439 HDFS-9168. Move client side unit test to hadoop-hdfs-client. Contributed by Haohui Mai. 2015-10-28 15:54:53 -07:00
Zhe Zhang
986dc718d3 HDFS-9295. Add a thorough test of the full KMS code path. Contributed by Daniel Templeton. 
Change-Id: Ife1abb2f8e9a30102c4f705c47646d9ac8328786
 2015-10-28 14:36:56 -07:00
Kihwal Lee
75bc53a86a HDFS-9279. Decomissioned capacity should not be considered for configured/used capacity. Contributed by Kihu Shukla . 
(cherry picked from commit 19a77f546657b086af8f41fa631099bdde7e010c)
 2015-10-28 11:59:36 -05:00