Apache
/
hadoop
1
0
Fork 0
Code Issues Pull Requests Packages Projects Releases Wiki Activity
10,771 Commits 326 Branches 371 Tags 1.5 GiB
Commit Graph

3756 Commits

Author SHA1 Message Date
cnauroth 9b039a5004 HDFS-9458. TestBackupNode always binds to port 50070, which can cause bind failures. Contributed by Xiao Chen. 
(cherry picked from commit 5c0ff69618)
 2015-12-22 10:29:40 -08:00
Colin Patrick Mccabe a56c5ef636 HDFS-9589. Block files which have been hardlinked should be duplicated before the DataNode appends to the them (cmccabe) 
(cherry picked from commit bb540ba85a)
 2015-12-22 09:37:30 -08:00
Akira Ajisaka 12caea3274 Move HDFS-8914 from 2.8.0 to 2.7.3 in CHANGES.txt. 
(cherry picked from commit 2cb5afffc4)
 2015-12-21 21:53:44 +09:00
Akira Ajisaka 7526a2a005 HDFS-9505. HDFS Architecture documentation needs to be refreshed. Contributed by Masatake Iwasaki. 
(cherry picked from commit fa544020f6)
 2015-12-21 21:23:28 +09:00
Uma Mahesh fc9e130825 YARN-4480. Clean up some inappropriate imports. (Kai Zheng via umamahesh) 
(cherry picked from commit 0f82b5d878)
 2015-12-19 23:17:33 -08:00
cnauroth dd91cf9fa5 HADOOP-12570. HDFS Secure Mode Documentation updates. Contributed by Arpit Agarwal. 
(cherry picked from commit 932956458d)
 2015-12-18 16:25:06 -08:00
Zhe Zhang 223b47b398 HDFS-9347. Invariant assumption in TestQuorumJournalManager.shutdown() is wrong. Contributed by Wei-Chiu Chuang. 
Change-Id: Idde8522e11c77f82b376e0c90a9bf33d163d5cb9
 2015-12-18 16:05:21 -08:00
Colin Patrick Mccabe 31d2fb91fc HDFS-9350. Avoid creating temprorary strings in Block.toString() and getBlockName() (Staffan Friberg via cmccabe) 
(cherry picked from commit e63388fdf2)
 2015-12-18 10:05:12 -08:00
Andrew Wang 71fb205735 HDFS-8477. describe dfs.ha.zkfc.port in hdfs-default.xml. Contributed by Kanaka Kumar Avvaru. 
(cherry picked from commit 058cfddebb)
 2015-12-18 09:14:23 -08:00
Vinayakumar B c887bcd1f0 HDFS-9393. After choosing favored nodes, choosing nodes for remaining replicas should go through BlockPlacementPolicy (Contributed by J.Andreina) 
(cherry picked from commit bfadf11b36)
 2015-12-18 11:39:46 +05:30
cnauroth 7e5306e795 HDFS-9571. Fix ASF Licence warnings in Jenkins reports. Contributed by Brahma Reddy Battula. 
(cherry picked from commit 6da9deec20)
 2015-12-17 16:26:08 -08:00
Kihwal Lee 3c0ae5914c HDFS-9533. seen_txid in the shared edits directory is modified during bootstrapping. Contributed by Kihwal Lee. 
(cherry picked from commit 5cb1e0118b)

Conflicts:
	hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/ha/BootstrapStandby.java
	hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/namenode/ha/TestBootstrapStandby.java
 2015-12-17 17:23:06 -06:00
cnauroth 294d0f6f6a HDFS-9572. Prevent DataNode log spam if a client connects on the data transfer port but sends no data. Contributed by Chris Nauroth 
(cherry picked from commit 03bab8dea1)
 2015-12-17 14:05:05 -08:00
Arpit Agarwal c0733db768 HDFS-9515. NPE when MiniDFSCluster#shutdown is invoked on uninitialized reference. (Contributed by Wei-Chiu Chuang) 2015-12-17 13:57:12 -08:00
cnauroth a40ff6d4fb Revert "HDFS-9569. Log the name of the fsimage being loaded for better supportability. Contributed by Yongjun Zhang." 
This reverts commit 03e10f1fb3.
 2015-12-17 13:53:25 -08:00
Allen Wittenauer a5c74cd921 HDFS-9570. Minor typos, grammar, and case sensitivity cleanup in HdfsPermissionsGuide.md's (Travis Campbell via aw) 
(cherry picked from commit 8d278d8d2e)

Conflicts:
	hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt
 2015-12-17 11:40:58 -08:00
cnauroth 940026fc6b HDFS-9552. Document types of permission checks performed for HDFS operations. Contributed by Chris Nauroth. 
(cherry picked from commit 724e71f63b)
 2015-12-17 10:43:54 -08:00
Arpit Agarwal 0c1d1e311c HDFS-9565. TestDistributedFileSystem.testLocatedFileStatusStorageIdsTypes is flaky due to race condition. (Contributed by Wei-Chiu Chuang) 2015-12-17 10:37:58 -08:00
Kihwal Lee 03e10f1fb3 HDFS-9569. Log the name of the fsimage being loaded for better supportability. Contributed by Yongjun Zhang. 
(cherry picked from commit eb6939cea0)
 2015-12-17 10:30:58 -06:00
Konstantin V Shvachko 3d0222eafd Update CHANGES.txt to move HDFS-9516 to 2.7.3 section. 2015-12-16 23:24:34 -08:00
Uma Mahesh 3be39d0229 HDFS-9198. Coalesce IBR processing in the NN. (Daryn Sharp via umamahesh) 
(cherry picked from commit f741476146)
 2015-12-16 22:04:19 -08:00
Uma Mahesh e82135df87 Revert this commit as there is compilation issue with this patch in branch-2 "HDFS-9198. Coalesce IBR processing in the NN. (Daryn Sharp via umamahesh)" 
This reverts commit 991ce3d630.
 2015-12-16 19:18:30 -08:00
Uma Mahesh 991ce3d630 HDFS-9198. Coalesce IBR processing in the NN. (Daryn Sharp via umamahesh) 
(cherry picked from commit f741476146)
 2015-12-16 18:25:29 -08:00
Andrew Wang c7cb29defe HDFS-9300. TestDirectoryScanner.testThrottle() is still a little flakey. Contributed by Daniel Templeton. 
(cherry picked from commit 3c0adac88c)
 2015-12-16 16:21:16 -08:00
cnauroth e8e3737ce8 HDFS-9557. Reduce object allocation in PB conversion. Contributed by Daryn Sharp. 
(cherry picked from commit c470c8953d)
 2015-12-16 11:11:27 -08:00
Andrew Wang 2796c1099f HDFS-8894. Set SO_KEEPALIVE on DN server sockets. Contributed by Kanaka Kumar Avvaru. 
(cherry picked from commit 49949a4bb0)
 2015-12-15 14:39:04 -08:00
Jing Zhao e0abb0a4e7 HDFS-9371. Code cleanup for DatanodeManager. Contributed by Jing Zhao. 
(cherry picked from commit 8602692338)
 2015-12-15 11:02:04 -08:00
Plamen Jeliazkov 15124c86d8 HDFS-9516. Truncate file fails with data dirs on multiple disks. Contributed by Plamen Jeliazkov. 2015-12-15 00:39:42 -08:00
Lei Xu 6df45118e5 Revert "Revert "HDFS-8860. Remove unused Replica copyOnWrite code (Lei (Eddy) Xu via Colin P. McCabe)"" 
This reverts commit fb909554a8.
 2015-12-14 11:26:32 -08:00
Jing Zhao e02ad5a618 HDFS-9535. Newly completed blocks in IBR should not be considered under-replicated too quickly. Contributed by Mingliang Liu. 
(cherry picked from commit e534569814)
 2015-12-14 11:18:28 -08:00
Lei Xu 0852d3562a HDFS-9281. Change TestDeleteBlockPool to not explicitly use File to check block pool existence. (lei) 
(cherry picked from commit db975af5fae506eb3d586bd9201f76adb3fd1281)
 2015-12-14 11:01:48 -08:00
Yongjun Zhang be167d0154 HDFS-9514. TestDistributedFileSystem.testDFSClientPeerWriteTimeout failing; exception being swallowed. (Wei-Chiu Chuang via Yongjun Zhang) 
(cherry picked from commit bf5295b118)
 2015-12-11 20:56:58 -08:00
Yongjun Zhang 3fc9593318 HDFS-9519. Some coding improvement in SecondaryNameNode#main. (Xiao Chen via Yongjun Zhang) 
(cherry picked from commit 2a4c7d4fac)
 2015-12-11 20:56:46 -08:00
Xiaoyu Yao 08616b3654 HDFS-8785. TestDistributedFileSystem is failing in trunk. Contributed by Xiaoyu Yao. 
(cherry picked from commit 2196e39e14)
 2015-12-11 20:56:12 -08:00
Uma Mahesh 5b1e1dfb3f HDFS-9528. Cleanup namenode audit/log/exception messages. (szetszwo via umamahesh) 
(cherry picked from commit 796a676d18)
 2015-12-11 19:18:25 -08:00
Yongjun Zhang e00888d5cb HDFS-9532. Detailed exception info is lost in reportTo methods of ErrorReportAction and ReportBadBlockAction. (Yongjun Zhang) 
(cherry picked from commit f5a911452f)

Conflicts:
	hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt
 2015-12-11 18:24:50 -08:00
Xiaoyu Yao 5595b912e4 HDFS-8326. Documentation about when checkpoints are run is out of date. (Misty Stanley-Jones via xyao) 
(cherry picked from commit d0e75e60fb)
 2015-12-11 08:41:30 -08:00
Kihwal Lee a72ef921f0 HDFS-9445. Datanode may deadlock while handling a bad volume. Contributed by Walter Su. 
(cherry picked from commit a48301791e)
 2015-12-11 08:46:03 -06:00
Uma Mahesh 7dc558b6a7 HDFS-9472. concat() API does not give proper exception messages on ./reserved relative path (Rakesh R via umamahesh) 
(cherry picked from commit e363417e7b)
 2015-12-11 00:01:02 -08:00
Lei Xu fb909554a8 Revert "HDFS-8860. Remove unused Replica copyOnWrite code (Lei (Eddy) Xu via Colin P. McCabe)" 
This reverts commit b9bf1a9391.

Conflicts:
	hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt
	hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/datanode/DataNodeTestUtils.java
 2015-12-10 11:23:02 -10:00
Tsz-Wo Nicholas Sze c65a796f88 HDFS-9527. The return type of FSNamesystem.getBlockCollection should be changed to INodeFile. 2015-12-09 18:10:34 -08:00
Vinod Kumar Vavilapalli e81cb68508 HDFS-9273. Moving to 2.6.3 CHANGES section to reflect the backport. 2015-12-08 11:34:05 -08:00
cnauroth 1072be12b9 HDFS-9414. Refactor reconfiguration of ClientDatanodeProtocol for reusability. Contributed by Xiaobing Zhou. 
(cherry picked from commit 303f80e656)
 2015-12-08 10:50:13 -08:00
Jing Zhao c993327483 HDFS-9129. Move the safemode block count into BlockManager. Contributed by Mingliang Liu. 2015-12-07 13:46:12 -08:00
Steve Loughran af14458de7 HADOOP-12321. Make JvmPauseMonitor an AbstractService. (Sunil G via Stevel) [includes HDFS-8947 MAPREDUCE-6462 and YARN-4072] 2015-12-06 17:42:56 +00:00
Arpit Agarwal 6999e764cb HDFS-9414. Add missing license header 2015-12-05 09:24:44 -08:00
Arpit Agarwal 71be31201a HDFS-9414. Refactor reconfiguration of ClientDatanodeProtocol for reusability. (Contributed by Xiaobing Zhou) 2015-12-04 20:53:43 -08:00
Lei Xu ce49ba39d2 HDFS-9491. Tests should get the number of pending async deletes via FsDatasetTestUtils. (Tony Wu via lei) 
(cherry picked from commit c92713dac56da0f836382005d84b577cfeb36a05)
 2015-12-04 18:05:51 -08:00
Xiaoyu Yao eb7f9901b4 HDFS-8831. Trash Support for deletion in HDFS encryption zone. Contributed by Xiaoyu Yao. 
(cherry picked from commit cbc7b6bf97)
 2015-12-04 16:57:58 -08:00
Arpit Agarwal 496c8969d4 HDFS-9214. Add missing license header 2015-12-04 15:33:24 -08:00
Arpit Agarwal 9ddd7a95b9 HDFS-9214. Support reconfiguring dfs.datanode.balance.max.concurrent.moves without DN restart. (Contributed by Xiaobing Zhou) 2015-12-04 14:46:54 -08:00
Yongjun Zhang 78a21e773d HDFS-9474. TestPipelinesFailover should not fail when printing debug message. (John Zhuge via Yongjun Zhang) 
(cherry picked from commit 59dbe8b3e9)
 2015-12-04 13:50:03 -08:00
Colin P. McCabe 3beedc177d HDFS-9267. TestDiskError should get stored replicas through FsDatasetTestUtils. (Lei (Eddy) Xu via Colin P. McCabe) 
(cherry picked from commit e02bbeb886)
 2015-12-04 12:51:55 -08:00
Lei Xu 58714f3b73 HDFS-9490. MiniDFSCluster should change block generation stamp via FsDatasetTestUtils. (Tony Wu via lei) 
(cherry picked from commit 0ac8fb4b33)
 2015-12-04 10:26:06 -08:00
Ming Ma a68faf1a02 HDFS-9430 Remove waitForLoadingFSImage since checkNNStartup has ensured image loaded and namenode started. (Brahma Reddy Battula via mingma) 
(cherry picked from commit 3fa33b5c2c)
 2015-12-04 09:55:02 -08:00
Konstantin V Shvachko 33d4588a7c HDFS-9484. NNThroughputBenchmark$BlockReportStats should not send empty block reports. Contributed by Mingliang Liu. 2015-12-03 14:56:00 -08:00
Konstantin V Shvachko 3aa73a867a HDFS-9436. Make NNThroughputBenchmark$BlockReportStats run with 10 datanodes by default. Contributed by Mingliang. 2015-12-03 14:54:06 -08:00
Tsz-Wo Nicholas Sze bac127a4e2 HDFS-9294. DFSClient deadlock when close file and failed to renew lease. Contributed by Brahma Reddy Battula 2015-12-02 17:49:16 -08:00
Colin Patrick Mccabe 9b516a2a05 HDFS-9429. Tests in TestDFSAdminWithHA intermittently fail with EOFException (Xiao Chen via Colin P. McCabe) 
(cherry picked from commit 53e3bf7e70)
 2015-12-01 23:58:09 -08:00
Arpit Agarwal 0a1ef49b33 HDFS-6533. TestBPOfferService#testBasicFunctionalitytest fails intermittently. (Contributed by Wei-Chiu Chuang) 2015-12-01 13:32:37 -08:00
Jing Zhao 7e416aa70d HDFS-9485. Make BlockManager#removeFromExcessReplicateMap accept BlockInfo instead of Block. Contributed by Mingliang Liu. 
(cherry picked from commit 830eb252aa)
 2015-12-01 13:07:19 -08:00
Zhe Zhang 9db4b77ea5 HDFS-9269. Update the documentation and wrapper for fuse-dfs. Contributed by Wei-Chiu Chuang. 
Change-Id: Ia9ec512de2464bf94725cc7c15c378c59d0f04c4
 2015-12-01 10:25:32 -08:00
Allen Wittenauer 45bb467918 HADOOP-12366. expose calculated paths (aw) 2015-12-01 08:16:20 -08:00
Junping Du 092c469059 Creating 2.6.4 entries in CHANGES.txt files. 
(cherry picked from commit 028894281b)
 2015-12-01 08:18:57 -08:00
Akira Ajisaka 402a2f2d64 HDFS-9336. deleteSnapshot throws NPE when snapshotname is null. Contributed by Brahma Reddy Battula. 
(cherry picked from commit 1c05393b51)
 2015-12-01 11:30:11 +08:00
Andrew Wang ce1111ceea HDFS-9470. Encryption zone on root not loaded from fsimage after NN restart. Xiao Chen via wang. 
(cherry picked from commit 9b8e50b424)
 2015-11-30 14:36:06 -08:00
Vinayakumar B 57e0c735f8 HDFS-9426. Rollingupgrade finalization is not backward compatible (Contributed by Kihwal Lee) 
(cherry picked from commit c62d42cd8b)

Conflicts:
	hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/protocolPB/DatanodeProtocolClientSideTranslatorPB.java
	hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/protocolPB/DatanodeProtocolServerSideTranslatorPB.java
	hadoop-hdfs-project/hadoop-hdfs/src/main/proto/DatanodeProtocol.proto

(cherry picked from commit 9f256d1d71)
 2015-11-26 09:40:44 +05:30
Vinod Kumar Vavilapalli b20e77eba3 Preparing for 2.9.0 development: mvn versions:set -DnewVersion=2.9.0 2015-11-25 17:47:22 -08:00
Vinod Kumar Vavilapalli 767815f317 Adding release 2.8.0 to CHANGES.txt 
(cherry picked from commit d57fd181c7)
 2015-11-25 17:37:47 -08:00
Jing Zhao 288cf8437b HDFS-9467. Fix data race accessing writeLockHeldTimeStamp in FSNamesystem. Contributed by Mingliang Liu. 
(cherry picked from commit e556c35b05)
 2015-11-25 14:23:06 -08:00
Konstantin V Shvachko 9c1ef648a9 HDFS-9407. TestFileTruncate should not use fixed NN port. Contributed by Brahma Reddy Battula. 2015-11-25 14:16:03 -08:00
Xiaoyu Yao 4f1f9f7eff HDFS-8512. WebHDFS : GETFILESTATUS should return LocatedBlock with storage type info. Contributed by Xiaoyu Yao. 
(cherry picked from commit e3d673901b)
 2015-11-25 13:54:22 -08:00
Haohui Mai 15caaa7d0a HDFS-9459. hadoop-hdfs-native-client fails test build on Windows after transition to ctest. Contributed by Chris Nauroth. 2015-11-25 10:32:13 -08:00
Yongjun Zhang 48b294c58e HDFS-9438. TestPipelinesFailover assumes Linux ifconfig. (John Zhuge via Yongjun Zhang) 
(cherry picked from commit 8176ea7dc6)
 2015-11-25 08:50:02 -08:00
Yongjun Zhang 7ac8d8f510 HDFS-6694. Addendum. Update CHANGES.txt for cherry-picking to 2.8. 
(cherry picked from commit 84d01ad7f4)

Conflicts:
	hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt
 2015-11-25 08:49:26 -08:00
arp 566ceac1ff HDFS-6694. TestPipelinesFailover.testPipelineRecoveryStress tests fail intermittently with various symptoms - debugging patch (Contributed by Yongjun Zhang) 
(cherry picked from commit c5d9a4a91e)

Conflicts:
	hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt
 2015-11-25 08:45:59 -08:00
Tom Zeng d6f7d22929 HADOOP-12415. Add io.netty dependency to hadoop-nfs and to hadoop-hdfs, needed to build Bigtop successfuly, see BIGTOP-2049 
Signed-off-by: Konstantin Boudnik <cos@boudnik.org>
(cherry picked from commit b4c6b511e7)
 2015-11-24 18:39:01 -08:00
Tsz-Wo Nicholas Sze 79e55fb47f HDFS-8807. dfs.datanode.data.dir does not handle spaces between storageType and URI correctly. Contributed by Anu Engineer 2015-11-24 16:05:21 -08:00
Haohui Mai b07c91a16b HDFS-8335. FSNamesystem should construct FSPermissionChecker only if permission is enabled. Contributed by Gabor Liptak. 2015-11-24 13:07:45 -08:00
Tsz-Wo Nicholas Sze 15ecbc95bf Move HDFS-9434 to 2.6.3 in CHANGES.txt. 2015-11-24 12:44:37 -08:00
Ming Ma 85d04dc464 HDFS-9314. Improve BlockPlacementPolicyDefault's picking of excess replicas. (Xiao Chen via mingma) 
(cherry picked from commit 0e54b164a8)
 2015-11-24 10:31:23 -08:00
cnauroth c79f017752 HDFS-6101. TestReplaceDatanodeOnFailure fails occasionally. Contributed by Wei-Chiu Chuang. 
(cherry picked from commit 1777608fa0)
 2015-11-24 09:39:37 -08:00
Uma Mahesh 52c889ac14 HDFS-9433. DFS getEZForPath API on a non-existent file should throw FileNotFoundException (Rakesh R via umamahesh) 
(cherry picked from commit 411e2b2e7c)
 2015-11-23 18:46:48 -08:00
Haohui Mai d469e9e4ca HDFS-7988. Replace usage of ExactSizeInputStream with LimitInputStream. Contributed by Walter Su. 2015-11-23 10:50:23 -08:00
Steve Loughran 331ed6998b Revert "HDFS-9423. Fix intermittent failure of TestEditLogTailer. Contributed by Masatake Iwasaki." 
This reverts commit de3b33e121.
 2015-11-23 12:28:25 +00:00
Walter Su de3b33e121 HDFS-9423. Fix intermittent failure of TestEditLogTailer. Contributed by Masatake Iwasaki. 
(cherry picked from commit 201f14e271)
 2015-11-23 18:50:29 +08:00
Walter Su 040c582b99 HDFS-9435. TestBlockRecovery#testRBWReplicas is failing intermittently. Contributed by Rakesh R. 
(cherry picked from commit 001ecf962c)
 2015-11-23 15:50:57 +08:00
Walter Su 846de7d842 HDFS-9428. Fix intermittent failure of TestDNFencing.testQueueingWithAppend. Contributed by Masatake Iwasaki. 
(cherry picked from commit 5aba093361)
 2015-11-23 15:38:41 +08:00
Haohui Mai c07fab5f6f HDFS-9024. Deprecate the TotalFiles metric. Contributed by Akira Ahisaka. 2015-11-22 23:03:05 -08:00
Haohui Mai 5ce7e72145 HDFS-9356. Decommissioning node does not have Last Contact value in the UI. Contributed by Surendra Singh Lilhore. 2015-11-22 22:59:12 -08:00
Haohui Mai 529103c51f HDFS-7897. Shutdown metrics when stopping JournalNode. Contributed by zhouyingchao. 2015-11-22 19:12:32 -08:00
Haohui Mai 8aa12772d4 HDFS-6885. Fix wrong use of BytesWritable in FSEditLogOp#RenameOp. Contributed by Yi Liu. 2015-11-22 17:17:01 -08:00
Haohui Mai 53f6a9e134 HDFS-3302. Review and improve HDFS trash documentation. Contributed by Harsh J. 2015-11-22 16:30:46 -08:00
Haohui Mai 2a4cd27bdc HDFS-7796. Include X-editable for slick contenteditable fields in the web UI. Contributed by Ravi Prakash. 2015-11-22 16:22:14 -08:00
Haohui Mai 469fef2a4f HDFS-9153. Pretty-format the output for DFSIO. Contributed by Kai Zheng. 2015-11-22 16:18:16 -08:00
Haohui Mai 049ccf71ce HDFS-8914. Document HA support in the HDFS HdfsDesign.md. Contributed by Lars Francke. 2015-11-22 16:10:38 -08:00
Haohui Mai 64add87f5c HADOOP-11218. Add TLSv1.1,TLSv1.2 to KMS, HttpFS, SSLFactory. Contributed by Vijay Singh. 2015-11-22 16:00:44 -08:00
Haohui Mai f504d95e87 HDFS-9402. Switch DataNode.LOG to use slf4j. Contributed by Walter Su. 2015-11-22 15:57:23 -08:00
Haohui Mai 2c185222f9 HDFS-9443. Disabling HDFS client socket cache causes logging message printed to console for CLI commands. Contributed by Chris Nauroth. 2015-11-22 15:06:36 -08:00
Tsz-Wo Nicholas Sze bf82a8de30 HDFS-9434. Recommission a datanode with 500k blocks may pause NN for 30 seconds for printing info log messags. 2015-11-19 14:54:59 -08:00