Apache
/
hadoop
1
0
Fork 0
Code Issues Pull Requests Packages Projects Releases Wiki Activity
10,649 Commits 326 Branches 371 Tags 1.5 GiB
Commit Graph

3690 Commits

Author SHA1 Message Date
Kihwal Lee a8dc35dfdf HDFS-9569. Log the name of the fsimage being loaded for better supportability. Contributed by Yongjun Zhang. 
(cherry picked from commit eb6939cea0)
 2015-12-17 10:31:37 -06:00
Plamen Jeliazkov cffa262f11 HDFS-9516. Truncate file fails with data dirs on multiple disks. Contributed by Plamen Jeliazkov. 2015-12-17 00:03:51 -08:00
Uma Mahesh e6dc6f24a2 HDFS-9198. Coalesce IBR processing in the NN. (Daryn Sharp via umamahesh) 
(cherry picked from commit f741476146)
 2015-12-16 22:50:12 -08:00
cnauroth d3e1161b8d HDFS-9557. Reduce object allocation in PB conversion. Contributed by Daryn Sharp. 
(cherry picked from commit c470c8953d)
(cherry picked from commit e8e3737ce8)
 2015-12-16 11:11:34 -08:00
Andrew Wang 8e5de6cc42 HDFS-8894. Set SO_KEEPALIVE on DN server sockets. Contributed by Kanaka Kumar Avvaru. 
(cherry picked from commit 49949a4bb0)
(cherry picked from commit 2796c1099f)
 2015-12-15 14:40:23 -08:00
Yongjun Zhang 4bc1aab49d HDFS-9514. TestDistributedFileSystem.testDFSClientPeerWriteTimeout failing; exception being swallowed. (Wei-Chiu Chuang via Yongjun Zhang) 
(cherry picked from commit bf5295b118)
 2015-12-14 15:40:55 -08:00
Yongjun Zhang 6c0e36b91c HDFS-9519. Some coding improvement in SecondaryNameNode#main. (Xiao Chen via Yongjun Zhang) 
(cherry picked from commit 2a4c7d4fac)

Conflicts:
	hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt
 2015-12-14 15:40:13 -08:00
Xiaoyu Yao cdf43bb438 HDFS-8785. TestDistributedFileSystem is failing in trunk. Contributed by Xiaoyu Yao. 
(cherry picked from commit 2196e39e14)
 2015-12-14 15:31:37 -08:00
Jing Zhao 73a055bb9f HDFS-9535. Newly completed blocks in IBR should not be considered under-replicated too quickly. Contributed by Mingliang Liu. 
(cherry picked from commit e534569814)
(cherry picked from commit e02ad5a618)
 2015-12-14 11:20:10 -08:00
Uma Mahesh 4c0b9c9062 HDFS-9528. Cleanup namenode audit/log/exception messages. (szetszwo via umamahesh) 
(cherry picked from commit 796a676d18)
 2015-12-11 19:32:40 -08:00
Yongjun Zhang d02886bf8d HDFS-9532. Detailed exception info is lost in reportTo methods of ErrorReportAction and ReportBadBlockAction. (Yongjun Zhang) 
(cherry picked from commit f5a911452f)

Conflicts:
	hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt

(cherry picked from commit e00888d5cb)
 2015-12-11 18:30:57 -08:00
Kihwal Lee 937a43dc29 HDFS-9445. Datanode may deadlock while handling a bad volume. Contributed by Walter Su. 
(cherry picked from commit a48301791e)
 2015-12-11 08:46:42 -06:00
Uma Mahesh d028295414 HDFS-9472. concat() API does not give proper exception messages on ./reserved relative path (Rakesh R via umamahesh) 
(cherry picked from commit e363417e7b)
 2015-12-11 00:02:49 -08:00
Tsz-Wo Nicholas Sze 26c8950fb4 HDFS-9527. The return type of FSNamesystem.getBlockCollection should be changed to INodeFile. 2015-12-09 18:14:07 -08:00
Vinod Kumar Vavilapalli 469bf56801 HDFS-9273. Moving to 2.6.3 CHANGES section to reflect the backport. 2015-12-08 11:35:28 -08:00
Arpit Agarwal 43241bf8e5 HDFS-9214. Add missing license header 2015-12-04 15:33:30 -08:00
Arpit Agarwal da3dfd77fd HDFS-9214. Support reconfiguring dfs.datanode.balance.max.concurrent.moves without DN restart. (Contributed by Xiaobing Zhou) 2015-12-04 14:47:01 -08:00
Yongjun Zhang 560d49da8f HDFS-9474. TestPipelinesFailover should not fail when printing debug message. (John Zhuge via Yongjun Zhang) 
(cherry picked from commit 59dbe8b3e9)
 2015-12-04 14:02:28 -08:00
Tsz-Wo Nicholas Sze afc76eb218 HDFS-9294. DFSClient deadlock when close file and failed to renew lease. Contributed by Brahma Reddy Battula 2015-12-03 11:35:29 -08:00
Colin Patrick Mccabe 228ad84e9c HDFS-9429. Tests in TestDFSAdminWithHA intermittently fail with EOFException (Xiao Chen via Colin P. McCabe) 
(cherry picked from commit 53e3bf7e70)
(cherry picked from commit 9b516a2a05)
 2015-12-02 15:40:49 -08:00
Junping Du 12fb8b4cc5 Add 2.6.4 as release entry to CHANGES.txt 2015-12-02 07:38:03 -08:00
Arpit Agarwal 05d7202b1e HDFS-6533. TestBPOfferService#testBasicFunctionalitytest fails intermittently. (Contributed by Wei-Chiu Chuang) 2015-12-01 15:29:42 -08:00
Kihwal Lee b70b380f37 HDFS-9426. Rollingupgrade finalization is not backward compatible (Contributed by Kihwal Lee) 
(cherry picked from commit c62d42cd8b)

Conflicts:
	hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/protocolPB/DatanodeProtocolClientSideTranslatorPB.java
	hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/protocolPB/DatanodeProtocolServerSideTranslatorPB.java
	hadoop-hdfs-project/hadoop-hdfs/src/main/proto/DatanodeProtocol.proto
(cherry picked from commit 9f256d1d71)
 2015-12-01 16:43:27 -06:00
Zhe Zhang 456426fe6b HDFS-9269. Update the documentation and wrapper for fuse-dfs. Contributed by Wei-Chiu Chuang. 
Conflicts:
	hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt

Change-Id: Idebd3b5d2ab09910c71cfde19e4b4d88f7ce1efb
 2015-12-01 14:41:57 -08:00
Jing Zhao 084ad2b420 HDFS-9485. Make BlockManager#removeFromExcessReplicateMap accept BlockInfo instead of Block. Contributed by Mingliang Liu. 
(cherry picked from commit 830eb252aa)
(cherry picked from commit 7e416aa70d)
 2015-12-01 13:10:56 -08:00
Allen Wittenauer 81a7ad45c9 HADOOP-12366. expose calculated paths (aw) 2015-12-01 08:15:57 -08:00
Akira Ajisaka 745772c975 HDFS-9336. deleteSnapshot throws NPE when snapshotname is null. Contributed by Brahma Reddy Battula. 
(cherry picked from commit 1c05393b51)
(cherry picked from commit 402a2f2d64)
 2015-12-01 11:30:26 +08:00
Andrew Wang 8676959d13 HDFS-9470. Encryption zone on root not loaded from fsimage after NN restart. Xiao Chen via wang. 
(cherry picked from commit 9b8e50b424)
(cherry picked from commit ce1111ceea)
 2015-11-30 18:04:46 -08:00
Vinod Kumar Vavilapalli efaef45a24 Reverting previous commit which added 2.9.0 entries. Don't need that in branch-2.8. 
Revert "Adding release 2.8.0 to CHANGES.txt"

This reverts commit 767815f317.
 2015-11-25 17:49:48 -08:00
Vinod Kumar Vavilapalli 767815f317 Adding release 2.8.0 to CHANGES.txt 
(cherry picked from commit d57fd181c7)
 2015-11-25 17:37:47 -08:00
Jing Zhao 288cf8437b HDFS-9467. Fix data race accessing writeLockHeldTimeStamp in FSNamesystem. Contributed by Mingliang Liu. 
(cherry picked from commit e556c35b05)
 2015-11-25 14:23:06 -08:00
Konstantin V Shvachko 9c1ef648a9 HDFS-9407. TestFileTruncate should not use fixed NN port. Contributed by Brahma Reddy Battula. 2015-11-25 14:16:03 -08:00
Xiaoyu Yao 4f1f9f7eff HDFS-8512. WebHDFS : GETFILESTATUS should return LocatedBlock with storage type info. Contributed by Xiaoyu Yao. 
(cherry picked from commit e3d673901b)
 2015-11-25 13:54:22 -08:00
Haohui Mai 15caaa7d0a HDFS-9459. hadoop-hdfs-native-client fails test build on Windows after transition to ctest. Contributed by Chris Nauroth. 2015-11-25 10:32:13 -08:00
Yongjun Zhang 48b294c58e HDFS-9438. TestPipelinesFailover assumes Linux ifconfig. (John Zhuge via Yongjun Zhang) 
(cherry picked from commit 8176ea7dc6)
 2015-11-25 08:50:02 -08:00
Yongjun Zhang 7ac8d8f510 HDFS-6694. Addendum. Update CHANGES.txt for cherry-picking to 2.8. 
(cherry picked from commit 84d01ad7f4)

Conflicts:
	hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt
 2015-11-25 08:49:26 -08:00
arp 566ceac1ff HDFS-6694. TestPipelinesFailover.testPipelineRecoveryStress tests fail intermittently with various symptoms - debugging patch (Contributed by Yongjun Zhang) 
(cherry picked from commit c5d9a4a91e)

Conflicts:
	hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt
 2015-11-25 08:45:59 -08:00
Tom Zeng d6f7d22929 HADOOP-12415. Add io.netty dependency to hadoop-nfs and to hadoop-hdfs, needed to build Bigtop successfuly, see BIGTOP-2049 
Signed-off-by: Konstantin Boudnik <cos@boudnik.org>
(cherry picked from commit b4c6b511e7)
 2015-11-24 18:39:01 -08:00
Tsz-Wo Nicholas Sze 79e55fb47f HDFS-8807. dfs.datanode.data.dir does not handle spaces between storageType and URI correctly. Contributed by Anu Engineer 2015-11-24 16:05:21 -08:00
Haohui Mai b07c91a16b HDFS-8335. FSNamesystem should construct FSPermissionChecker only if permission is enabled. Contributed by Gabor Liptak. 2015-11-24 13:07:45 -08:00
Tsz-Wo Nicholas Sze 15ecbc95bf Move HDFS-9434 to 2.6.3 in CHANGES.txt. 2015-11-24 12:44:37 -08:00
Ming Ma 85d04dc464 HDFS-9314. Improve BlockPlacementPolicyDefault's picking of excess replicas. (Xiao Chen via mingma) 
(cherry picked from commit 0e54b164a8)
 2015-11-24 10:31:23 -08:00
cnauroth c79f017752 HDFS-6101. TestReplaceDatanodeOnFailure fails occasionally. Contributed by Wei-Chiu Chuang. 
(cherry picked from commit 1777608fa0)
 2015-11-24 09:39:37 -08:00
Uma Mahesh 52c889ac14 HDFS-9433. DFS getEZForPath API on a non-existent file should throw FileNotFoundException (Rakesh R via umamahesh) 
(cherry picked from commit 411e2b2e7c)
 2015-11-23 18:46:48 -08:00
Haohui Mai d469e9e4ca HDFS-7988. Replace usage of ExactSizeInputStream with LimitInputStream. Contributed by Walter Su. 2015-11-23 10:50:23 -08:00
Steve Loughran 331ed6998b Revert "HDFS-9423. Fix intermittent failure of TestEditLogTailer. Contributed by Masatake Iwasaki." 
This reverts commit de3b33e121.
 2015-11-23 12:28:25 +00:00
Walter Su de3b33e121 HDFS-9423. Fix intermittent failure of TestEditLogTailer. Contributed by Masatake Iwasaki. 
(cherry picked from commit 201f14e271)
 2015-11-23 18:50:29 +08:00
Walter Su 040c582b99 HDFS-9435. TestBlockRecovery#testRBWReplicas is failing intermittently. Contributed by Rakesh R. 
(cherry picked from commit 001ecf962c)
 2015-11-23 15:50:57 +08:00
Walter Su 846de7d842 HDFS-9428. Fix intermittent failure of TestDNFencing.testQueueingWithAppend. Contributed by Masatake Iwasaki. 
(cherry picked from commit 5aba093361)
 2015-11-23 15:38:41 +08:00
Haohui Mai c07fab5f6f HDFS-9024. Deprecate the TotalFiles metric. Contributed by Akira Ahisaka. 2015-11-22 23:03:05 -08:00
Haohui Mai 5ce7e72145 HDFS-9356. Decommissioning node does not have Last Contact value in the UI. Contributed by Surendra Singh Lilhore. 2015-11-22 22:59:12 -08:00
Haohui Mai 529103c51f HDFS-7897. Shutdown metrics when stopping JournalNode. Contributed by zhouyingchao. 2015-11-22 19:12:32 -08:00
Haohui Mai 8aa12772d4 HDFS-6885. Fix wrong use of BytesWritable in FSEditLogOp#RenameOp. Contributed by Yi Liu. 2015-11-22 17:17:01 -08:00
Haohui Mai 53f6a9e134 HDFS-3302. Review and improve HDFS trash documentation. Contributed by Harsh J. 2015-11-22 16:30:46 -08:00
Haohui Mai 2a4cd27bdc HDFS-7796. Include X-editable for slick contenteditable fields in the web UI. Contributed by Ravi Prakash. 2015-11-22 16:22:14 -08:00
Haohui Mai 469fef2a4f HDFS-9153. Pretty-format the output for DFSIO. Contributed by Kai Zheng. 2015-11-22 16:18:16 -08:00
Haohui Mai 049ccf71ce HDFS-8914. Document HA support in the HDFS HdfsDesign.md. Contributed by Lars Francke. 2015-11-22 16:10:38 -08:00
Haohui Mai f504d95e87 HDFS-9402. Switch DataNode.LOG to use slf4j. Contributed by Walter Su. 2015-11-22 15:57:23 -08:00
Haohui Mai 2c185222f9 HDFS-9443. Disabling HDFS client socket cache causes logging message printed to console for CLI commands. Contributed by Chris Nauroth. 2015-11-22 15:06:36 -08:00
Tsz-Wo Nicholas Sze bf82a8de30 HDFS-9434. Recommission a datanode with 500k blocks may pause NN for 30 seconds for printing info log messags. 2015-11-19 14:54:59 -08:00
Yongjun Zhang 0664782025 HDFS-9439. Include status of closeAck into exception message in DataNode#run. (Xiao Chen via Yongjun Zhang) 
(cherry picked from commit 31dc5c650e)
 2015-11-19 14:22:44 -08:00
Ming Ma 5a3db21563 HDFS-8056. Decommissioned dead nodes should continue to be counted as dead after NN restart. (mingma) 
(cherry picked from commit 1c4951a7a0)
 2015-11-19 10:04:40 -08:00
Akira Ajisaka c74e42b4a2 HDFS-9431. DistributedFileSystem#concat fails if the target path is relative. Contributed by Kazuho Fujii. 
(cherry picked from commit ac1aa6c819)
 2015-11-19 20:55:36 +09:00
cnauroth 7b94ae17ab HDFS-9400. TestRollingUpgradeRollback fails on branch-2. Contributed by Brahma Reddy Battula. 
(cherry picked from commit bad2afe39a)
 2015-11-18 11:00:18 -08:00
Colin Patrick Mccabe 0f9f9ba3cb HDFS-9252. Change TestFileTruncate to use FsDatasetTestUtils to get block file size and genstamp. (Lei (Eddy) Xu via cmccabe) 
(cherry picked from commit dfbde3fc51)
 2015-11-17 10:54:32 -08:00
Akira Ajisaka 2babf12810 Move HDFS-8615 from 2.8.0 to 2.6.3 in CHANGES.txt. 
(cherry picked from commit b82a6cee4b)
 2015-11-17 16:34:04 +09:00
Arpit Agarwal efcaf9c28f HDFS-9397. Fix typo for readChecksum() LOG.warn in BlockSender.java. (Contributed by Enrique Flores) 2015-11-16 21:02:06 -08:00
Walter Su bff9d08a43 HDFS-9358. TestNodeCount#testNodeCount timed out. Contributed by Masatake Iwasaki. 
(cherry picked from commit 621cbb4f69)
 2015-11-17 10:53:50 +08:00
Xiaoyu Yao dceed6b0a8 HDFS-9421. NNThroughputBenchmark replication test NPE with -namenode option. Contributed by Mingliang Liu. 
(cherry picked from commit a4f62a2d58)
 2015-11-16 16:42:57 -08:00
Xiaoyu Yao 5ea3f03ecc HDFS-9387. Fix namenodeUri parameter parsing in NNThroughputBenchmark. Contributed by Mingliang Liu. 
(cherry picked from commit 2701f2d255)
 2015-11-16 16:31:06 -08:00
Ming Ma 42b55ff23e HDFS-9413. getContentSummary() on standby should throw StandbyException. (Brahma Reddy Battula via mingma) 
(cherry picked from commit 02653add98)
 2015-11-16 09:34:03 -08:00
Walter Su 0f5aae5fd8 HDFS-9410. Some tests should always reset sysout and syserr. Contributed by Xiao Chen. 
(cherry picked from commit cccf88480b)
 2015-11-13 14:29:25 +08:00
Kihwal Lee 8726a5e020 HDFS-9396. Total files and directories on jmx and web UI on standby is uninitialized. Contributed by Kihwal Lee. 
(cherry picked from commit a2a104f3d3)
 2015-11-11 15:37:11 -06:00
Xiaoyu Yao f3a075e053 HDFS-9245. Fix findbugs warnings in hdfs-nfs/WriteCtx. Contributed by Mingliang Liu. 
(cherry picked from commit 6e4562b844)
 2015-11-10 16:51:01 -08:00
Xiaoyu Yao 48a494cdc3 HDFS-9234. WebHdfs: getContentSummary() should give quota for storage types. Contributed by Surendra Singh Lilhore. 
(cherry picked from commit 41d3f8899d)
 2015-11-10 16:48:06 -08:00
Haohui Mai aaa38581f5 HDFS-9369. Use ctest to run tests for hadoop-hdfs-native-client. Contributed by Haohui Mai. 2015-11-10 14:53:22 -08:00
Zhe Zhang edd4b2d458 HDFS-9364. Unnecessary DNS resolution attempts when creating NameNodeProxies. Contributed by Xiao Chen. 
Change-Id: I9e42f724f27924cf73891425a832de37ce014a1e
 2015-11-10 09:55:39 -08:00
Walter Su e5420feb09 HDFS-9401. Fix findbugs warnings in BlockRecoveryWorker. Contributed by Brahma Reddy Battula. 
(cherry picked from commit 2fda45b9dc)
 2015-11-10 14:54:04 +08:00
Haohui Mai c91bd3aabb HDFS-2261. AOP unit tests are not getting compiled or run. Contributed by Haohui Mai. 2015-11-09 15:26:01 -08:00
Yongjun Zhang 6bca317157 HDFS-9249. NPE is thrown if an IOException is thrown in NameNode constructor. (Wei-Chiu Chuang via Yongjun Zhang) 
(cherry picked from commit 2741a2109b)
 2015-11-09 14:12:20 -08:00
Kihwal Lee ceb062f093 HDFS-9383. TestByteArrayManager#testByteArrayManager fails. Contributed by Tsz Wo Nicholas Sze. 
(cherry picked from commit ef926b2e38)
 2015-11-09 08:57:25 -06:00
cnauroth 385dcc4064 HDFS-9394. branch-2 hadoop-hdfs-client fails during FileSystem ServiceLoader initialization, because HftpFileSystem is missing. Contributed by Mingliang Liu. 
(cherry picked from commit 1ee2db5c41)
 2015-11-07 07:39:45 -08:00
Tsz-Wo Nicholas Sze 9766fc988f HDFS-9398. Make ByteArraryManager log message in one-line format. Contributed by Mingliang Liu 2015-11-07 22:22:36 +08:00
Arpit Agarwal 1e0746e756 HDFS-9379. Make NNThroughputBenchmark support more than 10 datanodes. (Contributed by Mingliang Liu) 2015-11-06 18:59:03 -08:00
Kihwal Lee 481e7248de HDFS-9318. considerLoad factor can be improved. Contributed by Kuhu Shukla. 
(cherry picked from commit bf6aa30a15)
 2015-11-06 14:08:10 -06:00
Yongjun Zhang 41d0d9a32c HDFS-9236. Missing sanity check for block size during block recovery. (Tony Wu via Yongjun Zhang) 
(cherry picked from commit b64242c0d2)
 2015-11-06 11:26:41 -08:00
Arpit Agarwal 33908c6f52 HDFS-6481. DatanodeManager#getDatanodeStorageInfos() should check the length of storageIDs. (Contributed by szetszwo) 2015-11-06 10:19:00 -08:00
cnauroth 75bcc8bcd8 HDFS-9384. TestWebHdfsContentLength intermittently hangs and fails due to TCP conversation mismatch between client and server. Contributed by Chris Nauroth. 
(cherry picked from commit 66c0967310)
 2015-11-05 22:07:34 -08:00
Yongjun Zhang 1bb89cbac7 HDFS-9377. Fix findbugs warnings in FSDirSnapshotOp. (Mingliang Liu via Yongjun Zhang) 
(cherry picked from commit 286cc6483f)
 2015-11-05 14:54:29 -08:00
Lei Xu 489c920c9a HDFS-9282. Make data directory count and storage raw capacity related tests FsDataset-agnostic. (Tony Wu via lei) 
(cherry picked from commit 5a1a54c9f5d9a8f7585000fe33ef106e62dbb43a)
 2015-11-05 10:46:46 -08:00
Lei Xu 7a2a17cb5f HDFS-9363. Add fetchReplica to FsDatasetTestUtils to return FsDataset-agnostic replica. (Tony Wu via lei) 
(cherry picked from commit 4976a02841592b61f2d2b40d2aadd5146571322c)
 2015-11-05 10:46:46 -08:00
Haohui Mai aa03030e4a HDFS-9372. Remove dead code in DataStorage.recoverTransitionRead. Contributed by Duo Zhang. 2015-11-05 10:00:18 -08:00
Kihwal Lee 37abd0e33b HDFS-4937. ReplicationMonitor can infinite-loop in BlockPlacementPolicyDefault#chooseRandom(). Contributed by Kihwal Lee. 
(cherry picked from commit ff47f35dee)
 2015-11-05 09:26:53 -06:00
cnauroth 6edd41ac69 HDFS-9378. hadoop-hdfs-client tests do not write logs. Contributed by Chris Nauroth. 
(cherry picked from commit c563401796)
 2015-11-05 06:48:32 -08:00
Xiaoyu Yao 891be39993 HDFS-9360. Storage type usage isn't updated properly after file deletion. Contributed by Ming Ma. 
(cherry picked from commit ea5bb48326)
 2015-11-05 01:40:05 -08:00
Lei Xu 8ed1fd2171 HDFS-9331. Modify TestNameNodeMXBean#testNameNodeMXBeanInfo() to account for filesystem entirely allocated for DFS use. (Tony Wu via lei) 
(cherry picked from commit e2a5441b06)
 2015-11-04 10:28:00 -08:00
Haohui Mai 6d92778191 Revert "HDFS-8855. Webhdfs client leaks active NameNode connections. Contributed by Xiaobing Zhou." 
This reverts commit e7203b6954.
 2015-11-04 10:22:21 -08:00
Lei Xu 3f2be7b18f HDFS-9007. Fix HDFS Balancer to honor upgrade domain policy. (Ming Ma via lei) 2015-11-04 10:21:58 -08:00
Kihwal Lee c8ffea3db7 fix up CHANGES.txt 2015-11-04 12:15:58 -06:00
Haohui Mai b61aa716e8 HDFS-9357. NN UI renders icons of decommissioned DN incorrectly. Contributed by Surendra Singh Lilhore. 2015-11-04 09:17:01 -08:00