Apache
/
hadoop
1
0
Fork 0
Code Issues Pull Requests Packages Projects Releases Wiki Activity
11,067 Commits 326 Branches 371 Tags 1.5 GiB
Commit Graph

3881 Commits

Author SHA1 Message Date
Andrew Wang bdb0e98483 HDFS-10297. Increase default balance bandwidth and concurrent moves. Contributed by John Zhuge. 2016-04-27 17:03:36 -07:00
Kihwal Lee dd7c9f5b8f HDFS-10330. Add Corrupt Blocks Information in Metasave output. Contributed by Kuhu Shukla. 
(cherry picked from commit e181092b86)
 2016-04-27 08:30:29 -05:00
Kihwal Lee 621be60592 HDFS-10329. Bad initialisation of StringBuffer in RequestHedgingProxyProvider. Contributed by Lin Yiqun. 
(cherry picked from commit c6a2430b47)
 2016-04-26 11:47:15 -05:00
Kihwal Lee 973c73aa68 HDFS-10318. TestJMXGet hides the real error in case of test failure. Contributed by Andras Bokor. 
(cherry picked from commit 10f0f7851a)
 2016-04-25 11:42:17 -05:00
Masatake Iwasaki c4d8090b72 HDFS-9905. WebHdfsFileSystem#runWithRetry should display original stack trace on error. (Wei-Chiu Chuang via iwasakims) 
(cherry picked from commit 6fcde2e38d)
 2016-04-23 23:55:27 +09:00
Kihwal Lee 3d3ed9b68e HDFS-9555. LazyPersistFileScrubber should still sleep if there are errors in the clear progress. Contributed by Phil Yang. 
(cherry picked from commit 5d4255a801)
 2016-04-22 11:10:36 -05:00
Tsz-Wo Nicholas Sze 60c68c65c1 HDFS-10309 Balancer doesn't honor dfs.blocksize value defined with suffix k(kilo), m(mega), g(giga). Contributed by Amit Anand 2016-04-21 15:45:36 -07:00
Tsz-Wo Nicholas Sze 704c57e3eb HDFS-10319. Balancer should not try to pair storages with different types. 2016-04-21 15:25:49 -07:00
Chris Nauroth 7fb1cfc591 HDFS-10312. Large block reports may fail to decode at NameNode due to 64 MB protobuf maximum length restriction. Contributed by Chris Nauroth. 
(cherry picked from commit 63ac2db59a)
(cherry picked from commit cc56bed686)

Conflicts:
	hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/datanode/fsdataset/impl/BlockPoolSlice.java
	hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/datanode/fsdataset/impl/FsDatasetImplTestUtils.java
 2016-04-20 14:00:06 -07:00
Colin Patrick Mccabe e2968f5da2 HDFS-10308. TestRetryCacheWithHA#testRetryCacheOnStandbyNN failing (Rakesh R via cmccabe) 
(cherry picked from commit ad36fa6f42)
(cherry picked from commit 76d963acb4)
 2016-04-20 11:52:49 -07:00
Arpit Agarwal a13628fe4a HDFS-10264. Logging improvements in FSImageFormatProtobuf.Saver. (Contributed by Xiaobing Zhou) 2016-04-19 11:27:46 -07:00
Walter Su 1860b8ceb5 HDFS-9744. TestDirectoryScanner#testThrottling occasionally time out after 300 seconds. Contributed by Lin Yiqun. 
(cherry picked from commit 90d1af1416)
 2016-04-19 18:56:23 +08:00
Colin Patrick Mccabe e1d9e5ab82 HDFS-10265. OEV tool fails to read edit xml file if OP_UPDATE_BLOCKS has no BLOCK tag (Wan Chang via cmccabe) 
(cherry picked from commit cb3ca460ef)
(cherry picked from commit a69b6b1e8b)
 2016-04-18 13:49:05 -07:00
Kihwal Lee 7ac2e5ec7b HDFS-10302. BlockPlacementPolicyDefault should use default replication considerload value. Contributed by Lin Yiqun. 
(cherry picked from commit d8b729e16f)
 2016-04-18 08:01:17 -05:00
Walter Su 09ffc65d94 HDFS-10275. TestDataNodeMetrics failing intermittently due to TotalWriteTime counted incorrectly. Contributed by Lin Yiqun. 
(cherry picked from commit ab903029a9)
 2016-04-18 20:32:36 +08:00
Steve Loughran e9699aa443 HDFS-10291 TestShortCircuitLocalRead failing (stevel) 2016-04-18 10:26:35 +01:00
Walter Su 7f47d984fa HDFS-9412. getBlocks occupies FSLock and takes too long to complete. Contributed by He Tianyi. 
(cherry picked from commit 67523ffcf4)

 Conflicts:
	hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/TestGetBlocks.java

(cherry picked from commit a6d0e9e56a)
 2016-04-18 09:39:46 +08:00
Kihwal Lee 8b1e7842e3 HDFS-10281. TestPendingCorruptDnMessages fails intermittently. Contributed by Mingliang Liu. 
(cherry picked from commit b9c9d03591)
 2016-04-14 15:30:59 -05:00
Kihwal Lee 7dd8798933 HDFS-10292. Add block id when client got Unable to close file exception. Contributed by Brahma Reddy Battula. 
(cherry picked from commit 2c155afe27)
 2016-04-14 14:26:59 -05:00
Kihwal Lee 019ca8f7c1 HDFS-10280. Document new dfsadmin command -evictWriters. Contributed by Wei-Chiu Chuang. 
(cherry picked from commit c970f1d005)

Conflicts:
	hadoop-hdfs-project/hadoop-hdfs/src/site/markdown/HDFSCommands.md
 2016-04-14 12:50:36 -05:00
Andrew Wang 1dee3e1200 HDFS-10279. Improve validation of the configured number of tolerated failed volumes. Contributed by Lin Yiqun. 
(cherry picked from commit 314aa21a89)
(cherry picked from commit 4c18f54c1d)
 2016-04-13 16:40:08 -07:00
Kihwal Lee ccaf746eea HDFS-10270. TestJMXGet:testNameNode() fails. Contributed by Gergely Novák 
(cherry picked from commit d2f3bbc290)
 2016-04-13 11:24:45 -05:00
Walter Su f3dbabb13a HDFS-9772. TestBlockReplacement#testThrottler doesn't work as expected. Contributed by Lin Yiqun. 
(cherry picked from commit 903428bf94)
 2016-04-13 17:50:47 +08:00
Colin Patrick Mccabe 59b0661c7f HADOOP-12973. Make DU pluggable. (Elliott Clark via cmccabe) 
(cherry picked from commit 35f0770555)
(cherry picked from commit 2b0b332e2f)

Conflicts:
	hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/datanode/fsdataset/impl/BlockPoolSlice.java
 2016-04-12 16:30:58 -07:00
Vinayakumar B b804b20843 HDFS-10271. Extra bytes are getting released from reservedSpace for append (Contributed by Brahma Reddy Battula) 
(cherry picked from commit a9a607f8fc)
(cherry picked from commit 04621537c9)
 2016-04-12 11:47:01 +05:30
Akira Ajisaka e2b8bdbe0a HDFS-10277. PositionedReadable test testReadFullyZeroByteFile failing in HDFS. Contributed by Steve Loughran. 
(cherry picked from commit a409508b3f)
(cherry picked from commit 8f968171bd)
 2016-04-12 14:55:42 +09:00
Chris Nauroth 1b5b85e560 HADOOP-12994. Specify PositionedReadable, add contract tests, fix problems. Contributed by Steve Loughran. 
(cherry picked from commit 843ee8d59d)
(cherry picked from commit 26a23eff82)

Conflicts:
	hadoop-hdfs-project/hadoop-hdfs-client/src/main/java/org/apache/hadoop/hdfs/DFSInputStream.java
 2016-04-08 13:44:27 -07:00
Tsz-Wo Nicholas Sze f872d519db HDFS-10186. DirectoryScanner: Improve logs by adding full path of both actual and expected block directories. Contributed by Rakesh R 2016-04-07 12:42:27 +08:00
Colin Patrick Mccabe 354801fe61 HDFS-10267. Extra "synchronized" on FsDatasetImpl#recoverAppend and FsDatasetImpl#recoverClose 
(cherry picked from commit 4bd7cbc29d)

Conflicts:
	hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/datanode/TestBlockRecovery.java

(cherry picked from commit b5e6ad457f)
 2016-04-06 21:11:48 -07:00
Eric Payne 497c65ad0c HDFS-9945. Datanode command for evicting writers. Contributed by Kihwal Lee 
(cherry picked from commit aede8c10ec)
 2016-04-06 20:33:40 +00:00
Colin Patrick Mccabe 8514ab8cb5 HDFS-6520. hdfs fsck passes invalid length value when creating BlockReader (Xiao Chen via cmccabe) 
(cherry picked from commit 188f65287d)
(cherry picked from commit 387d3f25c1)
 2016-04-06 11:38:19 -07:00
Akira Ajisaka e426d1bb3f HADOOP-12022. fix site -Pdocs -Pdist in hadoop-project-dist; cleanout remaining forrest bits. Contributed by Allen Wittenauer. 
(cherry picked from commit 1371f993a9)
 2016-04-06 17:44:54 +09:00
Kihwal Lee 8bb4655649 HDFS-10261. TestBookKeeperHACheckpoints doesn't handle ephemeral HTTP ports. Contributed by Eric Badger. 
(cherry picked from commit 9ba1e5af06)

Conflicts:
	hadoop-hdfs-project/hadoop-hdfs/src/contrib/bkjournal/src/test/java/org/apache/hadoop/contrib/bkjournal/TestBookKeeperHACheckpoints.java

(cherry picked from commit 6e37c5fe80)
 2016-04-05 16:54:46 -05:00
Ravi Prakash 7286c435c4 HDFS-10235. Last contact for Live Nodes should be relative time. Contributed by Brahma Reddy Battula. 
(cherry picked from commit 0cd320a846)
 2016-04-05 13:43:08 -07:00
Masatake Iwasaki 886273e123 HADOOP-12672. RPC timeout should not override IPC ping interval (iwasakims) 
(cherry picked from commit 85ec5573eb)
 2016-04-06 03:29:49 +09:00
Kihwal Lee f9764d073f HDFS-10239. Fsshell mv fails if port usage doesn't match in src and destination paths. Contributed by Kuhu Shukla. 
Conflicts:
	hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/TestDFSShell.java

(cherry picked from commit ef3da82357)
 2016-04-05 09:16:51 -05:00
Vinayakumar B 3deeb9a15f HDFS-9917. IBR accumulate more objects when SNN was down for sometime. (Contributed by Brahma Reddy Battula) 
(cherry picked from commit 818d6b799e)
(cherry picked from commit ad08114b92)
 2016-04-05 09:54:50 +08:00
Colin Patrick Mccabe 63fe2ae72c HDFS-8496. Calling stopWriter() with FSDatasetImpl lock held may block other threads (cmccabe) 
(cherry picked from commit f6b1a81812)

Conflicts:
	hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/datanode/TestBlockRecovery.java

(cherry picked from commit d55af8b5a3)
 2016-04-04 18:07:21 -07:00
Kihwal Lee 7d671cad3f HDFS-10178. Permanent write failures can happen if pipeline recoveries occur for the first packet. Contributed by Kihwal Lee. 
(cherry picked from commit a7d1fb0cd2)
 2016-04-04 16:41:23 -05:00
Masatake Iwasaki 97cda4077a HDFS-9599. TestDecommissioningStatus.testDecommissionStatus occasionally fails (Lin Yiqun via iwasakims) 
(cherry picked from commit 154d2532cf)
 2016-04-05 03:44:12 +09:00
Vinayakumar B bafcb584a8 HDFS-10253. Fix TestRefreshCallQueue failure (Contributed by Xiaoyu Yao) 
(cherry picked from commit 54b2e78fd2)
(cherry picked from commit aa38748997)
 2016-04-03 13:28:52 +05:30
Akira Ajisaka 3892efd3fb HDFS-10221. Add .json to the rat exclusions. Contributed by Ming Ma. 
(cherry picked from commit 32c0c3ecdf)
(cherry picked from commit 0795ee687b)
 2016-03-31 09:06:48 +09:00
Colin Patrick Mccabe 5cf55c5fb6 HDFS-10223. peerFromSocketAndKey performs SASL exchange before setting connection timeouts (cmccabe) 
(cherry picked from commit 37e23ce45c)

Conflicts:
	hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/datanode/erasurecode/ErasureCodingWorker.java

(cherry picked from commit be5894c12d)
 2016-03-30 13:45:05 -07:00
Vinayakumar B def143e1e1 HDFS-5177. blocksScheduled count should be decremented for abandoned blocks (Contributed by Vinayakumar B) 
(cherry picked from commit 09d63d5a19)
(cherry picked from commit 0852e44110)
 2016-03-30 14:23:56 +08:00
Arpit Agarwal bfe657e83a HDFS-10228. TestHDFSCLI fails. (Contributed by Akira AJISAKA) 2016-03-29 13:56:12 -07:00
Andrew Wang 026b84bf70 HDFS-10197. TestFsDatasetCache failing intermittently due to timeout. Contributed by Lin Yiqun. 
(cherry picked from commit f2aec4eb82)
(cherry picked from commit 1d79b57c14)
 2016-03-29 11:55:04 -07:00
Vinayakumar B 88b56cdfaa HDFS-9871. "Bytes Being Moved" -ve(-1 B) when cluster was already balanced. (Contributed by Brahma Reddy Battulla) 
(cherry picked from commit 1f004b3367)
(cherry picked from commit f74364e8b9)
 2016-03-29 11:31:18 +08:00
Walter Su d71843558b HDFS-10182. Hedged read might overwrite user's buf. Contributed by zhouyingchao. 
(cherry picked from commit d8383c687c)
 2016-03-28 15:45:40 +08:00
Andrew Wang d4aeddee31 HDFS-10200. Docs for WebHDFS still describe GETDELEGATIONTOKENS operation. Contributed by Wellington Chevreuil. 
(cherry picked from commit 8f85e5d212)
(cherry picked from commit c658deabb6)
 2016-03-23 12:56:17 -07:00
Colin Patrick Mccabe 4ca13e7b98 HDFS-10193. fuse_dfs segfaults if uid cannot be resolved to a username (John Thiltges via cmccabe) 
(cherry picked from commit 0d19a0ce98)
(cherry picked from commit 7ad3162df4)
 2016-03-23 11:20:12 -07:00