Apache
/
hadoop
1
0
Fork 0
Code Issues Pull Requests Packages Projects Releases Wiki Activity
11,121 Commits 326 Branches 371 Tags 1.5 GiB
Commit Graph

3866 Commits

Author SHA1 Message Date
Colin Patrick Mccabe 5f04795aed HADOOP-13065. Add a new interface for retrieving FS and FC Statistics (Mingliang Liu via cmccabe) 
(cherry picked from commit 687233f20d)

Conflicts:
	hadoop-hdfs-project/hadoop-hdfs-client/src/main/java/org/apache/hadoop/hdfs/web/WebHdfsFileSystem.java
	hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/TestDistributedFileSystem.java

(cherry picked from commit 15bfcde238)

Conflicts:
	hadoop-hdfs-project/hadoop-hdfs-client/src/main/java/org/apache/hadoop/hdfs/DistributedFileSystem.java
	hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/TestDistributedFileSystem.java
 2016-05-11 14:24:51 -07:00
Colin Patrick Mccabe 154332bb54 HDFS-10377. CacheReplicationMonitor shutdown log message should use INFO level (Yiqun Lin via cmccabe) 
(cherry picked from commit f0252ad252)
(cherry picked from commit 43f32d3f4a)
 2016-05-10 11:21:16 -07:00
Akira Ajisaka a84c4d7d4b HDFS-6187. Update the document of hftp/hsftp in branch-2 to mention that they are deprecated. Contributed by Gergely Novák. 
(cherry picked from commit 901588542b)
 2016-05-10 09:33:55 -07:00
Akira Ajisaka 1622fc1373 Revert "HDFS-6187. Update the document of hftp/hsftp in branch-2 to mention that they are deprecated." 
This reverts commit f25dcb935e.
 2016-05-10 09:33:18 -07:00
Akira Ajisaka f25dcb935e HDFS-6187. Update the document of hftp/hsftp in branch-2 to mention that they are deprecated. 
(cherry picked from commit c297df5995)
 2016-05-10 09:29:47 -07:00
Kihwal Lee d7cda6d031 HDFS-10372. Fix for failing TestFsDatasetImpl#testCleanShutdownOfVolume. Contributed by Rushabh Shah. 
(cherry picked from commit b9e5a32fa1)
 2016-05-09 15:52:12 -05:00
Masatake Iwasaki c80b47e73f HDFS-2043. TestHFlush failing intermittently. Contributed by Lin Yiqun. 
(cherry picked from commit 2835f146b5)
 2016-05-06 15:50:48 +09:00
Xiaoyu Yao 59eeee7501 HDFS-10324. Trash directory in an encryption zone should be pre-created with correct permissions. Contributed by Wei-Chiu Chuang. 
(cherry picked from commit dacd1f50fe)
(cherry picked from commit 35859872b7)
 2016-05-05 18:23:00 -07:00
Ming Ma d6e95ae47b HDFS-10320. Rack failures may result in NN terminate. (Xiao Chen via mingma) 
(cherry picked from commit 1268cf5fbe)
 2016-05-04 17:07:36 -07:00
Arpit Agarwal ce94203566 HDFS-9902. Support different values of dfs.datanode.du.reserved per storage type. (Contributed by Brahma Reddy Battula) 2016-05-03 16:53:07 -07:00
Xiaoyu Yao 5dfcfb2d2d HDFS-10344. DistributedFileSystem#getTrashRoots should skip encryption zone that does not have .Trash. Contributed by Xiaoyu Yao. 
(cherry picked from commit 45a753ccf7)
 2016-05-02 19:50:10 -07:00
Xiaoyu Yao 643d2fd3c1 HADOOP-12985. Support MetricsSource interface for DecayRpcScheduler Metrics. Contributed by Xiaoyu Yao. 
(cherry picked from commit 5bd7b592e5)
(cherry picked from commit a2a65922dd)
 2016-05-02 19:38:03 -07:00
Harsh J cf6efccebb HDFS-2580. NameNode#main(...) can make use of GenericOptionsParser. Contributed by harsh. (harsh) 
(cherry picked from commit 54e612bfb9)
 2016-05-02 17:53:10 -07:00
Tsz-Wo Nicholas Sze a1ba6eee38 HADOOP-12957. Limit the number of outstanding async calls. Contributed by Xiaobing Zhou 2016-05-02 11:34:59 -07:00
Tsz-Wo Nicholas Sze bdc45bef64 HDFS-10224. Implement asynchronous rename for DistributedFileSystem. Contributed by Xiaobing Zhou 2016-05-02 11:34:22 -07:00
Lei Xu 5477c31c5c HDFS-3702. Add an option for NOT writing the blocks locally if there is a datanode on the same box as the client. (Contributed by Lei (Eddy) Xu) 
(cherry picked from commit 0a152103f1)
(cherry picked from commit 4289cb8b36)
 2016-05-02 10:21:59 -07:00
Kihwal Lee e9cd88d484 HDFS-10347. Namenode report bad block method doesn't log the bad block or datanode. Contributed by Rushabh Shah. 
(cherry picked from commit 7da540d03e)
 2016-04-29 15:58:45 -05:00
Tsz-Wo Nicholas Sze 31ed34c982 HDFS-10335 Mover$Processor#chooseTarget() always chooses the first matching target storage group. Contributed by Mingliang Liu 2016-04-29 11:27:14 -07:00
Kihwal Lee a4db3729be HDFS-10260. TestFsDatasetImpl#testCleanShutdownOfVolume often fails. Contributed by Rushabh Shah. 
(cherry picked from commit af9b000535)
 2016-04-29 10:05:58 -05:00
Andrew Wang bdb0e98483 HDFS-10297. Increase default balance bandwidth and concurrent moves. Contributed by John Zhuge. 2016-04-27 17:03:36 -07:00
Kihwal Lee dd7c9f5b8f HDFS-10330. Add Corrupt Blocks Information in Metasave output. Contributed by Kuhu Shukla. 
(cherry picked from commit e181092b86)
 2016-04-27 08:30:29 -05:00
Kihwal Lee 621be60592 HDFS-10329. Bad initialisation of StringBuffer in RequestHedgingProxyProvider. Contributed by Lin Yiqun. 
(cherry picked from commit c6a2430b47)
 2016-04-26 11:47:15 -05:00
Kihwal Lee 973c73aa68 HDFS-10318. TestJMXGet hides the real error in case of test failure. Contributed by Andras Bokor. 
(cherry picked from commit 10f0f7851a)
 2016-04-25 11:42:17 -05:00
Masatake Iwasaki c4d8090b72 HDFS-9905. WebHdfsFileSystem#runWithRetry should display original stack trace on error. (Wei-Chiu Chuang via iwasakims) 
(cherry picked from commit 6fcde2e38d)
 2016-04-23 23:55:27 +09:00
Kihwal Lee 3d3ed9b68e HDFS-9555. LazyPersistFileScrubber should still sleep if there are errors in the clear progress. Contributed by Phil Yang. 
(cherry picked from commit 5d4255a801)
 2016-04-22 11:10:36 -05:00
Tsz-Wo Nicholas Sze 60c68c65c1 HDFS-10309 Balancer doesn't honor dfs.blocksize value defined with suffix k(kilo), m(mega), g(giga). Contributed by Amit Anand 2016-04-21 15:45:36 -07:00
Tsz-Wo Nicholas Sze 704c57e3eb HDFS-10319. Balancer should not try to pair storages with different types. 2016-04-21 15:25:49 -07:00
Chris Nauroth 7fb1cfc591 HDFS-10312. Large block reports may fail to decode at NameNode due to 64 MB protobuf maximum length restriction. Contributed by Chris Nauroth. 
(cherry picked from commit 63ac2db59a)
(cherry picked from commit cc56bed686)

Conflicts:
	hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/datanode/fsdataset/impl/BlockPoolSlice.java
	hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/datanode/fsdataset/impl/FsDatasetImplTestUtils.java
 2016-04-20 14:00:06 -07:00
Colin Patrick Mccabe e2968f5da2 HDFS-10308. TestRetryCacheWithHA#testRetryCacheOnStandbyNN failing (Rakesh R via cmccabe) 
(cherry picked from commit ad36fa6f42)
(cherry picked from commit 76d963acb4)
 2016-04-20 11:52:49 -07:00
Arpit Agarwal a13628fe4a HDFS-10264. Logging improvements in FSImageFormatProtobuf.Saver. (Contributed by Xiaobing Zhou) 2016-04-19 11:27:46 -07:00
Walter Su 1860b8ceb5 HDFS-9744. TestDirectoryScanner#testThrottling occasionally time out after 300 seconds. Contributed by Lin Yiqun. 
(cherry picked from commit 90d1af1416)
 2016-04-19 18:56:23 +08:00
Colin Patrick Mccabe e1d9e5ab82 HDFS-10265. OEV tool fails to read edit xml file if OP_UPDATE_BLOCKS has no BLOCK tag (Wan Chang via cmccabe) 
(cherry picked from commit cb3ca460ef)
(cherry picked from commit a69b6b1e8b)
 2016-04-18 13:49:05 -07:00
Kihwal Lee 7ac2e5ec7b HDFS-10302. BlockPlacementPolicyDefault should use default replication considerload value. Contributed by Lin Yiqun. 
(cherry picked from commit d8b729e16f)
 2016-04-18 08:01:17 -05:00
Walter Su 09ffc65d94 HDFS-10275. TestDataNodeMetrics failing intermittently due to TotalWriteTime counted incorrectly. Contributed by Lin Yiqun. 
(cherry picked from commit ab903029a9)
 2016-04-18 20:32:36 +08:00
Steve Loughran e9699aa443 HDFS-10291 TestShortCircuitLocalRead failing (stevel) 2016-04-18 10:26:35 +01:00
Walter Su 7f47d984fa HDFS-9412. getBlocks occupies FSLock and takes too long to complete. Contributed by He Tianyi. 
(cherry picked from commit 67523ffcf4)

 Conflicts:
	hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/TestGetBlocks.java

(cherry picked from commit a6d0e9e56a)
 2016-04-18 09:39:46 +08:00
Kihwal Lee 8b1e7842e3 HDFS-10281. TestPendingCorruptDnMessages fails intermittently. Contributed by Mingliang Liu. 
(cherry picked from commit b9c9d03591)
 2016-04-14 15:30:59 -05:00
Kihwal Lee 019ca8f7c1 HDFS-10280. Document new dfsadmin command -evictWriters. Contributed by Wei-Chiu Chuang. 
(cherry picked from commit c970f1d005)

Conflicts:
	hadoop-hdfs-project/hadoop-hdfs/src/site/markdown/HDFSCommands.md
 2016-04-14 12:50:36 -05:00
Andrew Wang 1dee3e1200 HDFS-10279. Improve validation of the configured number of tolerated failed volumes. Contributed by Lin Yiqun. 
(cherry picked from commit 314aa21a89)
(cherry picked from commit 4c18f54c1d)
 2016-04-13 16:40:08 -07:00
Kihwal Lee ccaf746eea HDFS-10270. TestJMXGet:testNameNode() fails. Contributed by Gergely Novák 
(cherry picked from commit d2f3bbc290)
 2016-04-13 11:24:45 -05:00
Walter Su f3dbabb13a HDFS-9772. TestBlockReplacement#testThrottler doesn't work as expected. Contributed by Lin Yiqun. 
(cherry picked from commit 903428bf94)
 2016-04-13 17:50:47 +08:00
Colin Patrick Mccabe 59b0661c7f HADOOP-12973. Make DU pluggable. (Elliott Clark via cmccabe) 
(cherry picked from commit 35f0770555)
(cherry picked from commit 2b0b332e2f)

Conflicts:
	hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/datanode/fsdataset/impl/BlockPoolSlice.java
 2016-04-12 16:30:58 -07:00
Vinayakumar B b804b20843 HDFS-10271. Extra bytes are getting released from reservedSpace for append (Contributed by Brahma Reddy Battula) 
(cherry picked from commit a9a607f8fc)
(cherry picked from commit 04621537c9)
 2016-04-12 11:47:01 +05:30
Tsz-Wo Nicholas Sze f872d519db HDFS-10186. DirectoryScanner: Improve logs by adding full path of both actual and expected block directories. Contributed by Rakesh R 2016-04-07 12:42:27 +08:00
Colin Patrick Mccabe 354801fe61 HDFS-10267. Extra "synchronized" on FsDatasetImpl#recoverAppend and FsDatasetImpl#recoverClose 
(cherry picked from commit 4bd7cbc29d)

Conflicts:
	hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/datanode/TestBlockRecovery.java

(cherry picked from commit b5e6ad457f)
 2016-04-06 21:11:48 -07:00
Eric Payne 497c65ad0c HDFS-9945. Datanode command for evicting writers. Contributed by Kihwal Lee 
(cherry picked from commit aede8c10ec)
 2016-04-06 20:33:40 +00:00
Colin Patrick Mccabe 8514ab8cb5 HDFS-6520. hdfs fsck passes invalid length value when creating BlockReader (Xiao Chen via cmccabe) 
(cherry picked from commit 188f65287d)
(cherry picked from commit 387d3f25c1)
 2016-04-06 11:38:19 -07:00
Akira Ajisaka e426d1bb3f HADOOP-12022. fix site -Pdocs -Pdist in hadoop-project-dist; cleanout remaining forrest bits. Contributed by Allen Wittenauer. 
(cherry picked from commit 1371f993a9)
 2016-04-06 17:44:54 +09:00
Kihwal Lee 8bb4655649 HDFS-10261. TestBookKeeperHACheckpoints doesn't handle ephemeral HTTP ports. Contributed by Eric Badger. 
(cherry picked from commit 9ba1e5af06)

Conflicts:
	hadoop-hdfs-project/hadoop-hdfs/src/contrib/bkjournal/src/test/java/org/apache/hadoop/contrib/bkjournal/TestBookKeeperHACheckpoints.java

(cherry picked from commit 6e37c5fe80)
 2016-04-05 16:54:46 -05:00
Ravi Prakash 7286c435c4 HDFS-10235. Last contact for Live Nodes should be relative time. Contributed by Brahma Reddy Battula. 
(cherry picked from commit 0cd320a846)
 2016-04-05 13:43:08 -07:00