Apache/hadoop
1
0
Fork 0
Code Issues Pull Requests Packages Projects Releases Wiki Activity
11,975 Commits 326 Branches 371 Tags
Commit Graph

4100 Commits

Author SHA1 Message Date
Tsz-Wo Nicholas Sze
a96ea528cc HDFS-10309 Balancer doesn't honor dfs.blocksize value defined with suffix k(kilo), m(mega), g(giga). Contributed by Amit Anand 2016-04-21 15:44:01 -07:00
Tsz-Wo Nicholas Sze
5245254a46 HDFS-10319. Balancer should not try to pair storages with different types. 2016-04-21 15:21:45 -07:00
Xiaoyu Yao
432531dcf7 HDFS-10207. Support enable Hadoop IPC backoff without namenode restart. Contributed by Xiaobing Zhou. 
(cherry picked from commit b4be288c5d6801988f555a566c2eb793c88a15a4)
 2016-04-21 10:19:35 -07:00
Xiaoyu Yao
a2a65922dd HADOOP-12985. Support MetricsSource interface for DecayRpcScheduler Metrics. Contributed by Xiaoyu Yao. 
(cherry picked from commit 5bd7b592e5fbe4d448fd127c15d29f3121b8a833)
 2016-04-20 23:11:42 -07:00
Chris Nauroth
cc56bed686 HDFS-10312. Large block reports may fail to decode at NameNode due to 64 MB protobuf maximum length restriction. Contributed by Chris Nauroth. 
(cherry picked from commit 63ac2db59af2b50e74dc892cae1dbc4d2e061423)
 2016-04-20 13:55:03 -07:00
Colin Patrick Mccabe
76d963acb4 HDFS-10308. TestRetryCacheWithHA#testRetryCacheOnStandbyNN failing (Rakesh R via cmccabe) 
(cherry picked from commit ad36fa6f42870c517526618a30204b443bfc6b5a)
 2016-04-20 11:52:26 -07:00
Arpit Agarwal
b776db36f7 HDFS-10264. Logging improvements in FSImageFormatProtobuf.Saver. (Contributed by Xiaobing Zhou) 2016-04-19 11:27:36 -07:00
Walter Su
09a508beee HDFS-9744. TestDirectoryScanner#testThrottling occasionally time out after 300 seconds. Contributed by Lin Yiqun. 
(cherry picked from commit 90d1af141666ba12f18d44464cb6169c445d0902)
 2016-04-19 18:56:08 +08:00
Walter Su
d31a0d7443 HDFS-10284. o.a.h.hdfs.server.blockmanagement.TestBlockManagerSafeMode.testCheckSafeMode fails intermittently. Contributed by Mingliang Liu. 
(cherry picked from commit 4b4df2cd48649b2d9cbdd97907c8d5f813cacdb1)
 2016-04-19 18:08:52 +08:00
Jing Zhao
bf4403e034 HDFS-10306. SafeModeMonitor should not leave safe mode if name system is starting active service. Contributed by Mingliang Liu. 
(cherry picked from commit be0bce1b7171c49e2dca22f56d4e750e606862fc)
 2016-04-18 18:06:10 -07:00
Colin Patrick Mccabe
a69b6b1e8b HDFS-10265. OEV tool fails to read edit xml file if OP_UPDATE_BLOCKS has no BLOCK tag (Wan Chang via cmccabe) 
(cherry picked from commit cb3ca460efb97be8c031bdb14bb7705cc25f2117)
 2016-04-18 13:48:24 -07:00
Kihwal Lee
aa846bd492 HDFS-10302. BlockPlacementPolicyDefault should use default replication considerload value. Contributed by Lin Yiqun. 
(cherry picked from commit d8b729e16fb253e6c84f414d419b5663d9219a43)
 2016-04-18 08:00:44 -05:00
Walter Su
a06ee5a71d HDFS-10275. TestDataNodeMetrics failing intermittently due to TotalWriteTime counted incorrectly. Contributed by Lin Yiqun. 
(cherry picked from commit ab903029a9d353677184ff5602966b11ffb408b9)
 2016-04-18 20:31:55 +08:00
Steve Loughran
dd64769699 HDFS-10291 TestShortCircuitLocalRead failing (stevel) 2016-04-18 10:26:58 +01:00
Walter Su
a6d0e9e56a HDFS-9412. getBlocks occupies FSLock and takes too long to complete. Contributed by He Tianyi. 
(cherry picked from commit 67523ffcf491f4f2db5335899c00a174d0caaa9b)

 Conflicts:
	hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/TestGetBlocks.java
 2016-04-18 09:34:01 +08:00
Jing Zhao
eba7ecc993 HDFS-10283. o.a.h.hdfs.server.namenode.TestFSImageWithSnapshot#testSaveLoadImageWithAppending fails intermittently. Contributed by Mingliang Liu. 
(cherry picked from commit 89a838769ff5b6c64565e6949b14d7fed05daf54)
 2016-04-15 10:50:58 -07:00
Kihwal Lee
9b3f1139ba HDFS-10281. TestPendingCorruptDnMessages fails intermittently. Contributed by Mingliang Liu. 
(cherry picked from commit b9c9d03591a49be31f3fbc738d01a31700bfdbc4)
 2016-04-14 15:27:40 -05:00
Xiaoyu Yao
79f7c03fe1 HDFS-10286. Fix TestDFSAdmin#testNameNodeGetReconfigurableProperties. Contributed by Xiaobing Zhou. 
(cherry picked from commit 809226752dd109e16956038017dece16ada6ee0f)
 2016-04-14 10:58:57 -07:00
Kihwal Lee
5e82431374 HDFS-10280. Document new dfsadmin command -evictWriters. Contributed by Wei-Chiu Chuang. 
(cherry picked from commit c970f1d00525e4273075cff7406dcbd71305abd5)
 2016-04-14 12:47:25 -05:00
Kihwal Lee
51fc96f5cf HDFS-10282. The VolumeScanner should warn about replica files which are misplaced. Contributed by Colin Patrick McCabe. 
(cherry picked from commit 0d1c1152f1ce2706f92109bfbdff0d62e98e6797)
 2016-04-14 07:59:41 -05:00
Xiaoyu Yao
1f75e62b6d HDFS-10209. Support enable caller context in HDFS namenode audit log without restart namenode. Contributed by Xiaobing Zhou. 
(cherry picked from commit 5566177c9af913baf380811dbbb1fa7e70235491)
 2016-04-13 16:52:22 -07:00
Andrew Wang
4c18f54c1d HDFS-10279. Improve validation of the configured number of tolerated failed volumes. Contributed by Lin Yiqun. 
(cherry picked from commit 314aa21a89134fac68ac3cb95efdeb56bd3d7b05)
 2016-04-13 16:39:59 -07:00
Kihwal Lee
4a7e52d302 HDFS-10270. TestJMXGet:testNameNode() fails. Contributed by Gergely Novák 
(cherry picked from commit d2f3bbc29046435904ad9418073795439c71b441)
 2016-04-13 11:24:06 -05:00
Walter Su
c195b1c9fe HDFS-9772. TestBlockReplacement#testThrottler doesn't work as expected. Contributed by Lin Yiqun. 
(cherry picked from commit 903428bf946827b4d58c7c577ed0c574a7cff029)
 2016-04-13 17:50:30 +08:00
Colin Patrick Mccabe
2b0b332e2f HADOOP-12973. Make DU pluggable. (Elliott Clark via cmccabe) 
(cherry picked from commit 35f07705552ef5636f8b8d2599a6af5ec1426203)
 2016-04-12 16:29:15 -07:00
Colin Patrick Mccabe
6e37e40dd2 HDFS-10273. Remove duplicate logSync() and log message in FSN#enterSafemode() (Vinayakumar B via cmccabe) 
(cherry picked from commit 600d129bb8d52ae820e7b74b8ce363aabd69d25c)
 2016-04-12 10:55:43 -07:00
Akira Ajisaka
bb1ed426d1 HDFS-8356. Document missing properties in hdfs-default.xml. Contributed by Ray Chiang. 2016-04-12 15:28:05 +09:00
Vinayakumar B
04621537c9 HDFS-10271. Extra bytes are getting released from reservedSpace for append (Contributed by Brahma Reddy Battula) 
(cherry picked from commit a9a607f8fc0d996af3fb37f7efa7591d6655900d)
 2016-04-12 11:45:44 +05:30
Tsz-Wo Nicholas Sze
bccc28914b HDFS-10186. DirectoryScanner: Improve logs by adding full path of both actual and expected block directories. Contributed by Rakesh R 2016-04-07 12:39:26 +08:00
Colin Patrick Mccabe
b5e6ad457f HDFS-10267. Extra "synchronized" on FsDatasetImpl#recoverAppend and FsDatasetImpl#recoverClose 
(cherry picked from commit 4bd7cbc29d142fc56324156333b9a8a7d7b68042)

Conflicts:
	hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/datanode/TestBlockRecovery.java
 2016-04-06 21:08:42 -07:00
Eric Payne
caef78afe6 HDFS-9945. Datanode command for evicting writers. Contributed by Kihwal Lee 
(cherry picked from commit aede8c10ecad4f2a8802a834e4bd0b8286cebade)
 2016-04-06 20:31:20 +00:00
Colin Patrick Mccabe
387d3f25c1 HDFS-6520. hdfs fsck passes invalid length value when creating BlockReader (Xiao Chen via cmccabe) 
(cherry picked from commit 188f65287d5b2f26a8862c88198f83ac59035016)
 2016-04-06 11:37:57 -07:00
Jing Zhao
7cac807eb3 HDFS-10192. Namenode safemode not coming out during failover. Contributed by Brahma Reddy Battula. 
(cherry picked from commit 221b3a8722f84f8e9ad0a98eea38a12cc4ad2f24)
 2016-04-06 10:43:38 -07:00
Akira Ajisaka
1371f993a9 HADOOP-12022. fix site -Pdocs -Pdist in hadoop-project-dist; cleanout remaining forrest bits. Contributed by Allen Wittenauer. 2016-04-06 17:43:31 +09:00
Kihwal Lee
6e37c5fe80 HDFS-10261. TestBookKeeperHACheckpoints doesn't handle ephemeral HTTP ports. Contributed by Eric Badger. 
(cherry picked from commit 9ba1e5af06070ba01dcf46e1a4c66713a1d43352)

Conflicts:
	hadoop-hdfs-project/hadoop-hdfs/src/contrib/bkjournal/src/test/java/org/apache/hadoop/contrib/bkjournal/TestBookKeeperHACheckpoints.java
 2016-04-05 16:49:59 -05:00
Ravi Prakash
2ca6251a6a HDFS-10235. Last contact for Live Nodes should be relative time. Contributed by Brahma Reddy Battula. 
(cherry picked from commit 0cd320a8463efe19a6228f9fe14693aa37ac8a10)
 2016-04-05 13:43:42 -07:00
Kihwal Lee
ef3da82357 HDFS-10239. Fsshell mv fails if port usage doesn't match in src and destination paths. Contributed by Kuhu Shukla. 
Conflicts:
	hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/TestDFSShell.java
 2016-04-05 09:16:05 -05:00
Vinayakumar B
ad08114b92 HDFS-9917. IBR accumulate more objects when SNN was down for sometime. (Contributed by Brahma Reddy Battula) 
(cherry picked from commit 818d6b799eead13a17a0214172df60a269b046fb)
 2016-04-05 09:53:57 +08:00
Colin Patrick Mccabe
d55af8b5a3 HDFS-8496. Calling stopWriter() with FSDatasetImpl lock held may block other threads (cmccabe) 
(cherry picked from commit f6b1a818124cc42688c4c5acaf537d96cf00e43b)

Conflicts:
	hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/datanode/TestBlockRecovery.java
 2016-04-04 18:07:01 -07:00
Kihwal Lee
9e91433295 HDFS-10178. Permanent write failures can happen if pipeline recoveries occur for the first packet. Contributed by Kihwal Lee. 
(cherry picked from commit a7d1fb0cd2fdbf830602eb4dbbd9bbe62f4d5584)
 2016-04-04 16:40:44 -05:00
Masatake Iwasaki
c981efb0f6 HDFS-9599. TestDecommissioningStatus.testDecommissionStatus occasionally fails (Lin Yiqun via iwasakims) 
(cherry picked from commit 154d2532cf015e9ab9141864bd3ab0d6100ef597)
 2016-04-05 03:39:10 +09:00
Vinayakumar B
aa38748997 HDFS-10253. Fix TestRefreshCallQueue failure (Contributed by Xiaoyu Yao) 
(cherry picked from commit 54b2e78fd28c9def42bec7f0418833bad352686c)
 2016-04-03 13:28:22 +05:30
Akira Ajisaka
0795ee687b HDFS-10221. Add .json to the rat exclusions. Contributed by Ming Ma. 
(cherry picked from commit 32c0c3ecdf72e89a63f4aee5e75d1c5a12714b89)
 2016-03-31 09:06:34 +09:00
Colin Patrick Mccabe
be5894c12d HDFS-10223. peerFromSocketAndKey performs SASL exchange before setting connection timeouts (cmccabe) 
(cherry picked from commit 37e23ce45c592f3c9c48a08a52a5f46787f6c0e9)

Conflicts:
	hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/datanode/erasurecode/ErasureCodingWorker.java
 2016-03-30 13:42:24 -07:00
Vinayakumar B
0852e44110 HDFS-5177. blocksScheduled count should be decremented for abandoned blocks (Contributed by Vinayakumar B) 
(cherry picked from commit 09d63d5a192b5d6b172f94ff6c94da348fd49ea6)
 2016-03-30 14:23:11 +08:00
Arpit Agarwal
ed15cb9a30 HDFS-9439. Support reconfiguring fs.protected.directories without NN restart. (Contributed by Xiaobing Zhou) 2016-03-29 13:56:02 -07:00
Arpit Agarwal
8ee2140706 HDFS-10228. TestHDFSCLI fails. (Contributed by Akira AJISAKA) 2016-03-29 13:56:00 -07:00
Andrew Wang
1d79b57c14 HDFS-10197. TestFsDatasetCache failing intermittently due to timeout. Contributed by Lin Yiqun. 
(cherry picked from commit f2aec4eb824647a01e14b4eede03af0babe65fb6)
 2016-03-29 11:54:52 -07:00
Vinayakumar B
f74364e8b9 HDFS-9871. "Bytes Being Moved" -ve(-1 B) when cluster was already balanced. (Contributed by Brahma Reddy Battulla) 
(cherry picked from commit 1f004b3367c57de9e8a67040a57efc31c9ba8ee2)
 2016-03-29 11:28:10 +08:00
Lei Xu
ae98314980 Add missing files from HDFS-9005. (lei) 2016-03-25 18:13:01 -07:00