Apache/hadoop
1
0
Fork 0
Code Issues Pull Requests Packages Projects Releases Wiki Activity
9,259 Commits 326 Branches 371 Tags
Commit Graph

3081 Commits

Author SHA1 Message Date
Masatake Iwasaki
dd8aba3fad HDFS-2043. TestHFlush failing intermittently. Contributed by Lin Yiqun. 2016-05-06 15:57:06 +09:00
Harsh J
d490cfc42c HDFS-2580. NameNode#main(...) can make use of GenericOptionsParser. Contributed by harsh. (harsh) 
(cherry picked from commit 54e612bfb9f877e58f7f153c43cb4147876826d3)
 2016-05-02 17:52:14 -07:00
Kihwal Lee
6347f8f4d4 HDFS-10347. Namenode report bad block method doesn't log the bad block or datanode. Contributed by Rushabh Shah. 
(cherry picked from commit 7da540d03eccc2f97950bf47e8b35ce8c889d1e0)
Updated CHANGES.txt
 2016-04-29 16:07:20 -05:00
Tsz-Wo Nicholas Sze
c927f938fe HDFS-10335 Mover$Processor#chooseTarget() always chooses the first matching target storage group. Contributed by Mingliang Liu 2016-04-29 11:28:23 -07:00
Kihwal Lee
a3ece8b5b4 HDFS-9958. BlockManager#createLocatedBlocks can throw NPE for corruptBlocks on failed storages. Contributed by Kuhu Shukla. 2016-04-28 16:47:04 -05:00
Akira Ajisaka
076215024b HDFS-10245. Fix the findbugs warnings in branch-2.7. Contributed by Brahma Reddy Battula. 2016-04-25 22:03:48 +09:00
Kihwal Lee
9e3b91b141 HDFS-9555. LazyPersistFileScrubber should still sleep if there are errors in the clear progress. Contributed by Phil Yang. 
(cherry picked from commit 5d4255a80156d2cacfea8184b41805070223d3a9)

Conflicts:
	hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/FSNamesystem.java
 2016-04-22 11:47:18 -05:00
Tsz-Wo Nicholas Sze
e7f1b8fcb3 HDFS-10319. Balancer should not try to pair storages with different types. 2016-04-21 15:36:28 -07:00
Arpit Agarwal
12c6c2c9a6 HDFS-10264. Logging improvements in FSImageFormatProtobuf.Saver. (Contributed by Xiaobing Zhou) 2016-04-19 11:28:01 -07:00
Walter Su
91ca90fdd1 HDFS-10275. TestDataNodeMetrics failing intermittently due to TotalWriteTime counted incorrectly. Contributed by Lin Yiqun. 
(cherry picked from commit ab903029a9d353677184ff5602966b11ffb408b9)
 2016-04-18 20:30:48 +08:00
Walter Su
2ee244ac28 HDFS-9772. TestBlockReplacement#testThrottler doesn't work as expected. Contributed by Lin Yiqun. 
(cherry picked from commit 903428bf946827b4d58c7c577ed0c574a7cff029)
 2016-04-13 17:51:01 +08:00
Vinayakumar B
b52ac3cb4d Addendum: HDFS-10271. 
Added CHANGES.txt entry
 2016-04-12 11:57:00 +05:30
Vinayakumar B
37e4e45390 HDFS-10271. Extra bytes are getting released from reservedSpace for append (Contributed by Brahma Reddy Battula) 2016-04-12 11:32:28 +05:30
Tsz-Wo Nicholas Sze
262d769b4d HDFS-10186. DirectoryScanner: Improve logs by adding full path of both actual and expected block directories. Contributed by Rakesh R 2016-04-07 12:43:09 +08:00
Kihwal Lee
8e784afb15 Revert "HDFS-8791. block ID-based DN storage layout can be very slow for datanode on ext4. Contributed by Chris Trezzo." 
This reverts commit 9bc9e13a979c775354b2394d926f466a0d5be514.

Conflicts:
	hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt
 2016-04-06 17:35:54 -05:00
Kihwal Lee
dd701c9800 HDFS-10261. TestBookKeeperHACheckpoints doesn't handle ephemeral HTTP ports. Contributed by Eric Badger. 
(cherry picked from commit 9ba1e5af06070ba01dcf46e1a4c66713a1d43352)

Conflicts:
	hadoop-hdfs-project/hadoop-hdfs/src/contrib/bkjournal/src/test/java/org/apache/hadoop/contrib/bkjournal/TestBookKeeperHACheckpoints.java

(cherry picked from commit 6e37c5fe80ee9ce46afc26838391cc27ca6b1e6f)
 2016-04-05 17:00:42 -05:00
Kihwal Lee
9608601330 HDFS-10239. Fsshell mv fails if port usage doesn't match in src and destination paths. Contributed by Kuhu Shukla. 
Conflicts:
	hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/TestDFSShell.java

(cherry picked from commit ef3da823573cbf16fd1d84479330dd457f95e0ff)
 2016-04-05 09:20:38 -05:00
Vinayakumar B
fc0ffc48d4 HDFS-9917. IBR accumulate more objects when SNN was down for sometime. (Contributed by Brahma Reddy Battula) 2016-04-05 09:59:37 +08:00
Kihwal Lee
69d4fa0deb HDFS-10178. Permanent write failures can happen if pipeline recoveries occur for the first packet. Contributed by Kihwal Lee. 
(cherry picked from commit a7d1fb0cd2fdbf830602eb4dbbd9bbe62f4d5584)
TestClientProtocolForPipelineRecovery modified to use DFSConfigKeys instead of HdfsClientConfigKeys.
 2016-04-04 16:54:04 -05:00
Kihwal Lee
efb210cc30 Fix CHANGES.txt 2016-03-28 11:31:07 -05:00
Walter Su
ec459e3966 HDFS-10182. Hedged read might overwrite user's buf. Contributed by zhouyingchao. 
(cherry picked from commit d8383c687c95dbb37effa307ab2d41497da1cfc2)
 2016-03-28 15:46:15 +08:00
Aaron T. Myers
17dbf82f54 HDFS-8101. DFSClient use of non-constant DFSConfigKeys pulls in WebHDFS classes at runtime. Contributed by Sean Busbey. 
(cherry picked from commit 3fe61e0bb0d025a6acbb754027f73f3084b2f4d1)
 2016-03-26 17:54:09 -07:00
Kihwal Lee
a66d2a2a85 HDFS-9874. Long living DataXceiver threads cause volume shutdown to block. Contributed by Rushabh Shah. 
(cherry picked from commit 63c966a3fbeb675959fc4101e65de9f57aecd17d)
(cherry picked from commit 242c7f1fee664b4d609a6c72e899f10816430f65)

Conflicts:
	hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/datanode/fsdataset/impl/FsDatasetImpl.java
	hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/datanode/fsdataset/impl/TestFsDatasetImpl.java
 2016-03-18 10:38:33 -05:00
Akira Ajisaka
6902789a92 HDFS-9860. Backport HDFS-9638 to branch-2.7. Contributed by Wei-Chiu Chuang. 2016-03-17 02:59:51 +09:00
Kihwal Lee
b8e73cff8f HDFS-9904. testCheckpointCancellationDuringUpload occasionally fails. Contributed by Lin Yiqun. 
(cherry picked from commit d4574017845cfa7521e703f80efd404afd09b8c4)
 2016-03-15 10:56:46 -05:00
Akira Ajisaka
7f43eb9547 HDFS-9812. Streamer threads leak if failure happens when closing DFSOutputStream. Contributed by Lin Yiqun. 
(cherry picked from commit 352d299cf8ebe330d24117df98d1e6a64ae38c26)
(cherry picked from commit fe0009a2bd17e26f9e9364ec3f772e9a3f138de8)
 2016-03-08 11:15:27 +09:00
Masatake Iwasaki
e768f8b771 HDFS-9865. TestBlockReplacement fails intermittently in trunk (Lin Yiqun via iwasakims) 2016-03-08 02:13:50 +09:00
Masatake Iwasaki
08fc048018 HDFS-9048. DistCp documentation is out-of-dated (Daisuke Kobayashi via iwasakims) 
(cherry picked from commit 33a412e8a4ab729d588a9576fb7eb90239c6e383)

 Conflicts:
	hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt

(cherry picked from commit 55f7ceb0db13a6ef7a29b54f63075ce05dc1b019)
 2016-03-03 19:00:50 +09:00
Akira Ajisaka
8f2d0d4cea HDFS-9851. NameNode throws NPE when setPermission is called on a path that does not exist. Contributed by Brahma Reddy Battula. 2016-03-02 14:45:33 +09:00
Akira Ajisaka
2f3ceb8c65 HDFS-9766. TestDataNodeMetrics#testDataNodeTimeSpend fails intermittently. Contributed by Xiao Chen. 
(cherry picked from commit 81fbc40f084eab85750cc7d4cc967d43d70d8333)
(cherry picked from commit e8401417269bc3515918566bb2860e3b70d1d3ed)
 2016-03-02 14:16:58 +09:00
Kihwal Lee
006ed54a16 HDFS-9880. TestDatanodeRegistration fails occasionally. Contributed by Kihwal Lee. 
(cherry picked from commit e76b13c415459e4062c4c9660a16759a11ffb34a)
 2016-03-01 17:02:43 -06:00
Kihwal Lee
9bc9e13a97 HDFS-8791. block ID-based DN storage layout can be very slow for datanode on ext4. Contributed by Chris Trezzo. 
(cherry picked from commit b8dbffbef87c0ee0bee5e17649586fc9f0e1f55b)
 2016-03-01 15:01:04 -06:00
Kihwal Lee
9fa300ad1f HDFS-9198. Coalesce IBR processing in the NN. Contributed by Daryn Sharp. 2016-03-01 13:02:05 -06:00
Akira Ajisaka
e14ab939bf HDFS-9864. Correct reference for RENEWDELEGATIONTOKEN and CANCELDELEGATIONTOKEN in webhdfs doc. Contributed by Brahma Reddy Battula. 
(cherry picked from commit 056f9013122dc3f0effc62d620261c312dd7f8ed)
(cherry picked from commit 6a9f8d95b7c4f9c66253d975ebbca1d186267d4e)
 2016-02-29 18:41:57 +09:00
Kihwal Lee
25b3531eb4 HDFS-9395. Make HDFS audit logging consistant. Contributed by Kuhu Shukla. 2016-02-26 09:45:52 -06:00
Vinayakumar B
33b961ee88 HDFS-8578. On upgrade, Datanode should process all storage/data dirs in parallel. Contributed by vinayakumarb and szetszwo 
Addendum to fix compilation
 2016-02-23 14:39:24 +05:30
Tsz-Wo Nicholas Sze
1b139593c5 HDFS-8578. On upgrade, Datanode should process all storage/data dirs in parallel. Contributed by vinayakumarb and szetszwo 2016-02-22 15:18:46 -08:00
Zhe Zhang
79027309c9 HDFS-9688. Test the effect of nested encryption zones in HDFS downgrade. 
Change-Id: I2a802325a2bc82c97012d2c9fb388b292fe3af8f
 2016-02-22 15:10:31 -08:00
Kihwal Lee
019770c769 HDFS-4946. Allow preferLocalNode in BlockPlacementPolicyDefault to be configurable. Contributed by James Kinley and Nathan Roberts. 
(cherry picked from commit ba6d5ed955aa34cdcb7aaa4b1025da6ee5e91355)

Conflicts:
	hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/DFSConfigKeys.java
	hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/blockmanagement/BlockPlacementPolicyDefault.java
 2016-02-18 14:10:43 -06:00
Kihwal Lee
a4492dc6ac HDFS-8845. DiskChecker should not traverse the entire tree (Chang Li via Colin P. McCabe) 
(cherry picked from commit ec183faadcf7edaf432aca3b25d24215d505c2ec)

Conflicts:
	hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt
 2016-02-17 09:18:55 -06:00
Akira Ajisaka
8377683b63 HDFS-9765. TestBlockScanner#testVolumeIteratorWithCaching fails intermittently. (aajisaka) 
(cherry picked from commit 8ed07bd3e6b6a8af3b82e9a085c74be29f8dfd87)
(cherry picked from commit 8dec72bfd73174bdc2dfa8ce845b28af48a6bca4)
 2016-02-15 19:10:03 +09:00
Junping Du
63069389ee Set the release date for 2.6.4. 
(cherry picked from commit ec12ce8f48944c38271399a6e452aa2c2a9f5b80)
(cherry picked from commit a28be9edc72cc4fe42fab107c17bcac6789af4e5)
 2016-02-13 01:50:30 -08:00
Kihwal Lee
a30043ca26 HDFS-9779 . TestReplicationPolicyWithNodeGroup NODE variable picks wrong rack value. Contributed by Kuhu Shukla. 
(cherry picked from commit a7fce9ab415fcc834ca1d91f913157ffd103fa5f)

Conflicts:
	hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt
 2016-02-11 08:15:35 -06:00
Akira Ajisaka
948dd27960 HDFS-9752. Permanent write failures may happen to slow writers during datanode rolling upgrades. (Contributed by Walter Su) 2016-02-10 03:52:42 +09:00
Akira Ajisaka
2a46d9c6c4 HDFS-9784. Example usage is not correct in Transparent Encryption document. Contributed by Ohnishi Takashi. 
(cherry picked from commit 60d2011b7c0fe55b8bc44a141660d3e2df37a68d)
(cherry picked from commit cb53dfcc95ca9f6c45482600b52758bc096810b9)
 2016-02-10 03:33:31 +09:00
Zhe Zhang
ef05096f95 HDFS-9347. Invariant assumption in TestQuorumJournalManager.shutdown() is wrong. Contributed by Wei-Chiu Chuang. 
Change-Id: Idde8522e11c77f82b376e0c90a9bf33d163d5cb9
(cherry picked from commit ef3f3f6bb14cf44bef1778f1091d8ed8a4b764a3)

Conflicts:
	hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt
 2016-02-09 10:42:21 +08:00
Haohui Mai
cb3aa41f53 HDFS-9724. Degraded performance in WebHDFS listing as it does not reuse ObjectMapper. Contributed by Akira Ajisaka. 2016-02-04 11:36:12 -08:00
Kihwal Lee
c10bf788eb HDFS-9730. Storage ID update does not happen when there is a layout change. Contributed by Tsz Wo Nicholas Sze. 
(cherry picked from commit 496f33de0ce80dc455cfd51f19612da6f9b914f9)

Conflicts:
	hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/datanode/DataStorage.java
 2016-02-04 13:32:11 -06:00
Vinayakumar B
bef53683b6 HDFS-9629. Update the footer of Web UI to show year 2016 (Contributed by Xiao Chen and Vinayakumar B) 
(cherry picked from commit ddcd6954d03d3caadc8b69f7b886a3a17c3c9d0c)
(cherry picked from commit 8cdfdbf815bf89eee3184ec7cd00e1924c1ea316)
 2016-02-04 13:20:27 +05:30
Kihwal Lee
dd11c8274d HDFS-9743. Fix TestLazyPersistFiles#testFallbackToDiskFull in branch-2.7. Contributed by Kihwal Lee. 2016-02-03 12:51:01 -06:00