Apache
/
hadoop
1
0
Fork 0
Code Issues Pull Requests Packages Projects Releases Wiki Activity
9,472 Commits 326 Branches 371 Tags 1.5 GiB
Commit Graph

3199 Commits

Author SHA1 Message Date
Zhe Zhang c7e1e92c1a HDFS-10544. Balancer doesn't work with IPFailoverProxyProvider. 
(cherry picked from commit 087290e6b1)
(cherry picked from commit 5b4e916b3c)
(cherry picked from commit af94dfaa53)
 2016-07-13 14:36:26 -07:00
Arpit Agarwal 771033babd HADOOP-12570. HDFS Secure Mode Documentation updates. (Arpit Agarwal) 2016-07-13 13:02:01 -07:00
Vinod Kumar Vavilapalli c454639abf HADOOP-13312. Updated CHANGES.txt to reflect all the changes in branch-2.7. Contributed by Akira Ajisaka. 2016-07-12 15:41:22 -07:00
Andrew Wang 5ebebc6524 HDFS-10488. Update WebHDFS documentation regarding CREATE and MKDIR default permissions. Contributed by Wellington Chevreuil. 
(cherry picked from commit 728623242f)
 2016-07-07 10:38:33 -07:00
Brahma Reddy Battula 6a1fc34595 HDFS-9530. ReservedSpace is not cleared for abandoned Blocks (Contributed by Brahma Reddy Battula) 2016-06-21 16:12:46 +05:30
Rohith Sharma K S 0bc531aa8f Adding-release-2.7.4-to-CHANGES.txt 2016-06-21 11:27:13 +05:30
Brahma Reddy Battula be94ed6ceb HDFS-10474. hftp copy fails when file name with Chinese+special char in branch-2 (Contributed by Brahma Reddy Battula) 2016-06-20 12:43:18 +05:30
Arpit Agarwal aca4310f32 HDFS-9906. Remove spammy log spew when a datanode is restarted. (Contributed by Brahma Reddy Battula) 
(cherry picked from commit 02949b2151)
 2016-06-14 23:13:48 -07:00
Vinod Kumar Vavilapalli f4b58eae94 Preparing for 2.7.4 development: Updating snapshot version to 2.7.4 on branch-2.7. 2016-06-14 19:14:02 -07:00
Kihwal Lee 8854cdd9ee HDFS-8581. ContentSummary on / skips further counts on yielding lock (contributed by J.Andreina) 
(cherry picked from commit 4014ce5990)

Conflicts:
	hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt
	hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/TestQuota.java
 2016-06-14 08:25:09 -05:00
Masatake Iwasaki 5a3fed0602 HDFS-9905. WebHdfsFileSystem#runWithRetry should display original stack trace on error. (Wei-Chiu Chuang via iwasakims) 
cherry-picked from 6fcde2e38d
 2016-06-09 20:48:58 +00:00
Zhe Zhang 9b68eda274 HDFS-10458. getFileEncryptionInfo should return quickly for non-encrypted cluster. 
(cherry picked from commit 6de9213df1)
(cherry picked from commit 06e38c835d)
(cherry picked from commit 0274636529)
 2016-06-06 16:09:14 -07:00
Xiaoyu Yao 97e79662b6 HDFS-8548. Minicluster throws NPE on shutdown. Contributed by surendra singh lilhore. 
(cherry picked from commit 6a76250b39)
 2016-06-03 12:55:31 -07:00
Akira Ajisaka fe08ba439e HDFS-9476. TestDFSUpgradeFromImage#testUpgradeFromRel1BBWImage occasionally fail. Contributed by Masatake Iwasaki. 
(cherry picked from commit 69555fca06)
(cherry picked from commit 3dd557e4b6)
 2016-06-02 18:55:36 +09:00
Tsz-Wo Nicholas Sze d3bdea7f7f HDFS-9365. Balaner does not work with the HDFS-6376 HA setup. 2016-05-24 13:52:05 -07:00
Kihwal Lee 6e62b6fd83 HDFS-8772. Fix TestStandbyIsHot#testDatanodeRestarts which occasionally fails. Contributed by Walter Su. 
(cherry picked from commit 0c4c7b3b62)

Conflicts:
	hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/MiniDFSCluster.java
 2016-05-12 11:26:27 -05:00
Colin Patrick Mccabe a61e04171f HDFS-10377. CacheReplicationMonitor shutdown log message should use INFO level (Yiqun Lin via cmccabe) 
(cherry picked from commit f0252ad252)
(cherry picked from commit 43f32d3f4a)
(cherry picked from commit 154332bb54)
 2016-05-10 11:22:13 -07:00
Kihwal Lee 0467efcc09 HDFS-10372. Fix for failing TestFsDatasetImpl#testCleanShutdownOfVolume. Contributed by Rushabh Shah. 
(cherry picked from commit b9e5a32fa1)
Added a CHANGES.txt entry.
 2016-05-09 15:56:51 -05:00
Masatake Iwasaki dd8aba3fad HDFS-2043. TestHFlush failing intermittently. Contributed by Lin Yiqun. 2016-05-06 15:57:06 +09:00
Harsh J d490cfc42c HDFS-2580. NameNode#main(...) can make use of GenericOptionsParser. Contributed by harsh. (harsh) 
(cherry picked from commit 54e612bfb9)
 2016-05-02 17:52:14 -07:00
Kihwal Lee 6347f8f4d4 HDFS-10347. Namenode report bad block method doesn't log the bad block or datanode. Contributed by Rushabh Shah. 
(cherry picked from commit 7da540d03e)
Updated CHANGES.txt
 2016-04-29 16:07:20 -05:00
Tsz-Wo Nicholas Sze c927f938fe HDFS-10335 Mover$Processor#chooseTarget() always chooses the first matching target storage group. Contributed by Mingliang Liu 2016-04-29 11:28:23 -07:00
Kihwal Lee a3ece8b5b4 HDFS-9958. BlockManager#createLocatedBlocks can throw NPE for corruptBlocks on failed storages. Contributed by Kuhu Shukla. 2016-04-28 16:47:04 -05:00
Akira Ajisaka 076215024b HDFS-10245. Fix the findbugs warnings in branch-2.7. Contributed by Brahma Reddy Battula. 2016-04-25 22:03:48 +09:00
Kihwal Lee 9e3b91b141 HDFS-9555. LazyPersistFileScrubber should still sleep if there are errors in the clear progress. Contributed by Phil Yang. 
(cherry picked from commit 5d4255a801)

Conflicts:
	hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/FSNamesystem.java
 2016-04-22 11:47:18 -05:00
Tsz-Wo Nicholas Sze e7f1b8fcb3 HDFS-10319. Balancer should not try to pair storages with different types. 2016-04-21 15:36:28 -07:00
Arpit Agarwal 12c6c2c9a6 HDFS-10264. Logging improvements in FSImageFormatProtobuf.Saver. (Contributed by Xiaobing Zhou) 2016-04-19 11:28:01 -07:00
Walter Su 91ca90fdd1 HDFS-10275. TestDataNodeMetrics failing intermittently due to TotalWriteTime counted incorrectly. Contributed by Lin Yiqun. 
(cherry picked from commit ab903029a9)
 2016-04-18 20:30:48 +08:00
Walter Su 2ee244ac28 HDFS-9772. TestBlockReplacement#testThrottler doesn't work as expected. Contributed by Lin Yiqun. 
(cherry picked from commit 903428bf94)
 2016-04-13 17:51:01 +08:00
Vinayakumar B b52ac3cb4d Addendum: HDFS-10271. 
Added CHANGES.txt entry
 2016-04-12 11:57:00 +05:30
Vinayakumar B 37e4e45390 HDFS-10271. Extra bytes are getting released from reservedSpace for append (Contributed by Brahma Reddy Battula) 2016-04-12 11:32:28 +05:30
Tsz-Wo Nicholas Sze 262d769b4d HDFS-10186. DirectoryScanner: Improve logs by adding full path of both actual and expected block directories. Contributed by Rakesh R 2016-04-07 12:43:09 +08:00
Kihwal Lee 8e784afb15 Revert "HDFS-8791. block ID-based DN storage layout can be very slow for datanode on ext4. Contributed by Chris Trezzo." 
This reverts commit 9bc9e13a97.

Conflicts:
	hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt
 2016-04-06 17:35:54 -05:00
Kihwal Lee dd701c9800 HDFS-10261. TestBookKeeperHACheckpoints doesn't handle ephemeral HTTP ports. Contributed by Eric Badger. 
(cherry picked from commit 9ba1e5af06)

Conflicts:
	hadoop-hdfs-project/hadoop-hdfs/src/contrib/bkjournal/src/test/java/org/apache/hadoop/contrib/bkjournal/TestBookKeeperHACheckpoints.java

(cherry picked from commit 6e37c5fe80)
 2016-04-05 17:00:42 -05:00
Kihwal Lee 9608601330 HDFS-10239. Fsshell mv fails if port usage doesn't match in src and destination paths. Contributed by Kuhu Shukla. 
Conflicts:
	hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/TestDFSShell.java

(cherry picked from commit ef3da82357)
 2016-04-05 09:20:38 -05:00
Vinayakumar B fc0ffc48d4 HDFS-9917. IBR accumulate more objects when SNN was down for sometime. (Contributed by Brahma Reddy Battula) 2016-04-05 09:59:37 +08:00
Kihwal Lee 69d4fa0deb HDFS-10178. Permanent write failures can happen if pipeline recoveries occur for the first packet. Contributed by Kihwal Lee. 
(cherry picked from commit a7d1fb0cd2)
TestClientProtocolForPipelineRecovery modified to use DFSConfigKeys instead of HdfsClientConfigKeys.
 2016-04-04 16:54:04 -05:00
Kihwal Lee efb210cc30 Fix CHANGES.txt 2016-03-28 11:31:07 -05:00
Walter Su ec459e3966 HDFS-10182. Hedged read might overwrite user's buf. Contributed by zhouyingchao. 
(cherry picked from commit d8383c687c)
 2016-03-28 15:46:15 +08:00
Aaron T. Myers 17dbf82f54 HDFS-8101. DFSClient use of non-constant DFSConfigKeys pulls in WebHDFS classes at runtime. Contributed by Sean Busbey. 
(cherry picked from commit 3fe61e0bb0)
 2016-03-26 17:54:09 -07:00
Kihwal Lee a66d2a2a85 HDFS-9874. Long living DataXceiver threads cause volume shutdown to block. Contributed by Rushabh Shah. 
(cherry picked from commit 63c966a3fb)
(cherry picked from commit 242c7f1fee)

Conflicts:
	hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/datanode/fsdataset/impl/FsDatasetImpl.java
	hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/datanode/fsdataset/impl/TestFsDatasetImpl.java
 2016-03-18 10:38:33 -05:00
Akira Ajisaka 6902789a92 HDFS-9860. Backport HDFS-9638 to branch-2.7. Contributed by Wei-Chiu Chuang. 2016-03-17 02:59:51 +09:00
Kihwal Lee b8e73cff8f HDFS-9904. testCheckpointCancellationDuringUpload occasionally fails. Contributed by Lin Yiqun. 
(cherry picked from commit d457401784)
 2016-03-15 10:56:46 -05:00
Akira Ajisaka 7f43eb9547 HDFS-9812. Streamer threads leak if failure happens when closing DFSOutputStream. Contributed by Lin Yiqun. 
(cherry picked from commit 352d299cf8)
(cherry picked from commit fe0009a2bd)
 2016-03-08 11:15:27 +09:00
Masatake Iwasaki e768f8b771 HDFS-9865. TestBlockReplacement fails intermittently in trunk (Lin Yiqun via iwasakims) 2016-03-08 02:13:50 +09:00
Masatake Iwasaki 08fc048018 HDFS-9048. DistCp documentation is out-of-dated (Daisuke Kobayashi via iwasakims) 
(cherry picked from commit 33a412e8a4)

 Conflicts:
	hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt

(cherry picked from commit 55f7ceb0db)
 2016-03-03 19:00:50 +09:00
Akira Ajisaka 8f2d0d4cea HDFS-9851. NameNode throws NPE when setPermission is called on a path that does not exist. Contributed by Brahma Reddy Battula. 2016-03-02 14:45:33 +09:00
Akira Ajisaka 2f3ceb8c65 HDFS-9766. TestDataNodeMetrics#testDataNodeTimeSpend fails intermittently. Contributed by Xiao Chen. 
(cherry picked from commit 81fbc40f084eab85750cc7d4cc967d43d70d8333)
(cherry picked from commit e840141726)
 2016-03-02 14:16:58 +09:00
Kihwal Lee 006ed54a16 HDFS-9880. TestDatanodeRegistration fails occasionally. Contributed by Kihwal Lee. 
(cherry picked from commit e76b13c415)
 2016-03-01 17:02:43 -06:00
Kihwal Lee 9bc9e13a97 HDFS-8791. block ID-based DN storage layout can be very slow for datanode on ext4. Contributed by Chris Trezzo. 
(cherry picked from commit b8dbffbef87c0ee0bee5e17649586fc9f0e1f55b)
 2016-03-01 15:01:04 -06:00
Kihwal Lee 9fa300ad1f HDFS-9198. Coalesce IBR processing in the NN. Contributed by Daryn Sharp. 2016-03-01 13:02:05 -06:00
Akira Ajisaka e14ab939bf HDFS-9864. Correct reference for RENEWDELEGATIONTOKEN and CANCELDELEGATIONTOKEN in webhdfs doc. Contributed by Brahma Reddy Battula. 
(cherry picked from commit 056f9013122dc3f0effc62d620261c312dd7f8ed)
(cherry picked from commit 6a9f8d95b7c4f9c66253d975ebbca1d186267d4e)
 2016-02-29 18:41:57 +09:00
Kihwal Lee 25b3531eb4 HDFS-9395. Make HDFS audit logging consistant. Contributed by Kuhu Shukla. 2016-02-26 09:45:52 -06:00
Vinayakumar B 33b961ee88 HDFS-8578. On upgrade, Datanode should process all storage/data dirs in parallel. Contributed by vinayakumarb and szetszwo 
Addendum to fix compilation
 2016-02-23 14:39:24 +05:30
Tsz-Wo Nicholas Sze 1b139593c5 HDFS-8578. On upgrade, Datanode should process all storage/data dirs in parallel. Contributed by vinayakumarb and szetszwo 2016-02-22 15:18:46 -08:00
Zhe Zhang 79027309c9 HDFS-9688. Test the effect of nested encryption zones in HDFS downgrade. 
Change-Id: I2a802325a2bc82c97012d2c9fb388b292fe3af8f
 2016-02-22 15:10:31 -08:00
Kihwal Lee 019770c769 HDFS-4946. Allow preferLocalNode in BlockPlacementPolicyDefault to be configurable. Contributed by James Kinley and Nathan Roberts. 
(cherry picked from commit ba6d5ed955)

Conflicts:
	hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/DFSConfigKeys.java
	hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/blockmanagement/BlockPlacementPolicyDefault.java
 2016-02-18 14:10:43 -06:00
Kihwal Lee a4492dc6ac HDFS-8845. DiskChecker should not traverse the entire tree (Chang Li via Colin P. McCabe) 
(cherry picked from commit ec183faadc)

Conflicts:
	hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt
 2016-02-17 09:18:55 -06:00
Akira Ajisaka 8377683b63 HDFS-9765. TestBlockScanner#testVolumeIteratorWithCaching fails intermittently. (aajisaka) 
(cherry picked from commit 8ed07bd3e6)
(cherry picked from commit 8dec72bfd7)
 2016-02-15 19:10:03 +09:00
Junping Du 63069389ee Set the release date for 2.6.4. 
(cherry picked from commit ec12ce8f48)
(cherry picked from commit a28be9edc7)
 2016-02-13 01:50:30 -08:00
Kihwal Lee a30043ca26 HDFS-9779 . TestReplicationPolicyWithNodeGroup NODE variable picks wrong rack value. Contributed by Kuhu Shukla. 
(cherry picked from commit a7fce9ab41)

Conflicts:
	hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt
 2016-02-11 08:15:35 -06:00
Akira Ajisaka 948dd27960 HDFS-9752. Permanent write failures may happen to slow writers during datanode rolling upgrades. (Contributed by Walter Su) 2016-02-10 03:52:42 +09:00
Akira Ajisaka 2a46d9c6c4 HDFS-9784. Example usage is not correct in Transparent Encryption document. Contributed by Ohnishi Takashi. 
(cherry picked from commit 60d2011b7c)
(cherry picked from commit cb53dfcc95)
 2016-02-10 03:33:31 +09:00
Zhe Zhang ef05096f95 HDFS-9347. Invariant assumption in TestQuorumJournalManager.shutdown() is wrong. Contributed by Wei-Chiu Chuang. 
Change-Id: Idde8522e11c77f82b376e0c90a9bf33d163d5cb9
(cherry picked from commit ef3f3f6bb1)

Conflicts:
	hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt
 2016-02-09 10:42:21 +08:00
Haohui Mai cb3aa41f53 HDFS-9724. Degraded performance in WebHDFS listing as it does not reuse ObjectMapper. Contributed by Akira Ajisaka. 2016-02-04 11:36:12 -08:00
Kihwal Lee c10bf788eb HDFS-9730. Storage ID update does not happen when there is a layout change. Contributed by Tsz Wo Nicholas Sze. 
(cherry picked from commit 496f33de0c)

Conflicts:
	hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/datanode/DataStorage.java
 2016-02-04 13:32:11 -06:00
Vinayakumar B bef53683b6 HDFS-9629. Update the footer of Web UI to show year 2016 (Contributed by Xiao Chen and Vinayakumar B) 
(cherry picked from commit ddcd6954d03d3caadc8b69f7b886a3a17c3c9d0c)
(cherry picked from commit 8cdfdbf815bf89eee3184ec7cd00e1924c1ea316)
 2016-02-04 13:20:27 +05:30
Kihwal Lee dd11c8274d HDFS-9743. Fix TestLazyPersistFiles#testFallbackToDiskFull in branch-2.7. Contributed by Kihwal Lee. 2016-02-03 12:51:01 -06:00
Vinayakumar B af9de19cfd HDFS-9740. Use a reasonable limit in DFSTestUtil.waitForMetric() (Contributed by Chang Li) 
(cherry picked from commit eb2fb943fd)

 Conflicts:
	hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/DFSTestUtil.java

(cherry picked from commit 6941e2ccbf)

 Conflicts:
	hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/DFSTestUtil.java
 2016-02-03 11:24:26 +05:30
Junping Du bfee573baf Adding release 2.6.5 to CHANGES.txt 2016-02-02 18:19:08 -08:00
Colin Patrick Mccabe 449d2ad85c HDFS-9669. TcpPeerServer should respect ipc.server.listen.queue.size (Elliot Clark via cmccabe) 
(cherry picked from commit 2da03b48eb)
(cherry picked from commit 60d3a3c30b)
(cherry picked from commit c4c94e1cf2)

Conflicts:
	hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/datanode/DataNode.java
	hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/datanode/SecureDataNodeStarter.java
 2016-02-02 11:17:28 -08:00
Kihwal Lee 2f031830e8 HDFS-9072. Fix random failures in TestJMXGet. Contributed by J.Andreina. 2016-02-02 11:07:48 -06:00
Yongjun Zhang 3d88293dda HDFS-9406. FSImage may get corrupted after deleting snapshot. (Contributed by Jing Zhao, Stanislav Antic, Vinayakumar B, Yongjun Zhang) 
(cherry picked from commit 34ab50ea92)

Conflicts:
	hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/namenode/TestINodeFile.java

(cherry picked from commit fc8d9cc758)

Conflicts:
	hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/INodeFile.java
	hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/snapshot/DirectoryWithSnapshotFeature.java
	hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/namenode/TestINodeFile.java
	hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/namenode/snapshot/TestSnapshotDeletion.java
 2016-02-01 15:29:03 -08:00
Tsz-Wo Nicholas Sze 672481e28c HDFS-9654. Code refactoring for HDFS-8578. 2016-01-28 11:09:05 +08:00
Vinayakumar B 07704aab84 HDFS-9690. ClientProtocol.addBlock is not idempotent after HDFS-8071.(Contributed by Tsz Wo Nicholas Sze) 2016-01-26 21:34:07 +05:30
Vinod Kumar Vavilapalli (I am also known as @tshooter.) 34c9289f6c Release process for 2.7.2: Set the release date for 2.7.2 
(cherry picked from commit ec4d2d9f40)
 2016-01-25 15:48:18 -08:00
Kihwal Lee e160ddfb8d HDFS-9634. webhdfs client side exceptions don't provide enough details. Contributed by Eric Payne. 
(cherry picked from commit 7b70500484574a565dd8cd5c7d8b5bc7c6d91154)

Conflicts:
	hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt
 2016-01-21 09:34:29 -06:00
Vinayakumar B ca29933405 HDFS-9625. set replication for empty file failed when set storage policy (Contributed by DENG FEI) 
(cherry picked from commit b7372b7166)
(cherry picked from commit 845acfd96c)
 2016-01-21 19:21:00 +05:30
Vinayakumar B eb3dcc0739 HDFS-9661. Deadlock in DN.FsDatasetImpl between moveBlockAcrossStorage and createRbw (Contributed by ade) 
(cherry picked from commit 1425578690)
 2016-01-20 21:11:39 +05:30
Tsz-Wo Nicholas Sze 2131b9820d In CHANGES.txt, move HDFS-9294 to Release 2.6.4. 2016-01-16 23:48:03 +08:00
Arpit Agarwal 6cd5fcb634 HDFS-9415. Document dfs.cluster.administrators and dfs.permissions.superusergroup. (Contributed by Xiaobing Zhou) 2016-01-14 14:19:51 -08:00
Yongjun Zhang 7a550db931 HDFS-9648. TestStartup.testImageChecksum is broken by HDFS-9569's message change. (Wei-Chiu Chuang via Yongjun Zhang) 
(cherry picked from commit 817cc1f02a)
 2016-01-14 12:45:42 -08:00
Kihwal Lee a90dcb0865 Fix up CHANGES.txt 2016-01-14 09:58:35 -06:00
Junping Du c2d1d69525 Fix yarn CHANGES.txt entry for 2.7.2 2016-01-14 06:17:42 -08:00
Junping Du 0e0272f120 Add missing CHANGES.txt entry for HDFS-8767. 
(cherry picked from commit e50dd19a9c)

Conflicts:

	hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt
(cherry picked from commit 62eabec549)
 2016-01-14 06:14:56 -08:00
Vinod Kumar Vavilapalli (I am also known as @tshooter.) faebc20e26 HADOOP-5323. Trash documentation should describe its directory structure and configurations. (Weiwei Yang via ozawa) 
Moved entry from HDFS CHANGES.txt to that of common.

(cherry picked from commit cb73c753da)
 2016-01-13 14:56:58 -08:00
Kihwal Lee 83a4791243 HDFS-9047. Retire libwebhdfs. 2016-01-13 11:01:35 -06:00
Yongjun Zhang 4c09a75f3e HDFS-9569. Log the name of the fsimage being loaded for better supportability. (Yongjun Zhang) 
(cherry picked from commit 25051c3bd0)

Conflicts:
	hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/FSImageFormat.java
 2016-01-12 09:40:56 -08:00
Kihwal Lee b06e39de4f HDFS-9574. Reduce client failures during datanode restart. Contributed by Kihwal Lee. 2016-01-08 11:31:49 -06:00
Vinayakumar B 5dc5bb2812 HDFS-9600. do not check replication if the block is under construction (Contributed by Phil Yang) 
(cherry picked from commit 34cd7cd765)
(cherry picked from commit aa710bd461)

Conflicts:
	hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/blockmanagement/BlockManager.java
	hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/blockmanagement/DecommissionManager.java
	hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/blockmanagement/TestBlockManager.java
 2016-01-07 11:50:12 +05:30
Akira Ajisaka 68c68e187f Set release date for 2.6.3 
(cherry picked from commit db99e30f67)
(cherry picked from commit 247ab92464)
 2016-01-04 16:14:30 +09:00
Akira Ajisaka 3a1c98bba3 HADOOP-11252. RPC client does not time out by default. Contributed by Wilfred Spiegelenburg and Masatake Iwasaki. 
(cherry picked from commit 64ae85fd2e)
(cherry picked from commit d98cfe611f)
 2016-01-04 14:35:54 +09:00
Kihwal Lee cdf9f1c662 HDFS-7163. WebHdfsFileSystem should retry reads according to the configured retry policy. Contributed by Eric Payne. 2015-12-28 10:57:31 -06:00
Haohui Mai e0e5a0fa82 HDFS-8914. Document HA support in the HDFS HdfsDesign.md. Contributed by Lars Francke. 
(cherry picked from commit 049ccf71ce)
 2015-12-21 21:44:46 +09:00
Akira Ajisaka 5ffad1551b HDFS-9505. HDFS Architecture documentation needs to be refreshed. Contributed by Masatake Iwasaki. 
(cherry picked from commit fa544020f6)
(cherry picked from commit 7526a2a005)
 2015-12-21 21:27:28 +09:00
Junping Du 803fdc9f5e Creating 2.6.4 entries in CHANGES.txt files. 
(cherry picked from commit 028894281b)

(cherry picked from commit 092c469059)
 2015-12-18 14:24:51 +09:00
Kihwal Lee 03292b234e HDFS-9533. seen_txid in the shared edits directory is modified during bootstrapping. Contributed by Kihwal Lee. 
(cherry picked from commit 5cb1e0118b)

Conflicts:
	hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/ha/BootstrapStandby.java
	hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/namenode/ha/TestBootstrapStandby.java
(cherry picked from commit 3c0ae5914c)
 2015-12-17 17:28:24 -06:00
cnauroth 56882e7b1c Revert "HDFS-9569. Log the name of the fsimage being loaded for better supportability. Contributed by Yongjun Zhang." 
This reverts commit 4d2ee51c52.
 2015-12-17 13:49:47 -08:00
Kihwal Lee 4d2ee51c52 HDFS-9569. Log the name of the fsimage being loaded for better supportability. Contributed by Yongjun Zhang. 
(cherry picked from commit eb6939cea0)
 2015-12-17 10:33:15 -06:00
Plamen Jeliazkov d97696e4c4 HDFS-9516. Truncate file fails with data dirs on multiple disks. Contributed by Plamen Jeliazkov. 2015-12-17 00:09:48 -08:00
Tom Zeng 6c62d422d6 HADOOP-12415. Add io.netty dependency to hadoop-nfs and to hadoop-hdfs, needed to build Bigtop successfuly, see BIGTOP-2049 
Signed-off-by: Konstantin Boudnik <cos@boudnik.org>
(cherry picked from commit b4c6b511e7)
 2015-12-16 14:27:30 -08:00
Kihwal Lee a02312546a HDFS-9445. Datanode may deadlock while handling a bad volume. Contributed by Walter Su. 
(cherry picked from commit a48301791e)
 2015-12-11 08:47:22 -06:00
Vinod Kumar Vavilapalli 067d14f503 HDFS-9273. Moving to 2.6.3 CHANGES section to reflect the backport. 2015-12-08 11:36:15 -08:00
cnauroth e23ab9d02b HDFS-9273. ACLs on root directory may be lost after NN restart. Contributed by Xiao Chen. 
(cherry picked from commit 1b525a9c32)
(cherry picked from commit faf1c60899)
 2015-12-08 11:22:19 -08:00
Zhe Zhang 670617f325 HDFS-9314. Improve BlockPlacementPolicyDefault's picking of excess replicas. Contributed by Xiao Chen. 
Change-Id: Idf99293085531165239369155c039b55db0eed83
 2015-12-04 09:49:43 -08:00
Zhe Zhang b39de3b0bd HDFS-9313. Possible NullPointerException in BlockManager if no excess replica can be chosen. Contributed by Ming Ma. Branch-2.7 patch contributed by Xiao Chen. 
Change-Id: Iadaee6be2b44f7b0322d7b0e58f8fc983ef463f6
 2015-12-03 23:19:17 -08:00
Zhe Zhang 1ae8a8342e HDFS-8647. Abstract BlockManager's rack policy into BlockPlacementPolicy. Contributed by Brahma Reddy Battula. Branch-2.7 patch contributed by Xiao Chen. 
Change-Id: I1b32627bc2a2f30f5debeaba7663fb2777958079
 2015-12-03 10:30:48 -08:00
Tsz-Wo Nicholas Sze 4123650e92 HDFS-9294. DFSClient deadlock when close file and failed to renew lease. Contributed by Brahma Reddy Battula 2015-12-02 17:55:10 -08:00
Andrew Wang 10e8a67d23 HDFS-9470. Encryption zone on root not loaded from fsimage after NN restart. Xiao Chen via wang. 
(cherry picked from commit 9b8e50b424)
(cherry picked from commit ce1111ceea)

 Conflicts:
	hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/FSDirectory.java
 2015-11-30 14:53:04 -08:00
Vinayakumar B 0c48c17ae9 HDFS-9426. Rollingupgrade finalization is not backward compatible (Contributed by Kihwal Lee) 
(cherry picked from commit c62d42cd8b)
 2015-11-26 09:42:26 +05:30
Tsz-Wo Nicholas Sze af811030e5 Move HDFS-9434 to 2.6.3 in CHANGES.txt. 2015-11-24 12:49:34 -08:00
Tsz-Wo Nicholas Sze 09f4a4eefd HDFS-9434. Recommission a datanode with 500k blocks may pause NN for 30 seconds for printing info log messags. 2015-11-24 12:47:50 -08:00
Akira Ajisaka bb6983e153 HDFS-9431. DistributedFileSystem#concat fails if the target path is relative. Contributed by Kazuho Fujii. 
(cherry picked from commit ac1aa6c819)
(cherry picked from commit c74e42b4a2)
 2015-11-19 20:55:52 +09:00
Ming Ma 1b0f277eae HDFS-9413. getContentSummary() on standby should throw StandbyException. (Brahma Reddy Battula via mingma) 
(cherry picked from commit 02653add98)
(cherry picked from commit 42b55ff23e)

 Conflicts:
	hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt

(cherry picked from commit 83200450f6)

 Conflicts:
	hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt
 2015-11-17 16:12:58 +05:30
Akira Ajisaka ab04c9ab5f HDFS-8615. Correct HTTP method in WebHDFS document. Contributed by Brahma Reddy Battula. 
(cherry picked from commit 1a169a26bc)
(cherry picked from commit 9f9315008e)
 2015-11-17 16:32:07 +09:00
Kihwal Lee 6f678deb86 HDFS-9383. TestByteArrayManager#testByteArrayManager fails. Contributed by Tsz Wo Nicholas Sze. 
(cherry picked from commit ef926b2e38)
 2015-11-09 08:58:58 -06:00
Arpit Agarwal 63ed399e1c HDFS-6481. DatanodeManager#getDatanodeStorageInfos() should check the length of storageIDs. (Contributed by szetszwo) 2015-11-06 10:32:34 -08:00
Kihwal Lee 616ed9084b HDFS-4937. ReplicationMonitor can infinite-loop in BlockPlacementPolicyDefault#chooseRandom(). Contributed by Kihwal Lee. 
(cherry picked from commit ff47f35dee)
 2015-11-05 09:27:36 -06:00
Kihwal Lee 397b554c36 HDFS-9289. Make DataStreamer#block thread safe and verify genStamp in commitBlock. Contributed by Chang Li. 2015-11-04 12:10:59 -06:00
Jason Lowe 7499f71e77 Add 2.7.3 release to CHANGES.txt 
(cherry picked from commit 0383a3973b)

Conflicts:

	hadoop-common-project/hadoop-common/CHANGES.txt
	hadoop-mapreduce-project/CHANGES.txt
	hadoop-yarn-project/CHANGES.txt
 2015-11-04 16:29:56 +00:00
Vinod Kumar Vavilapalli 09348c4bce Preparing for 2.7.3 development 2015-11-03 15:18:10 -08:00
Kihwal Lee 2c51bbda51 HDFS-9221. HdfsServerConstants#ReplicaState#getState should avoid calling values() since it creates a temporary array. (Staffan Friberg via yliu) 
(cherry picked from commit 0ff1216100)
 2015-11-02 09:07:16 -06:00
yliu e1798e8f21 Revert "HDFS-4937. ReplicationMonitor can infinite-loop in BlockPlacementPolicyDefault#chooseRandom(). Contributed by Kihwal Lee." 
This reverts commit c250b21c23.
 2015-10-31 16:25:53 +08:00
Sangjin Lee c8767fb68c Updated the 2.6.2 final release date. 
(cherry picked from commit a4a6b5b4b4)
 2015-10-30 18:53:10 -07:00
Jason Lowe e894be93f6 Creating 2.6.3 entries in CHANGES.txt files. 
(cherry picked from commit eadf7b3096)

Conflicts:

	hadoop-common-project/hadoop-common/CHANGES.txt
	hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt
	hadoop-mapreduce-project/CHANGES.txt
	hadoop-yarn-project/CHANGES.txt
 2015-10-30 14:55:33 +00:00
Kihwal Lee c250b21c23 HDFS-4937. ReplicationMonitor can infinite-loop in BlockPlacementPolicyDefault#chooseRandom(). Contributed by Kihwal Lee. 
(cherry picked from commit 43539b5ff4)
 2015-10-30 09:39:49 -05:00
Kihwal Lee 13b256ed22 HDFS-9083. Replication violates block placement policy. Contributed by Rushabh Shah. 2015-10-28 14:49:41 -05:00
Akira Ajisaka ce01a5951b HDFS-9317. Document fsck -blockId and -storagepolicy options in branch-2.7. (aajisaka) 2015-10-28 13:14:58 +09:00
Kihwal Lee 59a2072135 HDFS-8950. NameNode refresh doesn't remove DataNodes that are no longer in the allowed list. Contributed by Daniel Templeton. 2015-10-27 16:42:33 -05:00
Kihwal Lee 1d23e1ec07 HDFS-7725. Incorrect 'nodes in service' metrics caused all writes to fail. Contributed by Ming Ma. 
(cherry picked from commit 8104d52269)
 2015-10-27 11:09:05 -05:00
Arpit Agarwal 653ef52ef2 HDFS-9305. Delayed heartbeat processing causes storm of subsequent heartbeats. (Contributed by Arpit Agarwal) 2015-10-26 15:47:37 -07:00
Kihwal Lee f2fd011cc1 HDFS-9290. DFSClient#callAppend() is not backward compatible for slightly older NameNodes. Contributed by Tony Wu. 
(cherry picked from commit b9e0417bdf)

Conflicts:
	hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/DFSClient.java
 2015-10-23 16:46:14 -05:00
Kihwal Lee 4cf7f8441a HDFS-9220. Reading small file (< 512 bytes) that is open for append fails due to incorrect checksum. Contributed by Jing Zhao. 
(cherry picked from commit c7c36cbd62)
 2015-10-15 11:26:14 -05:00
Kihwal Lee efba749760 HDFS-8676. Delayed rolling upgrade finalization can cause heartbeat expiration. Contributed by Walter Su. 
(cherry picked from commit 5b43db47a3)
 2015-10-13 13:25:10 -05:00
Kihwal Lee 61388b41e7 HDFS-8656. Preserve compatibility of ClientProtocol#rollingUpgrade after finalization. 
(cherry picked from commit 60b858bfa6)

Conflicts:
	hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt
	hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/protocol/ClientProtocol.java
	hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/TestRollingUpgrade.java
 2015-10-13 10:46:20 -05:00
Kihwal Lee 3a5b9f49df HDFS-7645. Rolling upgrade is restoring blocks from trash multiple times (Contributed by Vinayakumar B and Keisuke Ogiwara) 
(cherry picked from commit abf3ad988d)
 2015-10-13 10:29:42 -05:00
Kihwal Lee 5cb80992fb HDFS-9178. Slow datanode I/O can cause a wrong node to be marked bad. Contributed by Kihwal Lee. 
(cherry picked from commit 99e5204ff5)

Conflicts:
	hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/datanode/BlockReceiver.java
	hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/TestClientProtocolForPipelineRecovery.java
 2015-10-07 10:24:55 -05:00
Kihwal Lee b2bc72a051 HDFS-8850. VolumeScanner thread exits with exception if there is no block pool to be scanned but there are suspicious blocks. (Colin Patrick McCabe via yliu) 
(cherry picked from commit d2b941f94a)

Conflicts:
	hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt
 2015-10-02 14:59:29 -05:00
Kihwal Lee 6e7c76a5bd HDFS-9106. Transfer failure during pipeline recovery causes permanent write failures. Contributed by Kihwal Lee. 2015-09-28 15:19:57 -05:00
Vinod Kumar Vavilapalli (I am also known as @tshooter.) 2b4c7fcc67 Release process for 2.6.1: Set the release date for 2.6.1 
(cherry picked from commit 1f707ecffd)
(cherry picked from commit f48af332a7)
 2015-09-23 15:13:09 -07:00
Vinayakumar B d800e2e9ed HDFS-9043. Doc updation for commands in HDFS Federation (Contributed by J.Andreina) 
(cherry picked from commit 57003fa971)
(cherry picked from commit 96e3fbf401)
 2015-09-22 16:40:19 +05:30
Vinayakumar B 7d3e882a49 HDFS-9042. Update document for the Storage policy name (Contributed by J.Andreina) 
(cherry picked from commit d8455479b8)
 2015-09-12 12:37:01 +05:30
Kihwal Lee f2cc6b5fdf HDFS-8099. Change "DFSInputStream has been closed already" message to debug log level (Charles Lamb via Colin P. McCabe) 
(cherry picked from commit 30acb7372a)

Conflicts:
	hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt
 2015-09-11 15:59:29 -05:00
Vinod Kumar Vavilapalli fb15053124 HDFS-7314. Moving to 2.6.1 CHANGES.txt section. 
(cherry picked from commit f103a70af5)
 2015-09-10 16:48:53 -07:00
Vinod Kumar Vavilapalli (I am also known as @tshooter.) c05ba079e3 Updating all CHANGES.txt files to move entires from future releases into 2.6.1 
section given the large number of backports to 2.6.1.

(cherry picked from commit fbbb7ff1ed)
(cherry picked from commit cbf90f93f9)
 2015-09-10 16:10:56 -07:00
Colin Patrick Mccabe ee830b0d27 HDFS-8846. Add a unit test for INotify functionality across a layout version upgrade (Zhe Zhang via Colin P. McCabe) 
(cherry picked from commit a4d9acc51d)
(cherry picked from commit 9264b7e119)
 2015-09-10 11:56:30 -07:00
Ming Ma decf8a6653 HDFS-7314. When the DFSClient lease cannot be renewed, abort open-for-write files rather than the entire DFSClient. (mingma) 
(cherry picked from commit fbd88f1062)
(cherry picked from commit 516bbf1c20)
 2015-09-10 11:56:29 -07:00
Jing Zhao 6c127b44ca HDFS-7609. Avoid retry cache collision when Standby NameNode loading edits. Contributed by Ming Ma. 
(cherry picked from commit 7817674a3a)
(cherry picked from commit 17fb442a4c)
 2015-09-10 11:56:28 -07:00
cnauroth 188096f1fe HDFS-8431. hdfs crypto class not found in Windows. Contributed by Anu Engineer. 
(cherry picked from commit 50eeea1300)

Conflicts:
	hadoop-hdfs-project/hadoop-hdfs/src/main/bin/hdfs.cmd

(cherry picked from commit 25db341278)
 2015-09-10 11:56:27 -07:00
Xiaoyu Yao d3562b4e1d HDFS-8219. setStoragePolicy with folder behavior is different after cluster restart. (surendra singh lilhore via Xiaoyu Yao) 
(cherry picked from commit 0100b15501)
(cherry picked from commit e68e8b3b5c)
 2015-09-10 11:56:22 -07:00