Apache
/
hadoop
1
0
Fork 0
Code Issues Pull Requests Packages Projects Releases Wiki Activity
9,499 Commits 326 Branches 371 Tags 1.5 GiB
Commit Graph

3236 Commits

Author SHA1 Message Date
Vinayakumar B fc0ffc48d4 HDFS-9917. IBR accumulate more objects when SNN was down for sometime. (Contributed by Brahma Reddy Battula) 2016-04-05 09:59:37 +08:00
Kihwal Lee 69d4fa0deb HDFS-10178. Permanent write failures can happen if pipeline recoveries occur for the first packet. Contributed by Kihwal Lee. 
(cherry picked from commit a7d1fb0cd2)
TestClientProtocolForPipelineRecovery modified to use DFSConfigKeys instead of HdfsClientConfigKeys.
 2016-04-04 16:54:04 -05:00
Kihwal Lee efb210cc30 Fix CHANGES.txt 2016-03-28 11:31:07 -05:00
Walter Su ec459e3966 HDFS-10182. Hedged read might overwrite user's buf. Contributed by zhouyingchao. 
(cherry picked from commit d8383c687c)
 2016-03-28 15:46:15 +08:00
Aaron T. Myers 17dbf82f54 HDFS-8101. DFSClient use of non-constant DFSConfigKeys pulls in WebHDFS classes at runtime. Contributed by Sean Busbey. 
(cherry picked from commit 3fe61e0bb0)
 2016-03-26 17:54:09 -07:00
Kihwal Lee a66d2a2a85 HDFS-9874. Long living DataXceiver threads cause volume shutdown to block. Contributed by Rushabh Shah. 
(cherry picked from commit 63c966a3fb)
(cherry picked from commit 242c7f1fee)

Conflicts:
	hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/datanode/fsdataset/impl/FsDatasetImpl.java
	hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/datanode/fsdataset/impl/TestFsDatasetImpl.java
 2016-03-18 10:38:33 -05:00
Akira Ajisaka 6902789a92 HDFS-9860. Backport HDFS-9638 to branch-2.7. Contributed by Wei-Chiu Chuang. 2016-03-17 02:59:51 +09:00
Kihwal Lee b8e73cff8f HDFS-9904. testCheckpointCancellationDuringUpload occasionally fails. Contributed by Lin Yiqun. 
(cherry picked from commit d457401784)
 2016-03-15 10:56:46 -05:00
Akira Ajisaka 7f43eb9547 HDFS-9812. Streamer threads leak if failure happens when closing DFSOutputStream. Contributed by Lin Yiqun. 
(cherry picked from commit 352d299cf8)
(cherry picked from commit fe0009a2bd)
 2016-03-08 11:15:27 +09:00
Masatake Iwasaki e768f8b771 HDFS-9865. TestBlockReplacement fails intermittently in trunk (Lin Yiqun via iwasakims) 2016-03-08 02:13:50 +09:00
Masatake Iwasaki 08fc048018 HDFS-9048. DistCp documentation is out-of-dated (Daisuke Kobayashi via iwasakims) 
(cherry picked from commit 33a412e8a4)

 Conflicts:
	hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt

(cherry picked from commit 55f7ceb0db)
 2016-03-03 19:00:50 +09:00
Akira Ajisaka 8f2d0d4cea HDFS-9851. NameNode throws NPE when setPermission is called on a path that does not exist. Contributed by Brahma Reddy Battula. 2016-03-02 14:45:33 +09:00
Akira Ajisaka 2f3ceb8c65 HDFS-9766. TestDataNodeMetrics#testDataNodeTimeSpend fails intermittently. Contributed by Xiao Chen. 
(cherry picked from commit 81fbc40f084eab85750cc7d4cc967d43d70d8333)
(cherry picked from commit e840141726)
 2016-03-02 14:16:58 +09:00
Kihwal Lee 006ed54a16 HDFS-9880. TestDatanodeRegistration fails occasionally. Contributed by Kihwal Lee. 
(cherry picked from commit e76b13c415)
 2016-03-01 17:02:43 -06:00
Kihwal Lee 9bc9e13a97 HDFS-8791. block ID-based DN storage layout can be very slow for datanode on ext4. Contributed by Chris Trezzo. 
(cherry picked from commit b8dbffbef87c0ee0bee5e17649586fc9f0e1f55b)
 2016-03-01 15:01:04 -06:00
Kihwal Lee 9fa300ad1f HDFS-9198. Coalesce IBR processing in the NN. Contributed by Daryn Sharp. 2016-03-01 13:02:05 -06:00
Akira Ajisaka e14ab939bf HDFS-9864. Correct reference for RENEWDELEGATIONTOKEN and CANCELDELEGATIONTOKEN in webhdfs doc. Contributed by Brahma Reddy Battula. 
(cherry picked from commit 056f9013122dc3f0effc62d620261c312dd7f8ed)
(cherry picked from commit 6a9f8d95b7c4f9c66253d975ebbca1d186267d4e)
 2016-02-29 18:41:57 +09:00
Kihwal Lee 25b3531eb4 HDFS-9395. Make HDFS audit logging consistant. Contributed by Kuhu Shukla. 2016-02-26 09:45:52 -06:00
Vinayakumar B 33b961ee88 HDFS-8578. On upgrade, Datanode should process all storage/data dirs in parallel. Contributed by vinayakumarb and szetszwo 
Addendum to fix compilation
 2016-02-23 14:39:24 +05:30
Tsz-Wo Nicholas Sze 1b139593c5 HDFS-8578. On upgrade, Datanode should process all storage/data dirs in parallel. Contributed by vinayakumarb and szetszwo 2016-02-22 15:18:46 -08:00
Zhe Zhang 79027309c9 HDFS-9688. Test the effect of nested encryption zones in HDFS downgrade. 
Change-Id: I2a802325a2bc82c97012d2c9fb388b292fe3af8f
 2016-02-22 15:10:31 -08:00
Kihwal Lee 019770c769 HDFS-4946. Allow preferLocalNode in BlockPlacementPolicyDefault to be configurable. Contributed by James Kinley and Nathan Roberts. 
(cherry picked from commit ba6d5ed955)

Conflicts:
	hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/DFSConfigKeys.java
	hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/blockmanagement/BlockPlacementPolicyDefault.java
 2016-02-18 14:10:43 -06:00
Kihwal Lee a4492dc6ac HDFS-8845. DiskChecker should not traverse the entire tree (Chang Li via Colin P. McCabe) 
(cherry picked from commit ec183faadc)

Conflicts:
	hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt
 2016-02-17 09:18:55 -06:00
Akira Ajisaka 8377683b63 HDFS-9765. TestBlockScanner#testVolumeIteratorWithCaching fails intermittently. (aajisaka) 
(cherry picked from commit 8ed07bd3e6)
(cherry picked from commit 8dec72bfd7)
 2016-02-15 19:10:03 +09:00
Junping Du 63069389ee Set the release date for 2.6.4. 
(cherry picked from commit ec12ce8f48)
(cherry picked from commit a28be9edc7)
 2016-02-13 01:50:30 -08:00
Kihwal Lee a30043ca26 HDFS-9779 . TestReplicationPolicyWithNodeGroup NODE variable picks wrong rack value. Contributed by Kuhu Shukla. 
(cherry picked from commit a7fce9ab41)

Conflicts:
	hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt
 2016-02-11 08:15:35 -06:00
Akira Ajisaka 948dd27960 HDFS-9752. Permanent write failures may happen to slow writers during datanode rolling upgrades. (Contributed by Walter Su) 2016-02-10 03:52:42 +09:00
Akira Ajisaka 2a46d9c6c4 HDFS-9784. Example usage is not correct in Transparent Encryption document. Contributed by Ohnishi Takashi. 
(cherry picked from commit 60d2011b7c)
(cherry picked from commit cb53dfcc95)
 2016-02-10 03:33:31 +09:00
Zhe Zhang ef05096f95 HDFS-9347. Invariant assumption in TestQuorumJournalManager.shutdown() is wrong. Contributed by Wei-Chiu Chuang. 
Change-Id: Idde8522e11c77f82b376e0c90a9bf33d163d5cb9
(cherry picked from commit ef3f3f6bb1)

Conflicts:
	hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt
 2016-02-09 10:42:21 +08:00
Haohui Mai cb3aa41f53 HDFS-9724. Degraded performance in WebHDFS listing as it does not reuse ObjectMapper. Contributed by Akira Ajisaka. 2016-02-04 11:36:12 -08:00
Kihwal Lee c10bf788eb HDFS-9730. Storage ID update does not happen when there is a layout change. Contributed by Tsz Wo Nicholas Sze. 
(cherry picked from commit 496f33de0c)

Conflicts:
	hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/datanode/DataStorage.java
 2016-02-04 13:32:11 -06:00
Vinayakumar B bef53683b6 HDFS-9629. Update the footer of Web UI to show year 2016 (Contributed by Xiao Chen and Vinayakumar B) 
(cherry picked from commit ddcd6954d03d3caadc8b69f7b886a3a17c3c9d0c)
(cherry picked from commit 8cdfdbf815bf89eee3184ec7cd00e1924c1ea316)
 2016-02-04 13:20:27 +05:30
Kihwal Lee dd11c8274d HDFS-9743. Fix TestLazyPersistFiles#testFallbackToDiskFull in branch-2.7. Contributed by Kihwal Lee. 2016-02-03 12:51:01 -06:00
Vinayakumar B af9de19cfd HDFS-9740. Use a reasonable limit in DFSTestUtil.waitForMetric() (Contributed by Chang Li) 
(cherry picked from commit eb2fb943fd)

 Conflicts:
	hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/DFSTestUtil.java

(cherry picked from commit 6941e2ccbf)

 Conflicts:
	hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/DFSTestUtil.java
 2016-02-03 11:24:26 +05:30
Junping Du bfee573baf Adding release 2.6.5 to CHANGES.txt 2016-02-02 18:19:08 -08:00
Colin Patrick Mccabe 449d2ad85c HDFS-9669. TcpPeerServer should respect ipc.server.listen.queue.size (Elliot Clark via cmccabe) 
(cherry picked from commit 2da03b48eb)
(cherry picked from commit 60d3a3c30b)
(cherry picked from commit c4c94e1cf2)

Conflicts:
	hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/datanode/DataNode.java
	hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/datanode/SecureDataNodeStarter.java
 2016-02-02 11:17:28 -08:00
Kihwal Lee 2f031830e8 HDFS-9072. Fix random failures in TestJMXGet. Contributed by J.Andreina. 2016-02-02 11:07:48 -06:00
Yongjun Zhang 3d88293dda HDFS-9406. FSImage may get corrupted after deleting snapshot. (Contributed by Jing Zhao, Stanislav Antic, Vinayakumar B, Yongjun Zhang) 
(cherry picked from commit 34ab50ea92)

Conflicts:
	hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/namenode/TestINodeFile.java

(cherry picked from commit fc8d9cc758)

Conflicts:
	hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/INodeFile.java
	hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/snapshot/DirectoryWithSnapshotFeature.java
	hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/namenode/TestINodeFile.java
	hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/namenode/snapshot/TestSnapshotDeletion.java
 2016-02-01 15:29:03 -08:00
Tsz-Wo Nicholas Sze 672481e28c HDFS-9654. Code refactoring for HDFS-8578. 2016-01-28 11:09:05 +08:00
Vinayakumar B 07704aab84 HDFS-9690. ClientProtocol.addBlock is not idempotent after HDFS-8071.(Contributed by Tsz Wo Nicholas Sze) 2016-01-26 21:34:07 +05:30
Vinod Kumar Vavilapalli (I am also known as @tshooter.) 34c9289f6c Release process for 2.7.2: Set the release date for 2.7.2 
(cherry picked from commit ec4d2d9f40)
 2016-01-25 15:48:18 -08:00
Kihwal Lee e160ddfb8d HDFS-9634. webhdfs client side exceptions don't provide enough details. Contributed by Eric Payne. 
(cherry picked from commit 7b70500484574a565dd8cd5c7d8b5bc7c6d91154)

Conflicts:
	hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt
 2016-01-21 09:34:29 -06:00
Vinayakumar B ca29933405 HDFS-9625. set replication for empty file failed when set storage policy (Contributed by DENG FEI) 
(cherry picked from commit b7372b7166)
(cherry picked from commit 845acfd96c)
 2016-01-21 19:21:00 +05:30
Vinayakumar B eb3dcc0739 HDFS-9661. Deadlock in DN.FsDatasetImpl between moveBlockAcrossStorage and createRbw (Contributed by ade) 
(cherry picked from commit 1425578690)
 2016-01-20 21:11:39 +05:30
Tsz-Wo Nicholas Sze 2131b9820d In CHANGES.txt, move HDFS-9294 to Release 2.6.4. 2016-01-16 23:48:03 +08:00
Arpit Agarwal 6cd5fcb634 HDFS-9415. Document dfs.cluster.administrators and dfs.permissions.superusergroup. (Contributed by Xiaobing Zhou) 2016-01-14 14:19:51 -08:00
Yongjun Zhang 7a550db931 HDFS-9648. TestStartup.testImageChecksum is broken by HDFS-9569's message change. (Wei-Chiu Chuang via Yongjun Zhang) 
(cherry picked from commit 817cc1f02a)
 2016-01-14 12:45:42 -08:00
Kihwal Lee a90dcb0865 Fix up CHANGES.txt 2016-01-14 09:58:35 -06:00
Junping Du c2d1d69525 Fix yarn CHANGES.txt entry for 2.7.2 2016-01-14 06:17:42 -08:00
Junping Du 0e0272f120 Add missing CHANGES.txt entry for HDFS-8767. 
(cherry picked from commit e50dd19a9c)

Conflicts:

	hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt
(cherry picked from commit 62eabec549)
 2016-01-14 06:14:56 -08:00
Vinod Kumar Vavilapalli (I am also known as @tshooter.) faebc20e26 HADOOP-5323. Trash documentation should describe its directory structure and configurations. (Weiwei Yang via ozawa) 
Moved entry from HDFS CHANGES.txt to that of common.

(cherry picked from commit cb73c753da)
 2016-01-13 14:56:58 -08:00
Kihwal Lee 83a4791243 HDFS-9047. Retire libwebhdfs. 2016-01-13 11:01:35 -06:00
Yongjun Zhang 4c09a75f3e HDFS-9569. Log the name of the fsimage being loaded for better supportability. (Yongjun Zhang) 
(cherry picked from commit 25051c3bd0)

Conflicts:
	hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/FSImageFormat.java
 2016-01-12 09:40:56 -08:00
Kihwal Lee b06e39de4f HDFS-9574. Reduce client failures during datanode restart. Contributed by Kihwal Lee. 2016-01-08 11:31:49 -06:00
Vinayakumar B 5dc5bb2812 HDFS-9600. do not check replication if the block is under construction (Contributed by Phil Yang) 
(cherry picked from commit 34cd7cd765)
(cherry picked from commit aa710bd461)

Conflicts:
	hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/blockmanagement/BlockManager.java
	hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/blockmanagement/DecommissionManager.java
	hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/blockmanagement/TestBlockManager.java
 2016-01-07 11:50:12 +05:30
Akira Ajisaka 68c68e187f Set release date for 2.6.3 
(cherry picked from commit db99e30f67)
(cherry picked from commit 247ab92464)
 2016-01-04 16:14:30 +09:00
Akira Ajisaka 3a1c98bba3 HADOOP-11252. RPC client does not time out by default. Contributed by Wilfred Spiegelenburg and Masatake Iwasaki. 
(cherry picked from commit 64ae85fd2e)
(cherry picked from commit d98cfe611f)
 2016-01-04 14:35:54 +09:00
Kihwal Lee cdf9f1c662 HDFS-7163. WebHdfsFileSystem should retry reads according to the configured retry policy. Contributed by Eric Payne. 2015-12-28 10:57:31 -06:00
Haohui Mai e0e5a0fa82 HDFS-8914. Document HA support in the HDFS HdfsDesign.md. Contributed by Lars Francke. 
(cherry picked from commit 049ccf71ce)
 2015-12-21 21:44:46 +09:00
Akira Ajisaka 5ffad1551b HDFS-9505. HDFS Architecture documentation needs to be refreshed. Contributed by Masatake Iwasaki. 
(cherry picked from commit fa544020f6)
(cherry picked from commit 7526a2a005)
 2015-12-21 21:27:28 +09:00
Junping Du 803fdc9f5e Creating 2.6.4 entries in CHANGES.txt files. 
(cherry picked from commit 028894281b)

(cherry picked from commit 092c469059)
 2015-12-18 14:24:51 +09:00
Kihwal Lee 03292b234e HDFS-9533. seen_txid in the shared edits directory is modified during bootstrapping. Contributed by Kihwal Lee. 
(cherry picked from commit 5cb1e0118b)

Conflicts:
	hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/ha/BootstrapStandby.java
	hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/namenode/ha/TestBootstrapStandby.java
(cherry picked from commit 3c0ae5914c)
 2015-12-17 17:28:24 -06:00
cnauroth 56882e7b1c Revert "HDFS-9569. Log the name of the fsimage being loaded for better supportability. Contributed by Yongjun Zhang." 
This reverts commit 4d2ee51c52.
 2015-12-17 13:49:47 -08:00
Kihwal Lee 4d2ee51c52 HDFS-9569. Log the name of the fsimage being loaded for better supportability. Contributed by Yongjun Zhang. 
(cherry picked from commit eb6939cea0)
 2015-12-17 10:33:15 -06:00
Plamen Jeliazkov d97696e4c4 HDFS-9516. Truncate file fails with data dirs on multiple disks. Contributed by Plamen Jeliazkov. 2015-12-17 00:09:48 -08:00
Tom Zeng 6c62d422d6 HADOOP-12415. Add io.netty dependency to hadoop-nfs and to hadoop-hdfs, needed to build Bigtop successfuly, see BIGTOP-2049 
Signed-off-by: Konstantin Boudnik <cos@boudnik.org>
(cherry picked from commit b4c6b511e7)
 2015-12-16 14:27:30 -08:00
Kihwal Lee a02312546a HDFS-9445. Datanode may deadlock while handling a bad volume. Contributed by Walter Su. 
(cherry picked from commit a48301791e)
 2015-12-11 08:47:22 -06:00
Vinod Kumar Vavilapalli 067d14f503 HDFS-9273. Moving to 2.6.3 CHANGES section to reflect the backport. 2015-12-08 11:36:15 -08:00
cnauroth e23ab9d02b HDFS-9273. ACLs on root directory may be lost after NN restart. Contributed by Xiao Chen. 
(cherry picked from commit 1b525a9c32)
(cherry picked from commit faf1c60899)
 2015-12-08 11:22:19 -08:00
Zhe Zhang 670617f325 HDFS-9314. Improve BlockPlacementPolicyDefault's picking of excess replicas. Contributed by Xiao Chen. 
Change-Id: Idf99293085531165239369155c039b55db0eed83
 2015-12-04 09:49:43 -08:00
Zhe Zhang b39de3b0bd HDFS-9313. Possible NullPointerException in BlockManager if no excess replica can be chosen. Contributed by Ming Ma. Branch-2.7 patch contributed by Xiao Chen. 
Change-Id: Iadaee6be2b44f7b0322d7b0e58f8fc983ef463f6
 2015-12-03 23:19:17 -08:00
Zhe Zhang 1ae8a8342e HDFS-8647. Abstract BlockManager's rack policy into BlockPlacementPolicy. Contributed by Brahma Reddy Battula. Branch-2.7 patch contributed by Xiao Chen. 
Change-Id: I1b32627bc2a2f30f5debeaba7663fb2777958079
 2015-12-03 10:30:48 -08:00
Tsz-Wo Nicholas Sze 4123650e92 HDFS-9294. DFSClient deadlock when close file and failed to renew lease. Contributed by Brahma Reddy Battula 2015-12-02 17:55:10 -08:00
Andrew Wang 10e8a67d23 HDFS-9470. Encryption zone on root not loaded from fsimage after NN restart. Xiao Chen via wang. 
(cherry picked from commit 9b8e50b424)
(cherry picked from commit ce1111ceea)

 Conflicts:
	hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/FSDirectory.java
 2015-11-30 14:53:04 -08:00
Vinayakumar B 0c48c17ae9 HDFS-9426. Rollingupgrade finalization is not backward compatible (Contributed by Kihwal Lee) 
(cherry picked from commit c62d42cd8b)
 2015-11-26 09:42:26 +05:30
Tsz-Wo Nicholas Sze af811030e5 Move HDFS-9434 to 2.6.3 in CHANGES.txt. 2015-11-24 12:49:34 -08:00
Tsz-Wo Nicholas Sze 09f4a4eefd HDFS-9434. Recommission a datanode with 500k blocks may pause NN for 30 seconds for printing info log messags. 2015-11-24 12:47:50 -08:00
Akira Ajisaka bb6983e153 HDFS-9431. DistributedFileSystem#concat fails if the target path is relative. Contributed by Kazuho Fujii. 
(cherry picked from commit ac1aa6c819)
(cherry picked from commit c74e42b4a2)
 2015-11-19 20:55:52 +09:00
Ming Ma 1b0f277eae HDFS-9413. getContentSummary() on standby should throw StandbyException. (Brahma Reddy Battula via mingma) 
(cherry picked from commit 02653add98)
(cherry picked from commit 42b55ff23e)

 Conflicts:
	hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt

(cherry picked from commit 83200450f6)

 Conflicts:
	hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt
 2015-11-17 16:12:58 +05:30
Akira Ajisaka ab04c9ab5f HDFS-8615. Correct HTTP method in WebHDFS document. Contributed by Brahma Reddy Battula. 
(cherry picked from commit 1a169a26bc)
(cherry picked from commit 9f9315008e)
 2015-11-17 16:32:07 +09:00
Kihwal Lee 6f678deb86 HDFS-9383. TestByteArrayManager#testByteArrayManager fails. Contributed by Tsz Wo Nicholas Sze. 
(cherry picked from commit ef926b2e38)
 2015-11-09 08:58:58 -06:00
Arpit Agarwal 63ed399e1c HDFS-6481. DatanodeManager#getDatanodeStorageInfos() should check the length of storageIDs. (Contributed by szetszwo) 2015-11-06 10:32:34 -08:00
Kihwal Lee 616ed9084b HDFS-4937. ReplicationMonitor can infinite-loop in BlockPlacementPolicyDefault#chooseRandom(). Contributed by Kihwal Lee. 
(cherry picked from commit ff47f35dee)
 2015-11-05 09:27:36 -06:00
Kihwal Lee 397b554c36 HDFS-9289. Make DataStreamer#block thread safe and verify genStamp in commitBlock. Contributed by Chang Li. 2015-11-04 12:10:59 -06:00
Jason Lowe 7499f71e77 Add 2.7.3 release to CHANGES.txt 
(cherry picked from commit 0383a3973b)

Conflicts:

	hadoop-common-project/hadoop-common/CHANGES.txt
	hadoop-mapreduce-project/CHANGES.txt
	hadoop-yarn-project/CHANGES.txt
 2015-11-04 16:29:56 +00:00
Vinod Kumar Vavilapalli 09348c4bce Preparing for 2.7.3 development 2015-11-03 15:18:10 -08:00
Kihwal Lee 2c51bbda51 HDFS-9221. HdfsServerConstants#ReplicaState#getState should avoid calling values() since it creates a temporary array. (Staffan Friberg via yliu) 
(cherry picked from commit 0ff1216100)
 2015-11-02 09:07:16 -06:00
yliu e1798e8f21 Revert "HDFS-4937. ReplicationMonitor can infinite-loop in BlockPlacementPolicyDefault#chooseRandom(). Contributed by Kihwal Lee." 
This reverts commit c250b21c23.
 2015-10-31 16:25:53 +08:00
Sangjin Lee c8767fb68c Updated the 2.6.2 final release date. 
(cherry picked from commit a4a6b5b4b4)
 2015-10-30 18:53:10 -07:00
Jason Lowe e894be93f6 Creating 2.6.3 entries in CHANGES.txt files. 
(cherry picked from commit eadf7b3096)

Conflicts:

	hadoop-common-project/hadoop-common/CHANGES.txt
	hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt
	hadoop-mapreduce-project/CHANGES.txt
	hadoop-yarn-project/CHANGES.txt
 2015-10-30 14:55:33 +00:00
Kihwal Lee c250b21c23 HDFS-4937. ReplicationMonitor can infinite-loop in BlockPlacementPolicyDefault#chooseRandom(). Contributed by Kihwal Lee. 
(cherry picked from commit 43539b5ff4)
 2015-10-30 09:39:49 -05:00
Kihwal Lee 13b256ed22 HDFS-9083. Replication violates block placement policy. Contributed by Rushabh Shah. 2015-10-28 14:49:41 -05:00
Akira Ajisaka ce01a5951b HDFS-9317. Document fsck -blockId and -storagepolicy options in branch-2.7. (aajisaka) 2015-10-28 13:14:58 +09:00
Kihwal Lee 59a2072135 HDFS-8950. NameNode refresh doesn't remove DataNodes that are no longer in the allowed list. Contributed by Daniel Templeton. 2015-10-27 16:42:33 -05:00
Kihwal Lee 1d23e1ec07 HDFS-7725. Incorrect 'nodes in service' metrics caused all writes to fail. Contributed by Ming Ma. 
(cherry picked from commit 8104d52269)
 2015-10-27 11:09:05 -05:00
Arpit Agarwal 653ef52ef2 HDFS-9305. Delayed heartbeat processing causes storm of subsequent heartbeats. (Contributed by Arpit Agarwal) 2015-10-26 15:47:37 -07:00
Kihwal Lee f2fd011cc1 HDFS-9290. DFSClient#callAppend() is not backward compatible for slightly older NameNodes. Contributed by Tony Wu. 
(cherry picked from commit b9e0417bdf)

Conflicts:
	hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/DFSClient.java
 2015-10-23 16:46:14 -05:00
Kihwal Lee 4cf7f8441a HDFS-9220. Reading small file (< 512 bytes) that is open for append fails due to incorrect checksum. Contributed by Jing Zhao. 
(cherry picked from commit c7c36cbd62)
 2015-10-15 11:26:14 -05:00
Kihwal Lee efba749760 HDFS-8676. Delayed rolling upgrade finalization can cause heartbeat expiration. Contributed by Walter Su. 
(cherry picked from commit 5b43db47a3)
 2015-10-13 13:25:10 -05:00
Kihwal Lee 61388b41e7 HDFS-8656. Preserve compatibility of ClientProtocol#rollingUpgrade after finalization. 
(cherry picked from commit 60b858bfa6)

Conflicts:
	hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt
	hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/protocol/ClientProtocol.java
	hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/TestRollingUpgrade.java
 2015-10-13 10:46:20 -05:00
Kihwal Lee 3a5b9f49df HDFS-7645. Rolling upgrade is restoring blocks from trash multiple times (Contributed by Vinayakumar B and Keisuke Ogiwara) 
(cherry picked from commit abf3ad988d)
 2015-10-13 10:29:42 -05:00
Kihwal Lee 5cb80992fb HDFS-9178. Slow datanode I/O can cause a wrong node to be marked bad. Contributed by Kihwal Lee. 
(cherry picked from commit 99e5204ff5)

Conflicts:
	hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/datanode/BlockReceiver.java
	hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/TestClientProtocolForPipelineRecovery.java
 2015-10-07 10:24:55 -05:00
Kihwal Lee b2bc72a051 HDFS-8850. VolumeScanner thread exits with exception if there is no block pool to be scanned but there are suspicious blocks. (Colin Patrick McCabe via yliu) 
(cherry picked from commit d2b941f94a)

Conflicts:
	hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt
 2015-10-02 14:59:29 -05:00
Kihwal Lee 6e7c76a5bd HDFS-9106. Transfer failure during pipeline recovery causes permanent write failures. Contributed by Kihwal Lee. 2015-09-28 15:19:57 -05:00
Vinod Kumar Vavilapalli (I am also known as @tshooter.) 2b4c7fcc67 Release process for 2.6.1: Set the release date for 2.6.1 
(cherry picked from commit 1f707ecffd)
(cherry picked from commit f48af332a7)
 2015-09-23 15:13:09 -07:00
Vinayakumar B d800e2e9ed HDFS-9043. Doc updation for commands in HDFS Federation (Contributed by J.Andreina) 
(cherry picked from commit 57003fa971)
(cherry picked from commit 96e3fbf401)
 2015-09-22 16:40:19 +05:30
Vinayakumar B 7d3e882a49 HDFS-9042. Update document for the Storage policy name (Contributed by J.Andreina) 
(cherry picked from commit d8455479b8)
 2015-09-12 12:37:01 +05:30
Kihwal Lee f2cc6b5fdf HDFS-8099. Change "DFSInputStream has been closed already" message to debug log level (Charles Lamb via Colin P. McCabe) 
(cherry picked from commit 30acb7372a)

Conflicts:
	hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt
 2015-09-11 15:59:29 -05:00
Vinod Kumar Vavilapalli fb15053124 HDFS-7314. Moving to 2.6.1 CHANGES.txt section. 
(cherry picked from commit f103a70af5)
 2015-09-10 16:48:53 -07:00
Vinod Kumar Vavilapalli (I am also known as @tshooter.) c05ba079e3 Updating all CHANGES.txt files to move entires from future releases into 2.6.1 
section given the large number of backports to 2.6.1.

(cherry picked from commit fbbb7ff1ed)
(cherry picked from commit cbf90f93f9)
 2015-09-10 16:10:56 -07:00
Colin Patrick Mccabe ee830b0d27 HDFS-8846. Add a unit test for INotify functionality across a layout version upgrade (Zhe Zhang via Colin P. McCabe) 
(cherry picked from commit a4d9acc51d)
(cherry picked from commit 9264b7e119)
 2015-09-10 11:56:30 -07:00
Ming Ma decf8a6653 HDFS-7314. When the DFSClient lease cannot be renewed, abort open-for-write files rather than the entire DFSClient. (mingma) 
(cherry picked from commit fbd88f1062)
(cherry picked from commit 516bbf1c20)
 2015-09-10 11:56:29 -07:00
Jing Zhao 6c127b44ca HDFS-7609. Avoid retry cache collision when Standby NameNode loading edits. Contributed by Ming Ma. 
(cherry picked from commit 7817674a3a)
(cherry picked from commit 17fb442a4c)
 2015-09-10 11:56:28 -07:00
cnauroth 188096f1fe HDFS-8431. hdfs crypto class not found in Windows. Contributed by Anu Engineer. 
(cherry picked from commit 50eeea1300)

Conflicts:
	hadoop-hdfs-project/hadoop-hdfs/src/main/bin/hdfs.cmd

(cherry picked from commit 25db341278)
 2015-09-10 11:56:27 -07:00
Xiaoyu Yao d3562b4e1d HDFS-8219. setStoragePolicy with folder behavior is different after cluster restart. (surendra singh lilhore via Xiaoyu Yao) 
(cherry picked from commit 0100b15501)
(cherry picked from commit e68e8b3b5c)
 2015-09-10 11:56:22 -07:00
Kihwal Lee cfb27d7f78 HDFS-8046. Allow better control of getContentSummary. Contributed by Kihwal Lee. 
(cherry picked from commit 285b31e75e)

(cherry picked from commit 7e622076d4)
 2015-09-10 10:53:19 -07:00
Akira Ajisaka c5a74c5c37 HDFS-9033. dfsadmin -metasave prints "NaN" for cache used%. Contributed by Brahma Reddy Battula. 
(cherry picked from commit 0113e4528d)
(cherry picked from commit eb9e06f8e3)
 2015-09-09 17:23:25 +09:00
Jing Zhao fd100ccbae HDFS-8384. Allow NN to startup if there are files having a lease but are not under construction. Contributed by Jing Zhao. 2015-09-08 10:52:42 -07:00
Tsuyoshi Ozawa 139fd21c83 HADOOP-5323. Trash documentation should describe its directory structure and configurations. Contributed by Weiwei Yang. 
(cherry picked from commit 9a87f81ce6)

Conflicts:
	hadoop-hdfs-project/hadoop-hdfs/src/site/markdown/HdfsDesign.md
 2015-09-03 22:39:12 +09:00
yliu 3ab43accaf HDFS-8995. Flaw in registration bookeeping can make DN die on reconnect. (Kihwal Lee via yliu) 2015-09-02 09:00:22 +08:00
Kihwal Lee 285dad32eb fix merge error in CHANGES.TXT 2015-08-28 15:37:07 -05:00
Kihwal Lee 42aa880df3 HDFS-8879. Quota by storage type usage incorrectly initialized upon namenode restart. Contributed by Xiaoyu Yao. 
(cherry picked from commit 3e715a4f4c)
 2015-08-28 13:27:17 -05:00
Jing Zhao ba0f66984c HDFS-8891. HDFS concat should keep srcs order. Contributed by Yong Zhang. 
(cherry picked from commit dc7a061668)
(cherry picked from commit 2edfc3ce71)
 2015-08-20 20:42:20 -07:00
Vinod Kumar Vavilapalli 9faee040f9 Creating 2.6.2 entries in CHANGES.txt files. 
(cherry picked from commit 33474939a0)
 2015-08-20 17:39:29 -07:00
yliu d4b6c4a3cd HDFS-8863. The remaining space check in BlockPlacementPolicyDefault is flawed. (Kihwal Lee via yliu) 2015-08-20 20:43:26 +08:00
Jing Zhao 4c005a4e16 HDFS-8867. Enable optimized block reports. Contributed by Daryn Sharp. 
(cherry picked from commit f61120d964)
(cherry picked from commit c0a4cd978a)
 2015-08-19 10:41:30 -07:00
Akira Ajisaka f98b5ae3bb HDFS-8852. HDFS architecture documentation of version 2.x is outdated about append write support. Contributed by Ajith S. 
(cherry picked from commit fc509f66d8)
(cherry picked from commit 38b344adfa)
 2015-08-18 23:33:28 +09:00
Vinayakumar B 4d94bc1d95 HDFS-8270. create() always retried with hardcoded timeout when file already exists with open lease (Contributed by J.Andreina) 
Moved to 2.6.1

(cherry picked from commit 84bf71295a)

Conflicts:
	hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt
 2015-08-14 16:15:10 +05:30
Vinayakumar B 90f3641728 HDFS-7225. Remove stale block invalidation work when DN re-registers with different UUID. (Zhe Zhang and Andrew Wang) 
Moved to 2.6.1

(cherry picked from commit 08bd4edf40)
 2015-08-14 12:38:59 +05:30
Vinayakumar B b06d342749 HDFS-7263. Snapshot read can reveal future bytes for appended files. Contributed by Tao Luo. 
Moved CHANGES.txt entry to 2.6.1

(cherry picked from commit fa2641143c)
 2015-08-14 11:46:58 +05:30
Vinayakumar B f40714f8d8 HDFS-7235. DataNode#transferBlock should report blocks that don't exist using reportBadBlock (yzhang via cmccabe) 
Moved CHANGES.txt entry to 2.6.1

(cherry picked from commit f2b4bc9b6a)
 2015-08-14 11:38:28 +05:30
Vinayakumar B 5ba4065345 HDFS-7213. processIncrementalBlockReport performance degradation. Contributed by Eric Payne. 
Moved CHANGES.TXT entry to 2.6.1

(cherry picked from commit d25cb8fe12)
(cherry picked from commit 2c0063d420)

Conflicts:
	hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt
 2015-08-14 11:27:46 +05:30
Akira Ajisaka b919c1c044 HDFS-8806. Inconsistent metrics: number of missing blocks with replication factor 1 not properly cleared. Contributed by Zhe Zhang. 
(cherry picked from commit 206d4933a5)
(cherry picked from commit b2a532f61c)
 2015-07-24 18:38:44 +09:00
Akira Ajisaka 3e793224f9 HDFS-6945. BlockManager should remove a block from excessReplicateMap and decrement ExcessBlocks metric when the block is removed. (aajisaka) 
(cherry picked from commit 18a91fe4df)
(cherry picked from commit b85bbca745)
 2015-07-21 11:42:21 +09:00
Kihwal Lee e263219829 HDFS-8722. Optimize datanode writes for small writes and flushes. Contributed by Kihwal Lee 
(cherry picked from commit 59388a8015)
 2015-07-14 14:08:15 -05:00
Vinod Kumar Vavilapalli (I am also known as @tshooter.) dcafd3ccd2 Release process for 2.7.1: Set the release date for 2.7.1. 
(cherry picked from commit bf89ddb9b8)
 2015-07-06 16:41:38 -07:00
Yongjun Zhang 849b47525d HDFS-8659. Block scanner INFO message is spamming logs. Contributed by Yongjun Zhang. 
(cherry picked from commit 43a1288fff)
 2015-06-29 14:56:14 -07:00
Vinod Kumar Vavilapalli a36f090c55 Preparing for 2.7.2 development: mvn versions:set -DnewVersion=2.7.2 2015-06-28 16:41:12 -07:00
Vinod Kumar Vavilapalli 2e99675c72 Adding release 2.7.2 to CHANGES.txt. 
(cherry picked from commit aad6a7d5db)
 2015-06-28 16:33:49 -07:00
Arpit Agarwal f6550361f7 HDFS-8681. BlockScanner is incorrectly disabled by default. (Contributed by Arpit Agarwal) 2015-06-28 14:58:07 -07:00
Arpit Agarwal 4a8db40f3b HDFS-8626. Add test case for branch-2.7 2015-06-23 10:03:44 -07:00
Colin Patrick Mccabe 86c01d90ba HDFS-8480. Fix performance and timeout issues in HDFS-7929 by using hard-links to preserve old edit logs, instead of copying them. (Zhe Zhang via Colin P. McCabe) 
(cherry picked from commit 7b424f938c)
(cherry picked from commit cbd11681ce)

Conflicts:
    hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/TestDFSUpgrade.java
 2015-06-22 14:46:57 -07:00
Arpit Agarwal 25b99e4816 HDFS-7164. Feature documentation for HDFS-6581. (Contributed by Arpit Agarwal) 2015-06-20 13:08:18 -07:00
Allen Wittenauer 0adb6d8292 HDFS-7546. Document, and set an accepting default for dfs.namenode.kerberos.principal.pattern (Harsh J via aw) 
(cherry picked from commit 63613c79c1)
 2015-06-20 07:45:39 -07:00
Arpit Agarwal dc4171cb66 HDFS-8626. Reserved RBW space is not released if creation of RBW File fails. (Contributed by kanaka kumar avvaru) 2015-06-19 18:30:53 -07:00
Yongjun Zhang 8c03a079df HDFS-8633. Fix setting of dfs.datanode.readahead.bytes in hdfs-default.xml to match DFSConfigKeys. Contributed by Ray Chiang. 
(cherry picked from commit b42f1ec13d)
 2015-06-19 09:51:48 -07:00
Kihwal Lee 6de4d52c31 HDFS-4660. Block corruption can happen during pipeline recovery. Contributed by Kihwal Lee. 
(cherry picked from commit c74517c46b)
 2015-06-16 15:44:38 -05:00
Xiaoyu Yao fe35ba1df9 HDFS-8597. Fix TestFSImage#testZeroBlockSize on Windows. Contributed by Xiaoyu Yao. 
(cherry picked from commit 4e88ff5b27)

Conflicts:
	hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt
 2015-06-16 12:51:34 -07:00
Yongjun Zhang f4ae56d605 HDFS-8544. Incorrect port specified in HFTP Guide document in branch-2. (Contributed by Brahma Reddy Battula) 
(cherry picked from commit b1d3ed02c1)
 2015-06-15 21:51:21 -07:00
Tsz-Wo Nicholas Sze 632cb542f6 HDFS-8361. Choose SSD over DISK in block placement. 2015-06-15 17:29:33 -07:00
Tsz-Wo Nicholas Sze 7a248f2e9e HDFS-8576. Lease recovery should return true if the lease can be released and the file can be closed. Contributed by J.Andreina 2015-06-15 16:13:56 -07:00
Arpit Agarwal a1810d75f0 HDFS-8600. TestWebHdfsFileSystemContract.testGetFileBlockLocations fails in branch-2.7. (Contributed by Arpit Agarwal) 2015-06-14 20:18:31 -07:00
Arpit Agarwal 1145d09e0a HDFS-8595. TestCommitBlockSynchronization fails in branch-2.7. (Patch applies to all branches). (Contributed by Arpit Agarwal) 2015-06-14 16:24:42 -07:00
Yongjun Zhang 6dda57a817 HDFS-8596. TestDistributedFileSystem et al tests are broken in branch-2 due to incorrect setting of "datanode" attribute. Contributed by Yongjun Zhang. 
(cherry picked from commit b0dc291961)

Conflicts:
	hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/datanode/web/DatanodeHttpServer.java

(cherry picked from commit cd578fa289)
 2015-06-14 12:04:16 -07:00
Haohui Mai 3651feb49d HDFS-8572. DN always uses HTTP/localhost@REALM principals in SPNEGO. Contributed by Haohui Mai. 2015-06-11 19:14:48 -07:00
Arpit Agarwal 3ec9b63586 HDFS-8583. Document that NFS gateway does not work with rpcbind on SLES 11. (Arpit Agarwal) 2015-06-11 15:08:15 -07:00
Colin Patrick Mccabe 0bcc68337b HDFS-8566. HDFS documentation about debug commands wrongly identifies them as "hdfs dfs" commands (Surendra Singh Lilhore via Colin P. McCabe) 
(cherry picked from commit 587309bbba)
(cherry picked from commit 754dd3496c)
 2015-06-11 14:30:21 -07:00
Arpit Agarwal 105cadb421 HADOOP-12078. The default retry policy does not handle RetriableException correctly. (Contributed by Arpit Agarwal) 2015-06-09 20:58:55 -07:00
Xiaoyu Yao 8c1a46ca4f HDFS-8522. Change heavily recorded NN logs from INFO to DEBUG level. Contributed by xyao. 2015-06-05 17:13:40 -07:00
Vinayakumar B dd98cfd328 HDFS-8523. Remove usage information on unsupported operation 'fsck -showprogress' from branch-2 (Contributed by J.Andreina) 
(cherry picked from commit 0ed9c2d8fe)

Conflicts:
	hadoop-hdfs-project/hadoop-hdfs/src/site/markdown/HDFSCommands.md
 2015-06-03 15:18:42 +05:30
Vinayakumar B 8090a6ee63 HDFS-8270. create() always retried with hardcoded timeout when file already exists with open lease (Contributed by J.Andreina) 
(cherry picked from commit 54f83d9bd9)

Conflicts:
	hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/DFSConfigKeys.java
	hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/NameNodeProxies.java
	hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/TestFileCreation.java
 2015-06-03 12:20:54 +05:30
Colin Patrick Mccabe a7058caa72 HDFS-8521. Add VisibleForTesting annotation to BlockPoolSlice#selectReplicaToDelete. (cmccabe) 
(cherry picked from commit 806e407ac8)
(cherry picked from commit 12497b0543)

Conflicts:
	hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/datanode/fsdataset/impl/BlockPoolSlice.java
 2015-06-02 20:07:35 -07:00
Colin Patrick Mccabe 323ffccfe9 HDFS-8486. DN startup may cause severe data loss (Daryn Sharp via Colin P. McCabe) 
(cherry picked from commit 03fb5c6425)
(cherry picked from commit 17e369511d)
 2015-06-02 11:46:12 -07:00
Kihwal Lee 1ff3f16ed0 HDFS-5215. dfs.datanode.du.reserved is not considered while computing 
available space ( Brahma Reddy Battula via Yongjun Zhang)
(cherry picked from commit 66763bb06f)

Conflicts:
	hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt
 2015-06-02 08:14:57 -05:00
Xiaoyu Yao c550825673 HDFS-8451. DFSClient probe for encryption testing interprets empty URI property for enabled. Contributed by Steve Loughran. 
(cherry picked from commit 05e04f34f2)

Conflicts:
	hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/DFSUtil.java
 2015-05-21 12:49:49 -07:00
Kihwal Lee 7ac66f8712 HDFS-8404. Pending block replication can get stuck using older genstamp. Contributed by Nathan Roberts. 
(cherry picked from commit 8860e352c3)

Conflicts:
	hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/blockmanagement/BlockManager.java
 2015-05-19 13:17:45 -05:00
Tsz-Wo Nicholas Sze 59d1b4a323 HDFS-8405. Fix a typo in NamenodeFsck. Contributed by Takanobu Asanuma 2015-05-19 03:17:51 +08:00
cnauroth 82530ba0f1 HADOOP-11966. Variable cygwin is undefined in hadoop-config.sh when executed through hadoop-daemon.sh. Contributed by Chris Nauroth. 
(cherry picked from commit ddc4afeff2)
 2015-05-13 12:26:02 -07:00
Tsz-Wo Nicholas Sze 2113e0a3f2 HDFS-8143. Mover should exit after some retry when failed to move blocks. Contributed by surendra singh lilhore 2015-05-13 11:59:01 -07:00
Vinayakumar B ef2488e7f0 HDFS-6300. Prevent multiple balancers from running simultaneously (Contributed by Rakesh R) 
(cherry picked from commit 065d8f2a34)
 2015-05-13 17:30:03 +05:30
Kihwal Lee 1e4a686ea9 HDFS-7916. 'reportBadBlocks' from datanodes to standby Node BPServiceActor goes for infinite loop. Contributed by Rushabh Shah. 
(cherry picked from commit ea11590aad)
 2015-05-11 17:11:51 -05:00
Kihwal Lee f57b1bfbdd HDFS-8245. Standby namenode doesn't process DELETED_BLOCK if the addblock request is in edit log. Contributed by Rushabh S Shah. 
(cherry picked from commit 2d4ae3d18b)
 2015-05-08 16:38:43 -05:00
Kihwal Lee bbb9f6aafe HDFS-7894. Rolling upgrade readiness is not updated in jmx until query command is issued. Contributed by Brahma Reddy Battula. 
(cherry picked from commit 6f622672b6)
 2015-05-08 09:32:44 -05:00
Vinayakumar B 1fbefe586b HDFS-8226. Non-HA rollback compatibility broken (Contributed by J.Andreina) 
Added one missed change, missed while resolving conflict

(cherry picked from commit 3ee1b50dad)
 2015-05-08 15:42:17 +05:30
Vinayakumar B 99309a9fec HDFS-8226. Non-HA rollback compatibility broken (Contributed by J.Andreina) 
(cherry picked from commit e01c1a979c)

Conflicts:
	hadoop-hdfs-project/hadoop-hdfs/src/main/bin/start-dfs.sh
	hadoop-hdfs-project/hadoop-hdfs/src/site/markdown/HdfsUserGuide.md

(cherry picked from commit 3a894b4563)
 2015-05-08 15:42:02 +05:30
Tsz-Wo Nicholas Sze 74bd9666d1 HDFS-7980. Incremental BlockReport will dramatically slow down namenode startup. Contributed by Walter Su 2015-05-07 11:38:15 -07:00
Colin Patrick Mccabe e056e0a3c1 HDFS-8305: HDFS INotify: the destination field of RenameOp should always end with the file name (cmccabe) 
(cherry picked from commit fcd4cb7516)
(cherry picked from commit 1f01d8347a)
 2015-05-05 11:07:13 -07:00
Vinayakumar B 69d837428c HDFS-7916. 'reportBadBlocks' from datanodes to standby Node BPServiceActor goes for infinite loop (Contributed by Vinayakumar B) 
Reverted earlier commit

(cherry picked from commit 318081ccd7)
 2015-05-05 11:07:59 +05:30
Colin Patrick Mccabe 472fd563e4 HDFS-8213. DFSClient should use hdfs.client.htrace HTrace configuration prefix rather than hadoop.htrace (cmccabe) 
(cherry picked from commit b82567d455)
(cherry picked from commit 9edea9507d)
 2015-05-01 11:52:08 -07:00
Arun Suresh 4e6ff8c78b HDFS-8091: ACLStatus and XAttributes should be presented to INodeAttributesProvider before returning to client (asuresh) 
(cherry picked from commit 922b7ed21d)
 2015-05-01 11:42:29 -07:00
Akira Ajisaka 185a1ff275 HDFS-7770. Need document for storage type label of data node storage locations under dfs.data.dir. Contributed by Xiaoyu Yao. 
(cherry picked from commit de9404f02f)
(cherry picked from commit fdb886d656)
 2015-04-30 20:13:47 +09:00
Haohui Mai 6ac2b5712b HDFS-8269. getBlockLocations() does not resolve the .reserved path and generates incorrect edit logs when updating the atime. Contributed by Haohui Mai. 2015-04-29 11:13:03 -07:00
Haohui Mai 4b9147ef8c HDFS-8273. FSNamesystem#Delete() should not call logSync() when holding the lock. Contributed by Haohui Mai. 2015-04-28 18:06:22 -07:00
Tsz-Wo Nicholas Sze 6b20b325f4 HDFS-8204. Mover/Balancer should not schedule two replicas to the same datanode. Contributed by Walter Su 2015-04-28 13:07:20 -07:00
Colin Patrick Mccabe 176d22b440 HDFS-8070. Pre-HDFS-7915 DFSClient cannot use short circuit on post-HDFS-7915 DataNode (cmccabe) 
(cherry picked from commit a8898445dc)
(cherry picked from commit 3d0385c3c8)
 2015-04-23 19:28:38 -07:00
Colin Patrick Mccabe c2e6d2589a HADOOP-11802. Fix renamed constant in test file in backport to 2.7.1 (cmccabe) 2015-04-23 19:16:28 -07:00
Colin Patrick Mccabe 496afb5e1a HADOOP-11802. DomainSocketWatcher thread terminates sometimes after there is an I/O error during requestShortCircuitShm (cmccabe) 
(cherry picked from commit a0e0a63209)
(cherry picked from commit 788b76761d)
 2015-04-23 19:09:50 -07:00
Tsz-Wo Nicholas Sze 5b4eda7573 HDFS-8147. StorageGroup in Dispatcher should override equals nad hashCode. Contributed by surendra singh lilhore 2015-04-23 11:57:16 -07:00
Arpit Agarwal b17d365fa1 HDFS-8163. Using monotonicNow for block report scheduling causes test failures on recently restarted systems. (Arpit Agarwal) 2015-04-21 11:17:27 -07:00
Vinod Kumar Vavilapalli 7dd5f42a76 Set the release date for 2.7.0 
(cherry picked from commit d52de61544)
 2015-04-20 20:18:38 -07:00
Vinayakumar B 1c8d54bc04 HDFS-7916. 'reportBadBlocks' from datanodes to standby Node BPServiceActor goes for infinite loop (Contributed by Vinayakumar B) 
(cherry picked from commit 867d5d2675)
 2015-04-21 07:45:59 +05:30
Arpit Agarwal cc42ad03d9 HDFS-8179. DFSClient#getServerDefaults returns null within 1 hour of system start. (Contributed by Xiaoyu Yao) 2015-04-20 15:42:58 -07:00
Jitendra Pandey 0743f128a0 HDFS-8153. Error Message points to wrong parent directory in case of path component name length error. Contributed by Anu Engineer. 
cherry picked from 369ddc67bd
 2015-04-16 22:23:35 -07:00
Akira Ajisaka cffc801ecb Revert "Merge r1588258 from trunk, HDFS-6153. Document "fileId" and "childrenNum" fields in the FileStatus Json schema" 
This reverts commit 641cbc0feb.
 2015-04-16 16:46:19 -05:00
Akira Ajisaka 35e1936f84 HDFS-8149. The footer of the Web UI "Hadoop, 2014" is old. Contributed by Brahma Reddy Battula. 
(cherry picked from commit de0f1700c1)
(cherry picked from commit 6d183561ff)
 2015-04-15 15:32:29 -05:00
Jing Zhao e1ce0700eb HDFS-7934. Update RollingUpgrade rollback documentation: should use bootstrapstandby for standby NN. Contributed by J. Andreina. 
(cherry picked from commit b172d03595)
(cherry picked from commit 725621d95a)
 2015-04-15 13:11:45 -07:00
Jing Zhao 9e11ae684a HDFS-8151. Always use snapshot path as source when invalid snapshot names are used for diff based distcp. Contributed by Jing Zhao. 
(cherry picked from commit 4c097e473b)
(cherry picked from commit d4dd97eabd)
 2015-04-15 12:41:34 -07:00
Jing Zhao 50c8d06314 HDFS-8127. NameNode Failover during HA upgrade can cause DataNode to finalize upgrade. Contributed by Jing Zhao. 
(cherry picked from commit fddd55279d)
(cherry picked from commit 38b031d6ba)
 2015-04-14 15:15:36 -07:00
Arun Suresh 0a95b4f8aa HDFS-7931. DistributedFileSystem should not look for keyProvider in cache if Encryption is disabled (asuresh) 
(cherry picked from commit 7660da95cb)
 2015-04-10 11:43:49 -07:00
Konstantin V Shvachko e964bbc856 HDFS-8081. Split getAdditionalBlock() into two methods. Contributed by Konstantin Shvachko 2015-04-09 22:04:48 -07:00