Apache
/
hadoop
1
0
Fork 0
Code Issues Pull Requests Packages Projects Releases Wiki Activity
9,053 Commits 326 Branches 371 Tags 1.5 GiB
Commit Graph

2995 Commits

Author SHA1 Message Date
Zhe Zhang 670617f325 HDFS-9314. Improve BlockPlacementPolicyDefault's picking of excess replicas. Contributed by Xiao Chen. 
Change-Id: Idf99293085531165239369155c039b55db0eed83
 2015-12-04 09:49:43 -08:00
Zhe Zhang b39de3b0bd HDFS-9313. Possible NullPointerException in BlockManager if no excess replica can be chosen. Contributed by Ming Ma. Branch-2.7 patch contributed by Xiao Chen. 
Change-Id: Iadaee6be2b44f7b0322d7b0e58f8fc983ef463f6
 2015-12-03 23:19:17 -08:00
Zhe Zhang 1ae8a8342e HDFS-8647. Abstract BlockManager's rack policy into BlockPlacementPolicy. Contributed by Brahma Reddy Battula. Branch-2.7 patch contributed by Xiao Chen. 
Change-Id: I1b32627bc2a2f30f5debeaba7663fb2777958079
 2015-12-03 10:30:48 -08:00
Tsz-Wo Nicholas Sze 4123650e92 HDFS-9294. DFSClient deadlock when close file and failed to renew lease. Contributed by Brahma Reddy Battula 2015-12-02 17:55:10 -08:00
Andrew Wang 10e8a67d23 HDFS-9470. Encryption zone on root not loaded from fsimage after NN restart. Xiao Chen via wang. 
(cherry picked from commit 9b8e50b424)
(cherry picked from commit ce1111ceea)

 Conflicts:
	hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/FSDirectory.java
 2015-11-30 14:53:04 -08:00
Vinayakumar B 0c48c17ae9 HDFS-9426. Rollingupgrade finalization is not backward compatible (Contributed by Kihwal Lee) 
(cherry picked from commit c62d42cd8b)
 2015-11-26 09:42:26 +05:30
Tsz-Wo Nicholas Sze af811030e5 Move HDFS-9434 to 2.6.3 in CHANGES.txt. 2015-11-24 12:49:34 -08:00
Tsz-Wo Nicholas Sze 09f4a4eefd HDFS-9434. Recommission a datanode with 500k blocks may pause NN for 30 seconds for printing info log messags. 2015-11-24 12:47:50 -08:00
Akira Ajisaka bb6983e153 HDFS-9431. DistributedFileSystem#concat fails if the target path is relative. Contributed by Kazuho Fujii. 
(cherry picked from commit ac1aa6c819)
(cherry picked from commit c74e42b4a2)
 2015-11-19 20:55:52 +09:00
Ming Ma 1b0f277eae HDFS-9413. getContentSummary() on standby should throw StandbyException. (Brahma Reddy Battula via mingma) 
(cherry picked from commit 02653add98)
(cherry picked from commit 42b55ff23e)

 Conflicts:
	hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt

(cherry picked from commit 83200450f6)

 Conflicts:
	hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt
 2015-11-17 16:12:58 +05:30
Akira Ajisaka ab04c9ab5f HDFS-8615. Correct HTTP method in WebHDFS document. Contributed by Brahma Reddy Battula. 
(cherry picked from commit 1a169a26bc)
(cherry picked from commit 9f9315008e)
 2015-11-17 16:32:07 +09:00
Kihwal Lee 6f678deb86 HDFS-9383. TestByteArrayManager#testByteArrayManager fails. Contributed by Tsz Wo Nicholas Sze. 
(cherry picked from commit ef926b2e38)
 2015-11-09 08:58:58 -06:00
Arpit Agarwal 63ed399e1c HDFS-6481. DatanodeManager#getDatanodeStorageInfos() should check the length of storageIDs. (Contributed by szetszwo) 2015-11-06 10:32:34 -08:00
Kihwal Lee 616ed9084b HDFS-4937. ReplicationMonitor can infinite-loop in BlockPlacementPolicyDefault#chooseRandom(). Contributed by Kihwal Lee. 
(cherry picked from commit ff47f35dee)
 2015-11-05 09:27:36 -06:00
Kihwal Lee 397b554c36 HDFS-9289. Make DataStreamer#block thread safe and verify genStamp in commitBlock. Contributed by Chang Li. 2015-11-04 12:10:59 -06:00
Jason Lowe 7499f71e77 Add 2.7.3 release to CHANGES.txt 
(cherry picked from commit 0383a3973b)

Conflicts:

	hadoop-common-project/hadoop-common/CHANGES.txt
	hadoop-mapreduce-project/CHANGES.txt
	hadoop-yarn-project/CHANGES.txt
 2015-11-04 16:29:56 +00:00
Vinod Kumar Vavilapalli 09348c4bce Preparing for 2.7.3 development 2015-11-03 15:18:10 -08:00
Kihwal Lee 2c51bbda51 HDFS-9221. HdfsServerConstants#ReplicaState#getState should avoid calling values() since it creates a temporary array. (Staffan Friberg via yliu) 
(cherry picked from commit 0ff1216100)
 2015-11-02 09:07:16 -06:00
yliu e1798e8f21 Revert "HDFS-4937. ReplicationMonitor can infinite-loop in BlockPlacementPolicyDefault#chooseRandom(). Contributed by Kihwal Lee." 
This reverts commit c250b21c23.
 2015-10-31 16:25:53 +08:00
Sangjin Lee c8767fb68c Updated the 2.6.2 final release date. 
(cherry picked from commit a4a6b5b4b4)
 2015-10-30 18:53:10 -07:00
Jason Lowe e894be93f6 Creating 2.6.3 entries in CHANGES.txt files. 
(cherry picked from commit eadf7b3096)

Conflicts:

	hadoop-common-project/hadoop-common/CHANGES.txt
	hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt
	hadoop-mapreduce-project/CHANGES.txt
	hadoop-yarn-project/CHANGES.txt
 2015-10-30 14:55:33 +00:00
Kihwal Lee c250b21c23 HDFS-4937. ReplicationMonitor can infinite-loop in BlockPlacementPolicyDefault#chooseRandom(). Contributed by Kihwal Lee. 
(cherry picked from commit 43539b5ff4)
 2015-10-30 09:39:49 -05:00
Kihwal Lee 13b256ed22 HDFS-9083. Replication violates block placement policy. Contributed by Rushabh Shah. 2015-10-28 14:49:41 -05:00
Akira Ajisaka ce01a5951b HDFS-9317. Document fsck -blockId and -storagepolicy options in branch-2.7. (aajisaka) 2015-10-28 13:14:58 +09:00
Kihwal Lee 59a2072135 HDFS-8950. NameNode refresh doesn't remove DataNodes that are no longer in the allowed list. Contributed by Daniel Templeton. 2015-10-27 16:42:33 -05:00
Kihwal Lee 1d23e1ec07 HDFS-7725. Incorrect 'nodes in service' metrics caused all writes to fail. Contributed by Ming Ma. 
(cherry picked from commit 8104d52269)
 2015-10-27 11:09:05 -05:00
Arpit Agarwal 653ef52ef2 HDFS-9305. Delayed heartbeat processing causes storm of subsequent heartbeats. (Contributed by Arpit Agarwal) 2015-10-26 15:47:37 -07:00
Kihwal Lee f2fd011cc1 HDFS-9290. DFSClient#callAppend() is not backward compatible for slightly older NameNodes. Contributed by Tony Wu. 
(cherry picked from commit b9e0417bdf)

Conflicts:
	hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/DFSClient.java
 2015-10-23 16:46:14 -05:00
Kihwal Lee 4cf7f8441a HDFS-9220. Reading small file (< 512 bytes) that is open for append fails due to incorrect checksum. Contributed by Jing Zhao. 
(cherry picked from commit c7c36cbd62)
 2015-10-15 11:26:14 -05:00
Kihwal Lee efba749760 HDFS-8676. Delayed rolling upgrade finalization can cause heartbeat expiration. Contributed by Walter Su. 
(cherry picked from commit 5b43db47a3)
 2015-10-13 13:25:10 -05:00
Kihwal Lee 61388b41e7 HDFS-8656. Preserve compatibility of ClientProtocol#rollingUpgrade after finalization. 
(cherry picked from commit 60b858bfa6)

Conflicts:
	hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt
	hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/protocol/ClientProtocol.java
	hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/TestRollingUpgrade.java
 2015-10-13 10:46:20 -05:00
Kihwal Lee 3a5b9f49df HDFS-7645. Rolling upgrade is restoring blocks from trash multiple times (Contributed by Vinayakumar B and Keisuke Ogiwara) 
(cherry picked from commit abf3ad988d)
 2015-10-13 10:29:42 -05:00
Kihwal Lee 5cb80992fb HDFS-9178. Slow datanode I/O can cause a wrong node to be marked bad. Contributed by Kihwal Lee. 
(cherry picked from commit 99e5204ff5)

Conflicts:
	hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/datanode/BlockReceiver.java
	hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/TestClientProtocolForPipelineRecovery.java
 2015-10-07 10:24:55 -05:00
Kihwal Lee b2bc72a051 HDFS-8850. VolumeScanner thread exits with exception if there is no block pool to be scanned but there are suspicious blocks. (Colin Patrick McCabe via yliu) 
(cherry picked from commit d2b941f94a)

Conflicts:
	hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt
 2015-10-02 14:59:29 -05:00
Kihwal Lee 6e7c76a5bd HDFS-9106. Transfer failure during pipeline recovery causes permanent write failures. Contributed by Kihwal Lee. 2015-09-28 15:19:57 -05:00
Vinod Kumar Vavilapalli (I am also known as @tshooter.) 2b4c7fcc67 Release process for 2.6.1: Set the release date for 2.6.1 
(cherry picked from commit 1f707ecffd)
(cherry picked from commit f48af332a7)
 2015-09-23 15:13:09 -07:00
Vinayakumar B d800e2e9ed HDFS-9043. Doc updation for commands in HDFS Federation (Contributed by J.Andreina) 
(cherry picked from commit 57003fa971)
(cherry picked from commit 96e3fbf401)
 2015-09-22 16:40:19 +05:30
Vinayakumar B 7d3e882a49 HDFS-9042. Update document for the Storage policy name (Contributed by J.Andreina) 
(cherry picked from commit d8455479b8)
 2015-09-12 12:37:01 +05:30
Kihwal Lee f2cc6b5fdf HDFS-8099. Change "DFSInputStream has been closed already" message to debug log level (Charles Lamb via Colin P. McCabe) 
(cherry picked from commit 30acb7372a)

Conflicts:
	hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt
 2015-09-11 15:59:29 -05:00
Vinod Kumar Vavilapalli fb15053124 HDFS-7314. Moving to 2.6.1 CHANGES.txt section. 
(cherry picked from commit f103a70af5)
 2015-09-10 16:48:53 -07:00
Vinod Kumar Vavilapalli (I am also known as @tshooter.) c05ba079e3 Updating all CHANGES.txt files to move entires from future releases into 2.6.1 
section given the large number of backports to 2.6.1.

(cherry picked from commit fbbb7ff1ed)
(cherry picked from commit cbf90f93f9)
 2015-09-10 16:10:56 -07:00
Colin Patrick Mccabe ee830b0d27 HDFS-8846. Add a unit test for INotify functionality across a layout version upgrade (Zhe Zhang via Colin P. McCabe) 
(cherry picked from commit a4d9acc51d)
(cherry picked from commit 9264b7e119)
 2015-09-10 11:56:30 -07:00
Ming Ma decf8a6653 HDFS-7314. When the DFSClient lease cannot be renewed, abort open-for-write files rather than the entire DFSClient. (mingma) 
(cherry picked from commit fbd88f1062)
(cherry picked from commit 516bbf1c20)
 2015-09-10 11:56:29 -07:00
Jing Zhao 6c127b44ca HDFS-7609. Avoid retry cache collision when Standby NameNode loading edits. Contributed by Ming Ma. 
(cherry picked from commit 7817674a3a)
(cherry picked from commit 17fb442a4c)
 2015-09-10 11:56:28 -07:00
cnauroth 188096f1fe HDFS-8431. hdfs crypto class not found in Windows. Contributed by Anu Engineer. 
(cherry picked from commit 50eeea1300)

Conflicts:
	hadoop-hdfs-project/hadoop-hdfs/src/main/bin/hdfs.cmd

(cherry picked from commit 25db341278)
 2015-09-10 11:56:27 -07:00
Xiaoyu Yao d3562b4e1d HDFS-8219. setStoragePolicy with folder behavior is different after cluster restart. (surendra singh lilhore via Xiaoyu Yao) 
(cherry picked from commit 0100b15501)
(cherry picked from commit e68e8b3b5c)
 2015-09-10 11:56:22 -07:00
Kihwal Lee cfb27d7f78 HDFS-8046. Allow better control of getContentSummary. Contributed by Kihwal Lee. 
(cherry picked from commit 285b31e75e)

(cherry picked from commit 7e622076d4)
 2015-09-10 10:53:19 -07:00
Akira Ajisaka c5a74c5c37 HDFS-9033. dfsadmin -metasave prints "NaN" for cache used%. Contributed by Brahma Reddy Battula. 
(cherry picked from commit 0113e4528d)
(cherry picked from commit eb9e06f8e3)
 2015-09-09 17:23:25 +09:00
Jing Zhao fd100ccbae HDFS-8384. Allow NN to startup if there are files having a lease but are not under construction. Contributed by Jing Zhao. 2015-09-08 10:52:42 -07:00
Tsuyoshi Ozawa 139fd21c83 HADOOP-5323. Trash documentation should describe its directory structure and configurations. Contributed by Weiwei Yang. 
(cherry picked from commit 9a87f81ce6)

Conflicts:
	hadoop-hdfs-project/hadoop-hdfs/src/site/markdown/HdfsDesign.md
 2015-09-03 22:39:12 +09:00