Apache/hadoop
1
0
Fork 0
Code Issues Pull Requests Packages Projects Releases Wiki Activity

HDFS-4937. ReplicationMonitor can infinite-loop in BlockPlacementPolicyDefault#chooseRandom(). Contributed by Kihwal Lee.

Browse Source 
(cherry picked from commit ff47f35deed14ba6463cba76f0e6a6c15abb3eca)
This commit is contained in:
Kihwal Lee 2015-11-05 09:27:36 -06:00
parent 397b554c36
commit 616ed9084b
2 changed files with 12 additions and 0 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

3
hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt
View File

@ -15,6 +15,9 @@ Release 2.7.3 - UNRELEASED
HDFS-9289. Make DataStreamer#block thread safe and verify genStamp in HDFS-9289. Make DataStreamer#block thread safe and verify genStamp in
commitBlock. (Chang Li via kihwal) commitBlock. (Chang Li via kihwal)
HDFS-4937. ReplicationMonitor can infinite-loop in
BlockPlacementPolicyDefault#chooseRandom(). (kihwal)
Release 2.7.2 - UNRELEASED Release 2.7.2 - UNRELEASED
INCOMPATIBLE CHANGES INCOMPATIBLE CHANGES

9
hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/blockmanagement/BlockPlacementPolicyDefault.java
View File

@ -622,6 +622,7 @@ protected DatanodeStorageInfo chooseRandom(int numOfReplicas,
int numOfAvailableNodes = clusterMap.countNumOfAvailableNodes( int numOfAvailableNodes = clusterMap.countNumOfAvailableNodes(
scope, excludedNodes); scope, excludedNodes);
int refreshCounter = numOfAvailableNodes;
StringBuilder builder = null; StringBuilder builder = null;
if (LOG.isDebugEnabled()) { if (LOG.isDebugEnabled()) {
builder = debugLoggingBuilder.get(); builder = debugLoggingBuilder.get();
@ -675,6 +676,14 @@ protected DatanodeStorageInfo chooseRandom(int numOfReplicas,
// If no candidate storage was found on this DN then set badTarget. // If no candidate storage was found on this DN then set badTarget.
badTarget = (i == storages.length); badTarget = (i == storages.length);
} }
// Refresh the node count. If the live node count became smaller,
// but it is not reflected in this loop, it may loop forever in case
// the replicas/rack cannot be satisfied.
if (--refreshCounter == 0) {
numOfAvailableNodes = clusterMap.countNumOfAvailableNodes(scope,
excludedNodes);
refreshCounter = numOfAvailableNodes;
}
} }
if (numOfReplicas>0) { if (numOfReplicas>0) {