Apache
/
hadoop
1
0
Fork 0
Code Issues Pull Requests Packages Projects Releases Wiki Activity

HDFS-5828. BlockPlacementPolicyWithNodeGroup can place multiple replicas on the same node group when dfs.namenode.avoid.write.stale.datanode is true. (Buddy via junping_du) 

git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1563640 13f79535-47bb-0310-9956-ffa450edef68
This commit is contained in:
Junping Du 2014-02-02 15:39:02 +00:00
parent 1079c37128
commit 94b512bc51
3 changed files with 7 additions and 1 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

4
hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt
View File

@ -324,6 +324,10 @@ Release 2.4.0 - UNRELEASED
HDFS-5856. DataNode.checkDiskError might throw NPE. HDFS-5856. DataNode.checkDiskError might throw NPE.
(Josh Elser via suresh) (Josh Elser via suresh)
HDFS-5828. BlockPlacementPolicyWithNodeGroup can place multiple replicas on
the same node group when dfs.namenode.avoid.write.stale.datanode is true.
(Buddy via junping_du)
Release 2.3.0 - UNRELEASED Release 2.3.0 - UNRELEASED
INCOMPATIBLE CHANGES INCOMPATIBLE CHANGES

2
hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/blockmanagement/BlockPlacementPolicyDefault.java
View File

@ -317,7 +317,7 @@ public class BlockPlacementPolicyDefault extends BlockPlacementPolicy {
// We need to additionally exclude the nodes that were added to the // We need to additionally exclude the nodes that were added to the
// result list in the successful calls to choose*() above. // result list in the successful calls to choose*() above.
for (DatanodeStorageInfo resultStorage : results) { for (DatanodeStorageInfo resultStorage : results) {
oldExcludedNodes.add(resultStorage.getDatanodeDescriptor()); addToExcludedNodes(resultStorage.getDatanodeDescriptor(), oldExcludedNodes);
} }
// Set numOfReplicas, since it can get out of sync with the result list // Set numOfReplicas, since it can get out of sync with the result list
// if the NotEnoughReplicasException was thrown in chooseRandom(). // if the NotEnoughReplicasException was thrown in chooseRandom().

2
hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/blockmanagement/TestReplicationPolicyWithNodeGroup.java
View File

@ -124,6 +124,8 @@ public class TestReplicationPolicyWithNodeGroup {
CONF.set(CommonConfigurationKeysPublic.NET_TOPOLOGY_IMPL_KEY, CONF.set(CommonConfigurationKeysPublic.NET_TOPOLOGY_IMPL_KEY,
NetworkTopologyWithNodeGroup.class.getName()); NetworkTopologyWithNodeGroup.class.getName());
CONF.setBoolean(DFSConfigKeys.DFS_NAMENODE_AVOID_STALE_DATANODE_FOR_WRITE_KEY, true);
File baseDir = PathUtils.getTestDir(TestReplicationPolicyWithNodeGroup.class); File baseDir = PathUtils.getTestDir(TestReplicationPolicyWithNodeGroup.class);
CONF.set(DFSConfigKeys.DFS_NAMENODE_NAME_DIR_KEY, CONF.set(DFSConfigKeys.DFS_NAMENODE_NAME_DIR_KEY,