From 479c09751cc854910e55971c86a32ff51d4b52ff Mon Sep 17 00:00:00 2001 From: Zhenxiao Luo Date: Thu, 23 Jan 2020 13:42:03 -0800 Subject: [PATCH] Add MostAvailableSizeStorageLocationSelectorStrategy (#8879) * Add MostAvailableSize LocationSelectorStrategy * Add doc for mostAvailableSize strategy * Fix docs for mostAvailableSize --- docs/configuration/index.md | 13 ++++- ...leSizeStorageLocationSelectorStrategy.java | 58 +++++++++++++++++++ .../StorageLocationSelectorStrategy.java | 3 +- .../StorageLocationSelectorStrategyTest.java | 47 +++++++++++++++ website/.spelling | 1 + 5 files changed, 119 insertions(+), 3 deletions(-) create mode 100644 server/src/main/java/org/apache/druid/segment/loading/MostAvailableSizeStorageLocationSelectorStrategy.java diff --git a/docs/configuration/index.md b/docs/configuration/index.md index 7abcf0871e9..dd0acc68696 100644 --- a/docs/configuration/index.md +++ b/docs/configuration/index.md @@ -1365,7 +1365,7 @@ These Historical configurations can be defined in the `historical/runtime.proper |Property|Description|Default| |--------|-----------|-------| |`druid.segmentCache.locations`|Segments assigned to a Historical process are first stored on the local file system (in a disk cache) and then served by the Historical process. These locations define where that local cache resides. This value cannot be NULL or EMPTY. Here is an example `druid.segmentCache.locations=[{"path": "/mnt/druidSegments", "maxSize": 10000, "freeSpacePercent": 1.0}]`. "freeSpacePercent" is optional, if provided then enforces that much of free disk partition space while storing segments. But, it depends on File.getTotalSpace() and File.getFreeSpace() methods, so enable if only if they work for your File System.| none | -|`druid.segmentCache.locationSelectorStrategy`|The strategy used to select a location from the configured `druid.segmentCache.locations` for segment distribution. Possible values are `leastBytesUsed` or `roundRobin` or `random`. |leastBytesUsed| +|`druid.segmentCache.locationSelectorStrategy`|The strategy used to select a location from the configured `druid.segmentCache.locations` for segment distribution. Possible values are `leastBytesUsed`, `roundRobin`, `random`, or `mostAvailableSize`. |leastBytesUsed| |`druid.segmentCache.deleteOnRemove`|Delete segment files from cache once a process is no longer serving a segment.|true| |`druid.segmentCache.dropSegmentDelayMillis`|How long a process delays before completely dropping segment.|30000 (30 seconds)| |`druid.segmentCache.infoDir`|Historical processes keep track of the segments they are serving so that when the process is restarted they can reload the same segments without waiting for the Coordinator to reassign. This path defines where this metadata is kept. Directory will be created if needed.|${first_location}/info_dir| @@ -1377,7 +1377,16 @@ These Historical configurations can be defined in the `historical/runtime.proper In `druid.segmentCache.locations`, *freeSpacePercent* was added because *maxSize* setting is only a theoretical limit and assumes that much space will always be available for storing segments. In case of any druid bug leading to unaccounted segment files left alone on disk or some other process writing stuff to disk, This check can start failing segment loading early before filling up the disk completely and leaving the host usable otherwise. -In `druid.segmentCache.locationSelectorStrategy`, one of leastBytesUsed or roundRobin could be specified to represent the strategy to distribute segments across multiple segment cache locations. The leastBytesUsed which is the default strategy always selects a location which has least bytes used in absolute terms. The roundRobin strategy selects a location in a round robin fashion oblivious to the bytes used or the capacity. Note that `if druid.segmentCache.numLoadingThreads` > 1, multiple threads can download different segments at the same time. In this case, with the leastBytesUsed strategy, historicals may select a sub-optimal storage location because each decision is based on a snapshot of the storage location status of when a segment is requested to download. +In `druid.segmentCache.locationSelectorStrategy`, one of leastBytesUsed, roundRobin, random, or mostAvailableSize could be specified to represent the strategy to distribute segments across multiple segment cache locations. + +|Strategy|Description| +|--------|-----------| +|`leastBytesUsed`|selects a location which has least bytes used in absolute terms.| +|`roundRobin`|selects a location in a round robin fashion oblivious to the bytes used or the capacity.| +|`random`|selects a segment cache location randomly each time among the available storage locations.| +|`mostAvailableSize`|selects a segment cache location that has most free space among the available storage locations.| + +Note that if `druid.segmentCache.numLoadingThreads` > 1, multiple threads can download different segments at the same time. In this case, with the leastBytesUsed strategy or mostAvailableSize strategy, historicals may select a sub-optimal storage location because each decision is based on a snapshot of the storage location status of when a segment is requested to download. #### Historical query configs diff --git a/server/src/main/java/org/apache/druid/segment/loading/MostAvailableSizeStorageLocationSelectorStrategy.java b/server/src/main/java/org/apache/druid/segment/loading/MostAvailableSizeStorageLocationSelectorStrategy.java new file mode 100644 index 00000000000..4790f90a33e --- /dev/null +++ b/server/src/main/java/org/apache/druid/segment/loading/MostAvailableSizeStorageLocationSelectorStrategy.java @@ -0,0 +1,58 @@ +/* + * Licensed to the Apache Software Foundation (ASF) under one + * or more contributor license agreements. See the NOTICE file + * distributed with this work for additional information + * regarding copyright ownership. The ASF licenses this file + * to you under the Apache License, Version 2.0 (the + * "License"); you may not use this file except in compliance + * with the License. You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, + * software distributed under the License is distributed on an + * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY + * KIND, either express or implied. See the License for the + * specific language governing permissions and limitations + * under the License. + */ + +package org.apache.druid.segment.loading; + +import com.google.common.collect.Ordering; + +import java.util.Comparator; +import java.util.Iterator; +import java.util.List; + +/** + * A {@link StorageLocation} selector strategy that selects a segment cache location that has most free space + * among the available storage locations. + */ +public class MostAvailableSizeStorageLocationSelectorStrategy implements StorageLocationSelectorStrategy +{ + private static final Ordering ORDERING = Ordering.from(Comparator + .comparingLong(StorageLocation::availableSizeBytes) + .reversed()); + + private List storageLocations; + + public MostAvailableSizeStorageLocationSelectorStrategy(List storageLocations) + { + this.storageLocations = storageLocations; + } + + @Override + public Iterator getLocations() + { + return ORDERING.sortedCopy(this.storageLocations).iterator(); + } + + @Override + public String toString() + { + return "MostAvailableSizeStorageLocationSelectorStrategy{" + + "storageLocations=" + storageLocations + + '}'; + } +} diff --git a/server/src/main/java/org/apache/druid/segment/loading/StorageLocationSelectorStrategy.java b/server/src/main/java/org/apache/druid/segment/loading/StorageLocationSelectorStrategy.java index f85f28d8a98..adc73887b87 100644 --- a/server/src/main/java/org/apache/druid/segment/loading/StorageLocationSelectorStrategy.java +++ b/server/src/main/java/org/apache/druid/segment/loading/StorageLocationSelectorStrategy.java @@ -39,7 +39,8 @@ import java.util.Iterator; @JsonSubTypes(value = { @JsonSubTypes.Type(name = "leastBytesUsed", value = LeastBytesUsedStorageLocationSelectorStrategy.class), @JsonSubTypes.Type(name = "roundRobin", value = RoundRobinStorageLocationSelectorStrategy.class), - @JsonSubTypes.Type(name = "random", value = RandomStorageLocationSelectorStrategy.class) + @JsonSubTypes.Type(name = "random", value = RandomStorageLocationSelectorStrategy.class), + @JsonSubTypes.Type(name = "mostAvailableSize", value = MostAvailableSizeStorageLocationSelectorStrategy.class) }) public interface StorageLocationSelectorStrategy { diff --git a/server/src/test/java/org/apache/druid/segment/loading/StorageLocationSelectorStrategyTest.java b/server/src/test/java/org/apache/druid/segment/loading/StorageLocationSelectorStrategyTest.java index 0561a418075..82cbe702597 100644 --- a/server/src/test/java/org/apache/druid/segment/loading/StorageLocationSelectorStrategyTest.java +++ b/server/src/test/java/org/apache/druid/segment/loading/StorageLocationSelectorStrategyTest.java @@ -209,4 +209,51 @@ public class StorageLocationSelectorStrategyTest Assert.assertArrayEquals(new File[]{localStorageFolder1, localStorageFolder2, localStorageFolder3}, result); } + @Test + public void testMostAvailableSizeLocationSelectorStrategy() throws Exception + { + List storageLocations = new ArrayList<>(); + + final File localStorageFolder1 = tmpFolder.newFolder("local_storage_folder_1"); + final File localStorageFolder2 = tmpFolder.newFolder("local_storage_folder_2"); + final File localStorageFolder3 = tmpFolder.newFolder("local_storage_folder_3"); + + StorageLocation storageLocation1 = new StorageLocation(localStorageFolder1, 10000000000L, null); + storageLocations.add(storageLocation1); + StorageLocation storageLocation2 = new StorageLocation(localStorageFolder2, 20000000000L, null); + storageLocations.add(storageLocation2); + StorageLocation storageLocation3 = new StorageLocation(localStorageFolder3, 15000000000L, null); + storageLocations.add(storageLocation3); + + StorageLocationSelectorStrategy mostAvailableStrategy = new MostAvailableSizeStorageLocationSelectorStrategy(storageLocations); + + Iterator locations = mostAvailableStrategy.getLocations(); + + StorageLocation loc1 = locations.next(); + Assert.assertEquals("The next element of the iterator should point to path local_storage_folder_2", + localStorageFolder2, loc1.getPath()); + + StorageLocation loc2 = locations.next(); + Assert.assertEquals("The next element of the iterator should point to path local_storage_folder_3", + localStorageFolder3, loc2.getPath()); + + StorageLocation loc3 = locations.next(); + Assert.assertEquals("The next element of the iterator should point to path local_storage_folder_1", + localStorageFolder1, loc3.getPath()); + + storageLocation2.reserve("tmp_loc2", "__seg2", 6000000000L); + locations = mostAvailableStrategy.getLocations(); + + loc1 = locations.next(); + Assert.assertEquals("The next element of the iterator should point to path local_storage_folder_3", + localStorageFolder3, loc1.getPath()); + + loc2 = locations.next(); + Assert.assertEquals("The next element of the iterator should point to path local_storage_folder_2", + localStorageFolder2, loc2.getPath()); + + loc3 = locations.next(); + Assert.assertEquals("The next element of the iterator should point to path local_storage_folder_1", + localStorageFolder1, loc3.getPath()); + } } diff --git a/website/.spelling b/website/.spelling index 07a6cd0a5ec..a721bd63dd6 100644 --- a/website/.spelling +++ b/website/.spelling @@ -271,6 +271,7 @@ mergeable metadata millis misconfiguration +mostAvailableSize multitenancy multitenant mysql