honeymoose/OpenSearch
1
0
Fork 0
mirror of https://github.com/honeymoose/OpenSearch.git synced 2025-02-08 22:14:59 +00:00
Code Issues Packages Projects Releases Wiki Activity

Deprecate fixed_auto_queue_size thread pool type (#52399)

Browse Source 
Relates #52280
This commit is contained in:
Yannick Welsch 2020-02-20 11:11:06 +01:00 committed by GitHub
parent 087ceb899b
commit d76358c875
No known key found for this signature in database
GPG Key ID: 4AEE18F83AFDEB23
3 changed files with 28 additions and 16 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

25
docs/reference/modules/threadpool.asciidoc
View File

@ -20,8 +20,8 @@ There are several thread pools, but the important ones include:
`1000`. `1000`.
[[search-throttled]]`search_throttled`:: [[search-throttled]]`search_throttled`::
For count/search/suggest/get operations on `search_throttled indices`. For count/search/suggest/get operations on `search_throttled indices`.
Thread pool type is `fixed_auto_queue_size` with a size of `1`, and initial Thread pool type is `fixed_auto_queue_size` with a size of `1`, and initial
queue_size of `100`. queue_size of `100`.
`get`:: `get`::
@ -30,7 +30,7 @@ There are several thread pools, but the important ones include:
queue_size of `1000`. queue_size of `1000`.
`analyze`:: `analyze`::
For analyze requests. Thread pool type is `fixed` with a size of `1`, queue For analyze requests. Thread pool type is `fixed` with a size of `1`, queue
size of `16`. size of `16`.
`write`:: `write`::
@ -51,8 +51,8 @@ There are several thread pools, but the important ones include:
keep-alive of `5m` and a max of `min(10, (# of available processors)/2)`. keep-alive of `5m` and a max of `min(10, (# of available processors)/2)`.
`listener`:: `listener`::
Mainly for java client executing of action when listener threaded is set to Mainly for java client executing of action when listener threaded is set to
`true`. Thread pool type is `scaling` with a default max of `true`. Thread pool type is `scaling` with a default max of
`min(10, (# of available processors)/2)`. `min(10, (# of available processors)/2)`.
`fetch_shard_started`:: `fetch_shard_started`::
@ -125,6 +125,9 @@ thread_pool:
experimental[] experimental[]
deprecated[7.7.0,The experimental `fixed_auto_queue_size` thread pool type is
deprecated and will be removed in 8.0.]
The `fixed_auto_queue_size` thread pool holds a fixed size of threads to handle The `fixed_auto_queue_size` thread pool holds a fixed size of threads to handle
the requests with a bounded queue for pending requests that have no threads to the requests with a bounded queue for pending requests that have no threads to
service them. It's similar to the `fixed` threadpool, however, the `queue_size` service them. It's similar to the `fixed` threadpool, however, the `queue_size`
@ -202,13 +205,13 @@ processors: 2
There are a few use-cases for explicitly overriding the `processors` There are a few use-cases for explicitly overriding the `processors`
setting: setting:
. If you are running multiple instances of {es} on the same host but want {es} . If you are running multiple instances of {es} on the same host but want {es}
to size its thread pools as if it only has a fraction of the CPU, you should to size its thread pools as if it only has a fraction of the CPU, you should
override the `processors` setting to the desired fraction, for example, if override the `processors` setting to the desired fraction, for example, if
you're running two instances of {es} on a 16-core machine, set `processors` to 8. you're running two instances of {es} on a 16-core machine, set `processors` to 8.
Note that this is an expert-level use case and there's a lot more involved Note that this is an expert-level use case and there's a lot more involved
than just setting the `processors` setting as there are other considerations than just setting the `processors` setting as there are other considerations
like changing the number of garbage collector threads, pinning processes to like changing the number of garbage collector threads, pinning processes to
cores, and so on. cores, and so on.
. Sometimes the number of processors is wrongly detected and in such . Sometimes the number of processors is wrongly detected and in such
cases explicitly setting the `processors` setting will workaround such cases explicitly setting the `processors` setting will workaround such

13
server/src/main/java/org/elasticsearch/threadpool/AutoQueueAdjustingExecutorBuilder.java
View File

@ -67,11 +67,13 @@ public final class AutoQueueAdjustingExecutorBuilder extends ExecutorBuilder<Aut
final String frameSizeKey = settingsKey(prefix, "auto_queue_frame_size"); final String frameSizeKey = settingsKey(prefix, "auto_queue_frame_size");
final String targetedResponseTimeKey = settingsKey(prefix, "target_response_time"); final String targetedResponseTimeKey = settingsKey(prefix, "target_response_time");
this.targetedResponseTimeSetting = Setting.timeSetting(targetedResponseTimeKey, TimeValue.timeValueSeconds(1), this.targetedResponseTimeSetting = Setting.timeSetting(targetedResponseTimeKey, TimeValue.timeValueSeconds(1),
TimeValue.timeValueMillis(10), Setting.Property.NodeScope); TimeValue.timeValueMillis(10), Setting.Property.NodeScope, Setting.Property.Deprecated);
this.queueSizeSetting = Setting.intSetting(queueSizeKey, initialQueueSize, Setting.Property.NodeScope); this.queueSizeSetting = Setting.intSetting(queueSizeKey, initialQueueSize, Setting.Property.NodeScope);
// These temp settings are used to validate the min and max settings below // These temp settings are used to validate the min and max settings below
Setting<Integer> tempMaxQueueSizeSetting = Setting.intSetting(maxSizeKey, maxQueueSize, Setting.Property.NodeScope); Setting<Integer> tempMaxQueueSizeSetting = Setting.intSetting(maxSizeKey, maxQueueSize, Setting.Property.NodeScope,
Setting<Integer> tempMinQueueSizeSetting = Setting.intSetting(minSizeKey, minQueueSize, Setting.Property.NodeScope); Setting.Property.Deprecated);
Setting<Integer> tempMinQueueSizeSetting = Setting.intSetting(minSizeKey, minQueueSize, Setting.Property.NodeScope,
Setting.Property.Deprecated);
this.minQueueSizeSetting = new Setting<>( this.minQueueSizeSetting = new Setting<>(
minSizeKey, minSizeKey,
@ -126,8 +128,9 @@ public final class AutoQueueAdjustingExecutorBuilder extends ExecutorBuilder<Aut
} }
}, },
Setting.Property.NodeScope); Setting.Property.NodeScope, Setting.Property.Deprecated);
this.frameSizeSetting = Setting.intSetting(frameSizeKey, frameSize, 100, Setting.Property.NodeScope); this.frameSizeSetting = Setting.intSetting(frameSizeKey, frameSize, 100, Setting.Property.NodeScope, Setting.Property.Deprecated,
Setting.Property.Deprecated);
} }
@Override @Override

6
server/src/test/java/org/elasticsearch/threadpool/AutoQueueAdjustingExecutorBuilderTests.java
View File

@ -78,6 +78,8 @@ public class AutoQueueAdjustingExecutorBuilderTests extends ESThreadPoolTestCase
} catch (IllegalArgumentException e) { } catch (IllegalArgumentException e) {
assertEquals(e.getMessage(), "Failed to parse value [100] for setting [thread_pool.test.min_queue_size] must be <= 99"); assertEquals(e.getMessage(), "Failed to parse value [100] for setting [thread_pool.test.min_queue_size] must be <= 99");
} }
assertSettingDeprecationsAndWarnings(new String[]{"thread_pool.test.max_queue_size"});
} }
public void testSetLowerSettings() { public void testSetLowerSettings() {
@ -89,6 +91,8 @@ public class AutoQueueAdjustingExecutorBuilderTests extends ESThreadPoolTestCase
AutoQueueAdjustingExecutorBuilder.AutoExecutorSettings s = test.getSettings(settings); AutoQueueAdjustingExecutorBuilder.AutoExecutorSettings s = test.getSettings(settings);
assertEquals(10, s.maxQueueSize); assertEquals(10, s.maxQueueSize);
assertEquals(10, s.minQueueSize); assertEquals(10, s.minQueueSize);
assertSettingDeprecationsAndWarnings(new String[]{"thread_pool.test.min_queue_size", "thread_pool.test.max_queue_size"});
} }
public void testSetHigherSettings() { public void testSetHigherSettings() {
@ -100,6 +104,8 @@ public class AutoQueueAdjustingExecutorBuilderTests extends ESThreadPoolTestCase
AutoQueueAdjustingExecutorBuilder.AutoExecutorSettings s = test.getSettings(settings); AutoQueueAdjustingExecutorBuilder.AutoExecutorSettings s = test.getSettings(settings);
assertEquals(3000, s.maxQueueSize); assertEquals(3000, s.maxQueueSize);
assertEquals(2000, s.minQueueSize); assertEquals(2000, s.minQueueSize);
assertSettingDeprecationsAndWarnings(new String[]{"thread_pool.test.min_queue_size", "thread_pool.test.max_queue_size"});
} }
} }