honeymoose
/
OpenSearch
mirror of https://github.com/honeymoose/OpenSearch.git
1
0
Fork 0
Code Issues Packages Projects Releases Wiki Activity

Change Watcher thread pool to be scaling 

Watcher uses a custom thread pool. This is because executing watches can
be long-running tasks that often block on I/O and it is best to not
consume the core thread pools with these tasks. Today this thread pool
is fixed, and sized at five times the bounded number of cores (so 160 on
a 32-core box). It makes sense for there to possibly be so many threads,
again because these tasks can block on I/O and having excess capacity
lets unblocked watches execute. It's the fixed size that can cause
problem, all these threads are always consuming resources even when
there are no or not that many watches running. This commit changes this
thread pool to be a scaling thread pool.

Relates elastic/elasticsearch#3660

Original commit: elastic/x-pack-elasticsearch@3cafab6e83
This commit is contained in:
Jason Tedor 2016-10-04 18:15:19 -04:00 committed by GitHub
parent a0e1d44a44
commit 00cecac86e
1 changed files with 7 additions and 5 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

12
elasticsearch/src/main/java/org/elasticsearch/xpack/watcher/Watcher.java
View File

@ -25,7 +25,7 @@ import org.elasticsearch.plugins.ScriptPlugin;
import org.elasticsearch.rest.RestHandler; import org.elasticsearch.rest.RestHandler;
import org.elasticsearch.script.ScriptContext; import org.elasticsearch.script.ScriptContext;
import org.elasticsearch.threadpool.ExecutorBuilder; import org.elasticsearch.threadpool.ExecutorBuilder;
import org.elasticsearch.threadpool.FixedExecutorBuilder; import org.elasticsearch.threadpool.ScalingExecutorBuilder;
import org.elasticsearch.xpack.XPackPlugin; import org.elasticsearch.xpack.XPackPlugin;
import org.elasticsearch.xpack.XPackSettings; import org.elasticsearch.xpack.XPackSettings;
import org.elasticsearch.xpack.watcher.actions.WatcherActionModule; import org.elasticsearch.xpack.watcher.actions.WatcherActionModule;
@ -165,12 +165,14 @@ public class Watcher implements ActionPlugin, ScriptPlugin {
public List<ExecutorBuilder<?>> getExecutorBuilders(final Settings settings) { public List<ExecutorBuilder<?>> getExecutorBuilders(final Settings settings) {
if (enabled) { if (enabled) {
final FixedExecutorBuilder builder = final ScalingExecutorBuilder builder =
new FixedExecutorBuilder( new ScalingExecutorBuilder(
settings,
InternalWatchExecutor.THREAD_POOL_NAME, InternalWatchExecutor.THREAD_POOL_NAME,
0,
// watcher threads can block on I/O for a long time, so we let this
// pool be large so that execution of unblocked watches can proceed
5 * EsExecutors.boundedNumberOfProcessors(settings), 5 * EsExecutors.boundedNumberOfProcessors(settings),
1000, TimeValue.timeValueMinutes(5),
"xpack.watcher.thread_pool"); "xpack.watcher.thread_pool");
return Collections.singletonList(builder); return Collections.singletonList(builder);
} }