honeymoose
/
OpenSearch
mirror of https://github.com/honeymoose/OpenSearch.git
1
0
Fork 0
Code Issues Packages Projects Releases Wiki Activity

Revert Benchmark API 

The benchmark api is being worked on feature/bench branch and will be merged from there when ready.
This commit is contained in:
Simon Willnauer 2014-09-16 14:45:13 +02:00 committed by Luca Cavanna
parent 66fa72f21c
commit 7257345db9
53 changed files with 9 additions and 6461 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

2
docs/reference/search.asciidoc
View File

@ -105,5 +105,3 @@ include::search/percolate.asciidoc[]
include::search/more-like-this.asciidoc[]
include::search/benchmark.asciidoc[]

247
docs/reference/search/benchmark.asciidoc
View File

@ -1,247 +0,0 @@
[[search-benchmark]]
== Benchmark
experimental[]
The benchmark API provides a standard mechanism for submitting queries and
measuring their performance relative to one another.
[IMPORTANT]
=====
To be eligible to run benchmarks nodes must be started with: `--node.bench true`. This is just a way to mark certain nodes as "executors". Searches will still be distributed out to the cluster in the normal manner. This is primarily a defensive measure to prevent production nodes from being flooded with potentially many requests. Typically one would start a single node with this setting and submit benchmark requests to it.
=====
[source,bash]
--------------------------------------------------
$ ./bin/elasticsearch --node.bench true
--------------------------------------------------
Benchmarking a search request is as simple as executing the following command:
[source,js]
--------------------------------------------------
$ curl -XPUT 'localhost:9200/_bench/?pretty=true' -d '{
"name": "my_benchmark",
"competitors": [ {
"name": "my_competitor",
"requests": [ {
"query": {
"match": { "_all": "a*" }
}
} ]
} ]
}'
--------------------------------------------------
Response:
[source,js]
--------------------------------------------------
{
"status" : "complete",
"competitors" : {
"my_competitor" : {
"summary" : {
"nodes" : [ "localhost" ],
"total_iterations" : 5,
"completed_iterations" : 5,
"total_queries" : 1000,
"concurrency" : 5,
"multiplier" : 100,
"avg_warmup_time" : 43.0,
"statistics" : {
"min" : 1,
"max" : 10,
"mean" : 4.19,
"qps" : 238.663,
"std_dev" : 1.938,
"millis_per_hit" : 1.064,
"percentile_10" : 2,
"percentile_25" : 3,
"percentile_50" : 4,
"percentile_75" : 5,
"percentile_90" : 7,
"percentile_99" : 10
}
}
}
}
}
--------------------------------------------------
A 'competitor' defines one or more search requests to execute along with parameters that describe how the search(es) should be run.
Multiple competitors may be submitted as a group in which case they will execute one after the other. This makes it easy to compare various
competing alternatives side-by-side.
There are several parameters which may be set at the competition level:
[horizontal]
`name`:: Unique name for the competition.
`iterations`:: Number of times to run the competitors. Defaults to `5`.
`concurrency`:: Within each iteration use this level of parallelism. Defaults to `5`.
`multiplier`:: Within each iteration run the query this many times. Defaults to `1000`.
`warmup`:: Perform warmup of query. Defaults to `true`.
`num_slowest`:: Record N slowest queries. Defaults to `1`.
`search_type`:: Type of search, e.g. "query_then_fetch", "dfs_query_then_fetch", "count". Defaults to `query_then_fetch`.
`requests`:: Query DSL describing search requests.
`clear_caches`:: Whether caches should be cleared on each iteration, and if so, how. Caches are not cleared by default.
`indices`:: Array of indices to search, e.g. ["my_index_1", "my_index_2", "my_index_3"].
`types`:: Array of index types to search, e.g. ["my_type_1", "my_type_2"].
Cache clearing parameters:
[horizontal]
`clear_caches`:: Set to 'false' to disable cache clearing completely.
`clear_caches.filter`:: Whether to clear the filter cache.
`clear_caches.field_data`:: Whether to clear the field data cache.
`clear_caches.id`:: Whether to clear the id cache.
`clear_caches.recycler`:: Whether to clear the recycler cache.
`clear_caches.fields`:: Array of fields to clear.
`clear_caches.filter_keys`:: Array of filter keys to clear.
Global parameters:
[horizontal]
`name`:: Unique name for the benchmark.
`num_executor_nodes`:: Number of cluster nodes from which to submit and time benchmarks. Allows user to run a benchmark simultaneously on one or more nodes and compare timings. Note that this does not control how many nodes a search request will actually execute on. Defaults to: 1.
`percentiles`:: Array of percentile values to report. Defaults to: [10, 25, 50, 75, 90, 99].
Additionally, the following competition-level parameters may be set globally: iteration, concurrency, multiplier, warmup, and clear_caches.
Using these parameters it is possible to describe precisely how to execute a benchmark under various conditions. In the following example we run a filtered query against two different indices using two different search types.
[source,js]
--------------------------------------------------
$ curl -XPUT 'localhost:9200/_bench/?pretty=true' -d '{
"name": "my_benchmark",
"num_executor_nodes": 1,
"percentiles" : [ 25, 50, 75 ],
"iterations": 5,
"multiplier": 1000,
"concurrency": 5,
"num_slowest": 0,
"warmup": true,
"clear_caches": false,
"requests": [ {
"query" : {
"filtered" : {
"query" : { "match" : { "_all" : "*" } },
"filter" : {
"and" : [ { "term" : { "title" : "Spain" } },
{ "term" : { "title" : "rain" } },
{ "term" : { "title" : "plain" } } ]
}
}
}
} ],
"competitors": [ {
"name": "competitor_1",
"search_type": "query_then_fetch",
"indices": [ "my_index_1" ],
"types": [ "my_type_1" ],
"clear_caches" : {
"filter" : true,
"field_data" : true,
"id" : true,
"recycler" : true,
"fields": ["title"]
}
}, {
"name": "competitor_2",
"search_type": "dfs_query_then_fetch",
"indices": [ "my_index_2" ],
"types": [ "my_type_2" ],
"clear_caches" : {
"filter" : true,
"field_data" : true,
"id" : true,
"recycler" : true,
"fields": ["title"]
}
} ]
}'
--------------------------------------------------
Response:
[source,js]
--------------------------------------------------
{
"status" : "complete",
"competitors" : {
"competitor_1" : {
"summary" : {
"nodes" : [ "localhost" ],
"total_iterations" : 5,
"completed_iterations" : 5,
"total_queries" : 5000,
"concurrency" : 5,
"multiplier" : 1000,
"avg_warmup_time" : 54.0,
"statistics" : {
"min" : 0,
"max" : 3,
"mean" : 0.533,
"qps" : 1872.659,
"std_dev" : 0.528,
"millis_per_hit" : 0.0,
"percentile_25" : 0.0,
"percentile_50" : 1.0,
"percentile_75" : 1.0
}
}
},
"competitor_2" : {
"summary" : {
"nodes" : [ "localhost" ],
"total_iterations" : 5,
"completed_iterations" : 5,
"total_queries" : 5000,
"concurrency" : 5,
"multiplier" : 1000,
"avg_warmup_time" : 4.0,
"statistics" : {
"min" : 0,
"max" : 4,
"mean" : 0.487,
"qps" : 2049.180,
"std_dev" : 0.545,
"millis_per_hit" : 0.0,
"percentile_25" : 0.0,
"percentile_50" : 0.0,
"percentile_75" : 1.0
}
}
}
}
}
--------------------------------------------------
In some cases it may be desirable to view the progress of a long-running benchmark and optionally terminate it early. To view all active benchmarks use:
[source,js]
--------------------------------------------------
$ curl -XGET 'localhost:9200/_bench?pretty'
--------------------------------------------------
This would display run-time statistics in the same format as the sample output above.
To abort a long-running benchmark use the 'abort' endpoint:
[source,js]
--------------------------------------------------
$ curl -XPOST 'localhost:9200/_bench/abort/my_benchmark?pretty'
--------------------------------------------------
Response:
[source,js]
--------------------------------------------------
{
"aborted_benchmarks" : [
"node" "localhost",
"benchmark_name", "my_benchmark",
"aborted", true
]
}
--------------------------------------------------

20
rest-api-spec/api/abort_benchmark.json
View File

@ -1,20 +0,0 @@
{
"abort_benchmark" : {
"documentation": "http://www.elasticsearch.org/guide/en/elasticsearch/reference/master/search-benchmark.html",
"methods": ["POST"],
"url": {
"path": "/_bench/abort/{name}",
"paths": [
"/_bench/abort/{name}"
],
"parts": {
"name": {
"type" : "string",
"description" : "A benchmark name"
}
},
"params": {}
},
"body": null
}
}

33
rest-api-spec/api/benchmark.json
View File

@ -1,33 +0,0 @@
{
"benchmark" : {
"documentation": "http://www.elasticsearch.org/guide/en/elasticsearch/reference/master/search-benchmark.html",
"methods": ["PUT"],
"url": {
"path": "/_bench",
"paths": [
"/_bench",
"/{index}/_bench",
"/{index}/{type}/_bench"
],
"parts": {
"index": {
"type" : "list",
"description" : "A comma-separated list of index names; use `_all` or empty string to perform the operation on all indices"
},
"type": {
"type" : "string",
"description" : "The name of the document type"
}
},
"params": {
"verbose": {
"type": "boolean",
"description": "Specify whether to return verbose statistics about each iteration (default: false)"
}
}
},
"body": {
"description": "The search definition using the Query DSL"
}
}
}

26
rest-api-spec/api/list_benchmarks.json
View File

@ -1,26 +0,0 @@
{
"list_benchmarks" : {
"documentation": "http://www.elasticsearch.org/guide/en/elasticsearch/reference/master/search-benchmark.html",
"methods": ["GET"],
"url": {
"path": "/_bench",
"paths": [
"/_bench",
"/{index}/_bench",
"/{index}/{type}/_bench"
],
"parts": {
"index": {
"type" : "list",
"description" : "A comma-separated list of index names; use `_all` or empty string to perform the operation on all indices"
},
"type": {
"type" : "string",
"description" : "The name of the document type"
}
},
"params": {}
},
"body": null
}
}

11
rest-api-spec/test/abort_benchmark/10_basic.yaml
View File

@ -1,11 +0,0 @@
---
"Test benchmark abort":
- skip:
features: "benchmark"
- do:
abort_benchmark:
name: my_benchmark
catch: missing

33
rest-api-spec/test/benchmark/10_basic.yaml
View File

@ -1,33 +0,0 @@
---
"Test benchmark submit":
- skip:
features: "benchmark"
- do:
indices.create:
index: test_1
body:
settings:
index:
number_of_replicas: 0
- do:
cluster.health:
wait_for_status: yellow
- do:
benchmark:
index: test_1
body:
"name": "my_benchmark"
"competitors":
-
"name": "my_competitor"
"requests":
-
"query":
"match": { "_all": "*" }
- match: { status: COMPLETE }

9
rest-api-spec/test/list_benchmarks/10_basic.yaml
View File

@ -1,9 +0,0 @@
---
"Test benchmark list":
- skip:
features: "benchmark"
- do:
list_benchmarks: {}

4
src/main/java/org/elasticsearch/action/ActionModule.java
View File

@ -121,7 +121,6 @@ import org.elasticsearch.action.admin.indices.warmer.get.GetWarmersAction;
import org.elasticsearch.action.admin.indices.warmer.get.TransportGetWarmersAction;
import org.elasticsearch.action.admin.indices.warmer.put.PutWarmerAction;
import org.elasticsearch.action.admin.indices.warmer.put.TransportPutWarmerAction;
import org.elasticsearch.action.bench.*;
import org.elasticsearch.action.bulk.BulkAction;
import org.elasticsearch.action.bulk.TransportBulkAction;
import org.elasticsearch.action.bulk.TransportShardBulkAction;
@ -310,9 +309,6 @@ public class ActionModule extends AbstractModule {
registerAction(ExplainAction.INSTANCE, TransportExplainAction.class);
registerAction(ClearScrollAction.INSTANCE, TransportClearScrollAction.class);
registerAction(RecoveryAction.INSTANCE, TransportRecoveryAction.class);
registerAction(BenchmarkAction.INSTANCE, TransportBenchmarkAction.class);
registerAction(AbortBenchmarkAction.INSTANCE, TransportAbortBenchmarkAction.class);
registerAction(BenchmarkStatusAction.INSTANCE, TransportBenchmarkStatusAction.class);
//Indexed scripts
registerAction(PutIndexedScriptAction.INSTANCE, TransportPutIndexedScriptAction.class);

46
src/main/java/org/elasticsearch/action/bench/AbortBenchmarkAction.java
View File

@ -1,46 +0,0 @@
/*
* Licensed to Elasticsearch under one or more contributor
* license agreements. See the NOTICE file distributed with
* this work for additional information regarding copyright
* ownership. Elasticsearch licenses this file to you under
* the Apache License, Version 2.0 (the "License"); you may
* not use this file except in compliance with the License.
* You may obtain a copy of the License at
*
* http://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing,
* software distributed under the License is distributed on an
* "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
* KIND, either express or implied. See the License for the
* specific language governing permissions and limitations
* under the License.
*/
package org.elasticsearch.action.bench;
import org.elasticsearch.action.ClientAction;
import org.elasticsearch.client.Client;
/**
* Abort benchmark action
*/
public class AbortBenchmarkAction extends ClientAction<AbortBenchmarkRequest, AbortBenchmarkResponse, AbortBenchmarkRequestBuilder> {
public static final AbortBenchmarkAction INSTANCE = new AbortBenchmarkAction();
public static final String NAME = "indices:data/benchmark/abort";
private AbortBenchmarkAction() {
super(NAME);
}
@Override
public AbortBenchmarkResponse newResponse() {
return new AbortBenchmarkResponse();
}
@Override
public AbortBenchmarkRequestBuilder newRequestBuilder(Client client) {
return new AbortBenchmarkRequestBuilder(client);
}
}

72
src/main/java/org/elasticsearch/action/bench/AbortBenchmarkRequest.java
View File

@ -1,72 +0,0 @@
/*
* Licensed to Elasticsearch under one or more contributor
* license agreements. See the NOTICE file distributed with
* this work for additional information regarding copyright
* ownership. Elasticsearch licenses this file to you under
* the Apache License, Version 2.0 (the "License"); you may
* not use this file except in compliance with the License.
* You may obtain a copy of the License at
*
* http://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing,
* software distributed under the License is distributed on an
* "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
* KIND, either express or implied. See the License for the
* specific language governing permissions and limitations
* under the License.
*/
package org.elasticsearch.action.bench;
import org.elasticsearch.action.ActionRequestValidationException;
import org.elasticsearch.action.ValidateActions;
import org.elasticsearch.action.support.master.AcknowledgedRequest;
import org.elasticsearch.common.Strings;
import org.elasticsearch.common.io.stream.StreamInput;
import org.elasticsearch.common.io.stream.StreamOutput;
import java.io.IOException;
/**
* A request to abort a specified benchmark
*/
public class AbortBenchmarkRequest extends AcknowledgedRequest {
private String[] benchmarkNames = Strings.EMPTY_ARRAY;
public AbortBenchmarkRequest() { }
public AbortBenchmarkRequest(String... benchmarkNames) {
this.benchmarkNames = benchmarkNames;
}
public void benchmarkNames(String... benchmarkNames) {
this.benchmarkNames = benchmarkNames;
}
public String[] benchmarkNames() {
return benchmarkNames;
}
@Override
public ActionRequestValidationException validate() {
if (benchmarkNames == null || benchmarkNames.length == 0) {
return ValidateActions.addValidationError("benchmarkNames must not be null or empty", null);
}
return null;
}
@Override
public void readFrom(StreamInput in) throws IOException {
super.readFrom(in);
benchmarkNames = in.readStringArray();
readTimeout(in);
}
@Override
public void writeTo(StreamOutput out) throws IOException {
super.writeTo(out);
out.writeStringArray(benchmarkNames);
writeTimeout(out);
}
}

45
src/main/java/org/elasticsearch/action/bench/AbortBenchmarkRequestBuilder.java
View File

@ -1,45 +0,0 @@
/*
* Licensed to Elasticsearch under one or more contributor
* license agreements. See the NOTICE file distributed with
* this work for additional information regarding copyright
* ownership. Elasticsearch licenses this file to you under
* the Apache License, Version 2.0 (the "License"); you may
* not use this file except in compliance with the License.
* You may obtain a copy of the License at
*
* http://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing,
* software distributed under the License is distributed on an
* "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
* KIND, either express or implied. See the License for the
* specific language governing permissions and limitations
* under the License.
*/
package org.elasticsearch.action.bench;
import org.elasticsearch.action.ActionListener;
import org.elasticsearch.action.ActionRequestBuilder;
import org.elasticsearch.client.Client;
/**
* Request builder for aborting a benchmark
*/
public class AbortBenchmarkRequestBuilder extends ActionRequestBuilder<AbortBenchmarkRequest, AbortBenchmarkResponse, AbortBenchmarkRequestBuilder, Client> {
public AbortBenchmarkRequestBuilder(Client client) {
super(client, new AbortBenchmarkRequest());
}
public AbortBenchmarkRequestBuilder setBenchmarkNames(String... benchmarkNames) {
request.benchmarkNames(benchmarkNames);
return this;
}
@Override
protected void doExecute(ActionListener<AbortBenchmarkResponse> listener) {
client.abortBench(request, listener);
}
}

50
src/main/java/org/elasticsearch/action/bench/AbortBenchmarkResponse.java
View File

@ -1,50 +0,0 @@
/*
* Licensed to Elasticsearch under one or more contributor
* license agreements. See the NOTICE file distributed with
* this work for additional information regarding copyright
* ownership. Elasticsearch licenses this file to you under
* the Apache License, Version 2.0 (the "License"); you may
* not use this file except in compliance with the License.
* You may obtain a copy of the License at
*
* http://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing,
* software distributed under the License is distributed on an
* "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
* KIND, either express or implied. See the License for the
* specific language governing permissions and limitations
* under the License.
*/
package org.elasticsearch.action.bench;
import org.elasticsearch.action.support.master.AcknowledgedResponse;
import org.elasticsearch.common.io.stream.StreamInput;
import org.elasticsearch.common.io.stream.StreamOutput;
import java.io.IOException;
/**
* Response for a benchmark abort request
*/
public class AbortBenchmarkResponse extends AcknowledgedResponse {
public AbortBenchmarkResponse() {
super();
}
public AbortBenchmarkResponse(boolean acknowledged) {
super(acknowledged);
}
@Override
public void readFrom(StreamInput in) throws IOException {
super.readFrom(in);
readAcknowledged(in);
}
@Override
public void writeTo(StreamOutput out) throws IOException {
super.writeTo(out);
writeAcknowledged(out);
}
}

46
src/main/java/org/elasticsearch/action/bench/BenchmarkAction.java
View File

@ -1,46 +0,0 @@
/*
* Licensed to Elasticsearch under one or more contributor
* license agreements. See the NOTICE file distributed with
* this work for additional information regarding copyright
* ownership. Elasticsearch licenses this file to you under
* the Apache License, Version 2.0 (the "License"); you may
* not use this file except in compliance with the License.
* You may obtain a copy of the License at
*
* http://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing,
* software distributed under the License is distributed on an
* "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
* KIND, either express or implied. See the License for the
* specific language governing permissions and limitations
* under the License.
*/
package org.elasticsearch.action.bench;
import org.elasticsearch.action.ClientAction;
import org.elasticsearch.client.Client;
import org.elasticsearch.common.Strings;
/**
* Benchmark action
*/
public class BenchmarkAction extends ClientAction<BenchmarkRequest, BenchmarkResponse, BenchmarkRequestBuilder> {
public static final BenchmarkAction INSTANCE = new BenchmarkAction();
public static final String NAME = "indices:data/benchmark/start";
private BenchmarkAction() {
super(NAME);
}
@Override
public BenchmarkResponse newResponse() {
return new BenchmarkResponse();
}
@Override
public BenchmarkRequestBuilder newRequestBuilder(Client client) {
return new BenchmarkRequestBuilder(client, Strings.EMPTY_ARRAY);
}
}

116
src/main/java/org/elasticsearch/action/bench/BenchmarkCompetitor.java
View File

@ -1,116 +0,0 @@
/*
* Licensed to Elasticsearch under one or more contributor
* license agreements. See the NOTICE file distributed with
* this work for additional information regarding copyright
* ownership. Elasticsearch licenses this file to you under
* the Apache License, Version 2.0 (the "License"); you may
* not use this file except in compliance with the License.
* You may obtain a copy of the License at
*
* http://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing,
* software distributed under the License is distributed on an
* "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
* KIND, either express or implied. See the License for the
* specific language governing permissions and limitations
* under the License.
*/
package org.elasticsearch.action.bench;
import org.elasticsearch.action.ActionRequestValidationException;
import org.elasticsearch.action.ValidateActions;
import org.elasticsearch.common.io.stream.StreamInput;
import org.elasticsearch.common.io.stream.StreamOutput;
import org.elasticsearch.common.io.stream.Streamable;
import java.io.IOException;
/**
* A benchmark competitor describes how to run a search benchmark. Multiple competitors may be
* submitted in a single benchmark request, with their results compared.
*
* Competitors are executed in two loops. The outer loop is the iteration loop. The number of times
* this runs is controlled by the 'iterations' variable.
*
* The inner loop is the multiplier loop. This is controlled by the 'multiplier' variable.
*
* The level of concurrency pertains to the number of simultaneous searches that may be executed within
* the inner multiplier loop. Iterations are never run concurrently; they run serially.
*/
public class BenchmarkCompetitor implements Streamable {
private String name;
private BenchmarkSettings settings = new BenchmarkSettings();
/**
* Constructs a competition across the given indices
* @param indices Indices
*/
BenchmarkCompetitor(String... indices) {
settings.indices(indices);
}
/**
* Constructs a competition
*/
BenchmarkCompetitor() { }
ActionRequestValidationException validate(ActionRequestValidationException validationException) {
if (name == null) {
validationException = ValidateActions.addValidationError("name must not be null", validationException);
}
if (settings.concurrency() < 1) {
validationException = ValidateActions.addValidationError("concurrent requests must be >= 1 but was [" + settings.concurrency() + "]", validationException);
}
if (settings.iterations() < 1) {
validationException = ValidateActions.addValidationError("iterations must be >= 1 but was [" + settings.iterations() + "]", validationException);
}
if (settings.multiplier() < 1) {
validationException = ValidateActions.addValidationError("multiplier must be >= 1 but was [" + settings.multiplier() + "]", validationException);
}
if (settings.numSlowest() < 0) {
validationException = ValidateActions.addValidationError("numSlowest must be >= 0 but was [" + settings.numSlowest() + "]", validationException);
}
if (settings.searchType() == null) {
validationException = ValidateActions.addValidationError("searchType must not be null", validationException);
}
return validationException;
}
/**
* Gets the user-supplied name
* @return Name
*/
public String name() {
return name;
}
/**
* Sets the user-supplied name
* @param name Name
*/
public void name(String name) {
this.name = name;
}
/**
* Gets the benchmark settings
* @return Settings
*/
public BenchmarkSettings settings() {
return settings;
}
@Override
public void readFrom(StreamInput in) throws IOException {
name = in.readString();
settings = in.readOptionalStreamable(new BenchmarkSettings());
}
@Override
public void writeTo(StreamOutput out) throws IOException {
out.writeString(name);
out.writeOptionalStreamable(settings);
}
}

173
src/main/java/org/elasticsearch/action/bench/BenchmarkCompetitorBuilder.java
View File

@ -1,173 +0,0 @@
/*
* Licensed to Elasticsearch under one or more contributor
* license agreements. See the NOTICE file distributed with
* this work for additional information regarding copyright
* ownership. Elasticsearch licenses this file to you under
* the Apache License, Version 2.0 (the "License"); you may
* not use this file except in compliance with the License.
* You may obtain a copy of the License at
*
* http://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing,
* software distributed under the License is distributed on an
* "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
* KIND, either express or implied. See the License for the
* specific language governing permissions and limitations
* under the License.
*/
package org.elasticsearch.action.bench;
import org.elasticsearch.action.search.SearchRequest;
import org.elasticsearch.action.search.SearchType;
import org.elasticsearch.action.bench.BenchmarkSettings.ClearCachesSettings;
/**
* Builder for a benchmark competitor
*/
public class BenchmarkCompetitorBuilder {
private final BenchmarkCompetitor competitor;
/**
* Constructs a new competitor builder to run a competition on the given indices
* @param indices Indices to run against
*/
public BenchmarkCompetitorBuilder(String... indices) {
competitor = new BenchmarkCompetitor(indices);
}
/**
* If true, competition will run a 'warmup' round. This is to prevent timings from a cold start.
* @param warmup Whether to do a warmup
* @return this
*/
public BenchmarkCompetitorBuilder setWarmup(boolean warmup) {
competitor.settings().warmup(warmup, true);
return this;
}
/**
* Sets the list if indices to execute against
* @param indices Indices to run against
* @return this
*/
public BenchmarkCompetitorBuilder setIndices(String... indices) {
competitor.settings().indices(indices);
return this;
}
/**
* Sets the types of the indices to execute against
* @param types Types of indices
* @return this
*/
public BenchmarkCompetitorBuilder setTypes(String... types) {
competitor.settings().types(types);
return this;
}
/**
* Whether a competitor is allowed to clear index caches. If true, and if a
* ClearCachesSettings has been set, the competitor will
* submit an index cache clear action at the top of each iteration.
* @param allowCacheClearing If true, allow caches to be cleared
* @return this
*/
public BenchmarkCompetitorBuilder setAllowCacheClearing(boolean allowCacheClearing) {
competitor.settings().allowCacheClearing(allowCacheClearing);
return this;
}
/**
* Describes how an index cache clear request should be executed.
* @param clearCachesSettings Description of how to clear caches
* @return this
*/
public BenchmarkCompetitorBuilder setClearCachesSettings(ClearCachesSettings clearCachesSettings) {
competitor.settings().clearCachesSettings(clearCachesSettings, true);
return this;
}
/**
* Sets the concurrency level with which to run the competition. This determines the number of
* actively executing searches which the competition will run in parallel.
* @param concurrency Number of searches to run concurrently
* @return this
*/
public BenchmarkCompetitorBuilder setConcurrency(int concurrency) {
competitor.settings().concurrency(concurrency, true);
return this;
}
/**
* Adds a search request to the competition
* @param searchRequest Search request
* @return this
*/
public BenchmarkCompetitorBuilder addSearchRequest(SearchRequest... searchRequest) {
competitor.settings().addSearchRequest(searchRequest);
return this;
}
/**
* Sets the number of times to run each competition
* @param iters Number of times to run the competition
* @return this
*/
public BenchmarkCompetitorBuilder setIterations(int iters) {
competitor.settings().iterations(iters, true);
return this;
}
/**
* A 'multiplier' for each iteration. This specifies how many times to execute
* the search requests within each iteration. The resulting number of total searches
* executed will be: iterations X multiplier. Setting a higher multiplier will
* smooth out results and dampen the effect of outliers.
* @param multiplier Iteration multiplier
* @return this
*/
public BenchmarkCompetitorBuilder setMultiplier(int multiplier) {
competitor.settings().multiplier(multiplier, true);
return this;
}
/**
* Sets the number of slowest requests to report
* @param numSlowest Number of slow requests to report
* @return this
*/
public BenchmarkCompetitorBuilder setNumSlowest(int numSlowest) {
competitor.settings().numSlowest(numSlowest, true);
return this;
}
/**
* Builds a competitor
* @return A new competitor
*/
public BenchmarkCompetitor build() {
return competitor;
}
/**
* Sets the type of search to execute
* @param searchType Search type
* @return this
*/
public BenchmarkCompetitorBuilder setSearchType(SearchType searchType) {
competitor.settings().searchType(searchType, true);
return this;
}
/**
* Sets the user-supplied name to the competitor
* @param name Name
* @return this
*/
public BenchmarkCompetitorBuilder setName(String name) {
competitor.name(name);
return this;
}
}

52
src/main/java/org/elasticsearch/action/bench/BenchmarkExecutionException.java
View File

@ -1,52 +0,0 @@
/*
* Licensed to Elasticsearch under one or more contributor
* license agreements. See the NOTICE file distributed with
* this work for additional information regarding copyright
* ownership. Elasticsearch licenses this file to you under
* the Apache License, Version 2.0 (the "License"); you may
* not use this file except in compliance with the License.
* You may obtain a copy of the License at
*
* http://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing,
* software distributed under the License is distributed on an
* "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
* KIND, either express or implied. See the License for the
* specific language governing permissions and limitations
* under the License.
*/
package org.elasticsearch.action.bench;
import org.elasticsearch.ElasticsearchException;
import org.elasticsearch.rest.RestStatus;
import java.util.ArrayList;
import java.util.List;
/**
* Indicates a benchmark failure due to too many failures being encountered.
*/
public class BenchmarkExecutionException extends ElasticsearchException {
private List<String> errorMessages = new ArrayList<>();
public BenchmarkExecutionException(String msg, Throwable cause) {
super(msg, cause);
}
public BenchmarkExecutionException(String msg, List<String> errorMessages) {
super(msg);
this.errorMessages.addAll(errorMessages);
}
public List<String> errorMessages() {
return errorMessages;
}
@Override
public RestStatus status() {
return RestStatus.INTERNAL_SERVER_ERROR;
}
}

449
src/main/java/org/elasticsearch/action/bench/BenchmarkExecutor.java
View File

@ -1,449 +0,0 @@
/*
* Licensed to Elasticsearch under one or more contributor
* license agreements. See the NOTICE file distributed with
* this work for additional information regarding copyright
* ownership. Elasticsearch licenses this file to you under
* the Apache License, Version 2.0 (the "License"); you may
* not use this file except in compliance with the License.
* You may obtain a copy of the License at
*
* http://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing,
* software distributed under the License is distributed on an
* "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
* KIND, either express or implied. See the License for the
* specific language governing permissions and limitations
* under the License.
*/
package org.elasticsearch.action.bench;
import com.google.common.collect.UnmodifiableIterator;
import org.apache.lucene.util.PriorityQueue;
import org.elasticsearch.ElasticsearchException;
import org.elasticsearch.ExceptionsHelper;
import org.elasticsearch.action.ActionListener;
import org.elasticsearch.action.search.SearchRequest;
import org.elasticsearch.action.search.SearchResponse;
import org.elasticsearch.client.Client;
import org.elasticsearch.cluster.ClusterService;
import org.elasticsearch.common.collect.ImmutableOpenMap;
import org.elasticsearch.common.logging.ESLogger;
import org.elasticsearch.common.logging.Loggers;
import java.util.*;
import java.util.concurrent.*;
/**
* Handles execution, listing, and aborting of benchmarks
*/
public class BenchmarkExecutor {
private static final ESLogger logger = Loggers.getLogger(BenchmarkExecutor.class);
private final Client client;
private String nodeName;
private final ClusterService clusterService;
private volatile ImmutableOpenMap<String, BenchmarkState> activeBenchmarks = ImmutableOpenMap.of();
private final Object activeStateLock = new Object();
public BenchmarkExecutor(Client client, ClusterService clusterService) {
this.client = client;
this.clusterService = clusterService;
}
private static class BenchmarkState {
final String id;
final StoppableSemaphore semaphore;
final BenchmarkResponse response;
BenchmarkState(BenchmarkRequest request, BenchmarkResponse response, StoppableSemaphore semaphore) {
this.id = request.benchmarkName();
this.response = response;
this.semaphore = semaphore;
}
}
/**
* Aborts benchmark(s) matching the given wildcard patterns
*
* @param names the benchmark names to abort
*/
public AbortBenchmarkResponse abortBenchmark(String[] names) {
synchronized (activeStateLock) {
for (String name : names) {
try {
final BenchmarkState state = activeBenchmarks.get(name);
if (state == null) {
continue;
}
state.semaphore.stop();
activeBenchmarks = ImmutableOpenMap.builder(activeBenchmarks).fRemove(name).build();
logger.debug("Aborted benchmark [{}] on [{}]", name, nodeName());
} catch (Throwable e) {
logger.warn("Error while aborting [{}]", name, e);
}
}
}
return new AbortBenchmarkResponse(true);
}
/**
* Reports status of all active benchmarks
*
* @return Benchmark status response
*/
public BenchmarkStatusNodeResponse benchmarkStatus() {
BenchmarkStatusNodeResponse response = new BenchmarkStatusNodeResponse();
final ImmutableOpenMap<String, BenchmarkState> activeBenchmarks = this.activeBenchmarks;
UnmodifiableIterator<String> iter = activeBenchmarks.keysIt();
while (iter.hasNext()) {
String id = iter.next();
BenchmarkState state = activeBenchmarks.get(id);
response.addBenchResponse(state.response);
}
logger.debug("Reporting [{}] active benchmarks on [{}]", response.activeBenchmarks(), nodeName());
return response;
}
/**
* Submits a search benchmark for execution
*
* @param request A benchmark request
* @return Summary response of executed benchmark
* @throws ElasticsearchException
*/
public BenchmarkResponse benchmark(BenchmarkRequest request) throws ElasticsearchException {
final StoppableSemaphore semaphore = new StoppableSemaphore(1);
final Map<String, CompetitionResult> competitionResults = new HashMap<String, CompetitionResult>();
final BenchmarkResponse benchmarkResponse = new BenchmarkResponse(request.benchmarkName(), competitionResults);
synchronized (activeStateLock) {
if (activeBenchmarks.containsKey(request.benchmarkName())) {
throw new ElasticsearchException("Benchmark [" + request.benchmarkName() + "] is already running on [" + nodeName() + "]");
}
activeBenchmarks = ImmutableOpenMap.builder(activeBenchmarks).fPut(
request.benchmarkName(), new BenchmarkState(request, benchmarkResponse, semaphore)).build();
}
try {
for (BenchmarkCompetitor competitor : request.competitors()) {
final BenchmarkSettings settings = competitor.settings();
final int iterations = settings.iterations();
logger.debug("Executing [iterations: {}] [multiplier: {}] for [{}] on [{}]",
iterations, settings.multiplier(), request.benchmarkName(), nodeName());
final List<CompetitionIteration> competitionIterations = new ArrayList<>(iterations);
final CompetitionResult competitionResult =
new CompetitionResult(competitor.name(), settings.concurrency(), settings.multiplier(), request.percentiles());
final CompetitionNodeResult competitionNodeResult =
new CompetitionNodeResult(competitor.name(), nodeName(), iterations, competitionIterations);
competitionResult.addCompetitionNodeResult(competitionNodeResult);
benchmarkResponse.competitionResults.put(competitor.name(), competitionResult);
final List<SearchRequest> searchRequests = competitor.settings().searchRequests();
if (settings.warmup()) {
final long beforeWarmup = System.nanoTime();
final List<String> warmUpErrors = warmUp(competitor, searchRequests, semaphore);
final long afterWarmup = System.nanoTime();
competitionNodeResult.warmUpTime(TimeUnit.MILLISECONDS.convert(afterWarmup - beforeWarmup, TimeUnit.NANOSECONDS));
if (!warmUpErrors.isEmpty()) {
throw new BenchmarkExecutionException("Failed to execute warmup phase", warmUpErrors);
}
}
final int numMeasurements = settings.multiplier() * searchRequests.size();
final long[] timeBuckets = new long[numMeasurements];
final long[] docBuckets = new long[numMeasurements];
for (int i = 0; i < iterations; i++) {
if (settings.allowCacheClearing() && settings.clearCaches() != null) {
try {
client.admin().indices().clearCache(settings.clearCaches()).get();
} catch (ExecutionException e) {
throw new BenchmarkExecutionException("Failed to clear caches", e);
}
}
// Run the iteration
CompetitionIteration ci =
runIteration(competitor, searchRequests, timeBuckets, docBuckets, semaphore);
ci.percentiles(request.percentiles());
competitionIterations.add(ci);
competitionNodeResult.incrementCompletedIterations();
}
competitionNodeResult.totalExecutedQueries(settings.multiplier() * searchRequests.size() * iterations);
}
benchmarkResponse.state(BenchmarkResponse.State.COMPLETE);
} catch (BenchmarkExecutionException e) {
benchmarkResponse.state(BenchmarkResponse.State.FAILED);
benchmarkResponse.errors(e.errorMessages().toArray(new String[e.errorMessages().size()]));
} catch (InterruptedException ex) {
Thread.currentThread().interrupt();
benchmarkResponse.state(BenchmarkResponse.State.ABORTED);
} catch (Throwable ex) {
logger.debug("Unexpected exception during benchmark", ex);
benchmarkResponse.state(BenchmarkResponse.State.FAILED);
benchmarkResponse.errors(ex.getMessage());
} finally {
synchronized (activeStateLock) {
semaphore.stop();
activeBenchmarks = ImmutableOpenMap.builder(activeBenchmarks).fRemove(request.benchmarkName()).build();
}
}
return benchmarkResponse;
}
private List<String> warmUp(BenchmarkCompetitor competitor, List<SearchRequest> searchRequests, StoppableSemaphore stoppableSemaphore)
throws InterruptedException {
final StoppableSemaphore semaphore = stoppableSemaphore.reset(competitor.settings().concurrency());
final CountDownLatch totalCount = new CountDownLatch(searchRequests.size());
final CopyOnWriteArrayList<String> errorMessages = new CopyOnWriteArrayList<>();
for (SearchRequest searchRequest : searchRequests) {
semaphore.acquire();
client.search(searchRequest, new BoundsManagingActionListener<SearchResponse>(semaphore, totalCount, errorMessages) { } );
}
totalCount.await();
return errorMessages;
}
private CompetitionIteration runIteration(BenchmarkCompetitor competitor, List<SearchRequest> searchRequests,
final long[] timeBuckets, final long[] docBuckets,
StoppableSemaphore stoppableSemaphore) throws InterruptedException {
assert timeBuckets.length == competitor.settings().multiplier() * searchRequests.size();
assert docBuckets.length == competitor.settings().multiplier() * searchRequests.size();
final StoppableSemaphore semaphore = stoppableSemaphore.reset(competitor.settings().concurrency());
Arrays.fill(timeBuckets, -1); // wipe CPU cache ;)
Arrays.fill(docBuckets, -1); // wipe CPU cache ;)
int id = 0;
final CountDownLatch totalCount = new CountDownLatch(timeBuckets.length);
final CopyOnWriteArrayList<String> errorMessages = new CopyOnWriteArrayList<>();
final long beforeRun = System.nanoTime();
for (int i = 0; i < competitor.settings().multiplier(); i++) {
for (SearchRequest searchRequest : searchRequests) {
StatisticCollectionActionListener statsListener =
new StatisticCollectionActionListener(semaphore, timeBuckets, docBuckets, id++, totalCount, errorMessages);
semaphore.acquire();
client.search(searchRequest, statsListener);
}
}
totalCount.await();
assert id == timeBuckets.length;
final long afterRun = System.nanoTime();
if (!errorMessages.isEmpty()) {
throw new BenchmarkExecutionException("Too many execution failures", errorMessages);
}
final long totalTime = TimeUnit.MILLISECONDS.convert(afterRun - beforeRun, TimeUnit.NANOSECONDS);
CompetitionIterationData iterationData = new CompetitionIterationData(timeBuckets);
long sumDocs = new CompetitionIterationData(docBuckets).sum();
// Don't track slowest request if there is only one request as that is redundant
CompetitionIteration.SlowRequest[] topN = null;
if ((competitor.settings().numSlowest() > 0) && (searchRequests.size() > 1)) {
topN = getTopN(timeBuckets, searchRequests, competitor.settings().multiplier(), competitor.settings().numSlowest());
}
CompetitionIteration round =
new CompetitionIteration(topN, totalTime, timeBuckets.length, sumDocs, iterationData);
return round;
}
private CompetitionIteration.SlowRequest[] getTopN(long[] buckets, List<SearchRequest> requests, int multiplier, int topN) {
final int numRequests = requests.size();
// collect the top N
final PriorityQueue<IndexAndTime> topNQueue = new PriorityQueue<IndexAndTime>(topN) {
@Override
protected boolean lessThan(IndexAndTime a, IndexAndTime b) {
return a.avgTime < b.avgTime;
}
};
assert multiplier > 0;
for (int i = 0; i < numRequests; i++) {
long sum = 0;
long max = Long.MIN_VALUE;
for (int j = 0; j < multiplier; j++) {
final int base = (numRequests * j);
sum += buckets[i + base];
max = Math.max(buckets[i + base], max);
}
final long avg = sum / multiplier;
if (topNQueue.size() < topN) {
topNQueue.add(new IndexAndTime(i, max, avg));
} else if (topNQueue.top().avgTime < max) {
topNQueue.top().update(i, max, avg);
topNQueue.updateTop();
}
}
final CompetitionIteration.SlowRequest[] slowRequests = new CompetitionIteration.SlowRequest[topNQueue.size()];
int i = topNQueue.size() - 1;
while (topNQueue.size() > 0) {
IndexAndTime pop = topNQueue.pop();
CompetitionIteration.SlowRequest slow =
new CompetitionIteration.SlowRequest(pop.avgTime, pop.maxTime, requests.get(pop.index));
slowRequests[i--] = slow;
}
return slowRequests;
}
private static class IndexAndTime {
int index;
long maxTime;
long avgTime;
public IndexAndTime(int index, long maxTime, long avgTime) {
this.index = index;
this.maxTime = maxTime;
this.avgTime = avgTime;
}
public void update(int index, long maxTime, long avgTime) {
this.index = index;
this.maxTime = maxTime;
this.avgTime = avgTime;
}
}
private static abstract class BoundsManagingActionListener<Response> implements ActionListener<Response> {
private final StoppableSemaphore semaphore;
private final CountDownLatch latch;
private final CopyOnWriteArrayList<String> errorMessages;
public BoundsManagingActionListener(StoppableSemaphore semaphore, CountDownLatch latch, CopyOnWriteArrayList<String> errorMessages) {
this.semaphore = semaphore;
this.latch = latch;
this.errorMessages = errorMessages;
}
private void manage() {
try {
semaphore.release();
} finally {
latch.countDown();
}
}
@Override
public void onResponse(Response response) {
manage();
}
@Override
public void onFailure(Throwable e) {
try {
if (errorMessages.size() < 5) {
logger.debug("Failed to execute benchmark [{}]", e.getMessage(), e);
e = ExceptionsHelper.unwrapCause(e);
errorMessages.add(e.getLocalizedMessage());
}
} finally {
manage(); // first add the msg then call the count down on the latch otherwise we might iss one error
}
}
}
private static class StatisticCollectionActionListener extends BoundsManagingActionListener<SearchResponse> {
private final long[] timeBuckets;
private final int bucketId;
private final long[] docBuckets;
public StatisticCollectionActionListener(StoppableSemaphore semaphore, long[] timeBuckets, long[] docs,
int bucketId, CountDownLatch totalCount,
CopyOnWriteArrayList<String> errorMessages) {
super(semaphore, totalCount, errorMessages);
this.bucketId = bucketId;
this.timeBuckets = timeBuckets;
this.docBuckets = docs;
}
@Override
public void onResponse(SearchResponse searchResponse) {
super.onResponse(searchResponse);
timeBuckets[bucketId] = searchResponse.getTookInMillis();
if (searchResponse.getHits() != null) {
docBuckets[bucketId] = searchResponse.getHits().getTotalHits();
}
}
@Override
public void onFailure(Throwable e) {
try {
timeBuckets[bucketId] = -1;
docBuckets[bucketId] = -1;
} finally {
super.onFailure(e);
}
}
}
private final static class StoppableSemaphore {
private Semaphore semaphore;
private volatile boolean stopped = false;
public StoppableSemaphore(int concurrency) {
semaphore = new Semaphore(concurrency);
}
public StoppableSemaphore reset(int concurrency) {
semaphore = new Semaphore(concurrency);
return this;
}
public void acquire() throws InterruptedException {
if (stopped) {
throw new InterruptedException("Benchmark Interrupted");
}
semaphore.acquire();
}
public void release() {
semaphore.release();
}
public void stop() {
stopped = true;
}
}
private String nodeName() {
if (nodeName == null) {
nodeName = clusterService.localNode().name();
}
return nodeName;
}
private final boolean assertBuckets(long[] buckets) {
for (int i = 0; i < buckets.length; i++) {
assert buckets[i] >= 0 : "Bucket value was negative: " + buckets[i] + " bucket id: " + i;
}
return true;
}
}

38
src/main/java/org/elasticsearch/action/bench/BenchmarkMissingException.java
View File

@ -1,38 +0,0 @@
/*
* Licensed to Elasticsearch under one or more contributor
* license agreements. See the NOTICE file distributed with
* this work for additional information regarding copyright
* ownership. Elasticsearch licenses this file to you under
* the Apache License, Version 2.0 (the "License"); you may
* not use this file except in compliance with the License.
* You may obtain a copy of the License at
*
* http://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing,
* software distributed under the License is distributed on an
* "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
* KIND, either express or implied. See the License for the
* specific language governing permissions and limitations
* under the License.
*/
package org.elasticsearch.action.bench;
import org.elasticsearch.ElasticsearchException;
import org.elasticsearch.rest.RestStatus;
/**
* Thrown when a client tries to access a benchmark which does not exist
*/
public class BenchmarkMissingException extends ElasticsearchException {
public BenchmarkMissingException(String msg) {
super(msg);
}
@Override
public RestStatus status() {
return RestStatus.NOT_FOUND;
}
}

48
src/main/java/org/elasticsearch/action/bench/BenchmarkModule.java
View File

@ -1,48 +0,0 @@
/*
* Licensed to Elasticsearch under one or more contributor
* license agreements. See the NOTICE file distributed with
* this work for additional information regarding copyright
* ownership. Elasticsearch licenses this file to you under
* the Apache License, Version 2.0 (the "License"); you may
* not use this file except in compliance with the License.
* You may obtain a copy of the License at
*
* http://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing,
* software distributed under the License is distributed on an
* "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
* KIND, either express or implied. See the License for the
* specific language governing permissions and limitations
* under the License.
*/
package org.elasticsearch.action.bench;
import org.elasticsearch.common.inject.AbstractModule;
import org.elasticsearch.common.settings.Settings;
/**
* Benchmark module
*/
public class BenchmarkModule extends AbstractModule {
private final Settings settings;
public static final String BENCHMARK_SERVICE_KEY = "benchmark.service.impl";
public BenchmarkModule(Settings settings) {
this.settings = settings;
}
@Override
protected void configure() {
final Class<? extends BenchmarkService> service = settings.getAsClass(BENCHMARK_SERVICE_KEY, BenchmarkService.class);
if (!BenchmarkService.class.equals(service)) {
bind(BenchmarkService.class).to(service).asEagerSingleton();
} else {
bind(BenchmarkService.class).asEagerSingleton();
}
}
}

38
src/main/java/org/elasticsearch/action/bench/BenchmarkNodeMissingException.java
View File

@ -1,38 +0,0 @@
/*
* Licensed to Elasticsearch under one or more contributor
* license agreements. See the NOTICE file distributed with
* this work for additional information regarding copyright
* ownership. Elasticsearch licenses this file to you under
* the Apache License, Version 2.0 (the "License"); you may
* not use this file except in compliance with the License.
* You may obtain a copy of the License at
*
* http://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing,
* software distributed under the License is distributed on an
* "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
* KIND, either express or implied. See the License for the
* specific language governing permissions and limitations
* under the License.
*/
package org.elasticsearch.action.bench;
import org.elasticsearch.ElasticsearchException;
import org.elasticsearch.rest.RestStatus;
/**
* Indicates that a benchmark cannot be executed due to a lack of candidate nodes.
*/
public class BenchmarkNodeMissingException extends ElasticsearchException {
public BenchmarkNodeMissingException(String msg) {
super(msg);
}
@Override
public RestStatus status() {
return RestStatus.SERVICE_UNAVAILABLE;
}
}

287
src/main/java/org/elasticsearch/action/bench/BenchmarkRequest.java
View File

@ -1,287 +0,0 @@
/*
* Licensed to Elasticsearch under one or more contributor
* license agreements. See the NOTICE file distributed with
* this work for additional information regarding copyright
* ownership. Elasticsearch licenses this file to you under
* the Apache License, Version 2.0 (the "License"); you may
* not use this file except in compliance with the License.
* You may obtain a copy of the License at
*
* http://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing,
* software distributed under the License is distributed on an
* "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
* KIND, either express or implied. See the License for the
* specific language governing permissions and limitations
* under the License.
*/
package org.elasticsearch.action.bench;
import com.google.common.collect.Lists;
import org.elasticsearch.action.ActionRequestValidationException;
import org.elasticsearch.action.CompositeIndicesRequest;
import org.elasticsearch.action.IndicesRequest;
import org.elasticsearch.action.ValidateActions;
import org.elasticsearch.action.search.SearchRequest;
import org.elasticsearch.action.support.master.MasterNodeOperationRequest;
import org.elasticsearch.common.io.stream.StreamInput;
import org.elasticsearch.common.io.stream.StreamOutput;
import java.io.IOException;
import java.util.ArrayList;
import java.util.List;
/**
* A benchmark request contains one or more competitors, which are descriptions of how to
* perform an individual benchmark. Each competitor has its own settings such as concurrency,
* number of iterations to perform, and what type of search to perform.
*/
public class BenchmarkRequest extends MasterNodeOperationRequest<BenchmarkRequest> implements CompositeIndicesRequest {
private String benchmarkName;
private boolean verbose;
private int numExecutorNodes = 1; // How many nodes to run the benchmark on
private double[] percentiles = BenchmarkSettings.DEFAULT_PERCENTILES;
// Global settings which can be overwritten at the competitor level
private BenchmarkSettings settings = new BenchmarkSettings();
private List<BenchmarkCompetitor> competitors = new ArrayList<>();
/**
* Constructs a benchmark request
*/
public BenchmarkRequest() { }
/**
* Constructs a benchmark request
*/
public BenchmarkRequest(String... indices) {
settings().indices(indices);
}
/**
* Validate benchmark request
*
* @return Null if benchmark request is OK, exception otherwise
*/
@Override
public ActionRequestValidationException validate() {
ActionRequestValidationException validationException = null;
if (benchmarkName == null) {
validationException = ValidateActions.addValidationError("benchmarkName must not be null", validationException);
}
if (competitors.isEmpty()) {
validationException = ValidateActions.addValidationError("competitors must not be empty", validationException);
}
if (numExecutorNodes <= 0) {
validationException = ValidateActions.addValidationError("num_executors must not be less than 1", validationException);
}
for (BenchmarkCompetitor competitor : competitors) {
validationException = competitor.validate(validationException);
if (validationException != null) {
break;
}
}
return validationException;
}
@Override
public List<? extends IndicesRequest> subRequests() {
List<SearchRequest> searchRequests = Lists.newArrayList();
for (BenchmarkCompetitor competitor : competitors) {
for (SearchRequest searchRequest : competitor.settings().searchRequests()) {
searchRequests.add(searchRequest);
}
}
return searchRequests;
}
/**
* Cascade top-level benchmark settings to individual competitors while taking care
* not to overwrite any settings which the competitors specifically set.
*/
public void cascadeGlobalSettings() {
for (BenchmarkCompetitor competitor : competitors) {
competitor.settings().merge(settings);
if (competitor.settings().searchRequests().isEmpty()) {
for (SearchRequest defaultSearchRequest : settings.searchRequests()) {
SearchRequest copy = new SearchRequest();
if (defaultSearchRequest.indices() != null) {
copy.indices(defaultSearchRequest.indices());
}
copy.types(defaultSearchRequest.types());
copy.searchType(defaultSearchRequest.searchType());
copy.source(defaultSearchRequest.source(), true);
copy.extraSource(defaultSearchRequest.extraSource(), true);
copy.routing(defaultSearchRequest.routing());
copy.preference(defaultSearchRequest.preference());
competitor.settings().addSearchRequest(copy);
}
}
}
}
/**
* Apply late binding for certain settings. Indices and types passed will override previously
* set values. Cache clear requests cannot be constructed until we know the final set of
* indices so do this last.
*
* @param indices List of indices to execute on
* @param types List of types to execute on
*/
public void applyLateBoundSettings(String[] indices, String[] types) {
if (indices != null && indices.length > 0) {
settings.indices(indices);
}
if (types != null && types.length > 0) {
settings.types(types);
}
for (SearchRequest searchRequest : settings.searchRequests()) {
if (indices != null && indices.length > 0) {
searchRequest.indices(indices);
}
if (types != null && types.length > 0) {
searchRequest.types(types);
}
if (settings.clearCachesSettings() != null) {
settings.buildClearCachesRequestFromSettings();
}
}
for (BenchmarkCompetitor competitor : competitors) {
if (indices != null && indices.length > 0) {
competitor.settings().indices(indices);
competitor.settings().types(null);
}
if (types != null && types.length > 0) {
competitor.settings().types(types);
}
competitor.settings().buildSearchRequestsFromSettings();
if (competitor.settings().clearCachesSettings() != null) {
competitor.settings().buildClearCachesRequestFromSettings();
}
}
}
/**
* Gets the benchmark settings
* @return Settings
*/
public BenchmarkSettings settings() {
return settings;
}
/**
* Gets the number of nodes in the cluster to execute on.
* @return Number of nodes
*/
public int numExecutorNodes() {
return numExecutorNodes;
}
/**
* Sets the number of nodes in the cluster to execute the benchmark on. We will attempt to
* execute on this many eligible nodes, but will not fail if fewer nodes are available.
* @param numExecutorNodes Number of nodes
*/
public void numExecutorNodes(int numExecutorNodes) {
this.numExecutorNodes = numExecutorNodes;
}
/**
* Gets the name of the benchmark
* @return Benchmark name
*/
public String benchmarkName() {
return benchmarkName;
}
/**
* Sets the name of the benchmark
* @param benchmarkId Benchmark name
*/
public void benchmarkName(String benchmarkId) {
this.benchmarkName = benchmarkId;
}
/**
* Whether to report detailed statistics
* @return True if verbose on
*/
public boolean verbose() {
return verbose;
}
/**
* Whether to report detailed statistics
* @param verbose True/false
*/
public void verbose(boolean verbose) {
this.verbose = verbose;
}
/**
* Gets the list of percentiles to report
* @return The list of percentiles to report
*/
public double[] percentiles() {
return percentiles;
}
/**
* Sets the list of percentiles to report
* @param percentiles The list of percentiles to report
*/
public void percentiles(double[] percentiles) {
this.percentiles = percentiles;
}
/**
* Gets the list of benchmark competitions
* @return Competitions
*/
public List<BenchmarkCompetitor> competitors() {
return competitors;
}
/**
* Add a benchmark competition
* @param competitor Competition
*/
public void addCompetitor(BenchmarkCompetitor competitor) {
this.competitors.add(competitor);
}
@Override
public void readFrom(StreamInput in) throws IOException {
super.readFrom(in);
benchmarkName = in.readString();
numExecutorNodes = in.readVInt();
verbose = in.readBoolean();
percentiles = in.readDoubleArray();
int size = in.readVInt();
competitors.clear();
for (int i = 0; i < size; i++) {
BenchmarkCompetitor competitor = new BenchmarkCompetitor();
competitor.readFrom(in);
competitors.add(competitor);
}
}
@Override
public void writeTo(StreamOutput out) throws IOException {
super.writeTo(out);
out.writeString(benchmarkName);
out.writeVInt(numExecutorNodes);
out.writeBoolean(verbose);
if (percentiles != null) {
out.writeDoubleArray(percentiles);
} else {
out.writeVInt(0);
}
out.writeVInt(competitors.size());
for (BenchmarkCompetitor competitor : competitors) {
competitor.writeTo(out);
}
}
}

118
src/main/java/org/elasticsearch/action/bench/BenchmarkRequestBuilder.java
View File

@ -1,118 +0,0 @@
/*
* Licensed to Elasticsearch under one or more contributor
* license agreements. See the NOTICE file distributed with
* this work for additional information regarding copyright
* ownership. Elasticsearch licenses this file to you under
* the Apache License, Version 2.0 (the "License"); you may
* not use this file except in compliance with the License.
* You may obtain a copy of the License at
*
* http://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing,
* software distributed under the License is distributed on an
* "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
* KIND, either express or implied. See the License for the
* specific language governing permissions and limitations
* under the License.
*/
package org.elasticsearch.action.bench;
import org.elasticsearch.action.ActionListener;
import org.elasticsearch.action.ActionRequestBuilder;
import org.elasticsearch.action.search.SearchRequest;
import org.elasticsearch.action.search.SearchType;
import org.elasticsearch.client.Client;
/**
* Request builder for benchmarks
*/
public class BenchmarkRequestBuilder extends ActionRequestBuilder<BenchmarkRequest, BenchmarkResponse, BenchmarkRequestBuilder, Client> {
public BenchmarkRequestBuilder(Client client, String[] indices) {
super(client, new BenchmarkRequest(indices));
}
public BenchmarkRequestBuilder(Client client) {
super(client, new BenchmarkRequest());
}
public BenchmarkRequestBuilder setAllowCacheClearing(boolean allowCacheClearing) {
request.settings().allowCacheClearing(allowCacheClearing);
return this;
}
public BenchmarkRequestBuilder setClearCachesSettings(BenchmarkSettings.ClearCachesSettings clearCachesSettings) {
request.settings().clearCachesSettings(clearCachesSettings, false);
return this;
}
public BenchmarkRequestBuilder addSearchRequest(SearchRequest... searchRequest) {
request.settings().addSearchRequest(searchRequest);
return this;
}
public BenchmarkRequestBuilder addCompetitor(BenchmarkCompetitor competitor) {
request.addCompetitor(competitor);
return this;
}
public BenchmarkRequestBuilder addCompetitor(BenchmarkCompetitorBuilder competitorBuilder) {
return addCompetitor(competitorBuilder.build());
}
public BenchmarkRequestBuilder setNumExecutorNodes(int numExecutorNodes) {
request.numExecutorNodes(numExecutorNodes);
return this;
}
public BenchmarkRequestBuilder setIterations(int iterations) {
request.settings().iterations(iterations, false);
return this;
}
public BenchmarkRequestBuilder setConcurrency(int concurrency) {
request.settings().concurrency(concurrency, false);
return this;
}
public BenchmarkRequestBuilder setMultiplier(int multiplier) {
request.settings().multiplier(multiplier, false);
return this;
}
public BenchmarkRequestBuilder setNumSlowest(int numSlowest) {
request.settings().numSlowest(numSlowest, false);
return this;
}
public BenchmarkRequestBuilder setWarmup(boolean warmup) {
request.settings().warmup(warmup, false);
return this;
}
public BenchmarkRequestBuilder setBenchmarkId(String benchmarkId) {
request.benchmarkName(benchmarkId);
return this;
}
public BenchmarkRequestBuilder setSearchType(SearchType searchType) {
request.settings().searchType(searchType, false);
return this;
}
public BenchmarkRequestBuilder setVerbose(boolean verbose) {
request.verbose(verbose);
return this;
}
public BenchmarkRequestBuilder setPercentiles(double[] percentiles) {
request.percentiles(percentiles);
return this;
}
@Override
protected void doExecute(ActionListener<BenchmarkResponse> listener) {
client.bench(request, listener);
}
}

253
src/main/java/org/elasticsearch/action/bench/BenchmarkResponse.java
View File

@ -1,253 +0,0 @@
/*
* Licensed to Elasticsearch under one or more contributor
* license agreements. See the NOTICE file distributed with
* this work for additional information regarding copyright
* ownership. Elasticsearch licenses this file to you under
* the Apache License, Version 2.0 (the "License"); you may
* not use this file except in compliance with the License.
* You may obtain a copy of the License at
*
* http://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing,
* software distributed under the License is distributed on an
* "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
* KIND, either express or implied. See the License for the
* specific language governing permissions and limitations
* under the License.
*/
package org.elasticsearch.action.bench;
import org.elasticsearch.ElasticsearchIllegalArgumentException;
import org.elasticsearch.action.ActionResponse;
import org.elasticsearch.common.Strings;
import org.elasticsearch.common.io.stream.StreamInput;
import org.elasticsearch.common.io.stream.StreamOutput;
import org.elasticsearch.common.io.stream.Streamable;
import org.elasticsearch.common.xcontent.ToXContent;
import org.elasticsearch.common.xcontent.XContentBuilder;
import org.elasticsearch.common.xcontent.XContentBuilderString;
import org.elasticsearch.common.xcontent.XContentFactory;
import java.io.IOException;
import java.util.*;
/**
* Benchmark response.
*
* A benchmark response will contain a mapping of names to results for each competition.
*/
public class BenchmarkResponse extends ActionResponse implements Streamable, ToXContent {
private String benchmarkName;
private State state = State.RUNNING;
private boolean verbose;
private String[] errors = Strings.EMPTY_ARRAY;
Map<String, CompetitionResult> competitionResults;
public BenchmarkResponse() {
competitionResults = new HashMap<>();
}
public BenchmarkResponse(String benchmarkName, Map<String, CompetitionResult> competitionResults) {
this.benchmarkName = benchmarkName;
this.competitionResults = competitionResults;
}
/**
* Benchmarks can be in one of:
* RUNNING - executing normally
* COMPLETE - completed normally
* ABORTED - aborted
* FAILED - execution failed
*/
public static enum State {
RUNNING((byte) 0),
COMPLETE((byte) 1),
ABORTED((byte) 2),
FAILED((byte) 3);
private final byte id;
private static final State[] STATES = new State[State.values().length];
static {
for (State state : State.values()) {
assert state.id() < STATES.length && state.id() >= 0;
STATES[state.id] = state;
}
}
State(byte id) {
this.id = id;
}
public byte id() {
return id;
}
public static State fromId(byte id) throws ElasticsearchIllegalArgumentException {
if (id < 0 || id >= STATES.length) {
throw new ElasticsearchIllegalArgumentException("No mapping for id [" + id + "]");
}
return STATES[id];
}
}
/**
* Name of the benchmark
* @return Name of the benchmark
*/
public String benchmarkName() {
return benchmarkName;
}
/**
* Sets the benchmark name
* @param benchmarkName Benchmark name
*/
public void benchmarkName(String benchmarkName) {
this.benchmarkName = benchmarkName;
}
/**
* Benchmark state
* @return Benchmark state
*/
public State state() {
return state;
}
/**
* Sets the state of the benchmark
* @param state State
*/
public void state(State state) {
this.state = state;
}
/**
* Possibly replace the existing state with the new state depending on the severity
* of the new state. More severe states, such as FAILED, will over-write less severe
* ones, such as COMPLETED.
* @param newState New candidate state
* @return The merged state
*/
public State mergeState(State newState) {
if (state.compareTo(newState) < 0) {
state = newState;
}
return state;
}
/**
* Map of competition names to competition results
* @return Map of competition names to competition results
*/
public Map<String, CompetitionResult> competitionResults() {
return competitionResults;
}
/**
* Whether to report verbose statistics
*/
public boolean verbose() {
return verbose;
}
/**
* Sets whether to report verbose statistics
*/
public void verbose(boolean verbose) {
this.verbose = verbose;
}
/**
* Whether the benchmark encountered error conditions
* @return Whether the benchmark encountered error conditions
*/
public boolean hasErrors() {
return (errors != null && errors.length > 0);
}
/**
* Error messages
* @return Error messages
*/
public String[] errors() {
return this.errors;
}
/**
* Sets error messages
* @param errors Error messages
*/
public void errors(String... errors) {
this.errors = (errors == null) ? Strings.EMPTY_ARRAY : errors;
}
@Override
public XContentBuilder toXContent(XContentBuilder builder, Params params) throws IOException {
builder.field(Fields.STATUS, state.toString());
if (errors != null) {
builder.array(Fields.ERRORS, errors);
}
builder.startObject(Fields.COMPETITORS);
if (competitionResults != null) {
for (Map.Entry<String, CompetitionResult> entry : competitionResults.entrySet()) {
entry.getValue().verbose(verbose);
entry.getValue().toXContent(builder, params);
}
}
builder.endObject();
return builder;
}
@Override
public void readFrom(StreamInput in) throws IOException {
super.readFrom(in);
benchmarkName = in.readString();
state = State.fromId(in.readByte());
errors = in.readStringArray();
int size = in.readVInt();
for (int i = 0; i < size; i++) {
String s = in.readString();
CompetitionResult cr = new CompetitionResult();
cr.readFrom(in);
competitionResults.put(s, cr);
}
}
@Override
public void writeTo(StreamOutput out) throws IOException {
super.writeTo(out);
out.writeString(benchmarkName);
out.writeByte(state.id());
out.writeStringArray(errors);
out.write(competitionResults.size());
for (Map.Entry<String, CompetitionResult> entry : competitionResults.entrySet()) {
out.writeString(entry.getKey());
entry.getValue().writeTo(out);
}
}
@Override
public String toString() {
try {
XContentBuilder builder = XContentFactory.jsonBuilder().prettyPrint();
builder.startObject();
toXContent(builder, EMPTY_PARAMS);
builder.endObject();
return builder.string();
} catch (IOException e) {
return "{ \"error\" : \"" + e.getMessage() + "\"}";
}
}
static final class Fields {
static final XContentBuilderString STATUS = new XContentBuilderString("status");
static final XContentBuilderString ERRORS = new XContentBuilderString("errors");
static final XContentBuilderString COMPETITORS = new XContentBuilderString("competitors");
}
}

774
src/main/java/org/elasticsearch/action/bench/BenchmarkService.java
View File

@ -1,774 +0,0 @@
/*
* Licensed to Elasticsearch under one or more contributor
* license agreements. See the NOTICE file distributed with
* this work for additional information regarding copyright
* ownership. Elasticsearch licenses this file to you under
* the Apache License, Version 2.0 (the "License"); you may
* not use this file except in compliance with the License.
* You may obtain a copy of the License at
*
* http://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing,
* software distributed under the License is distributed on an
* "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
* KIND, either express or implied. See the License for the
* specific language governing permissions and limitations
* under the License.
*/
package org.elasticsearch.action.bench;
import com.google.common.collect.ImmutableList;
import com.google.common.collect.ImmutableMap;
import org.elasticsearch.ElasticsearchException;
import org.elasticsearch.ElasticsearchIllegalStateException;
import org.elasticsearch.action.ActionListener;
import org.elasticsearch.action.support.master.MasterNodeOperationRequest;
import org.elasticsearch.client.Client;
import org.elasticsearch.cluster.ClusterService;
import org.elasticsearch.cluster.ClusterState;
import org.elasticsearch.cluster.ProcessedClusterStateUpdateTask;
import org.elasticsearch.cluster.TimeoutClusterStateUpdateTask;
import org.elasticsearch.cluster.metadata.BenchmarkMetaData;
import org.elasticsearch.cluster.metadata.MetaData;
import org.elasticsearch.cluster.node.DiscoveryNode;
import org.elasticsearch.cluster.node.DiscoveryNodes;
import org.elasticsearch.common.collect.ImmutableOpenMap;
import org.elasticsearch.common.component.AbstractLifecycleComponent;
import org.elasticsearch.common.inject.Inject;
import org.elasticsearch.common.io.stream.StreamInput;
import org.elasticsearch.common.io.stream.StreamOutput;
import org.elasticsearch.common.regex.Regex;
import org.elasticsearch.common.settings.Settings;
import org.elasticsearch.common.unit.TimeValue;
import org.elasticsearch.common.util.concurrent.CountDown;
import org.elasticsearch.threadpool.ThreadPool;
import org.elasticsearch.transport.*;
import java.io.IOException;
import java.util.*;
import java.util.concurrent.CopyOnWriteArrayList;
/**
* Service component for running benchmarks
*/
public class BenchmarkService extends AbstractLifecycleComponent<BenchmarkService> {
private final ThreadPool threadPool;
private final ClusterService clusterService;
private final TransportService transportService;
protected final BenchmarkExecutor executor;
public static final String ABORT_ACTION_NAME = "indices:data/benchmark/executor/abort";
public static final String STATUS_ACTION_NAME = "indices:data/benchmark/executor/status";
public static final String START_ACTION_NAME = "indices:data/benchmark/executor/start";
/**
* Constructs a service component for running benchmarks
*
* @param settings Settings
* @param clusterService Cluster service
* @param threadPool Thread pool
* @param client Client
* @param transportService Transport service
*/
@Inject
public BenchmarkService(Settings settings, ClusterService clusterService, ThreadPool threadPool,
Client client, TransportService transportService) {
super(settings);
this.threadPool = threadPool;
this.executor = new BenchmarkExecutor(client, clusterService);
this.clusterService = clusterService;
this.transportService = transportService;
transportService.registerHandler(START_ACTION_NAME, new BenchExecutionHandler());
transportService.registerHandler(ABORT_ACTION_NAME, new AbortExecutionHandler());
transportService.registerHandler(STATUS_ACTION_NAME, new StatusExecutionHandler());
}
@Override
protected void doStart() throws ElasticsearchException {
}
@Override
protected void doStop() throws ElasticsearchException {
}
@Override
protected void doClose() throws ElasticsearchException {
}
/**
* Lists actively running benchmarks on the cluster
*
* @param request Status request
* @param listener Response listener
*/
public void listBenchmarks(final BenchmarkStatusRequest request, final ActionListener<BenchmarkStatusResponse> listener) {
final List<DiscoveryNode> nodes = availableBenchmarkNodes();
if (nodes.size() == 0) {
listener.onResponse(new BenchmarkStatusResponse());
} else {
BenchmarkStatusAsyncHandler async = new BenchmarkStatusAsyncHandler(nodes.size(), request, listener);
for (DiscoveryNode node : nodes) {
assert isBenchmarkNode(node);
transportService.sendRequest(node, STATUS_ACTION_NAME, new NodeStatusRequest(request), async);
}
}
}
/**
* Aborts actively running benchmarks on the cluster
*
* @param benchmarkNames Benchmark name(s) to abort
* @param listener Response listener
*/
public void abortBenchmark(final String[] benchmarkNames, final ActionListener<AbortBenchmarkResponse> listener) {
final List<DiscoveryNode> nodes = availableBenchmarkNodes();
if (nodes.size() == 0) {
listener.onFailure(new BenchmarkNodeMissingException("No available nodes for executing benchmarks"));
} else {
BenchmarkStateListener benchmarkStateListener = new BenchmarkStateListener() {
@Override
public void onResponse(final ClusterState newState, final List<BenchmarkMetaData.Entry> changed) {
if (!changed.isEmpty()) {
threadPool.executor(ThreadPool.Names.GENERIC).execute(new Runnable() {
@Override
public void run() {
Set<String> names = new HashSet<>();
Set<String> nodeNames = new HashSet<>();
final ImmutableOpenMap<String, DiscoveryNode> nodes = newState.nodes().nodes();
for (BenchmarkMetaData.Entry e : changed) {
names.add(e.benchmarkId());
nodeNames.addAll(Arrays.asList(e.nodes()));
}
BenchmarkAbortAsyncHandler asyncHandler = new BenchmarkAbortAsyncHandler(nodeNames.size(), listener);
String[] benchmarkNames = names.toArray(new String[names.size()]);
for (String nodeId : nodeNames) {
final DiscoveryNode node = nodes.get(nodeId);
if (node != null) {
transportService.sendRequest(node, ABORT_ACTION_NAME, new NodeAbortRequest(benchmarkNames), asyncHandler);
} else {
asyncHandler.countDown.countDown();
logger.debug("Node for ID [" + nodeId + "] not found in cluster state - skipping");
}
}
}
});
} else {
listener.onFailure(new BenchmarkMissingException("No benchmarks found for " + Arrays.toString(benchmarkNames)));
}
}
@Override
public void onFailure(Throwable t) {
listener.onFailure(t);
}
};
clusterService.submitStateUpdateTask("abort_benchmark", new AbortBenchmarkTask(benchmarkNames, benchmarkStateListener));
}
}
/**
* Executes benchmarks on the cluster
*
* @param request Benchmark request
* @param listener Response listener
*/
public void startBenchmark(final BenchmarkRequest request, final ActionListener<BenchmarkResponse> listener) {
final List<DiscoveryNode> nodes = availableBenchmarkNodes();
if (nodes.size() == 0) {
listener.onFailure(new BenchmarkNodeMissingException("No available nodes for executing benchmark [" +
request.benchmarkName() + "]"));
} else {
final BenchmarkStateListener benchListener = new BenchmarkStateListener() {
@Override
public void onResponse(final ClusterState newState, final List<BenchmarkMetaData.Entry> entries) {
threadPool.executor(ThreadPool.Names.GENERIC).execute(new Runnable() {
@Override
public void run() {
assert entries.size() == 1;
BenchmarkMetaData.Entry entry = entries.get(0);
final ImmutableOpenMap<String, DiscoveryNode> nodes = newState.nodes().nodes();
final BenchmarkSearchAsyncHandler async = new BenchmarkSearchAsyncHandler(entry.nodes().length, request, listener);
for (String nodeId : entry.nodes()) {
final DiscoveryNode node = nodes.get(nodeId);
if (node == null) {
async.handleExceptionInternal(
new ElasticsearchIllegalStateException("Node for ID [" + nodeId + "] not found in cluster state - skipping"));
} else {
logger.debug("Starting benchmark [{}] node [{}]", request.benchmarkName(), node.name());
transportService.sendRequest(node, START_ACTION_NAME, new NodeBenchRequest(request), async);
}
}
}
});
}
@Override
public void onFailure(Throwable t) {
listener.onFailure(t);
}
};
clusterService.submitStateUpdateTask("start_benchmark", new StartBenchmarkTask(request, benchListener));
}
}
private void finishBenchmark(final BenchmarkResponse benchmarkResponse, final String benchmarkId, final ActionListener<BenchmarkResponse> listener) {
clusterService.submitStateUpdateTask("finish_benchmark", new FinishBenchmarkTask("finish_benchmark", benchmarkId, new BenchmarkStateListener() {
@Override
public void onResponse(ClusterState newClusterState, List<BenchmarkMetaData.Entry> changed) {
listener.onResponse(benchmarkResponse);
}
@Override
public void onFailure(Throwable t) {
listener.onFailure(t);
}
}, (benchmarkResponse.state() != BenchmarkResponse.State.ABORTED) &&
(benchmarkResponse.state() != BenchmarkResponse.State.FAILED)));
}
private final boolean isBenchmarkNode(DiscoveryNode node) {
ImmutableMap<String, String> attributes = node.getAttributes();
if (attributes.containsKey("bench")) {
String bench = attributes.get("bench");
return Boolean.parseBoolean(bench);
}
return false;
}
private List<DiscoveryNode> findNodes(BenchmarkRequest request) {
final int numNodes = request.numExecutorNodes();
final DiscoveryNodes nodes = clusterService.state().nodes();
DiscoveryNode localNode = nodes.localNode();
List<DiscoveryNode> benchmarkNodes = new ArrayList<DiscoveryNode>();
if (isBenchmarkNode(localNode)) {
benchmarkNodes.add(localNode);
}
for (DiscoveryNode node : nodes) {
if (benchmarkNodes.size() >= numNodes) {
return benchmarkNodes;
}
if (node != localNode && isBenchmarkNode(node)) {
benchmarkNodes.add(node);
}
}
return benchmarkNodes;
}
private class BenchExecutionHandler extends BaseTransportRequestHandler<NodeBenchRequest> {
@Override
public NodeBenchRequest newInstance() {
return new NodeBenchRequest();
}
@Override
public void messageReceived(NodeBenchRequest request, TransportChannel channel) throws Exception {
BenchmarkResponse response = executor.benchmark(request.request);
channel.sendResponse(response);
}
@Override
public String executor() {
return ThreadPool.Names.BENCH;
}
}
private class StatusExecutionHandler extends BaseTransportRequestHandler<NodeStatusRequest> {
@Override
public NodeStatusRequest newInstance() {
return new NodeStatusRequest();
}
@Override
public void messageReceived(NodeStatusRequest request, TransportChannel channel) throws Exception {
BenchmarkStatusNodeResponse nodeResponse = executor.benchmarkStatus();
nodeResponse.nodeName(clusterService.localNode().name());
channel.sendResponse(nodeResponse);
}
@Override
public String executor() {
// Perform management tasks on GENERIC so as not to block pending acquisition of a thread from BENCH.
return ThreadPool.Names.GENERIC;
}
}
private class AbortExecutionHandler extends BaseTransportRequestHandler<NodeAbortRequest> {
@Override
public NodeAbortRequest newInstance() {
return new NodeAbortRequest();
}
@Override
public void messageReceived(NodeAbortRequest request, TransportChannel channel) throws Exception {
AbortBenchmarkResponse nodeResponse = executor.abortBenchmark(request.benchmarkNames);
channel.sendResponse(nodeResponse);
}
@Override
public String executor() {
// Perform management tasks on GENERIC so as not to block pending acquisition of a thread from BENCH.
return ThreadPool.Names.GENERIC;
}
}
public static class NodeAbortRequest extends TransportRequest {
private String[] benchmarkNames;
public NodeAbortRequest(String[] benchmarkNames) {
this.benchmarkNames = benchmarkNames;
}
public NodeAbortRequest() {
this(null);
}
@Override
public void readFrom(StreamInput in) throws IOException {
super.readFrom(in);
benchmarkNames = in.readStringArray();
}
@Override
public void writeTo(StreamOutput out) throws IOException {
super.writeTo(out);
out.writeStringArray(benchmarkNames);
}
}
public static class NodeStatusRequest extends TransportRequest {
final BenchmarkStatusRequest request;
public NodeStatusRequest(BenchmarkStatusRequest request) {
this.request = request;
}
public NodeStatusRequest() {
this(new BenchmarkStatusRequest());
}
@Override
public void readFrom(StreamInput in) throws IOException {
super.readFrom(in);
request.readFrom(in);
}
@Override
public void writeTo(StreamOutput out) throws IOException {
super.writeTo(out);
request.writeTo(out);
}
}
public static class NodeBenchRequest extends TransportRequest {
final BenchmarkRequest request;
public NodeBenchRequest(BenchmarkRequest request) {
this.request = request;
}
public NodeBenchRequest() {
this(new BenchmarkRequest());
}
@Override
public void readFrom(StreamInput in) throws IOException {
super.readFrom(in);
request.readFrom(in);
}
@Override
public void writeTo(StreamOutput out) throws IOException {
super.writeTo(out);
request.writeTo(out);
}
}
private abstract class CountDownAsyncHandler<T extends TransportResponse> implements TransportResponseHandler<T> {
protected final CountDown countDown;
protected final CopyOnWriteArrayList<T> responses = new CopyOnWriteArrayList<T>();
protected final CopyOnWriteArrayList<Throwable> failures = new CopyOnWriteArrayList<Throwable>();
protected CountDownAsyncHandler(int size) {
countDown = new CountDown(size);
}
@Override
public abstract T newInstance();
protected abstract void sendResponse();
@Override
public void handleResponse(T t) {
responses.add(t);
if (countDown.countDown()) {
sendResponse();
}
}
@Override
public void handleException(TransportException t) {
failures.add(t);
logger.error(t.getMessage(), t);
if (countDown.countDown()) {
sendResponse();
}
}
@Override
public String executor() {
return ThreadPool.Names.SAME;
}
}
private class BenchmarkAbortAsyncHandler extends CountDownAsyncHandler<AbortBenchmarkResponse> {
private final ActionListener<AbortBenchmarkResponse> listener;
public BenchmarkAbortAsyncHandler(int size, ActionListener<AbortBenchmarkResponse> listener) {
super(size);
this.listener = listener;
}
@Override
public AbortBenchmarkResponse newInstance() {
return new AbortBenchmarkResponse();
}
@Override
protected void sendResponse() {
boolean acked = true;
for (AbortBenchmarkResponse nodeResponse : responses) {
if (!nodeResponse.isAcknowledged()) {
acked = false;
break;
}
}
listener.onResponse(new AbortBenchmarkResponse(acked));
}
}
private class BenchmarkStatusAsyncHandler extends CountDownAsyncHandler<BenchmarkStatusNodeResponse> {
private final BenchmarkStatusRequest request;
private final ActionListener<BenchmarkStatusResponse> listener;
public BenchmarkStatusAsyncHandler(int nodeCount, final BenchmarkStatusRequest request, ActionListener<BenchmarkStatusResponse> listener) {
super(nodeCount);
this.request = request;
this.listener = listener;
}
@Override
public BenchmarkStatusNodeResponse newInstance() {
return new BenchmarkStatusNodeResponse();
}
@Override
protected void sendResponse() {
int activeBenchmarks = 0;
BenchmarkStatusResponse consolidatedResponse = new BenchmarkStatusResponse();
Map<String, List<BenchmarkResponse>> nameNodeResponseMap = new HashMap<>();
// Group node responses by benchmark name
for (BenchmarkStatusNodeResponse nodeResponse : responses) {
for (BenchmarkResponse benchmarkResponse : nodeResponse.benchResponses()) {
List<BenchmarkResponse> benchmarkResponses = nameNodeResponseMap.get(benchmarkResponse.benchmarkName());
if (benchmarkResponses == null) {
benchmarkResponses = new ArrayList<>();
nameNodeResponseMap.put(benchmarkResponse.benchmarkName(), benchmarkResponses);
}
benchmarkResponses.add(benchmarkResponse);
}
activeBenchmarks += nodeResponse.activeBenchmarks();
}
for (Map.Entry<String, List<BenchmarkResponse>> entry : nameNodeResponseMap.entrySet()) {
BenchmarkResponse consolidated = consolidateBenchmarkResponses(entry.getValue());
consolidatedResponse.addBenchResponse(consolidated);
}
consolidatedResponse.totalActiveBenchmarks(activeBenchmarks);
listener.onResponse(consolidatedResponse);
}
}
private BenchmarkResponse consolidateBenchmarkResponses(List<BenchmarkResponse> responses) {
BenchmarkResponse response = new BenchmarkResponse();
// Merge node responses into a single consolidated response
List<String> errors = new ArrayList<>();
for (BenchmarkResponse r : responses) {
for (Map.Entry<String, CompetitionResult> entry : r.competitionResults.entrySet()) {
if (!response.competitionResults.containsKey(entry.getKey())) {
response.competitionResults.put(entry.getKey(),
new CompetitionResult(
entry.getKey(), entry.getValue().concurrency(), entry.getValue().multiplier(),
false, entry.getValue().percentiles()));
}
CompetitionResult cr = response.competitionResults.get(entry.getKey());
cr.nodeResults().addAll(entry.getValue().nodeResults());
}
if (r.hasErrors()) {
for (String error : r.errors()) {
errors.add(error);
}
}
if (response.benchmarkName() == null) {
response.benchmarkName(r.benchmarkName());
}
assert response.benchmarkName().equals(r.benchmarkName());
if (!errors.isEmpty()) {
response.errors(errors.toArray(new String[errors.size()]));
}
response.mergeState(r.state());
assert errors.isEmpty() || response.state() != BenchmarkResponse.State.COMPLETE : "Response can't be complete since it has errors";
}
return response;
}
private class BenchmarkSearchAsyncHandler extends CountDownAsyncHandler<BenchmarkResponse> {
private final ActionListener<BenchmarkResponse> listener;
private final BenchmarkRequest request;
public BenchmarkSearchAsyncHandler(int size, BenchmarkRequest request, ActionListener<BenchmarkResponse> listener) {
super(size);
this.listener = listener;
this.request = request;
}
@Override
public BenchmarkResponse newInstance() {
return new BenchmarkResponse();
}
@Override
protected void sendResponse() {
BenchmarkResponse response = consolidateBenchmarkResponses(responses);
response.benchmarkName(request.benchmarkName());
response.verbose(request.verbose());
finishBenchmark(response, request.benchmarkName(), listener);
}
public void handleExceptionInternal(Throwable t) {
failures.add(t);
if (countDown.countDown()) {
sendResponse();
}
}
}
public static interface BenchmarkStateListener {
void onResponse(ClusterState newClusterState, List<BenchmarkMetaData.Entry> changed);
void onFailure(Throwable t);
}
public final class StartBenchmarkTask extends BenchmarkStateChangeAction<BenchmarkRequest> {
private final BenchmarkStateListener stateListener;
private List<BenchmarkMetaData.Entry> newBenchmark = new ArrayList<>();
public StartBenchmarkTask(BenchmarkRequest request, BenchmarkStateListener stateListener) {
super(request);
this.stateListener = stateListener;
}
@Override
public ClusterState execute(ClusterState currentState) {
MetaData metaData = currentState.getMetaData();
BenchmarkMetaData bmd = metaData.custom(BenchmarkMetaData.TYPE);
MetaData.Builder mdBuilder = MetaData.builder(metaData);
ImmutableList.Builder<BenchmarkMetaData.Entry> builder = ImmutableList.builder();
if (bmd != null) {
for (BenchmarkMetaData.Entry entry : bmd.entries()) {
if (request.benchmarkName().equals(entry.benchmarkId())) {
if (entry.state() != BenchmarkMetaData.State.SUCCESS && entry.state() != BenchmarkMetaData.State.FAILED) {
throw new ElasticsearchException("A benchmark with ID [" + request.benchmarkName() + "] is already running in state [" + entry.state() + "]");
}
// just drop the entry it it has finished successfully or it failed!
} else {
builder.add(entry);
}
}
}
List<DiscoveryNode> nodes = findNodes(request);
String[] nodeIds = new String[nodes.size()];
int i = 0;
for (DiscoveryNode node : nodes) {
nodeIds[i++] = node.getId();
}
BenchmarkMetaData.Entry entry = new BenchmarkMetaData.Entry(request.benchmarkName(), BenchmarkMetaData.State.STARTED, nodeIds);
newBenchmark.add(entry);
bmd = new BenchmarkMetaData(builder.add(entry).build());
mdBuilder.putCustom(BenchmarkMetaData.TYPE, bmd);
return ClusterState.builder(currentState).metaData(mdBuilder).build();
}
@Override
public void onFailure(String source, Throwable t) {
logger.warn("Failed to start benchmark: [{}]", t, request.benchmarkName());
newBenchmark = null;
stateListener.onFailure(t);
}
@Override
public void clusterStateProcessed(String source, ClusterState oldState, final ClusterState newState) {
if (newBenchmark != null) {
stateListener.onResponse(newState, newBenchmark);
}
}
@Override
public TimeValue timeout() {
return request.masterNodeTimeout();
}
}
public final class FinishBenchmarkTask extends UpdateBenchmarkStateTask {
private final boolean success;
public FinishBenchmarkTask(String reason, String benchmarkId, BenchmarkStateListener listener, boolean success) {
super(reason, benchmarkId, listener);
this.success = success;
}
@Override
protected BenchmarkMetaData.Entry process(BenchmarkMetaData.Entry entry) {
BenchmarkMetaData.State state = entry.state();
assert state == BenchmarkMetaData.State.STARTED || state == BenchmarkMetaData.State.ABORTED : "Expected state: STARTED or ABORTED but was: " + entry.state();
if (success) {
return new BenchmarkMetaData.Entry(entry, BenchmarkMetaData.State.SUCCESS);
} else {
return new BenchmarkMetaData.Entry(entry, BenchmarkMetaData.State.FAILED);
}
}
}
public final class AbortBenchmarkTask extends UpdateBenchmarkStateTask {
private final String[] patterns;
public AbortBenchmarkTask(String[] patterns, BenchmarkStateListener listener) {
super("abort_benchmark", null, listener);
this.patterns = patterns;
}
@Override
protected boolean match(BenchmarkMetaData.Entry entry) {
return entry.state() == BenchmarkMetaData.State.STARTED && Regex.simpleMatch(this.patterns, benchmarkId);
}
@Override
protected BenchmarkMetaData.Entry process(BenchmarkMetaData.Entry entry) {
return new BenchmarkMetaData.Entry(entry, BenchmarkMetaData.State.ABORTED);
}
}
public abstract class UpdateBenchmarkStateTask extends ProcessedClusterStateUpdateTask {
private final String reason;
protected final String benchmarkId;
private final BenchmarkStateListener listener;
private final List<BenchmarkMetaData.Entry> instances = new ArrayList<>();
protected UpdateBenchmarkStateTask(String reason, String benchmarkId, BenchmarkStateListener listener) {
this.reason = reason;
this.listener = listener;
this.benchmarkId = benchmarkId;
}
protected boolean match(BenchmarkMetaData.Entry entry) {
return entry.benchmarkId().equals(this.benchmarkId);
}
@Override
public ClusterState execute(ClusterState currentState) {
MetaData metaData = currentState.getMetaData();
BenchmarkMetaData bmd = metaData.custom(BenchmarkMetaData.TYPE);
MetaData.Builder mdBuilder = MetaData.builder(metaData);
if (bmd != null && !bmd.entries().isEmpty()) {
ImmutableList.Builder<BenchmarkMetaData.Entry> builder = new ImmutableList.Builder<BenchmarkMetaData.Entry>();
for (BenchmarkMetaData.Entry e : bmd.entries()) {
if (benchmarkId == null || match(e)) {
e = process(e);
instances.add(e);
}
// Don't keep finished benchmarks around in cluster state
if (e != null && (e.state() != BenchmarkMetaData.State.SUCCESS &&
e.state() != BenchmarkMetaData.State.ABORTED &&
e.state() != BenchmarkMetaData.State.FAILED)) {
builder.add(e);
}
}
if (instances.isEmpty()) {
throw new ElasticsearchException("No Benchmark found for id: [" + benchmarkId + "]");
}
bmd = new BenchmarkMetaData(builder.build());
}
if (bmd != null) {
mdBuilder.putCustom(BenchmarkMetaData.TYPE, bmd);
}
return ClusterState.builder(currentState).metaData(mdBuilder).build();
}
protected abstract BenchmarkMetaData.Entry process(BenchmarkMetaData.Entry entry);
@Override
public void onFailure(String source, Throwable t) {
logger.warn("Failed updating benchmark state for ID [{}] triggered by: [{}]", t, benchmarkId, reason);
listener.onFailure(t);
}
@Override
public void clusterStateProcessed(String source, ClusterState oldState, final ClusterState newState) {
listener.onResponse(newState, instances);
}
public String reason() {
return reason;
}
}
public abstract class BenchmarkStateChangeAction<R extends MasterNodeOperationRequest> extends TimeoutClusterStateUpdateTask {
protected final R request;
public BenchmarkStateChangeAction(R request) {
this.request = request;
}
@Override
public TimeValue timeout() {
return request.masterNodeTimeout();
}
}
private List<DiscoveryNode> availableBenchmarkNodes() {
DiscoveryNodes nodes = clusterService.state().nodes();
List<DiscoveryNode> benchmarkNodes = new ArrayList<>(nodes.size());
for (DiscoveryNode node : nodes) {
if (isBenchmarkNode(node)) {
benchmarkNodes.add(node);
}
}
return benchmarkNodes;
}
}

394
src/main/java/org/elasticsearch/action/bench/BenchmarkSettings.java
View File

@ -1,394 +0,0 @@
/*
* Licensed to Elasticsearch under one or more contributor
* license agreements. See the NOTICE file distributed with
* this work for additional information regarding copyright
* ownership. Elasticsearch licenses this file to you under
* the Apache License, Version 2.0 (the "License"); you may
* not use this file except in compliance with the License.
* You may obtain a copy of the License at
*
* http://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing,
* software distributed under the License is distributed on an
* "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
* KIND, either express or implied. See the License for the
* specific language governing permissions and limitations
* under the License.
*/
package org.elasticsearch.action.bench;
import org.elasticsearch.action.admin.indices.cache.clear.ClearIndicesCacheRequest;
import org.elasticsearch.action.search.SearchRequest;
import org.elasticsearch.action.search.SearchType;
import org.elasticsearch.common.Strings;
import org.elasticsearch.common.io.stream.StreamInput;
import org.elasticsearch.common.io.stream.StreamOutput;
import org.elasticsearch.common.io.stream.Streamable;
import java.io.IOException;
import java.util.ArrayList;
import java.util.Arrays;
import java.util.List;
/**
* Settings that define how a benchmark should be executed.
*/
public class BenchmarkSettings implements Streamable {
public static final int DEFAULT_CONCURRENCY = 5;
public static final int DEFAULT_ITERATIONS = 5;
public static final int DEFAULT_MULTIPLIER = 1000;
public static final int DEFAULT_NUM_SLOWEST = 1;
public static final boolean DEFAULT_WARMUP = true;
public static final SearchType DEFAULT_SEARCH_TYPE = SearchType.DEFAULT;
public static final double[] DEFAULT_PERCENTILES = { 10, 25, 50, 75, 90, 99 };
private int concurrency = DEFAULT_CONCURRENCY;
private int iterations = DEFAULT_ITERATIONS;
private int multiplier = DEFAULT_MULTIPLIER;
private int numSlowest = DEFAULT_NUM_SLOWEST;
private boolean warmup = DEFAULT_WARMUP;
private SearchType searchType = DEFAULT_SEARCH_TYPE;
private ClearIndicesCacheRequest clearCaches = null;
private ClearCachesSettings clearCachesSettings = null;
private String[] indices = Strings.EMPTY_ARRAY;
private String[] types = Strings.EMPTY_ARRAY;
private List<SearchRequest> searchRequests = new ArrayList<>();
private boolean concurrencyFinalized = false;
private boolean iterationsFinalized = false;
private boolean multiplierFinalized = false;
private boolean numSlowestFinalized = false;
private boolean warmupFinalized = false;
private boolean searchTypeFinalized = false;
private boolean clearCachesSettingsFinalized = false;
private boolean allowCacheClearing = true;
/**
* List of indices to execute on
* @return Indices
*/
public String[] indices() {
return indices;
}
/**
* Sets the indices to execute on
* @param indices Indices
*/
public void indices(String[] indices) {
this.indices = (indices == null) ? Strings.EMPTY_ARRAY : indices;
}
/**
* List of types to execute on
* @return Types
*/
public String[] types() {
return types;
}
/**
* Sets the types to execute on
* @param types Types
*/
public void types(String[] types) {
this.types = (types == null) ? Strings.EMPTY_ARRAY : types;
}
/**
* Gets the concurrency level
* @return Concurrency
*/
public int concurrency() {
return concurrency;
}
/**
* Sets the concurrency level; determines how many threads will be executing the competition concurrently.
* @param concurrency Concurrency
* @param finalize If true, value cannot be overwritten by subsequent calls
*/
public void concurrency(int concurrency, boolean finalize) {
if (concurrencyFinalized) {
return;
}
this.concurrency = concurrency;
concurrencyFinalized = finalize;
}
/**
* How many times to the competition.
* @return Iterations
*/
public int iterations() {
return iterations;
}
/**
* How many times to run the competition. Requests will be executed a total of (iterations * multiplier).
* @param iterations Iterations
* @param finalize If true, value cannot be overwritten by subsequent calls
*/
public void iterations(int iterations, boolean finalize) {
if (iterationsFinalized) {
return;
}
this.iterations = iterations;
iterationsFinalized = finalize;
}
/**
* Gets the multiplier
* @return Multiplier
*/
public int multiplier() {
return multiplier;
}
/**
* Sets the multiplier. The multiplier determines how many times each iteration will be run.
* @param multiplier Multiplier
* @param finalize If true, value cannot be overwritten by subsequent calls
*/
public void multiplier(int multiplier, boolean finalize) {
if (multiplierFinalized) {
return;
}
this.multiplier = multiplier;
multiplierFinalized = finalize;
}
/**
* Gets number of slow requests to track
* @return Number of slow requests to track
*/
public int numSlowest() {
return numSlowest;
}
/**
* How many slow requests to track
* @param numSlowest Number of slow requests to track
* @param finalize If true, value cannot be overwritten by subsequent calls
*/
public void numSlowest(int numSlowest, boolean finalize) {
if (numSlowestFinalized) {
return;
}
this.numSlowest = numSlowest;
numSlowestFinalized = finalize;
}
/**
* Whether to run a warmup search request
* @return True/false
*/
public boolean warmup() {
return warmup;
}
/**
* Whether to run a warmup search request
* @param warmup True/false
* @param finalize If true, value cannot be overwritten by subsequent calls
*/
public void warmup(boolean warmup, boolean finalize) {
if (warmupFinalized) {
return;
}
this.warmup = warmup;
warmupFinalized = finalize;
}
/**
* Gets the search type
* @return Search type
*/
public SearchType searchType() {
return searchType;
}
/**
* Sets the search type
* @param searchType Search type
* @param finalize If true, value cannot be overwritten by subsequent calls
*/
public void searchType(SearchType searchType, boolean finalize) {
if (searchTypeFinalized) {
return;
}
this.searchType = searchType;
searchTypeFinalized = finalize;
}
/**
* Gets the list of search requests to benchmark
* @return Search requests
*/
public List<SearchRequest> searchRequests() {
return searchRequests;
}
/**
* Adds a search request to benchmark
* @param searchRequest Search request
*/
public void addSearchRequest(SearchRequest... searchRequest) {
searchRequests.addAll(Arrays.asList(searchRequest));
}
/**
* Gets the clear caches request
* @return The clear caches request
*/
public ClearIndicesCacheRequest clearCaches() {
return clearCaches;
}
public boolean allowCacheClearing() {
return allowCacheClearing;
}
public void allowCacheClearing(boolean allowCacheClearing) {
this.allowCacheClearing = allowCacheClearing;
}
/**
* Gets the clear caches settings
* @return Clear caches settings
*/
public ClearCachesSettings clearCachesSettings() {
return clearCachesSettings;
}
/**
* Sets the clear caches settings
* @param clearCachesSettings Clear caches settings
* @param finalize If true, value cannot be overwritten by subsequent calls
*/
public void clearCachesSettings(ClearCachesSettings clearCachesSettings, boolean finalize) {
if (clearCachesSettingsFinalized) {
return;
}
this.clearCachesSettings = clearCachesSettings;
clearCachesSettingsFinalized = finalize;
}
/**
* Builds a clear cache request from the settings
*/
public void buildClearCachesRequestFromSettings() {
clearCaches = new ClearIndicesCacheRequest(indices);
clearCaches.filterCache(clearCachesSettings.filterCache);
clearCaches.fieldDataCache(clearCachesSettings.fieldDataCache);
clearCaches.idCache(clearCachesSettings.idCache);
clearCaches.recycler(clearCachesSettings.recyclerCache);
clearCaches.fields(clearCachesSettings.fields);
clearCaches.filterKeys(clearCachesSettings.filterKeys);
}
public void buildSearchRequestsFromSettings() {
for (SearchRequest searchRequest : searchRequests) {
searchRequest.indices(indices);
searchRequest.types(types);
searchRequest.searchType(searchType);
}
}
/**
* Merge another settings object with this one, ignoring fields which have been finalized.
* This is a convenience so that a global settings object can cascade it's settings
* down to specific competitors w/o inadvertently overwriting anything that has already been set.
* @param otherSettings The settings to merge into this
*/
public void merge(BenchmarkSettings otherSettings) {
if (!concurrencyFinalized) {
concurrency = otherSettings.concurrency;
}
if (!iterationsFinalized) {
iterations = otherSettings.iterations;
}
if (!multiplierFinalized) {
multiplier = otherSettings.multiplier;
}
if (!numSlowestFinalized) {
numSlowest = otherSettings.numSlowest;
}
if (!warmupFinalized) {
warmup = otherSettings.warmup;
}
if (!searchTypeFinalized) {
searchType = otherSettings.searchType;
}
if (!clearCachesSettingsFinalized) {
clearCachesSettings = otherSettings.clearCachesSettings;
}
}
public static class ClearCachesSettings {
private boolean filterCache = false;
private boolean fieldDataCache = false;
private boolean idCache = false;
private boolean recyclerCache = false;
private String[] fields = null;
private String[] filterKeys = null;
public void filterCache(boolean filterCache) {
this.filterCache = filterCache;
}
public void fieldDataCache(boolean fieldDataCache) {
this.fieldDataCache = fieldDataCache;
}
public void idCache(boolean idCache) {
this.idCache = idCache;
}
public void recycler(boolean recyclerCache) {
this.recyclerCache = recyclerCache;
}
public void fields(String[] fields) {
this.fields = fields;
}
public void filterKeys(String[] filterKeys) {
this.filterKeys = filterKeys;
}
}
@Override
public void readFrom(StreamInput in) throws IOException {
concurrency = in.readVInt();
iterations = in.readVInt();
multiplier = in.readVInt();
numSlowest = in.readVInt();
warmup = in.readBoolean();
indices = in.readStringArray();
types = in.readStringArray();
searchType = SearchType.fromId(in.readByte());
clearCaches = in.readOptionalStreamable(new ClearIndicesCacheRequest());
final int size = in.readVInt();
for (int i = 0; i < size; i++) {
SearchRequest searchRequest = new SearchRequest();
searchRequest.readFrom(in);
searchRequests.add(searchRequest);
}
}
@Override
public void writeTo(StreamOutput out) throws IOException {
out.writeVInt(concurrency);
out.writeVInt(iterations);
out.writeVInt(multiplier);
out.writeVInt(numSlowest);
out.writeBoolean(warmup);
out.writeStringArray(indices);
out.writeStringArray(types);
out.writeByte(searchType.id());
out.writeOptionalStreamable(clearCaches);
out.writeVInt(searchRequests.size());
for (SearchRequest searchRequest : searchRequests) {
searchRequest.writeTo(out);
}
}
}

46
src/main/java/org/elasticsearch/action/bench/BenchmarkStatusAction.java
View File

@ -1,46 +0,0 @@
/*
* Licensed to Elasticsearch under one or more contributor
* license agreements. See the NOTICE file distributed with
* this work for additional information regarding copyright
* ownership. Elasticsearch licenses this file to you under
* the Apache License, Version 2.0 (the "License"); you may
* not use this file except in compliance with the License.
* You may obtain a copy of the License at
*
* http://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing,
* software distributed under the License is distributed on an
* "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
* KIND, either express or implied. See the License for the
* specific language governing permissions and limitations
* under the License.
*/
package org.elasticsearch.action.bench;
import org.elasticsearch.action.ClientAction;
import org.elasticsearch.client.Client;
/**
* Benchmark status action
*/
public class BenchmarkStatusAction extends ClientAction<BenchmarkStatusRequest, BenchmarkStatusResponse, BenchmarkStatusRequestBuilder> {
public static final BenchmarkStatusAction INSTANCE = new BenchmarkStatusAction();
public static final String NAME = "indices:data/benchmark/status";
public BenchmarkStatusAction() {
super(NAME);
}
@Override
public BenchmarkStatusResponse newResponse() {
return new BenchmarkStatusResponse();
}
@Override
public BenchmarkStatusRequestBuilder newRequestBuilder(Client client) {
return new BenchmarkStatusRequestBuilder(client);
}
}

111
src/main/java/org/elasticsearch/action/bench/BenchmarkStatusNodeResponse.java
View File

@ -1,111 +0,0 @@
/*
* Licensed to Elasticsearch under one or more contributor
* license agreements. See the NOTICE file distributed with
* this work for additional information regarding copyright
* ownership. Elasticsearch licenses this file to you under
* the Apache License, Version 2.0 (the "License"); you may
* not use this file except in compliance with the License.
* You may obtain a copy of the License at
*
* http://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing,
* software distributed under the License is distributed on an
* "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
* KIND, either express or implied. See the License for the
* specific language governing permissions and limitations
* under the License.
*/
package org.elasticsearch.action.bench;
import org.elasticsearch.action.ActionResponse;
import org.elasticsearch.common.io.stream.StreamInput;
import org.elasticsearch.common.io.stream.StreamOutput;
import org.elasticsearch.common.io.stream.Streamable;
import org.elasticsearch.common.xcontent.ToXContent;
import org.elasticsearch.common.xcontent.XContentBuilder;
import org.elasticsearch.common.xcontent.XContentFactory;
import java.io.IOException;
import java.util.List;
import java.util.ArrayList;
/**
* Node-level response for the status of an on-going benchmark
*/
public class BenchmarkStatusNodeResponse extends ActionResponse implements Streamable, ToXContent {
private String nodeName;
private List<BenchmarkResponse> benchmarkResponses;
public BenchmarkStatusNodeResponse() {
benchmarkResponses = new ArrayList<>();
}
public void nodeName(String nodeName) {
this.nodeName = nodeName;
}
public String nodeName() {
return nodeName;
}
public void addBenchResponse(BenchmarkResponse benchmarkResponse) {
benchmarkResponses.add(benchmarkResponse);
}
public List<BenchmarkResponse> benchResponses() {
return benchmarkResponses;
}
public int activeBenchmarks() {
return benchResponses().size();
}
@Override
public XContentBuilder toXContent(XContentBuilder builder, Params params) throws IOException {
builder.field("node", nodeName);
builder.startArray("responses");
for (BenchmarkResponse benchmarkResponse : benchmarkResponses) {
benchmarkResponse.toXContent(builder, params);
}
builder.endArray();
return builder;
}
@Override
public void readFrom(StreamInput in) throws IOException {
super.readFrom(in);
nodeName = in.readString();
int size = in.readVInt();
for (int i = 0; i < size; i++) {
BenchmarkResponse br = new BenchmarkResponse();
br.readFrom(in);
benchmarkResponses.add(br);
}
}
@Override
public void writeTo(StreamOutput out) throws IOException {
super.writeTo(out);
out.writeString(nodeName);
out.writeVInt(benchmarkResponses.size());
for (BenchmarkResponse br : benchmarkResponses) {
br.writeTo(out);
}
}
@Override
public String toString() {
try {
XContentBuilder builder = XContentFactory.jsonBuilder().prettyPrint();
builder.startObject();
toXContent(builder, EMPTY_PARAMS);
builder.endObject();
return builder.string();
} catch (IOException e) {
return "{ \"error\" : \"" + e.getMessage() + "\"}";
}
}
}

48
src/main/java/org/elasticsearch/action/bench/BenchmarkStatusRequest.java
View File

@ -1,48 +0,0 @@
/*
* Licensed to Elasticsearch under one or more contributor
* license agreements. See the NOTICE file distributed with
* this work for additional information regarding copyright
* ownership. Elasticsearch licenses this file to you under
* the Apache License, Version 2.0 (the "License"); you may
* not use this file except in compliance with the License.
* You may obtain a copy of the License at
*
* http://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing,
* software distributed under the License is distributed on an
* "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
* KIND, either express or implied. See the License for the
* specific language governing permissions and limitations
* under the License.
*/
package org.elasticsearch.action.bench;
import org.elasticsearch.action.ActionRequestValidationException;
import org.elasticsearch.action.support.master.MasterNodeOperationRequest;
import org.elasticsearch.common.io.stream.StreamInput;
import org.elasticsearch.common.io.stream.StreamOutput;
import java.io.IOException;
/**
* Request for status about a running benchmark
*/
public class BenchmarkStatusRequest extends MasterNodeOperationRequest<BenchmarkStatusRequest> {
@Override
public ActionRequestValidationException validate() {
return null;
}
@Override
public void readFrom(StreamInput in) throws IOException {
super.readFrom(in);
}
@Override
public void writeTo(StreamOutput out) throws IOException {
super.writeTo(out);
}
}

39
src/main/java/org/elasticsearch/action/bench/BenchmarkStatusRequestBuilder.java
View File

@ -1,39 +0,0 @@
/*
* Licensed to Elasticsearch under one or more contributor
* license agreements. See the NOTICE file distributed with
* this work for additional information regarding copyright
* ownership. Elasticsearch licenses this file to you under
* the Apache License, Version 2.0 (the "License"); you may
* not use this file except in compliance with the License.
* You may obtain a copy of the License at
*
* http://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing,
* software distributed under the License is distributed on an
* "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
* KIND, either express or implied. See the License for the
* specific language governing permissions and limitations
* under the License.
*/
package org.elasticsearch.action.bench;
import org.elasticsearch.client.Client;
import org.elasticsearch.action.ActionListener;
import org.elasticsearch.action.ActionRequestBuilder;
/**
* Request builder for benchmark status
*/
public class BenchmarkStatusRequestBuilder extends ActionRequestBuilder<BenchmarkStatusRequest, BenchmarkStatusResponse, BenchmarkStatusRequestBuilder, Client> {
public BenchmarkStatusRequestBuilder(Client client) {
super(client, new BenchmarkStatusRequest());
}
@Override
protected void doExecute(ActionListener<BenchmarkStatusResponse> listener) {
client.benchStatus(request, listener);
}
}

96
src/main/java/org/elasticsearch/action/bench/BenchmarkStatusResponse.java
View File

@ -1,96 +0,0 @@
/*
* Licensed to Elasticsearch under one or more contributor
* license agreements. See the NOTICE file distributed with
* this work for additional information regarding copyright
* ownership. Elasticsearch licenses this file to you under
* the Apache License, Version 2.0 (the "License"); you may
* not use this file except in compliance with the License.
* You may obtain a copy of the License at
*
* http://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing,
* software distributed under the License is distributed on an
* "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
* KIND, either express or implied. See the License for the
* specific language governing permissions and limitations
* under the License.
*/
package org.elasticsearch.action.bench;
import org.elasticsearch.action.ActionResponse;
import org.elasticsearch.common.io.stream.StreamInput;
import org.elasticsearch.common.io.stream.StreamOutput;
import org.elasticsearch.common.io.stream.Streamable;
import org.elasticsearch.common.xcontent.ToXContent;
import org.elasticsearch.common.xcontent.XContentBuilder;
import java.io.IOException;
import java.util.Arrays;
import java.util.List;
import java.util.ArrayList;
/**
* Benchmark status response
*/
public class BenchmarkStatusResponse extends ActionResponse implements Streamable, ToXContent {
private int totalActiveBenchmarks = 0;
private final List<BenchmarkResponse> benchmarkResponses = new ArrayList<>();
public BenchmarkStatusResponse() { }
public void addBenchResponse(BenchmarkResponse response) {
benchmarkResponses.add(response);
}
public List<BenchmarkResponse> benchmarkResponses() {
return benchmarkResponses;
}
public void totalActiveBenchmarks(int totalActiveBenchmarks) {
this.totalActiveBenchmarks = totalActiveBenchmarks;
}
public int totalActiveBenchmarks() {
return totalActiveBenchmarks;
}
@Override
public XContentBuilder toXContent(XContentBuilder builder, Params params) throws IOException {
if (benchmarkResponses.size() > 0) {
builder.startObject("active_benchmarks");
for (BenchmarkResponse benchmarkResponse : benchmarkResponses) {
builder.startObject(benchmarkResponse.benchmarkName());
benchmarkResponse.toXContent(builder, params);
builder.endObject();
}
builder.endObject();
}
return builder;
}
@Override
public void readFrom(StreamInput in) throws IOException {
super.readFrom(in);
totalActiveBenchmarks = in.readVInt();
int size = in.readVInt();
for (int i = 0; i < size; i++) {
BenchmarkResponse br = new BenchmarkResponse();
br.readFrom(in);
benchmarkResponses.add(br);
}
}
@Override
public void writeTo(StreamOutput out) throws IOException {
super.writeTo(out);
out.writeVInt(totalActiveBenchmarks);
out.writeVInt(benchmarkResponses.size());
for (BenchmarkResponse br : benchmarkResponses) {
br.writeTo(out);
}
}
}

157
src/main/java/org/elasticsearch/action/bench/CompetitionDetails.java
View File

@ -1,157 +0,0 @@
/*
* Licensed to Elasticsearch under one or more contributor
* license agreements. See the NOTICE file distributed with
* this work for additional information regarding copyright
* ownership. Elasticsearch licenses this file to you under
* the Apache License, Version 2.0 (the "License"); you may
* not use this file except in compliance with the License.
* You may obtain a copy of the License at
*
* http://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing,
* software distributed under the License is distributed on an
* "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
* KIND, either express or implied. See the License for the
* specific language governing permissions and limitations
* under the License.
*/
package org.elasticsearch.action.bench;
import org.elasticsearch.common.xcontent.ToXContent;
import org.elasticsearch.common.xcontent.XContentBuilder;
import org.elasticsearch.common.xcontent.XContentBuilderString;
import java.io.IOException;
import java.util.List;
import java.util.Map;
/**
* Detailed statistics for each iteration of a benchmark search competition.
*/
public class CompetitionDetails implements ToXContent {
private List<CompetitionNodeResult> nodeResults;
public CompetitionDetails(List<CompetitionNodeResult> nodeResults) {
this.nodeResults = nodeResults;
}
/**
* Gets node-level competition results
* @return A list of node-level competition results
*/
public List<CompetitionNodeResult> getNodeResults() {
return nodeResults;
}
@Override
public XContentBuilder toXContent(XContentBuilder builder, Params params) throws IOException {
final int highestIteration = highestCompletedIteration();
computeAllStatistics();
CompetitionIteration prototypical = prototypicalIteration();
builder.startObject(Fields.ITERATIONS);
for (int i = 0; i < highestIteration; i++) {
builder.field(Fields.ITERATION, i);
builder.startObject();
builder.startObject(Fields.MIN);
for (CompetitionNodeResult nodeResult : nodeResults) {
CompetitionIteration iteration = nodeResult.iterations().get(i);
builder.field(nodeResult.nodeName(), iteration == null ? Fields.NULL : iteration.min());
}
builder.endObject();
builder.startObject(Fields.MAX);
for (CompetitionNodeResult nodeResult : nodeResults) {
CompetitionIteration iteration = nodeResult.iterations().get(i);
builder.field(nodeResult.nodeName(), iteration == null ? Fields.NULL : iteration.max());
}
builder.endObject();
builder.startObject(Fields.MEAN);
for (CompetitionNodeResult nodeResult : nodeResults) {
CompetitionIteration iteration = nodeResult.iterations().get(i);
builder.field(nodeResult.nodeName(), iteration == null ? Fields.NULL : iteration.mean());
}
builder.endObject();
builder.startObject(Fields.TOTAL_TIME);
for (CompetitionNodeResult nodeResult : nodeResults) {
CompetitionIteration iteration = nodeResult.iterations().get(i);
builder.field(nodeResult.nodeName(), iteration == null ? Fields.NULL : iteration.totalTime());
}
builder.endObject();
builder.startObject(Fields.QPS);
for (CompetitionNodeResult nodeResult : nodeResults) {
CompetitionIteration iteration = nodeResult.iterations().get(i);
builder.field(nodeResult.nodeName(), iteration == null ? Fields.NULL : iteration.queriesPerSecond());
}
builder.endObject();
// All nodes track the same percentiles, so just choose a prototype to iterate
if (prototypical != null) {
for (Map.Entry<Double, Double> entry : prototypical.percentileValues().entrySet()) {
// Change back to integral value for display purposes
builder.startObject(new XContentBuilderString("percentile_" + entry.getKey().longValue()));
for (CompetitionNodeResult nodeResult : nodeResults) {
CompetitionIteration iteration = nodeResult.iterations().get(i);
if (iteration != null) {
Double value = iteration.percentileValues().get(entry.getKey());
builder.field(nodeResult.nodeName(), (value.isNaN()) ? 0.0 : value);
} else {
builder.field(nodeResult.nodeName(), Fields.NULL);
}
}
builder.endObject();
}
}
builder.endObject();
}
return builder;
}
/**
* Calculates detailed statistics for each iteration. Should be called prior to
* accessing individual measurements.
*/
public void computeAllStatistics() {
for (CompetitionNodeResult nodeResult : nodeResults) {
for (CompetitionIteration iteration : nodeResult.iterations()) {
iteration.computeStatistics();
}
}
}
private CompetitionIteration prototypicalIteration() {
if (nodeResults != null && nodeResults.size() > 0) {
CompetitionNodeResult nodeResult = nodeResults.get(0);
if (nodeResult.iterations() != null && nodeResult.iterations().size() > 0) {
return nodeResult.iterations().get(0);
}
}
return null;
}
private int highestCompletedIteration() {
int count = 0;
for (CompetitionNodeResult nodeResult : nodeResults) {
count = Math.max(count, nodeResult.completedIterations());
}
return count;
}
static final class Fields {
static final XContentBuilderString ITERATIONS = new XContentBuilderString("iterations");
static final XContentBuilderString ITERATION = new XContentBuilderString("iteration");
static final XContentBuilderString TOTAL_TIME = new XContentBuilderString("total_time");
static final XContentBuilderString MEAN = new XContentBuilderString("mean");
static final XContentBuilderString MIN = new XContentBuilderString("min");
static final XContentBuilderString MAX = new XContentBuilderString("max");
static final XContentBuilderString QPS = new XContentBuilderString("qps");
static final XContentBuilderString NULL = new XContentBuilderString("null");
}
}

227
src/main/java/org/elasticsearch/action/bench/CompetitionIteration.java
View File

@ -1,227 +0,0 @@
/*
* Licensed to Elasticsearch under one or more contributor
* license agreements. See the NOTICE file distributed with
* this work for additional information regarding copyright
* ownership. Elasticsearch licenses this file to you under
* the Apache License, Version 2.0 (the "License"); you may
* not use this file except in compliance with the License.
* You may obtain a copy of the License at
*
* http://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing,
* software distributed under the License is distributed on an
* "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
* KIND, either express or implied. See the License for the
* specific language governing permissions and limitations
* under the License.
*/
package org.elasticsearch.action.bench;
import org.elasticsearch.action.search.SearchRequest;
import org.elasticsearch.common.io.stream.StreamInput;
import org.elasticsearch.common.io.stream.StreamOutput;
import org.elasticsearch.common.io.stream.Streamable;
import org.elasticsearch.common.xcontent.ToXContent;
import org.elasticsearch.common.xcontent.XContentBuilder;
import org.elasticsearch.common.xcontent.XContentBuilderString;
import org.elasticsearch.common.xcontent.XContentHelper;
import java.io.IOException;
import java.util.Map;
import java.util.TreeMap;
/**
* Represents a single iteration of a search benchmark competition
*/
public class CompetitionIteration implements Streamable {
private long numQueries;
private SlowRequest[] slowRequests;
private long totalTime;
private long sum;
private long sumTotalHits;
private double stddev;
private long min;
private long max;
private double mean;
private double qps;
private double millisPerHit;
private double[] percentiles = new double[0];
private Map<Double, Double> percentileValues = new TreeMap<>();
private CompetitionIterationData iterationData;
public CompetitionIteration() { }
public CompetitionIteration(SlowRequest[] slowestRequests, long totalTime, long numQueries, long sumTotalHits,
CompetitionIterationData iterationData) {
this.totalTime = totalTime;
this.sumTotalHits = sumTotalHits;
this.slowRequests = slowestRequests;
this.numQueries = numQueries;
this.iterationData = iterationData;
this.millisPerHit = totalTime / (double)sumTotalHits;
}
public void computeStatistics() {
final SinglePassStatistics single = new SinglePassStatistics();
for (long datum : iterationData.data()) {
if (datum > -1) { // ignore unset values in the underlying array
single.push(datum);
}
}
sum = single.sum();
stddev = single.stddev();
min = single.min();
max = single.max();
mean = single.mean();
qps = numQueries * (1000.d / (double) sum);
for (double percentile : percentiles) {
percentileValues.put(percentile, single.percentile(percentile / 100));
}
}
public CompetitionIterationData competitionIterationData() {
return iterationData;
}
public long numQueries() {
return numQueries;
}
public long totalTime() {
return totalTime;
}
public long sumTotalHits() {
return sumTotalHits;
}
public double millisPerHit() {
return millisPerHit;
}
public double queriesPerSecond() {
return qps;
}
public SlowRequest[] slowRequests() {
return slowRequests;
}
public long min() {
return min;
}
public long max() {
return max;
}
public double mean() {
return mean;
}
public Map<Double, Double> percentileValues() {
return percentileValues;
}
public void percentiles(double[] percentiles) {
this.percentiles = percentiles;
}
@Override
public void readFrom(StreamInput in) throws IOException {
totalTime = in.readVLong();
sumTotalHits = in.readVLong();
numQueries = in.readVLong();
millisPerHit = in.readDouble();
iterationData = in.readOptionalStreamable(new CompetitionIterationData());
int size = in.readVInt();
slowRequests = new SlowRequest[size];
for (int i = 0; i < size; i++) {
slowRequests[i] = new SlowRequest();
slowRequests[i].readFrom(in);
}
percentiles = in.readDoubleArray();
}
@Override
public void writeTo(StreamOutput out) throws IOException {
out.writeVLong(totalTime);
out.writeVLong(sumTotalHits);
out.writeVLong(numQueries);
out.writeDouble(millisPerHit);
out.writeOptionalStreamable(iterationData);
out.writeVInt(slowRequests == null ? 0 : slowRequests.length);
if (slowRequests != null) {
for (SlowRequest slowRequest : slowRequests) {
slowRequest.writeTo(out);
}
}
out.writeDoubleArray(percentiles);
}
/**
* Represents a 'slow' search request
*/
public static class SlowRequest implements ToXContent, Streamable {
private long maxTimeTaken;
private long avgTimeTaken;
private SearchRequest searchRequest;
public SlowRequest() { }
public SlowRequest(long avgTimeTaken, long maxTimeTaken, SearchRequest searchRequest) {
this.avgTimeTaken = avgTimeTaken;
this.maxTimeTaken = maxTimeTaken;
this.searchRequest = searchRequest;
}
public long avgTimeTaken() {
return avgTimeTaken;
}
public long maxTimeTaken() {
return maxTimeTaken;
}
public SearchRequest searchRequest() {
return searchRequest;
}
@Override
public void readFrom(StreamInput in) throws IOException {
avgTimeTaken = in.readVLong();
maxTimeTaken = in.readVLong();
searchRequest = new SearchRequest();
searchRequest.readFrom(in);
}
@Override
public void writeTo(StreamOutput out) throws IOException {
out.writeVLong(avgTimeTaken);
out.writeVLong(maxTimeTaken);
searchRequest.writeTo(out);
}
@Override
public XContentBuilder toXContent(XContentBuilder builder, Params params) throws IOException {
builder.field(Fields.MAX_TIME, maxTimeTaken);
builder.field(Fields.AVG_TIME, avgTimeTaken);
XContentHelper.writeRawField("request", searchRequest.source(), builder, params);
return builder;
}
}
static final class Fields {
static final XContentBuilderString MAX_TIME = new XContentBuilderString("max_time");
static final XContentBuilderString AVG_TIME = new XContentBuilderString("avg_time");
}
}

87
src/main/java/org/elasticsearch/action/bench/CompetitionIterationData.java
View File

@ -1,87 +0,0 @@
/*
* Licensed to Elasticsearch under one or more contributor
* license agreements. See the NOTICE file distributed with
* this work for additional information regarding copyright
* ownership. Elasticsearch licenses this file to you under
* the Apache License, Version 2.0 (the "License"); you may
* not use this file except in compliance with the License.
* You may obtain a copy of the License at
*
* http://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing,
* software distributed under the License is distributed on an
* "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
* KIND, either express or implied. See the License for the
* specific language governing permissions and limitations
* under the License.
*/
package org.elasticsearch.action.bench;
import org.elasticsearch.common.io.stream.StreamInput;
import org.elasticsearch.common.io.stream.StreamOutput;
import org.elasticsearch.common.io.stream.Streamable;
import java.io.IOException;
/**
* Holds data points for a single benchmark iteration
*
* Note that the underlying data array may be actively populated, so we take care not
* to compute statistics on uninitialized elements (which are indicated by a value of -1).
* Initialized elements (those with values > -1) will not be changed once set.
*/
public class CompetitionIterationData implements Streamable {
private long[] data;
public CompetitionIterationData() { }
public CompetitionIterationData(long[] data) {
this.data = data;
}
public long[] data() {
return data;
}
/**
* The number of data values currently holding valid values
*
* @return Number of data values currently holding valid values
*/
public long length() {
long length = 0;
for (int i = 0; i < data.length; i++) {
if (data[i] < 0) { // Data values can be invalid when computing statistics on actively running benchmarks
continue;
}
length++;
}
return length;
}
/**
* The sum of all currently set values
*
* @return The sum of all currently set values
*/
public long sum() {
long sum = 0;
for (int i = 0; i < data.length; i++) {
sum += Math.max(0, data[i]); // Data values can be invalid when computing statistics on actively running benchmarks
}
return sum;
}
@Override
public void readFrom(StreamInput in) throws IOException {
data = in.readLongArray();
}
@Override
public void writeTo(StreamOutput out) throws IOException {
out.writeLongArray(data);
}
}

126
src/main/java/org/elasticsearch/action/bench/CompetitionNodeResult.java
View File

@ -1,126 +0,0 @@
/*
* Licensed to Elasticsearch under one or more contributor
* license agreements. See the NOTICE file distributed with
* this work for additional information regarding copyright
* ownership. Elasticsearch licenses this file to you under
* the Apache License, Version 2.0 (the "License"); you may
* not use this file except in compliance with the License.
* You may obtain a copy of the License at
*
* http://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing,
* software distributed under the License is distributed on an
* "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
* KIND, either express or implied. See the License for the
* specific language governing permissions and limitations
* under the License.
*/
package org.elasticsearch.action.bench;
import org.elasticsearch.action.ActionResponse;
import org.elasticsearch.common.io.stream.StreamInput;
import org.elasticsearch.common.io.stream.StreamOutput;
import org.elasticsearch.common.io.stream.Streamable;
import java.io.IOException;
import java.util.ArrayList;
import java.util.List;
/**
* Per-node result of competition iterations
*/
public class CompetitionNodeResult extends ActionResponse implements Streamable {
private String competitionName;
private String nodeName;
private int totalIterations = 0;
private int completedIterations = 0;
private int totalExecutedQueries = 0;
private long warmUpTime = 0;
private List<CompetitionIteration> iterations;
public CompetitionNodeResult() {
iterations = new ArrayList<>();
}
public CompetitionNodeResult(String competitionName, String nodeName, int totalIterations, List<CompetitionIteration> iterations) {
this.competitionName = competitionName;
this.nodeName = nodeName;
this.iterations = iterations;
this.totalIterations = totalIterations;
}
public String competitionName() {
return competitionName;
}
public String nodeName() {
return nodeName;
}
public int totalIterations() {
return totalIterations;
}
public int completedIterations() {
return completedIterations;
}
public void incrementCompletedIterations() {
completedIterations++;
}
public long warmUpTime() {
return warmUpTime;
}
public void warmUpTime(long warmUpTime) {
this.warmUpTime = warmUpTime;
}
public int totalExecutedQueries() {
return totalExecutedQueries;
}
public void totalExecutedQueries(int totalExecutedQueries) {
this.totalExecutedQueries = totalExecutedQueries;
}
public List<CompetitionIteration> iterations() {
return iterations;
}
@Override
public void readFrom(StreamInput in) throws IOException {
super.readFrom(in);
competitionName = in.readString();
nodeName = in.readString();
totalIterations = in.readVInt();
completedIterations = in.readVInt();
totalExecutedQueries = in.readVInt();
warmUpTime = in.readVLong();
int size = in.readVInt();
for (int i = 0; i < size; i++) {
CompetitionIteration iteration = new CompetitionIteration();
iteration.readFrom(in);
iterations.add(iteration);
}
}
@Override
public void writeTo(StreamOutput out) throws IOException {
super.writeTo(out);
out.writeString(competitionName);
out.writeString(nodeName);
out.writeVInt(totalIterations);
out.writeVInt(completedIterations);
out.writeVInt(totalExecutedQueries);
out.writeVLong(warmUpTime);
out.writeVInt(iterations.size());
for (CompetitionIteration iteration : iterations) {
iteration.writeTo(out);
}
}
}

206
src/main/java/org/elasticsearch/action/bench/CompetitionResult.java
View File

@ -1,206 +0,0 @@
/*
* Licensed to Elasticsearch under one or more contributor
* license agreements. See the NOTICE file distributed with
* this work for additional information regarding copyright
* ownership. Elasticsearch licenses this file to you under
* the Apache License, Version 2.0 (the "License"); you may
* not use this file except in compliance with the License.
* You may obtain a copy of the License at
*
* http://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing,
* software distributed under the License is distributed on an
* "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
* KIND, either express or implied. See the License for the
* specific language governing permissions and limitations
* under the License.
*/
package org.elasticsearch.action.bench;
import org.elasticsearch.common.io.stream.StreamInput;
import org.elasticsearch.common.io.stream.StreamOutput;
import org.elasticsearch.common.io.stream.Streamable;
import org.elasticsearch.common.xcontent.ToXContent;
import org.elasticsearch.common.xcontent.XContentBuilder;
import java.io.IOException;
import java.util.ArrayList;
import java.util.List;
/**
* The result of a benchmark competition. Maintains per-node results.
*/
public class CompetitionResult implements Streamable, ToXContent {
private String competitionName;
private int concurrency;
private int multiplier;
private boolean verbose;
private double[] percentiles = BenchmarkSettings.DEFAULT_PERCENTILES;
private List<CompetitionNodeResult> nodeResults = new ArrayList<>();
private CompetitionSummary competitionSummary;
private CompetitionDetails competitionDetails;
public CompetitionResult() { }
/**
* Constructs a competition result
* @param competitionName Competition name
* @param concurrency Concurrency
* @param multiplier Internal multiplier; each iteration will run this many times to smooth out measurements
* @param percentiles Which percentiles to report on
*/
public CompetitionResult(String competitionName, int concurrency, int multiplier, double[] percentiles) {
this(competitionName, concurrency, multiplier, false, percentiles);
}
/**
* Constructs a competition result
* @param competitionName Competition name
* @param concurrency Concurrency
* @param multiplier Internal multiplier; each iteration will run this many times to smooth out measurements
* @param verbose Whether to report detailed statistics
* @param percentiles Which percentiles to report on
*/
public CompetitionResult(String competitionName, int concurrency, int multiplier, boolean verbose, double[] percentiles) {
this.competitionName = competitionName;
this.concurrency = concurrency;
this.multiplier = multiplier;
this.verbose = verbose;
this.percentiles = (percentiles != null && percentiles.length > 0) ? percentiles : BenchmarkSettings.DEFAULT_PERCENTILES;
this.competitionDetails = new CompetitionDetails(nodeResults);
this.competitionSummary = new CompetitionSummary(nodeResults, concurrency, multiplier, percentiles);
}
/**
* Adds a node-level competition result
* @param nodeResult Node result
*/
public void addCompetitionNodeResult(CompetitionNodeResult nodeResult) {
nodeResults.add(nodeResult);
}
/**
* Gets detailed statistics for the competition
* @return Detailed statistics
*/
public CompetitionDetails competitionDetails() {
return competitionDetails;
}
/**
* Gets summary statistics for the competition
* @return Summary statistics
*/
public CompetitionSummary competitionSummary() {
return competitionSummary;
}
/**
* Gets the name of the competition
* @return Name
*/
public String competitionName() {
return competitionName;
}
/**
* Gets the concurrency level; determines how many threads will be executing the competition concurrently.
* @return Concurrency
*/
public int concurrency() {
return concurrency;
}
/**
* Gets the multiplier. The multiplier determines how many times each iteration will be run.
* @return Multiplier
*/
public int multiplier() {
return multiplier;
}
/**
* Whether to report detailed statistics
* @return True/false
*/
public boolean verbose() {
return verbose;
}
/**
* Sets whether to report detailed statistics
* @param verbose True/false
*/
public void verbose(boolean verbose) {
this.verbose = verbose;
}
/**
* Gets list of percentiles to report
* @return List of percentiles
*/
public double[] percentiles() {
return percentiles;
}
/**
* Sets the list of percentiles to report
* @param percentiles Percentiles
*/
public void percentiles(double[] percentiles) {
this.percentiles = percentiles;
}
/**
* Gets the results for each cluster node that the competition executed on.
* @return Node-level results
*/
public List<CompetitionNodeResult> nodeResults() {
return nodeResults;
}
@Override
public XContentBuilder toXContent(XContentBuilder builder, Params params) throws IOException {
builder.startObject(competitionName);
competitionSummary.toXContent(builder, params);
if (verbose) {
competitionDetails.toXContent(builder, params);
}
builder.endObject();
return builder;
}
@Override
public void readFrom(StreamInput in) throws IOException {
competitionName = in.readString();
concurrency = in.readVInt();
multiplier = in.readVInt();
verbose = in.readBoolean();
int size = in.readVInt();
for (int i = 0; i < size; i++) {
CompetitionNodeResult result = new CompetitionNodeResult();
result.readFrom(in);
nodeResults.add(result);
}
percentiles = in.readDoubleArray();
competitionSummary = new CompetitionSummary(nodeResults, concurrency, multiplier, percentiles);
}
@Override
public void writeTo(StreamOutput out) throws IOException {
out.writeString(competitionName);
out.writeVInt(concurrency);
out.writeVInt(multiplier);
out.writeBoolean(verbose);
out.writeVInt(nodeResults.size());
for (CompetitionNodeResult result : nodeResults) {
result.writeTo(out);
}
out.writeDoubleArray(percentiles);
}
}

364
src/main/java/org/elasticsearch/action/bench/CompetitionSummary.java
View File

@ -1,364 +0,0 @@
/*
* Licensed to Elasticsearch under one or more contributor
* license agreements. See the NOTICE file distributed with
* this work for additional information regarding copyright
* ownership. Elasticsearch licenses this file to you under
* the Apache License, Version 2.0 (the "License"); you may
* not use this file except in compliance with the License.
* You may obtain a copy of the License at
*
* http://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing,
* software distributed under the License is distributed on an
* "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
* KIND, either express or implied. See the License for the
* specific language governing permissions and limitations
* under the License.
*/
package org.elasticsearch.action.bench;
import org.elasticsearch.common.collect.Tuple;
import org.elasticsearch.common.xcontent.ToXContent;
import org.elasticsearch.common.xcontent.XContentBuilder;
import org.elasticsearch.common.xcontent.XContentBuilderString;
import org.apache.lucene.util.CollectionUtil;
import java.io.IOException;
import java.util.*;
/**
* Summary statistics for a benchmark search competition.
*
* Statistics are calculated over all iteration results for all nodes
* that executed the competition.
*
* Since values are calculated lazily on first access, users of this class
* should first call computeSummaryStatistics() prior to accessing individual
* measurements.
*/
public class CompetitionSummary implements ToXContent {
private List<CompetitionNodeResult> nodeResults;
private boolean computed = false;
private long min = 0;
private long max = 0;
private long totalTime = 0;
private long sumTotalHits = 0;
private long totalIterations = 0;
private long completedIterations = 0;
private long totalQueries = 0;
private double avgWarmupTime = 0;
private int concurrency = 0;
private int multiplier = 0;
private double mean = 0;
private double millisPerHit = 0.0;
private double stdDeviation = 0.0;
private double queriesPerSecond = 0.0;
private double[] percentiles;
Map<Double, Double> percentileValues = new TreeMap<>();
private List<Tuple<String, CompetitionIteration.SlowRequest>> slowest = new ArrayList<>();
public CompetitionSummary() { }
public CompetitionSummary(List<CompetitionNodeResult> nodeResults, int concurrency, int multiplier, double[] percentiles) {
this.nodeResults = nodeResults;
this.concurrency = concurrency;
this.multiplier = multiplier;
this.percentiles = percentiles;
}
/**
* Gets node-level competition results
* @return A list of node-level competition results
*/
public List<CompetitionNodeResult> nodeResults() {
return nodeResults;
}
/**
* Calculates statistical measures from raw measurements. Should be called prior to accessing
* individual measurements.
*/
public void computeSummaryStatistics() {
if (computed) {
return;
}
long totalWarmupTime = 0;
final SinglePassStatistics single = new SinglePassStatistics();
for (CompetitionNodeResult nodeResult : nodeResults) {
totalWarmupTime += nodeResult.warmUpTime();
totalIterations += nodeResult.totalIterations();
completedIterations += nodeResult.completedIterations();
// only calculate statistics for iterations completed thus far
for (int i = 0; i < nodeResult.completedIterations(); i++) {
CompetitionIteration competitionIteration = nodeResult.iterations().get(i);
CompetitionIterationData iterationData = competitionIteration.competitionIterationData();
long[] data = iterationData.data();
for (long datum : data) {
if (datum > -1) { // ignore unset values in the underlying array
single.push(datum);
}
}
totalQueries += competitionIteration.numQueries();
totalTime += competitionIteration.totalTime();
sumTotalHits += competitionIteration.sumTotalHits();
// keep track of slowest requests
if (competitionIteration.slowRequests() != null) {
for (CompetitionIteration.SlowRequest slow : competitionIteration.slowRequests()) {
slowest.add(new Tuple<>(nodeResult.nodeName(), slow));
}
}
}
}
min = single.min();
max = single.max();
mean = single.mean();
stdDeviation = single.stddev();
avgWarmupTime = (nodeResults.size() > 0) ? totalWarmupTime / nodeResults.size() : 0.0;
queriesPerSecond = (single.sum() > 0) ? (totalQueries * (1000.0 / (double) single.sum())) : 0.0;
millisPerHit = (sumTotalHits > 0) ? (totalTime / (double) sumTotalHits) : 0.0;
for (double percentile : percentiles) {
percentileValues.put(percentile, single.percentile(percentile / 100.0d));
}
CollectionUtil.timSort(slowest, new Comparator<Tuple<String, CompetitionIteration.SlowRequest>>() {
@Override
public int compare(Tuple<String, CompetitionIteration.SlowRequest> o1, Tuple<String, CompetitionIteration.SlowRequest> o2) {
return Long.compare(o2.v2().maxTimeTaken(), o1.v2().maxTimeTaken());
}
});
computed = true;
}
@Override
public XContentBuilder toXContent(XContentBuilder builder, Params params) throws IOException {
computeSummaryStatistics();
builder.startObject(Fields.SUMMARY);
builder.startArray(Fields.NODES);
for (CompetitionNodeResult nodeResult : nodeResults) {
builder.field(nodeResult.nodeName());
}
builder.endArray();
builder.field(Fields.TOTAL_ITERATIONS, totalIterations);
builder.field(Fields.COMPLETED_ITERATIONS, completedIterations);
builder.field(Fields.TOTAL_QUERIES, totalQueries);
builder.field(Fields.CONCURRENCY, concurrency);
builder.field(Fields.MULTIPLIER, multiplier);
builder.field(Fields.AVG_WARMUP_TIME, avgWarmupTime);
builder.startObject(Fields.STATISTICS);
builder.field(Fields.MIN, min == Long.MAX_VALUE ? 0 : min);
builder.field(Fields.MAX, max == Long.MIN_VALUE ? 0 : max);
builder.field(Fields.MEAN, mean);
builder.field(Fields.QPS, queriesPerSecond);
builder.field(Fields.STD_DEV, stdDeviation);
builder.field(Fields.MILLIS_PER_HIT, millisPerHit);
for (Map.Entry<Double, Double> entry : percentileValues.entrySet()) {
// Change back to integral value for display purposes
builder.field(new XContentBuilderString("percentile_" + entry.getKey().longValue()),
(entry.getValue().isNaN()) ? 0.0 : entry.getValue());
}
builder.endObject();
if (totalIterations > 0 && slowest.size() > 0) {
builder.startArray(Fields.SLOWEST);
int n = (int) (slowest.size() / totalIterations);
for (int i = 0; i < n; i++) {
builder.startObject();
builder.field(Fields.NODE, slowest.get(i).v1());
slowest.get(i).v2().toXContent(builder, params);
builder.endObject();
}
builder.endArray();
}
builder.endObject();
return builder;
}
/**
* List of per-node competition results
* @return Per-node competition results
*/
public List<CompetitionNodeResult> getNodeResults() {
return nodeResults;
}
/**
* Shortest execution time of any search
* @return Shortest execution time of any search
*/
public long getMin() {
return min;
}
/**
* Longest execution time of any search
* @return Longest execution time of any search
*/
public long getMax() {
return max;
}
/**
* Total execution time
* @return Total execution time
*/
public long getTotalTime() {
return totalTime;
}
/**
* Total hit count
* @return Total hit count
*/
public long getSumTotalHits() {
return sumTotalHits;
}
/**
* Number of requested iterations
* @return Number of requested iterations
*/
public long getTotalIterations() {
return totalIterations;
}
/**
* Number of iterations actually completed
* @return Number of iterations actually completed
*/
public long getCompletedIterations() {
return completedIterations;
}
/**
* Total number of queries actually executed
* @return Number of queries actually executed
*/
public long getTotalQueries() {
return totalQueries;
}
/**
* Mean average of warmup times across all nodes
* @return Average warmup time
*/
public double getAvgWarmupTime() {
return avgWarmupTime;
}
/**
* Number of concurrent searches
* @return Concurrency
*/
public int getConcurrency() {
return concurrency;
}
/**
* Loop multiplier
* @return Multiplier
*/
public int getMultiplier() {
return multiplier;
}
/**
* Mean average
* @return Mean average
*/
public double getMean() {
return mean;
}
/**
* Total time considered as a percentage of total hits
* @return Milliseconds-per-hit
*/
public double getMillisPerHit() {
return millisPerHit;
}
/**
* Standard deviation from the mean of all measurements
* @return Standard deviation
*/
public double getStdDeviation() {
return stdDeviation;
}
/**
* Measurement of the queries-per-second calculated as: numQueries * (1000.0 / totalTime)
* @return Queries-per-second
*/
public double getQueriesPerSecond() {
return queriesPerSecond;
}
/**
* The user-requested percentiles to measure
* @return Array of percentiles to measure
*/
public double[] getPercentiles() {
return percentiles;
}
/**
* A map of percentiles and their measurements.
* @return A map of entries of (percentile, measurement)
*/
public Map<Double, Double> getPercentileValues() {
return percentileValues;
}
/**
* A list of the N slowest requests and the node that each executed on.
* @return A list of pairs of (node, request)
*/
public List<Tuple<String, CompetitionIteration.SlowRequest>> getSlowest() {
return slowest;
}
static final class Fields {
static final XContentBuilderString SUMMARY = new XContentBuilderString("summary");
static final XContentBuilderString NODES = new XContentBuilderString("nodes");
static final XContentBuilderString TOTAL_ITERATIONS = new XContentBuilderString("total_iterations");
static final XContentBuilderString COMPLETED_ITERATIONS = new XContentBuilderString("completed_iterations");
static final XContentBuilderString TOTAL_QUERIES = new XContentBuilderString("total_queries");
static final XContentBuilderString CONCURRENCY = new XContentBuilderString("concurrency");
static final XContentBuilderString MULTIPLIER = new XContentBuilderString("multiplier");
static final XContentBuilderString AVG_WARMUP_TIME = new XContentBuilderString("avg_warmup_time");
static final XContentBuilderString STATISTICS = new XContentBuilderString("statistics");
static final XContentBuilderString MIN = new XContentBuilderString("min");
static final XContentBuilderString MAX = new XContentBuilderString("max");
static final XContentBuilderString MEAN = new XContentBuilderString("mean");
static final XContentBuilderString QPS = new XContentBuilderString("qps");
static final XContentBuilderString STD_DEV = new XContentBuilderString("std_dev");
static final XContentBuilderString MILLIS_PER_HIT = new XContentBuilderString("millis_per_hit");
static final XContentBuilderString SLOWEST = new XContentBuilderString("slowest");
static final XContentBuilderString NODE = new XContentBuilderString("node");
}
}

143
src/main/java/org/elasticsearch/action/bench/SinglePassStatistics.java
View File

@ -1,143 +0,0 @@
/*
* Licensed to Elasticsearch under one or more contributor
* license agreements. See the NOTICE file distributed with
* this work for additional information regarding copyright
* ownership. Elasticsearch licenses this file to you under
* the Apache License, Version 2.0 (the "License"); you may
* not use this file except in compliance with the License.
* You may obtain a copy of the License at
*
* http://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing,
* software distributed under the License is distributed on an
* "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
* KIND, either express or implied. See the License for the
* specific language governing permissions and limitations
* under the License.
*/
package org.elasticsearch.action.bench;
import org.elasticsearch.search.aggregations.metrics.percentiles.tdigest.TDigestState;
/**
* Utility class for accurately measuring statistical variance in cases where
* it is not possible or sensible to retain an entire data set in memory.
*
* Mean and variance algorithms taken from Donald Knuth's Art of Computer Programming, Vol 2, page 232, 3rd edition.
* Based on reference implementation http://www.johndcook.com/standard_deviation.html.
*
* For each input, the running mean and running sum-of-squares variance is calculated according
* to:
* Running mean: Mk = Mk-1+ (xk - Mk-1)/k
* Running variance: Sk = Sk-1 + (xk - Mk-1)*(xk - Mk)
*
* Percentile computations use T-Digest: https://github.com/tdunning/t-digest
*/
public class SinglePassStatistics {
private long count = 0;
private double runningMean = 0.0;
private double runningVariance = 0.0;
private long runningSum = 0;
TDigestState tdigest = new TDigestState(100.0);
/**
* Adds a new value onto the running calculation
*
* @param value New value to add to calculation
*/
public void push(long value) {
count++;
if (count == 1) {
runningMean = value;
runningVariance = 0.0;
} else {
double newMean = runningMean + (( value - runningMean) / count ); // Mk = Mk-1 + ((xk - Mk-1) / k)
double newVariance = runningVariance + ( (value - runningMean) * (value - newMean) ); // Sk = Sk-1 + (xk - Mk-1)*(xk - Mk)
runningMean = newMean;
runningVariance = newVariance;
}
runningSum += value;
tdigest.add(value);
}
/**
* Current value for the running mean
*
* @return Current value of running mean
*/
public double mean() {
return runningMean;
}
/**
* Current value for the running variance from the mean
*
* @return Current value for the running variance
*/
public double variance() {
if (count > 1) {
return runningVariance / (count - 1);
} else {
return 0.0;
}
}
/**
* Current running value of standard deviation
*
* @return Current running value of standard deviation
*/
public double stddev() {
return Math.sqrt(variance());
}
/**
* Minimum value in data set
*
* @return Minimum value in data set
*/
public long min() {
return (long) tdigest.quantile(0.0);
}
/**
* Maximum value in data set
* @return Maximum value in data set
*/
public long max() {
return (long) tdigest.quantile(1.0);
}
/**
* Total number of values seen
*
* @return Total number of values seen
*/
public long count() {
return count;
}
/**
* Running sum of all values
*
* @return Running sum of all values
*/
public long sum() {
return runningSum;
}
/**
* Running percentile
* @param q Percentile to calculate
* @return Running percentile
*/
public double percentile(double q) {
return tdigest.quantile(q);
}
}

72
src/main/java/org/elasticsearch/action/bench/TransportAbortBenchmarkAction.java
View File

@ -1,72 +0,0 @@
/*
* Licensed to Elasticsearch under one or more contributor
* license agreements. See the NOTICE file distributed with
* this work for additional information regarding copyright
* ownership. Elasticsearch licenses this file to you under
* the Apache License, Version 2.0 (the "License"); you may
* not use this file except in compliance with the License.
* You may obtain a copy of the License at
*
* http://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing,
* software distributed under the License is distributed on an
* "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
* KIND, either express or implied. See the License for the
* specific language governing permissions and limitations
* under the License.
*/
package org.elasticsearch.action.bench;
import org.elasticsearch.ElasticsearchException;
import org.elasticsearch.action.ActionListener;
import org.elasticsearch.action.support.ActionFilters;
import org.elasticsearch.action.support.master.TransportMasterNodeOperationAction;
import org.elasticsearch.cluster.ClusterService;
import org.elasticsearch.cluster.ClusterState;
import org.elasticsearch.cluster.block.ClusterBlockException;
import org.elasticsearch.cluster.block.ClusterBlockLevel;
import org.elasticsearch.common.inject.Inject;
import org.elasticsearch.common.settings.Settings;
import org.elasticsearch.threadpool.ThreadPool;
import org.elasticsearch.transport.TransportService;
/**
* Transport action for benchmark abort requests
*/
public class TransportAbortBenchmarkAction extends TransportMasterNodeOperationAction<AbortBenchmarkRequest, AbortBenchmarkResponse> {
private final BenchmarkService service;
@Inject
public TransportAbortBenchmarkAction(Settings settings, TransportService transportService, ClusterService clusterService,
ThreadPool threadPool, BenchmarkService service, ActionFilters actionFilters) {
super(settings, AbortBenchmarkAction.NAME, transportService, clusterService, threadPool, actionFilters);
this.service = service;
}
@Override
protected String executor() {
return ThreadPool.Names.GENERIC;
}
@Override
protected ClusterBlockException checkBlock(AbortBenchmarkRequest request, ClusterState state) {
return state.blocks().globalBlockedException(ClusterBlockLevel.METADATA);
}
@Override
protected AbortBenchmarkRequest newRequest() {
return new AbortBenchmarkRequest();
}
@Override
protected AbortBenchmarkResponse newResponse() {
return new AbortBenchmarkResponse();
}
@Override
protected void masterOperation(AbortBenchmarkRequest request, ClusterState state, final ActionListener<AbortBenchmarkResponse> listener) throws ElasticsearchException {
service.abortBenchmark(request.benchmarkNames(), listener);
}
}

73
src/main/java/org/elasticsearch/action/bench/TransportBenchmarkAction.java
View File

@ -1,73 +0,0 @@
/*
* Licensed to Elasticsearch under one or more contributor
* license agreements. See the NOTICE file distributed with
* this work for additional information regarding copyright
* ownership. Elasticsearch licenses this file to you under
* the Apache License, Version 2.0 (the "License"); you may
* not use this file except in compliance with the License.
* You may obtain a copy of the License at
*
* http://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing,
* software distributed under the License is distributed on an
* "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
* KIND, either express or implied. See the License for the
* specific language governing permissions and limitations
* under the License.
*/
package org.elasticsearch.action.bench;
import org.elasticsearch.ElasticsearchException;
import org.elasticsearch.action.ActionListener;
import org.elasticsearch.action.support.ActionFilters;
import org.elasticsearch.action.support.master.TransportMasterNodeOperationAction;
import org.elasticsearch.cluster.ClusterService;
import org.elasticsearch.cluster.ClusterState;
import org.elasticsearch.cluster.block.ClusterBlockException;
import org.elasticsearch.cluster.block.ClusterBlockLevel;
import org.elasticsearch.common.inject.Inject;
import org.elasticsearch.common.settings.Settings;
import org.elasticsearch.threadpool.ThreadPool;
import org.elasticsearch.transport.TransportService;
/**
* Transport action for benchmarks
*/
public class TransportBenchmarkAction extends TransportMasterNodeOperationAction<BenchmarkRequest, BenchmarkResponse> {
private final BenchmarkService service;
@Inject
public TransportBenchmarkAction(Settings settings, TransportService transportService, ClusterService clusterService,
ThreadPool threadPool, BenchmarkService service, ActionFilters actionFilters) {
super(settings, BenchmarkAction.NAME, transportService, clusterService, threadPool, actionFilters);
this.service = service;
}
@Override
protected String executor() {
return ThreadPool.Names.GENERIC;
}
@Override
protected ClusterBlockException checkBlock(BenchmarkRequest request, ClusterState state) {
return state.blocks().globalBlockedException(ClusterBlockLevel.METADATA);
}
@Override
protected BenchmarkRequest newRequest() {
return new BenchmarkRequest();
}
@Override
protected BenchmarkResponse newResponse() {
return new BenchmarkResponse();
}
@Override
protected void masterOperation(BenchmarkRequest request, ClusterState state, ActionListener<BenchmarkResponse> listener) throws ElasticsearchException {
service.startBenchmark(request, listener);
}
}

75
src/main/java/org/elasticsearch/action/bench/TransportBenchmarkStatusAction.java
View File

@ -1,75 +0,0 @@
/*
* Licensed to Elasticsearch under one or more contributor
* license agreements. See the NOTICE file distributed with
* this work for additional information regarding copyright
* ownership. Elasticsearch licenses this file to you under
* the Apache License, Version 2.0 (the "License"); you may
* not use this file except in compliance with the License.
* You may obtain a copy of the License at
*
* http://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing,
* software distributed under the License is distributed on an
* "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
* KIND, either express or implied. See the License for the
* specific language governing permissions and limitations
* under the License.
*/
package org.elasticsearch.action.bench;
import org.elasticsearch.ElasticsearchException;
import org.elasticsearch.action.ActionListener;
import org.elasticsearch.action.admin.cluster.state.ClusterStateRequest;
import org.elasticsearch.action.support.ActionFilters;
import org.elasticsearch.cluster.ClusterService;
import org.elasticsearch.cluster.ClusterState;
import org.elasticsearch.cluster.block.ClusterBlockException;
import org.elasticsearch.cluster.block.ClusterBlockLevel;
import org.elasticsearch.common.inject.Inject;
import org.elasticsearch.common.settings.Settings;
import org.elasticsearch.threadpool.ThreadPool;
import org.elasticsearch.transport.TransportService;
import org.elasticsearch.action.support.master.TransportMasterNodeOperationAction;
/**
* Transport action for benchmark status requests
*/
public class TransportBenchmarkStatusAction extends TransportMasterNodeOperationAction<BenchmarkStatusRequest, BenchmarkStatusResponse> {
private final BenchmarkService service;
@Inject
public TransportBenchmarkStatusAction(Settings settings, TransportService transportService, ClusterService clusterService,
ThreadPool threadPool, BenchmarkService service, ActionFilters actionFilters) {
super(settings, BenchmarkStatusAction.NAME, transportService, clusterService, threadPool, actionFilters);
this.service = service;
}
@Override
protected String executor() {
return ThreadPool.Names.GENERIC;
}
@Override
protected ClusterBlockException checkBlock(BenchmarkStatusRequest request, ClusterState state) {
return state.blocks().globalBlockedException(ClusterBlockLevel.METADATA);
}
@Override
protected BenchmarkStatusRequest newRequest() {
return new BenchmarkStatusRequest();
}
@Override
protected BenchmarkStatusResponse newResponse() {
return new BenchmarkStatusResponse();
}
@Override
protected void masterOperation(BenchmarkStatusRequest request, ClusterState state, ActionListener<BenchmarkStatusResponse> listener)
throws ElasticsearchException {
service.listBenchmarks(request, listener);
}
}

36
src/main/java/org/elasticsearch/client/Client.java
View File

@ -20,7 +20,6 @@
package org.elasticsearch.client;
import org.elasticsearch.action.*;
import org.elasticsearch.action.bench.*;
import org.elasticsearch.action.bulk.BulkRequest;
import org.elasticsearch.action.bulk.BulkRequestBuilder;
import org.elasticsearch.action.bulk.BulkResponse;
@ -683,41 +682,6 @@ public interface Client extends ElasticsearchClient<Client>, Releasable {
*/
void clearScroll(ClearScrollRequest request, ActionListener<ClearScrollResponse> listener);
/**
* Runs a benchmark on the server
*/
void bench(BenchmarkRequest request, ActionListener<BenchmarkResponse> listener);
/**
* Runs a benchmark on the server
*/
ActionFuture<BenchmarkResponse> bench(BenchmarkRequest request);
/**
* Runs a benchmark on the server
*/
BenchmarkRequestBuilder prepareBench(String... indices);
/**
* Aborts a benchmark run on the server
*/
void abortBench(AbortBenchmarkRequest request, ActionListener<AbortBenchmarkResponse> listener);
/**
* Aborts a benchmark run on the server
*/
AbortBenchmarkRequestBuilder prepareAbortBench(String... benchmarkNames);
/**
* Reports on status of actively running benchmarks
*/
void benchStatus(BenchmarkStatusRequest request, ActionListener<BenchmarkStatusResponse> listener);
/**
* Reports on status of actively running benchmarks
*/
BenchmarkStatusRequestBuilder prepareBenchStatus();
/**
* Returns this clients settings
*/

36
src/main/java/org/elasticsearch/client/support/AbstractClient.java
View File

@ -20,7 +20,6 @@
package org.elasticsearch.client.support;
import org.elasticsearch.action.*;
import org.elasticsearch.action.bench.*;
import org.elasticsearch.action.bulk.BulkAction;
import org.elasticsearch.action.bulk.BulkRequest;
import org.elasticsearch.action.bulk.BulkRequestBuilder;
@ -551,39 +550,4 @@ public abstract class AbstractClient implements Client {
public ClearScrollRequestBuilder prepareClearScroll() {
return new ClearScrollRequestBuilder(this);
}
@Override
public void bench(BenchmarkRequest request, ActionListener<BenchmarkResponse> listener) {
execute(BenchmarkAction.INSTANCE, request, listener);
}
@Override
public ActionFuture<BenchmarkResponse> bench(BenchmarkRequest request) {
return execute(BenchmarkAction.INSTANCE, request);
}
@Override
public BenchmarkRequestBuilder prepareBench(String... indices) {
return new BenchmarkRequestBuilder(this, indices);
}
@Override
public void abortBench(AbortBenchmarkRequest request, ActionListener<AbortBenchmarkResponse> listener) {
execute(AbortBenchmarkAction.INSTANCE, request, listener);
}
@Override
public AbortBenchmarkRequestBuilder prepareAbortBench(String... benchmarkNames) {
return new AbortBenchmarkRequestBuilder(this).setBenchmarkNames(benchmarkNames);
}
@Override
public void benchStatus(BenchmarkStatusRequest request, ActionListener<BenchmarkStatusResponse> listener) {
execute(BenchmarkStatusAction.INSTANCE, request, listener);
}
@Override
public BenchmarkStatusRequestBuilder prepareBenchStatus() {
return new BenchmarkStatusRequestBuilder(this);
}
}

13
src/main/java/org/elasticsearch/client/transport/TransportClient.java
View File

@ -23,9 +23,6 @@ import com.google.common.collect.ImmutableList;
import org.elasticsearch.ElasticsearchException;
import org.elasticsearch.Version;
import org.elasticsearch.action.*;
import org.elasticsearch.action.bench.BenchmarkRequest;
import org.elasticsearch.action.bench.BenchmarkRequestBuilder;
import org.elasticsearch.action.bench.BenchmarkResponse;
import org.elasticsearch.action.bulk.BulkRequest;
import org.elasticsearch.action.bulk.BulkResponse;
import org.elasticsearch.action.count.CountRequest;
@ -481,14 +478,4 @@ public class TransportClient extends AbstractClient {
public void explain(ExplainRequest request, ActionListener<ExplainResponse> listener) {
internalClient.explain(request, listener);
}
@Override
public void bench(BenchmarkRequest request, ActionListener<BenchmarkResponse> listener) {
internalClient.bench(request, listener);
}
@Override
public BenchmarkRequestBuilder prepareBench(String... indices) {
return internalClient.prepareBench(indices);
}
}

2
src/main/java/org/elasticsearch/node/Node.java
View File

@ -24,7 +24,6 @@ import org.elasticsearch.ElasticsearchException;
import org.elasticsearch.ElasticsearchIllegalStateException;
import org.elasticsearch.Version;
import org.elasticsearch.action.ActionModule;
import org.elasticsearch.action.bench.BenchmarkModule;
import org.elasticsearch.cache.recycler.PageCacheRecycler;
import org.elasticsearch.cache.recycler.PageCacheRecyclerModule;
import org.elasticsearch.client.Client;
@ -197,7 +196,6 @@ public class Node implements Releasable {
modules.add(new ResourceWatcherModule());
modules.add(new RepositoriesModule());
modules.add(new TribeModule());
modules.add(new BenchmarkModule(settings));
injector = modules.createInjector();

3
src/main/java/org/elasticsearch/rest/action/RestActionModule.java
View File

@ -80,7 +80,6 @@ import org.elasticsearch.rest.action.admin.indices.warmer.delete.RestDeleteWarme
import org.elasticsearch.rest.action.admin.indices.warmer.get.RestGetWarmerAction;
import org.elasticsearch.rest.action.admin.indices.warmer.put.RestPutWarmerAction;
import org.elasticsearch.rest.action.admin.indices.recovery.RestRecoveryAction;
import org.elasticsearch.rest.action.bench.RestBenchAction;
import org.elasticsearch.rest.action.bulk.RestBulkAction;
import org.elasticsearch.rest.action.cat.*;
import org.elasticsearch.rest.action.delete.RestDeleteAction;
@ -220,8 +219,6 @@ public class RestActionModule extends AbstractModule {
bind(RestExplainAction.class).asEagerSingleton();
bind(RestRecoveryAction.class).asEagerSingleton();
// Benchmark API
bind(RestBenchAction.class).asEagerSingleton();
// Templates API
bind(RestGetSearchTemplateAction.class).asEagerSingleton();

381
src/main/java/org/elasticsearch/rest/action/bench/RestBenchAction.java
View File

@ -1,381 +0,0 @@
/*
* Licensed to Elasticsearch under one or more contributor
* license agreements. See the NOTICE file distributed with
* this work for additional information regarding copyright
* ownership. Elasticsearch licenses this file to you under
* the Apache License, Version 2.0 (the "License"); you may
* not use this file except in compliance with the License.
* You may obtain a copy of the License at
*
* http://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing,
* software distributed under the License is distributed on an
* "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
* KIND, either express or implied. See the License for the
* specific language governing permissions and limitations
* under the License.
*/
package org.elasticsearch.rest.action.bench;
import com.google.common.primitives.Doubles;
import org.elasticsearch.ElasticsearchParseException;
import org.elasticsearch.action.bench.*;
import org.elasticsearch.action.search.SearchRequest;
import org.elasticsearch.action.search.SearchType;
import org.elasticsearch.client.Client;
import org.elasticsearch.common.Strings;
import org.elasticsearch.common.bytes.BytesReference;
import org.elasticsearch.common.inject.Inject;
import org.elasticsearch.common.settings.Settings;
import org.elasticsearch.common.xcontent.XContent;
import org.elasticsearch.common.xcontent.XContentBuilder;
import org.elasticsearch.common.xcontent.XContentFactory;
import org.elasticsearch.common.xcontent.XContentParser;
import org.elasticsearch.rest.*;
import org.elasticsearch.rest.action.admin.indices.cache.clear.RestClearIndicesCacheAction;
import org.elasticsearch.rest.action.support.AcknowledgedRestListener;
import org.elasticsearch.rest.action.support.RestBuilderListener;
import java.io.IOException;
import java.util.ArrayList;
import java.util.List;
import static org.elasticsearch.common.xcontent.json.JsonXContent.contentBuilder;
import static org.elasticsearch.rest.RestRequest.Method.*;
import static org.elasticsearch.rest.RestStatus.*;
/**
* REST handler for benchmark actions.
*/
public class RestBenchAction extends BaseRestHandler {
@Inject
public RestBenchAction(Settings settings, RestController controller, Client client) {
super(settings, controller, client);
// List active benchmarks
controller.registerHandler(GET, "/_bench", this);
controller.registerHandler(GET, "/{index}/_bench", this);
controller.registerHandler(GET, "/{index}/{type}/_bench", this);
// Submit benchmark
controller.registerHandler(PUT, "/_bench", this);
controller.registerHandler(PUT, "/{index}/_bench", this);
controller.registerHandler(PUT, "/{index}/{type}/_bench", this);
// Abort benchmark
controller.registerHandler(POST, "/_bench/abort/{name}", this);
}
@Override
public void handleRequest(final RestRequest request, final RestChannel channel, final Client client) {
switch (request.method()) {
case POST:
handleAbortRequest(request, channel, client);
break;
case PUT:
handleSubmitRequest(request, channel, client);
break;
case GET:
handleStatusRequest(request, channel, client);
break;
default:
// Politely ignore methods we don't support
channel.sendResponse(new BytesRestResponse(METHOD_NOT_ALLOWED));
}
}
/**
* Reports on the status of all actively running benchmarks
*/
private void handleStatusRequest(final RestRequest request, final RestChannel channel, final Client client) {
BenchmarkStatusRequest benchmarkStatusRequest = new BenchmarkStatusRequest();
client.benchStatus(benchmarkStatusRequest, new RestBuilderListener<BenchmarkStatusResponse>(channel) {
@Override
public RestResponse buildResponse(BenchmarkStatusResponse response, XContentBuilder builder) throws Exception {
builder.startObject();
response.toXContent(builder, request);
builder.endObject();
return new BytesRestResponse(OK, builder);
}
});
}
/**
* Aborts an actively running benchmark
*/
private void handleAbortRequest(final RestRequest request, final RestChannel channel, final Client client) {
final String[] benchmarkNames = Strings.splitStringByCommaToArray(request.param("name"));
AbortBenchmarkRequest abortBenchmarkRequest = new AbortBenchmarkRequest(benchmarkNames);
client.abortBench(abortBenchmarkRequest, new AcknowledgedRestListener<AbortBenchmarkResponse>(channel));
}
/**
* Submits a benchmark for execution
*/
private void handleSubmitRequest(final RestRequest request, final RestChannel channel, final Client client) {
String[] indices = Strings.splitStringByCommaToArray(request.param("index"));
String[] types = Strings.splitStringByCommaToArray(request.param("type"));
final BenchmarkRequest benchmarkRequest;
try {
BenchmarkRequestBuilder builder = new BenchmarkRequestBuilder(client);
builder.setVerbose(request.paramAsBoolean("verbose", false));
benchmarkRequest = parse(builder, request.content(), request.contentUnsafe());
benchmarkRequest.cascadeGlobalSettings(); // Make sure competitors inherit global settings
benchmarkRequest.applyLateBoundSettings(indices, types); // Some settings cannot be applied until after parsing
Exception ex = benchmarkRequest.validate();
if (ex != null) {
throw ex;
}
benchmarkRequest.listenerThreaded(false);
} catch (Exception e) {
logger.debug("failed to parse search request parameters", e);
try {
channel.sendResponse(new BytesRestResponse(BAD_REQUEST, contentBuilder().startObject().field("error", e.getMessage()).endObject()));
} catch (IOException e1) {
logger.error("Failed to send failure response", e1);
}
return;
}
client.bench(benchmarkRequest, new RestBuilderListener<BenchmarkResponse>(channel) {
@Override
public RestResponse buildResponse(BenchmarkResponse response, XContentBuilder builder) throws Exception {
builder.startObject();
response.toXContent(builder, request);
builder.endObject();
return new BytesRestResponse(OK, builder);
}
});
}
public static BenchmarkRequest parse(BenchmarkRequestBuilder builder, BytesReference data, boolean contentUnsafe) throws Exception {
XContent xContent = XContentFactory.xContent(data);
XContentParser p = xContent.createParser(data);
XContentParser.Token token = p.nextToken();
assert token == XContentParser.Token.START_OBJECT;
String fieldName = null;
while ((token = p.nextToken()) != XContentParser.Token.END_OBJECT) {
switch (token) {
case START_ARRAY:
if ("requests".equals(fieldName)) {
while ((token = p.nextToken()) != XContentParser.Token.END_ARRAY) {
assert token == XContentParser.Token.START_OBJECT;
XContentBuilder payloadBuilder = XContentFactory.contentBuilder(p.contentType()).copyCurrentStructure(p);
SearchRequest req = new SearchRequest();
req.source(payloadBuilder.bytes(), contentUnsafe);
builder.addSearchRequest(req);
}
} else if ("competitors".equals(fieldName)) {
while (p.nextToken() != XContentParser.Token.END_ARRAY) {
builder.addCompetitor(parse(p, contentUnsafe));
}
} else if ("percentiles".equals(fieldName)) {
List<Double> percentiles = new ArrayList<>();
while (p.nextToken() != XContentParser.Token.END_ARRAY) {
percentiles.add(p.doubleValue());
}
builder.setPercentiles(Doubles.toArray(percentiles));
} else {
throw new ElasticsearchParseException("Failed parsing array field [" + fieldName + "] field is not recognized");
}
break;
case START_OBJECT:
if ("clear_caches".equals(fieldName)) {
BenchmarkSettings.ClearCachesSettings clearCachesSettings = new BenchmarkSettings.ClearCachesSettings();
builder.setClearCachesSettings(clearCachesSettings);
parseClearCaches(p, clearCachesSettings);
} else {
throw new ElasticsearchParseException("Failed parsing object field [" + fieldName + "] field is not recognized");
}
break;
case FIELD_NAME:
fieldName = p.text();
break;
case VALUE_NUMBER:
if ("num_executor_nodes".equals(fieldName)) {
builder.setNumExecutorNodes(p.intValue());
} else if ("iterations".equals(fieldName)) {
builder.setIterations(p.intValue());
} else if ("concurrency".equals(fieldName)) {
builder.setConcurrency(p.intValue());
} else if ("multiplier".equals(fieldName)) {
builder.setMultiplier(p.intValue());
} else if ("num_slowest".equals(fieldName)) {
builder.setNumSlowest(p.intValue());
} else {
throw new ElasticsearchParseException("Failed parsing numeric field [" + fieldName + "] field is not recognized");
}
break;
case VALUE_BOOLEAN:
if ("warmup".equals(fieldName)) {
builder.setWarmup(p.booleanValue());
} else if ("clear_caches".equals(fieldName)) {
if (p.booleanValue()) {
throw new ElasticsearchParseException("Failed parsing field [" + fieldName + "] must specify which caches to clear");
} else {
builder.setAllowCacheClearing(false);
}
} else {
throw new ElasticsearchParseException("Failed parsing boolean field [" + fieldName + "] field is not recognized");
}
break;
case VALUE_STRING:
if ("name".equals(fieldName)) {
builder.setBenchmarkId(p.text());
} else {
throw new ElasticsearchParseException("Failed parsing string field [" + fieldName + "] field is not recognized");
}
break;
default:
throw new ElasticsearchParseException("Failed parsing " + token.name() + " field [" + fieldName + "] field is not recognized");
}
}
return builder.request();
}
private static BenchmarkCompetitorBuilder parse(XContentParser p, boolean contentUnsafe) throws Exception {
XContentParser.Token token = p.currentToken();
BenchmarkCompetitorBuilder builder = new BenchmarkCompetitorBuilder();
assert token == XContentParser.Token.START_OBJECT;
String fieldName = null;
while ((token = p.nextToken()) != XContentParser.Token.END_OBJECT) {
switch (token) {
case START_ARRAY:
if ("requests".equals(fieldName)) {
while ((token = p.nextToken()) != XContentParser.Token.END_ARRAY) {
assert token == XContentParser.Token.START_OBJECT;
XContentBuilder payloadBuilder = XContentFactory.contentBuilder(p.contentType()).copyCurrentStructure(p);
SearchRequest req = new SearchRequest();
req.source(payloadBuilder.bytes(), contentUnsafe);
builder.addSearchRequest(req);
}
} else if ("indices".equals(fieldName)) {
List<String> perCompetitorIndices = new ArrayList<>();
while ((token = p.nextToken()) != XContentParser.Token.END_ARRAY) {
if (token == XContentParser.Token.VALUE_STRING) {
perCompetitorIndices.add(p.text());
} else {
throw new ElasticsearchParseException("Failed parsing array field [" + fieldName + "] expected string values but got: " + token);
}
}
builder.setIndices(perCompetitorIndices.toArray(new String[perCompetitorIndices.size()]));
} else if ("types".equals(fieldName)) {
List<String> perCompetitorTypes = new ArrayList<>();
while ((token = p.nextToken()) != XContentParser.Token.END_ARRAY) {
if (token == XContentParser.Token.VALUE_STRING) {
perCompetitorTypes.add(p.text());
} else {
throw new ElasticsearchParseException("Failed parsing array field [" + fieldName + "] expected string values but got: " + token);
}
}
builder.setTypes(perCompetitorTypes.toArray(new String[perCompetitorTypes.size()]));
} else {
throw new ElasticsearchParseException("Failed parsing array field [" + fieldName + "] field is not recognized");
}
break;
case START_OBJECT:
if ("clear_caches".equals(fieldName)) {
BenchmarkSettings.ClearCachesSettings clearCachesSettings = new BenchmarkSettings.ClearCachesSettings();
builder.setClearCachesSettings(clearCachesSettings);
parseClearCaches(p, clearCachesSettings);
} else {
throw new ElasticsearchParseException("Failed parsing object field [" + fieldName + "] field is not recognized");
}
break;
case FIELD_NAME:
fieldName = p.text();
break;
case VALUE_NUMBER:
if ("multiplier".equals(fieldName)) {
builder.setMultiplier(p.intValue());
} else if ("num_slowest".equals(fieldName)) {
builder.setNumSlowest(p.intValue());
} else if ("iterations".equals(fieldName)) {
builder.setIterations(p.intValue());
} else if ("concurrency".equals(fieldName)) {
builder.setConcurrency(p.intValue());
} else {
throw new ElasticsearchParseException("Failed parsing numeric field [" + fieldName + "] field is not recognized");
}
break;
case VALUE_BOOLEAN:
if ("warmup".equals(fieldName)) {
builder.setWarmup(p.booleanValue());
} else if ("clear_caches".equals(fieldName)) {
if (p.booleanValue()) {
throw new ElasticsearchParseException("Failed parsing field [" + fieldName + "] must specify which caches to clear");
} else {
builder.setAllowCacheClearing(false);
}
} else {
throw new ElasticsearchParseException("Failed parsing boolean field [" + fieldName + "] field is not recognized");
}
break;
case VALUE_STRING:
if ("name".equals(fieldName)) {
builder.setName(p.text());
} else if ("search_type".equals(fieldName) || "searchType".equals(fieldName)) {
builder.setSearchType(SearchType.fromString(p.text()));
} else {
throw new ElasticsearchParseException("Failed parsing string field [" + fieldName + "] field is not recognized");
}
break;
default:
throw new ElasticsearchParseException("Failed parsing " + token.name() + " field [" + fieldName + "] field is not recognized");
}
}
return builder;
}
private static void parseClearCaches(XContentParser p, BenchmarkSettings.ClearCachesSettings clearCachesSettings) throws Exception {
XContentParser.Token token;
String fieldName = null;
while ((token = p.nextToken()) != XContentParser.Token.END_OBJECT) {
switch (token) {
case START_OBJECT:
break;
case VALUE_BOOLEAN:
if (RestClearIndicesCacheAction.Fields.FILTER.match(fieldName)) {
clearCachesSettings.filterCache(p.booleanValue());
} else if (RestClearIndicesCacheAction.Fields.FIELD_DATA.match(fieldName)) {
clearCachesSettings.fieldDataCache(p.booleanValue());
} else if (RestClearIndicesCacheAction.Fields.ID.match(fieldName)) {
clearCachesSettings.idCache(p.booleanValue());
} else if (RestClearIndicesCacheAction.Fields.RECYCLER.match(fieldName)) {
clearCachesSettings.recycler(p.booleanValue());
} else {
throw new ElasticsearchParseException("Failed parsing " + token.name() + " field [" + fieldName + "] field is not recognized");
}
break;
case START_ARRAY:
List<String> fields = new ArrayList<>();
while ((token = p.nextToken()) != XContentParser.Token.END_ARRAY) {
fields.add(p.text());
}
if (RestClearIndicesCacheAction.Fields.FIELDS.match(fieldName)) {
clearCachesSettings.fields(fields.toArray(new String[fields.size()]));
} else if (RestClearIndicesCacheAction.Fields.FILTER_KEYS.match(fieldName)) {
clearCachesSettings.filterKeys(fields.toArray(new String[fields.size()]));
} else {
throw new ElasticsearchParseException("Failed parsing " + token.name() + " field [" + fieldName + "] field is not recognized");
}
break;
case FIELD_NAME:
fieldName = p.text();
break;
default:
throw new ElasticsearchParseException("Failed parsing " + token.name() + " field [" + fieldName + "] field is not recognized");
}
}
}
}

447
src/test/java/org/elasticsearch/action/bench/BenchmarkIntegrationTest.java
View File

@ -1,447 +0,0 @@
/*
* Licensed to Elasticsearch under one or more contributor
* license agreements. See the NOTICE file distributed with
* this work for additional information regarding copyright
* ownership. Elasticsearch licenses this file to you under
* the Apache License, Version 2.0 (the "License"); you may
* not use this file except in compliance with the License.
* You may obtain a copy of the License at
*
* http://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing,
* software distributed under the License is distributed on an
* "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
* KIND, either express or implied. See the License for the
* specific language governing permissions and limitations
* under the License.
*/
package org.elasticsearch.action.bench;
import com.google.common.base.Predicate;
import org.apache.lucene.util.English;
import org.elasticsearch.action.ActionFuture;
import org.elasticsearch.action.index.IndexRequestBuilder;
import org.elasticsearch.action.search.SearchRequest;
import org.elasticsearch.common.Strings;
import org.elasticsearch.common.settings.ImmutableSettings;
import org.elasticsearch.common.settings.Settings;
import org.elasticsearch.index.query.FilterBuilders;
import org.elasticsearch.index.query.functionscore.script.ScriptScoreFunctionBuilder;
import org.elasticsearch.script.groovy.GroovyScriptEngineService;
import org.elasticsearch.test.ElasticsearchIntegrationTest;
import org.junit.After;
import org.junit.Before;
import org.junit.Ignore;
import org.junit.Test;
import java.util.*;
import java.util.concurrent.CountDownLatch;
import static org.elasticsearch.client.Requests.searchRequest;
import static org.elasticsearch.index.query.QueryBuilders.functionScoreQuery;
import static org.elasticsearch.index.query.functionscore.ScoreFunctionBuilders.scriptFunction;
import static org.elasticsearch.search.builder.SearchSourceBuilder.searchSource;
import static org.elasticsearch.test.hamcrest.ElasticsearchAssertions.assertAcked;
import static org.hamcrest.Matchers.*;
/**
* Integration tests for benchmark API
*/
@ElasticsearchIntegrationTest.ClusterScope(scope = ElasticsearchIntegrationTest.Scope.SUITE)
@Ignore
public class BenchmarkIntegrationTest extends ElasticsearchIntegrationTest {
private static final String BENCHMARK_NAME = "test_benchmark";
private static final String BENCHMARK_NAME_WILDCARD = "test_*";
private static final String COMPETITOR_PREFIX = "competitor_";
private static final String INDEX_PREFIX = "test_index_";
private static final String INDEX_TYPE = "test_type";
private int numExecutorNodes = 0;
private Map<String, BenchmarkSettings> competitionSettingsMap;
private String[] indices = Strings.EMPTY_ARRAY;
private HashMap<Integer, Boolean> benchNodes = new HashMap<>();
@Override
protected synchronized Settings nodeSettings(int nodeOrdinal) {
if (nodeOrdinal == 0) { // at least one
return ImmutableSettings.builder().put(super.nodeSettings(nodeOrdinal))
.put("node.bench", true).put(GroovyScriptEngineService.GROOVY_SCRIPT_SANDBOX_ENABLED, false).build();
} else {
if (benchNodes.containsKey(nodeOrdinal)) {
return ImmutableSettings.builder().put(super.nodeSettings(nodeOrdinal))
.put("node.bench", benchNodes.get(nodeOrdinal)).put(GroovyScriptEngineService.GROOVY_SCRIPT_SANDBOX_ENABLED, false).build();
} else {
boolean b = randomBoolean();
benchNodes.put(nodeOrdinal, b);
return ImmutableSettings.builder().put(super.nodeSettings(nodeOrdinal))
.put("node.bench", b).put(GroovyScriptEngineService.GROOVY_SCRIPT_SANDBOX_ENABLED, false).build();
}
}
}
@After
public void afterBenchmarkIntegrationTests() throws Exception {
final BenchmarkStatusResponse statusResponse = client().prepareBenchStatus().execute().actionGet();
assertThat("Some benchmarks are still running", statusResponse.benchmarkResponses(), is(empty()));
}
@Before
public void beforeBenchmarkIntegrationTests() throws Exception {
waitForTestLatch = null;
waitForQuery = null;
numExecutorNodes = internalCluster().numBenchNodes();
competitionSettingsMap = new HashMap<>();
logger.info("--> indexing random data");
indices = randomData();
}
@Test
public void testSubmitBenchmark() throws Exception {
final int iters = between(1, 3); // we run this more than once to make sure metadata is cleaned up propperly
for (int i = 0; i < iters ; i++) {
final BenchmarkRequest request =
BenchmarkTestUtil.randomRequest(client(), indices, numExecutorNodes, competitionSettingsMap);
logger.info("--> Submitting benchmark - competitors [{}] iterations [{}]", request.competitors().size(),
request.settings().iterations());
final BenchmarkResponse response = client().bench(request).actionGet();
assertThat(response, notNullValue());
assertThat(response.state(), equalTo(BenchmarkResponse.State.COMPLETE));
assertFalse(response.hasErrors());
assertThat(response.benchmarkName(), equalTo(BENCHMARK_NAME));
assertThat(response.competitionResults().size(), equalTo(request.competitors().size()));
for (CompetitionResult result : response.competitionResults().values()) {
assertThat(result.nodeResults().size(), equalTo(numExecutorNodes));
validateCompetitionResult(result, competitionSettingsMap.get(result.competitionName()), true);
}
}
}
@Test
public void testListBenchmarks() throws Exception {
SearchRequest searchRequest = prepareBlockingScriptQuery();
final BenchmarkRequest request =
BenchmarkTestUtil.randomRequest(client(), indices, numExecutorNodes, competitionSettingsMap, searchRequest);
logger.info("--> Submitting benchmark - competitors [{}] iterations [{}]", request.competitors().size(),
request.settings().iterations());
final ActionFuture<BenchmarkResponse> future = client().bench(request);
try {
waitForQuery.await();
final BenchmarkStatusResponse statusResponse = client().prepareBenchStatus().execute().actionGet();
waitForTestLatch.countDown();
assertThat(statusResponse.benchmarkResponses().size(), equalTo(1));
for (BenchmarkResponse benchmarkResponse : statusResponse.benchmarkResponses()) {
assertThat(benchmarkResponse.benchmarkName(), equalTo(BENCHMARK_NAME));
assertThat(benchmarkResponse.state(), equalTo(BenchmarkResponse.State.RUNNING));
assertFalse(benchmarkResponse.hasErrors());
for (CompetitionResult result : benchmarkResponse.competitionResults().values()) {
assertThat(result.nodeResults().size(), lessThanOrEqualTo(numExecutorNodes));
validateCompetitionResult(result, competitionSettingsMap.get(result.competitionName()), false);
}
}
} finally {
if (waitForTestLatch.getCount() == 1) {
waitForTestLatch.countDown();
}
client().prepareAbortBench(BENCHMARK_NAME).get();
// Confirm that there are no active benchmarks in the cluster
assertThat(client().prepareBenchStatus().execute().actionGet().totalActiveBenchmarks(), equalTo(0));
assertThat(waitForTestLatch.getCount(), is(0l));
}
// Confirm that benchmark was indeed aborted
assertThat(future.get().state(), isOneOf(BenchmarkResponse.State.ABORTED, BenchmarkResponse.State.COMPLETE));
}
public static CountDownLatch waitForTestLatch;
public static CountDownLatch waitForQuery;
private SearchRequest prepareBlockingScriptQuery() {
/* Chuck Norris back in the house!! - this is super evil but the only way at this
point to ensure we actually call abort / list while a benchmark is executing
without doing busy waiting etc. This Script calls the two static latches above and this test
will not work if somebody messes around with them but it's much faster and less resource intensive / hardware
dependent to run massive benchmarks and do busy waiting. */
internalCluster(); // mark that we need a JVM local cluster!
waitForQuery = new CountDownLatch(1);
waitForTestLatch = new CountDownLatch(1);
String className = "BenchmarkIntegrationTest";
ScriptScoreFunctionBuilder scriptFunction = scriptFunction("import " + this.getClass().getName() + "; \n" +
className + ".waitForQuery.countDown(); \n" + className + ".waitForTestLatch.await(); \n return 1.0;");
SearchRequest searchRequest = searchRequest().source(
searchSource()
.query(functionScoreQuery(FilterBuilders.matchAllFilter(), scriptFunction)));
return searchRequest;
}
@Test
public void testBenchmarkWithErrors() {
List<SearchRequest> reqList = new ArrayList<>();
int numQueries = scaledRandomIntBetween(20, 100);
int numErrors = scaledRandomIntBetween(1, numQueries);
final boolean containsFatal = randomBoolean();
if (containsFatal) {
ScriptScoreFunctionBuilder scriptFunction = scriptFunction("DOES NOT COMPILE - fails on any shard");
SearchRequest searchRequest = searchRequest().source(
searchSource()
.query(functionScoreQuery(FilterBuilders.matchAllFilter(), scriptFunction)));
reqList.add(searchRequest);
}
for (int i = 0; reqList.size() < numErrors; i++) {
ScriptScoreFunctionBuilder scriptFunction = scriptFunction("throw new RuntimeException();");
SearchRequest searchRequest = searchRequest().source(
searchSource()
.query(functionScoreQuery(FilterBuilders.matchAllFilter(), scriptFunction)));
reqList.add(searchRequest);
}
logger.info("--> run with [{}] errors ", numErrors);
for (int i = 0; reqList.size() < numQueries; i++) {
reqList.add(BenchmarkTestUtil.randomSearch(client(), indices));
}
Collections.shuffle(reqList, getRandom());
final BenchmarkRequest request =
BenchmarkTestUtil.randomRequest(client(),indices, numExecutorNodes, competitionSettingsMap, reqList.toArray(new SearchRequest[0]));
logger.info("--> Submitting benchmark - competitors [{}] iterations [{}]", request.competitors().size(),
request.settings().iterations());
final BenchmarkResponse response = client().bench(request).actionGet();
assertThat(response, notNullValue());
if (response.hasErrors() || containsFatal) {
assertThat(response.state(), equalTo(BenchmarkResponse.State.FAILED));
} else {
assertThat(response.state(), equalTo(BenchmarkResponse.State.COMPLETE));
for (CompetitionResult result : response.competitionResults().values()) {
assertThat(result.nodeResults().size(), equalTo(numExecutorNodes));
validateCompetitionResult(result, competitionSettingsMap.get(result.competitionName()), true);
}
}
assertThat(response.benchmarkName(), equalTo(BENCHMARK_NAME));
}
@Test
public void testAbortByPattern() throws Exception {
final int iters = between(1, 3); // we run this more than once to make sure metadata is cleaned up propperly
for (int i = 0; i < iters ; i++) {
List<BenchmarkRequest> requests = new ArrayList<>();
List<ActionFuture<BenchmarkResponse>> responses = new ArrayList<>();
SearchRequest searchRequest = prepareBlockingScriptQuery();
final int benches = between(1, 3);
String[] names = new String[benches];
for (int k = 0; k < benches; k++) {
final BenchmarkRequest request =
BenchmarkTestUtil.randomRequest(client(), indices, numExecutorNodes, competitionSettingsMap, searchRequest);
request.settings().iterations(Integer.MAX_VALUE, true); // massive amount of iterations
names[k] = BENCHMARK_NAME + Integer.toString(k);
request.benchmarkName(names[k]);
requests.add(request);
logger.info("--> Submitting benchmark - competitors [{}] iterations [{}]", request.competitors().size(),
request.settings().iterations());
}
boolean aborted = false;
for (BenchmarkRequest r : requests) {
final ActionFuture<BenchmarkResponse> benchmarkResponse = client().bench(r);
responses.add(benchmarkResponse);
}
try {
waitForQuery.await();
if (benches > 1) {
awaitBusy(new Predicate<Object>() {
@Override
public boolean apply(java.lang.Object input) {
return client().prepareBenchStatus().get().benchmarkResponses().size() == benches;
}
});
}
final String badPatternA = "*z";
final String badPatternB = "xxx";
final String[] patterns;
switch (getRandom().nextInt(3)) {
case 0:
patterns = new String [] {"*"};
break;
case 1:
patterns = new String[] {BENCHMARK_NAME_WILDCARD, badPatternA, badPatternB };
break;
case 2:
patterns = names;
break;
default:
patterns = new String [] {BENCHMARK_NAME_WILDCARD};
}
final AbortBenchmarkResponse abortResponse = client().prepareAbortBench(patterns).get();
aborted = true;
assertAcked(abortResponse);
// Confirm that there are no active benchmarks in the cluster
final BenchmarkStatusResponse statusResponse = client().prepareBenchStatus().execute().actionGet();
waitForTestLatch.countDown(); // let the queries go - we already aborted and got the status
assertThat(statusResponse.totalActiveBenchmarks(), equalTo(0));
// Confirm that benchmark was indeed aborted
for (ActionFuture<BenchmarkResponse> r : responses) {
assertThat(r.get().state(), is(BenchmarkResponse.State.ABORTED));
}
} finally {
if (waitForTestLatch.getCount() == 1) {
waitForTestLatch.countDown();
}
if (!aborted) {
client().prepareAbortBench(BENCHMARK_NAME).get();
}
assertThat(waitForTestLatch.getCount(), is(0l));
}
}
}
@Test
public void testAbortBenchmark() throws Exception {
final int iters = between(1, 3); // we run this more than once to make sure metadata is cleaned up propperly
for (int i = 0; i < iters ; i++) {
SearchRequest searchRequest = prepareBlockingScriptQuery();
final BenchmarkRequest request =
BenchmarkTestUtil.randomRequest(client(), indices, numExecutorNodes, competitionSettingsMap, searchRequest);
request.settings().iterations(Integer.MAX_VALUE, true); // massive amount of iterations
logger.info("--> Submitting benchmark - competitors [{}] iterations [{}]", request.competitors().size(),
request.settings().iterations());
boolean aborted = false;
final ActionFuture<BenchmarkResponse> benchmarkResponse = client().bench(request);
try {
waitForQuery.await();
final AbortBenchmarkResponse abortResponse =
client().prepareAbortBench(BENCHMARK_NAME).get();
aborted = true;
// Confirm that the benchmark was actually aborted and did not finish on its own
assertAcked(abortResponse);
// Confirm that there are no active benchmarks in the cluster
final BenchmarkStatusResponse statusResponse = client().prepareBenchStatus().execute().actionGet();
waitForTestLatch.countDown(); // let the queries go - we already aborted and got the status
assertThat(statusResponse.totalActiveBenchmarks(), equalTo(0));
// Confirm that benchmark was indeed aborted
assertThat(benchmarkResponse.get().state(), is(BenchmarkResponse.State.ABORTED));
} finally {
if (waitForTestLatch.getCount() == 1) {
waitForTestLatch.countDown();
}
if (!aborted) {
client().prepareAbortBench(BENCHMARK_NAME).get();
}
assertThat(waitForTestLatch.getCount(), is(0l));
}
}
}
@Test(expected = BenchmarkMissingException.class)
public void testAbortNoSuchBenchmark() throws Exception {
client().prepareAbortBench(BENCHMARK_NAME).execute().actionGet();
}
private void validateCompetitionResult(CompetitionResult result, BenchmarkSettings requestedSettings, boolean strict) {
// Validate settings
assertTrue(result.competitionName().startsWith(COMPETITOR_PREFIX));
assertThat(result.concurrency(), equalTo(requestedSettings.concurrency()));
assertThat(result.multiplier(), equalTo(requestedSettings.multiplier()));
// Validate node-level responses
for (CompetitionNodeResult nodeResult : result.nodeResults()) {
assertThat(nodeResult.nodeName(), notNullValue());
assertThat(nodeResult.totalIterations(), equalTo(requestedSettings.iterations()));
if (strict) {
assertThat(nodeResult.completedIterations(), equalTo(requestedSettings.iterations()));
final int expectedQueryCount = requestedSettings.multiplier() *
nodeResult.totalIterations() * requestedSettings.searchRequests().size();
assertThat(nodeResult.totalExecutedQueries(), equalTo(expectedQueryCount));
assertThat(nodeResult.iterations().size(), equalTo(requestedSettings.iterations()));
}
assertThat(nodeResult.warmUpTime(), greaterThanOrEqualTo(0L));
for (CompetitionIteration iteration : nodeResult.iterations()) {
// Basic sanity checks
iteration.computeStatistics();
assertThat(iteration.totalTime(), greaterThanOrEqualTo(0L));
assertThat(iteration.min(), greaterThanOrEqualTo(0L));
assertThat(iteration.max(), greaterThanOrEqualTo(iteration.min()));
assertThat(iteration.mean(), greaterThanOrEqualTo((double) iteration.min()));
assertThat(iteration.mean(), lessThanOrEqualTo((double) iteration.max()));
assertThat(iteration.queriesPerSecond(), greaterThanOrEqualTo(0.0));
assertThat(iteration.millisPerHit(), greaterThanOrEqualTo(0.0));
validatePercentiles(iteration.percentileValues());
}
}
// Validate summary statistics
final CompetitionSummary summary = result.competitionSummary();
summary.computeSummaryStatistics();
assertThat(summary, notNullValue());
assertThat(summary.getMin(), greaterThanOrEqualTo(0L));
assertThat(summary.getMax(), greaterThanOrEqualTo(summary.getMin()));
assertThat(summary.getMean(), greaterThanOrEqualTo((double) summary.getMin()));
assertThat(summary.getMean(), lessThanOrEqualTo((double) summary.getMax()));
assertThat(summary.getTotalTime(), greaterThanOrEqualTo(0L));
assertThat(summary.getQueriesPerSecond(), greaterThanOrEqualTo(0.0));
assertThat(summary.getMillisPerHit(), greaterThanOrEqualTo(0.0));
assertThat(summary.getAvgWarmupTime(), greaterThanOrEqualTo(0.0));
if (strict) {
assertThat((int) summary.getTotalIterations(), equalTo(requestedSettings.iterations() * summary.nodeResults().size()));
assertThat((int) summary.getCompletedIterations(), equalTo(requestedSettings.iterations() * summary.nodeResults().size()));
assertThat((int) summary.getTotalQueries(), equalTo(requestedSettings.iterations() * requestedSettings.multiplier() *
requestedSettings.searchRequests().size() * summary.nodeResults().size()));
validatePercentiles(summary.percentileValues);
}
}
private void validatePercentiles(Map<Double, Double> percentiles) {
int i = 0;
double last = Double.NEGATIVE_INFINITY;
for (Map.Entry<Double, Double> entry : percentiles.entrySet()) {
assertThat(entry.getKey(), equalTo(BenchmarkSettings.DEFAULT_PERCENTILES[i++]));
// This is a hedge against rounding errors. Sometimes two adjacent percentile values will
// be nearly equivalent except for some insignificant decimal places. In such cases we
// want the two values to compare as equal.
assertThat(entry.getValue(), greaterThanOrEqualTo(last - 1e-6));
last = entry.getValue();
}
}
private String[] randomData() throws Exception {
final int numIndices = scaledRandomIntBetween(1, 5);
final String[] indices = new String[numIndices];
for (int i = 0; i < numIndices; i++) {
indices[i] = INDEX_PREFIX + i;
final int numDocs = scaledRandomIntBetween(1, 100);
final IndexRequestBuilder[] docs = new IndexRequestBuilder[numDocs];
for (int j = 0; j < numDocs; j++) {
docs[j] = client().prepareIndex(indices[i], INDEX_TYPE).
setSource(BenchmarkTestUtil.TestIndexField.INT_FIELD.toString(), randomInt(),
BenchmarkTestUtil.TestIndexField.FLOAT_FIELD.toString(), randomFloat(),
BenchmarkTestUtil.TestIndexField.BOOLEAN_FIELD.toString(), randomBoolean(),
BenchmarkTestUtil.TestIndexField.STRING_FIELD.toString(), English.intToEnglish(j));
}
indexRandom(true, docs);
}
flushAndRefresh();
return indices;
}
}

62
src/test/java/org/elasticsearch/action/bench/BenchmarkNegativeTest.java
View File

@ -1,62 +0,0 @@
/*
* Licensed to Elasticsearch under one or more contributor
* license agreements. See the NOTICE file distributed with
* this work for additional information regarding copyright
* ownership. Elasticsearch licenses this file to you under
* the Apache License, Version 2.0 (the "License"); you may
* not use this file except in compliance with the License.
* You may obtain a copy of the License at
*
* http://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing,
* software distributed under the License is distributed on an
* "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
* KIND, either express or implied. See the License for the
* specific language governing permissions and limitations
* under the License.
*/
package org.elasticsearch.action.bench;
import org.elasticsearch.common.settings.ImmutableSettings;
import org.elasticsearch.common.settings.Settings;
import org.elasticsearch.test.ElasticsearchIntegrationTest;
import org.junit.Test;
import static org.hamcrest.Matchers.*;
import static org.elasticsearch.test.ElasticsearchIntegrationTest.*;
/**
* Tests for negative situations where we cannot run benchmarks
*/
@ClusterScope(scope = Scope.SUITE, enableRandomBenchNodes = false)
public class BenchmarkNegativeTest extends ElasticsearchIntegrationTest {
private static final String INDEX_NAME = "test_index";
@Override
protected Settings nodeSettings(int nodeOrdinal) {
return ImmutableSettings.builder().put(super.nodeSettings(nodeOrdinal))
.put("node.bench", false).build();
}
@Test(expected = BenchmarkNodeMissingException.class)
public void testSubmitBenchmarkNegative() {
client().bench(BenchmarkTestUtil.randomRequest(
client(), new String[] {INDEX_NAME}, internalCluster().size(), null)).actionGet();
}
public void testListBenchmarkNegative() {
final BenchmarkStatusResponse response =
client().prepareBenchStatus().execute().actionGet();
assertThat(response.benchmarkResponses().size(), equalTo(0));
}
@Test(expected = BenchmarkNodeMissingException.class)
public void testAbortBenchmarkNegative() throws Exception {
client().prepareAbortBench(BenchmarkTestUtil.BENCHMARK_NAME).execute().actionGet();
}
}

221
src/test/java/org/elasticsearch/action/bench/BenchmarkTestUtil.java
View File

@ -1,221 +0,0 @@
/*
* Licensed to Elasticsearch under one or more contributor
* license agreements. See the NOTICE file distributed with
* this work for additional information regarding copyright
* ownership. Elasticsearch licenses this file to you under
* the Apache License, Version 2.0 (the "License"); you may
* not use this file except in compliance with the License.
* You may obtain a copy of the License at
*
* http://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing,
* software distributed under the License is distributed on an
* "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
* KIND, either express or implied. See the License for the
* specific language governing permissions and limitations
* under the License.
*/
package org.elasticsearch.action.bench;
import org.elasticsearch.action.search.SearchRequest;
import org.elasticsearch.action.search.SearchRequestBuilder;
import org.elasticsearch.action.search.SearchType;
import org.elasticsearch.client.Client;
import org.elasticsearch.index.query.QueryBuilder;
import org.elasticsearch.index.query.QueryBuilders;
import java.util.Map;
import static org.elasticsearch.test.ElasticsearchIntegrationTest.between;
import static org.elasticsearch.test.ElasticsearchIntegrationTest.randomFrom;
import static org.elasticsearch.test.ElasticsearchIntegrationTest.randomBoolean;
import static org.elasticsearch.test.ElasticsearchIntegrationTest.randomAsciiOfLengthBetween;
/**
* Utilities for building randomized benchmark tests.
*/
public class BenchmarkTestUtil {
public static final String BENCHMARK_NAME = "test_benchmark";
public static final String COMPETITOR_PREFIX = "competitor_";
public static final String INDEX_TYPE = "test_type";
public static final SearchType[] searchTypes = { SearchType.DFS_QUERY_THEN_FETCH,
SearchType.QUERY_THEN_FETCH,
SearchType.QUERY_AND_FETCH,
SearchType.DFS_QUERY_AND_FETCH,
SearchType.COUNT };
public static enum TestIndexField {
INT_FIELD("int_field"),
FLOAT_FIELD("float_field"),
BOOLEAN_FIELD("boolean_field"),
STRING_FIELD("string_field");
final String name;
TestIndexField(String name) {
this.name = name;
}
@Override
public String toString() {
return name;
}
}
public static enum TestQueryType {
MATCH_ALL {
@Override
QueryBuilder getQuery() {
return QueryBuilders.matchAllQuery();
}
},
MATCH {
@Override
QueryBuilder getQuery() {
return QueryBuilders.matchQuery(TestIndexField.STRING_FIELD.toString(),
randomAsciiOfLengthBetween(1, 3));
}
},
TERM {
@Override
QueryBuilder getQuery() {
return QueryBuilders.termQuery(TestIndexField.STRING_FIELD.toString(),
randomAsciiOfLengthBetween(1, 3));
}
},
QUERY_STRING {
@Override
QueryBuilder getQuery() {
return QueryBuilders.queryStringQuery(
randomAsciiOfLengthBetween(1, 3));
}
},
WILDCARD {
@Override
QueryBuilder getQuery() {
return QueryBuilders.wildcardQuery(
TestIndexField.STRING_FIELD.toString(), randomBoolean() ? "*" : "?");
}
};
abstract QueryBuilder getQuery();
}
public static BenchmarkRequest randomRequest(Client client, String[] indices, int numExecutorNodes,
Map<String, BenchmarkSettings> competitionSettingsMap,
int lowRandomIntervalBound, int highRandomIntervalBound, SearchRequest... requests) {
final BenchmarkRequestBuilder builder = new BenchmarkRequestBuilder(client, indices);
final BenchmarkSettings settings = randomSettings(lowRandomIntervalBound, highRandomIntervalBound);
builder.setIterations(settings.iterations());
builder.setConcurrency(settings.concurrency());
builder.setMultiplier(settings.multiplier());
builder.setSearchType(settings.searchType());
builder.setWarmup(settings.warmup());
builder.setNumExecutorNodes(numExecutorNodes);
final int numCompetitors = between(lowRandomIntervalBound, highRandomIntervalBound);
for (int i = 0; i < numCompetitors; i++) {
builder.addCompetitor(randomCompetitor(client, COMPETITOR_PREFIX + i, indices,
competitionSettingsMap, lowRandomIntervalBound, highRandomIntervalBound, requests));
}
final BenchmarkRequest request = builder.request();
request.benchmarkName(BENCHMARK_NAME);
request.cascadeGlobalSettings();
request.applyLateBoundSettings(indices, new String[] { INDEX_TYPE });
return request;
}
public static BenchmarkRequest randomRequest(Client client, String[] indices, int numExecutorNodes,
Map<String, BenchmarkSettings> competitionSettingsMap, SearchRequest... requests) {
return randomRequest(client, indices, numExecutorNodes,
competitionSettingsMap, 1, 3, requests);
}
public static SearchRequest randomSearch(Client client, String[] indices) {
final SearchRequestBuilder builder = new SearchRequestBuilder(client);
builder.setIndices(indices);
builder.setTypes(INDEX_TYPE);
builder.setQuery(randomFrom(TestQueryType.values()).getQuery());
return builder.request();
}
public static BenchmarkCompetitor randomCompetitor(Client client, String name, String[] indices,
Map<String, BenchmarkSettings> competitionSettingsMap,
int lowRandomIntervalBound, int highRandomIntervalBound, SearchRequest... requests) {
final BenchmarkCompetitorBuilder builder = new BenchmarkCompetitorBuilder();
final BenchmarkSettings settings = randomSettings(lowRandomIntervalBound, highRandomIntervalBound);
builder.setClearCachesSettings(randomCacheSettings());
builder.setIterations(settings.iterations());
builder.setConcurrency(settings.concurrency());
builder.setMultiplier(settings.multiplier());
builder.setSearchType(settings.searchType());
builder.setWarmup(settings.warmup());
builder.setName(name);
if (requests != null && requests.length != 0) {
for (int i = 0; i < requests.length; i++) {
builder.addSearchRequest(requests[i]);
settings.addSearchRequest(requests[i]);
}
} else {
final int numSearches = between(lowRandomIntervalBound, highRandomIntervalBound);
for (int i = 0; i < numSearches; i++) {
final SearchRequest searchRequest = randomSearch(client, indices);
builder.addSearchRequest(searchRequest);
settings.addSearchRequest(searchRequest);
}
}
if (competitionSettingsMap != null) {
competitionSettingsMap.put(name, settings);
}
return builder.build();
}
public static BenchmarkSettings.ClearCachesSettings randomCacheSettings() {
final BenchmarkSettings.ClearCachesSettings settings = new BenchmarkSettings.ClearCachesSettings();
settings.filterCache(randomBoolean());
settings.fieldDataCache(randomBoolean());
settings.idCache(randomBoolean());
settings.recycler(randomBoolean());
if (randomBoolean()) {
final int numFieldsToClear = between(1, TestIndexField.values().length);
final String[] fields = new String[numFieldsToClear];
for (int i = 0; i < numFieldsToClear; i++) {
fields[i] = TestIndexField.values()[i].toString();
}
settings.fields(fields);
}
return settings;
}
public static BenchmarkSettings randomSettings(int lowRandomIntervalBound, int highRandomIntervalBound) {
final BenchmarkSettings settings = new BenchmarkSettings();
settings.concurrency(between(lowRandomIntervalBound, highRandomIntervalBound), true);
settings.iterations(between(lowRandomIntervalBound, highRandomIntervalBound), true);
settings.multiplier(between(1, 50), true);
settings.warmup(randomBoolean(), true);
settings.searchType(searchTypes[between(0, searchTypes.length - 1)], true);
return settings;
}
}

6
src/test/java/org/elasticsearch/client/AbstractClientHeadersTests.java
View File

@ -39,7 +39,6 @@ import org.elasticsearch.action.admin.indices.flush.FlushAction;
import org.elasticsearch.action.admin.indices.flush.FlushResponse;
import org.elasticsearch.action.admin.indices.stats.IndicesStatsAction;
import org.elasticsearch.action.admin.indices.stats.IndicesStatsResponse;
import org.elasticsearch.action.bench.*;
import org.elasticsearch.action.delete.DeleteAction;
import org.elasticsearch.action.delete.DeleteResponse;
import org.elasticsearch.action.get.GetAction;
@ -78,7 +77,7 @@ public abstract class AbstractClientHeadersTests extends ElasticsearchTestCase {
private static final GenericAction[] ACTIONS = new GenericAction[] {
// client actions
GetAction.INSTANCE, SearchAction.INSTANCE, DeleteAction.INSTANCE, DeleteIndexedScriptAction.INSTANCE,
IndexAction.INSTANCE, AbortBenchmarkAction.INSTANCE, BenchmarkAction.INSTANCE, BenchmarkStatusAction.INSTANCE,
IndexAction.INSTANCE,
// cluster admin actions
ClusterStatsAction.INSTANCE, CreateSnapshotAction.INSTANCE, NodesShutdownAction.INSTANCE, ClusterRerouteAction.INSTANCE,
@ -116,9 +115,6 @@ public abstract class AbstractClientHeadersTests extends ElasticsearchTestCase {
client.prepareDelete("idx", "type", "id").execute().addListener(new AssertingActionListener<DeleteResponse>(DeleteAction.NAME));
client.prepareDeleteIndexedScript("lang", "id").execute().addListener(new AssertingActionListener<DeleteIndexedScriptResponse>(DeleteIndexedScriptAction.NAME));
client.prepareIndex("idx", "type", "id").setSource("source").execute().addListener(new AssertingActionListener<IndexResponse>(IndexAction.NAME));
client.prepareAbortBench("bname").execute().addListener(new AssertingActionListener<AbortBenchmarkResponse>(AbortBenchmarkAction.NAME));
client.prepareBench("idx").setBenchmarkId("id").addCompetitor(new BenchmarkCompetitorBuilder().setName("name")).execute().addListener(new AssertingActionListener<BenchmarkResponse>(BenchmarkAction.NAME));
client.prepareBenchStatus().execute().addListener(new AssertingActionListener<BenchmarkStatusResponse>(BenchmarkStatusAction.NAME));
// choosing arbitrary cluster admin actions to test
client.admin().cluster().prepareClusterStats().execute().addListener(new AssertingActionListener<ClusterStatsResponse>(ClusterStatsAction.NAME));

1
src/test/java/org/elasticsearch/rest/HeadersAndContextCopyClientTests.java
View File

@ -254,7 +254,6 @@ public class HeadersAndContextCopyClientTests extends ElasticsearchTestCase {
client.prepareIndex(),
client.prepareClearScroll(),
client.prepareMultiGet(),
client.prepareBenchStatus()
};
for (ActionRequestBuilder requestBuilder : requestBuilders) {

8
src/test/java/org/elasticsearch/transport/ActionNamesTests.java
View File

@ -19,6 +19,14 @@
package org.elasticsearch.transport;
import com.google.common.collect.Lists;
import org.elasticsearch.Version;
import org.elasticsearch.action.admin.indices.get.GetIndexAction;
import org.elasticsearch.action.admin.cluster.repositories.verify.VerifyRepositoryAction;
import org.elasticsearch.action.exists.ExistsAction;
import org.elasticsearch.discovery.zen.ping.unicast.UnicastZenPing;
import org.elasticsearch.search.action.SearchServiceTransportAction;
import org.elasticsearch.repositories.VerifyNodeRepositoryAction;
import org.elasticsearch.test.ElasticsearchIntegrationTest;
import org.junit.Test;