honeymoose/OpenSearch
1
0
Fork 0
mirror of https://github.com/honeymoose/OpenSearch.git synced 2025-03-09 14:34:43 +00:00
Code Issues Packages Projects Releases Wiki Activity

[7.x] Backport ValuesSourceRegistry and related work (#54922)

Browse Source 
* Add ValuesSource Registry and associated logic (#54281)

* Remove ValuesSourceType argument to ValuesSourceAggregationBuilder (#48638)

* ValuesSourceRegistry Prototype (#48758)

* Remove generics from ValuesSource related classes (#49606)

* fix percentile aggregation tests (#50712)

* Basic thread safety for ValuesSourceRegistry (#50340)

* Remove target value type from ValuesSourceAggregationBuilder (#49943)

* Cleanup default values source type (#50992)

* CoreValuesSourceType no longer implements Writable (#51276)

* Remove genereics & hard coded ValuesSource references from Matrix Stats (#51131)

* Put values source types on fields (#51503)

* Remove VST Any (#51539)

* Rewire terms agg to use new VS registry (#51182)

Also adds some basic AggTestCases for untested code
paths (and boilerplate for future tests once the IT are
converted over)

* Wire Cardinality aggregation to work with the ValuesSourceRegistry (#51337)

* Wire Percentiles aggregator into new VS framework (#51639)

This required a bit of a refactor to percentiles itself.  Before,
the Builder would switch on the chosen algo to generate an
algo-specific factory.  This doesn't work (or at least, would be
difficult) in the new VS framework.

This refactor consolidates both factories together and introduces
a PercentilesConfig object to act as a standardized way to pass
algo-specific parameters through the factory.  This object
is then used when deciding which kind of aggregator to create

Note: CoreValuesSourceType.HISTOGRAM still lives in core, and will
be moved in a subsequent PR.

* Remove generics and target value type from MultiVSAB (#51647)

* fix checkstyle after merge (#52008)

* Plumb ValuesSourceRegistry through to QuerySearchContext (#51710)

* Convert RareTerms to new VS registry (#52166)

* Wire up Value Count (#52225)

* Wire up Max & Min aggregations (#52219)

* ValuesSource refactoring: Wire up Sum aggregation (#52571)

* ValuesSource refactoring: Wire up SigTerms aggregation (#52590)

* Soft immutability for VSConfig (#52729)

* Unmute testSupportedFieldTypes, fix Percentiles/Ranks/Terms tests (#52734)

Also fixes Percentiles which was incorrectly specified to only accept
numeric, but in fact also accepts Boolean and Date (because those are
numeric on master - thanks `testSupportedFieldTypes` for catching it!)

* VS refactoring: Wire up stats aggregation (#52891)

* ValuesSource refactoring: Wire up string_stats aggregation (#52875)

* VS refactoring: Wire up median (MAD) aggregation (#52945)

* fix valuesourcetype issue with constant_keyword field (#53041)x-pack/plugin/rollup/src/main/java/org/elasticsearch/xpack/rollup/job/RollupIndexer.java

this commit implements `getValuesSourceType` for
the ConstantKeyword field type.

master was merged into feature/extensible-values-source
introducing a new field type that was not implementing
`getValuesSourceType`.

* ValuesSource refactoring: Wire up Avg aggregation (#52752)

* Wire PercentileRanks aggregator into new VS framework  (#51693)

* Add a VSConfig resolver for aggregations not using the registry (#53038)

* Vs refactor wire up ranges and date ranges (#52918)

* Wire up geo_bounds aggregation to ValuesSourceRegistry (#53034)

This commit updates the geo_bounds aggregation to depend
on registering itself in the ValuesSourceRegistry

relates #42949.

* VS refactoring: convert Boxplot to new registry (#53132)

* Wire-up geotile_grid and geohash_grid to ValuesSourceRegistry (#53037)

This commit updates the geo*_grid aggregations to depend
on registering itself in the ValuesSourceRegistry

relates to the values-source refactoring meta issue #42949.

* Wire-up geo_centroid agg to ValuesSourceRegistry (#53040)

This commit updates the geo_centroid aggregation to depend
on registering itself in the ValuesSourceRegistry.

relates to the values-source refactoring meta issue #42949.

* Fix type tests for Missing aggregation (#53501)

* ValuesSource Refactor: move histo VSType into XPack module (#53298)

- Introduces a new API (`getBareAggregatorRegistrar()`) which allows plugins to register aggregations against existing agg definitions defined in Core.
- This moves the histogram VSType over to XPack where it belongs. `getHistogramValues()` still remains as a Core concept
- Moves the histo-specific bits over to xpack (e.g. the actual aggregator logic). This requires extra boilerplate since we need to create a new "Analytics" Percentile/Rank aggregators to deal with the histo field. Doubly-so since percentiles/ranks are extra boiler-plate'y... should be much lighter for other aggs

* Wire up DateHistogram to the ValuesSourceRegistry (#53484)

* Vs refactor parser cleanup (#53198)

Co-authored-by: Zachary Tong <polyfractal@elastic.co>
Co-authored-by: Zachary Tong <zach@elastic.co>
Co-authored-by: Christos Soulios <1561376+csoulios@users.noreply.github.com>
Co-authored-by: Tal Levy <JubBoy333@gmail.com>

* First batch of easy fixes

* Remove List.of from ValuesSourceRegistry

Note that we intend to have a follow up PR dealing with the mutability
of the registry, so I didn't even try to address that here.

* More compiler fixes

* More compiler fixes

* More compiler fixes

* Precommit is happy and so am I

* Add new Core VSTs to tests

* Disabled supported type test on SigTerms until we can backport it's fix

* fix checkstyle

* Fix test failure from semantic merge issue

* Fix some metaData->metadata replacements that got lost

* Fix list of supported types for MinAggregator

* Fix list of supported types for Avg

* remove unused import

Co-authored-by: Zachary Tong <polyfractal@elastic.co>
Co-authored-by: Zachary Tong <zach@elastic.co>
Co-authored-by: Christos Soulios <1561376+csoulios@users.noreply.github.com>
Co-authored-by: Tal Levy <JubBoy333@gmail.com>
This commit is contained in:
Mark Tozzi 2020-04-16 16:54:46 -04:00 committed by GitHub
parent 8abdf7c7d3
commit 22c55180c1
No known key found for this signature in database
GPG Key ID: 4AEE18F83AFDEB23
261 changed files with 5631 additions and 2391 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

15
client/rest-high-level/src/main/java/org/elasticsearch/client/analytics/StringStatsAggregationBuilder.java
View File

@ -30,12 +30,10 @@ import org.elasticsearch.search.aggregations.AggregationBuilder;
import org.elasticsearch.search.aggregations.AggregatorFactories.Builder;
import org.elasticsearch.search.aggregations.AggregatorFactory;
import org.elasticsearch.search.aggregations.support.CoreValuesSourceType;
import org.elasticsearch.search.aggregations.support.ValueType;
import org.elasticsearch.search.aggregations.support.ValuesSource;
import org.elasticsearch.search.aggregations.support.ValuesSource.Bytes;
import org.elasticsearch.search.aggregations.support.ValuesSourceAggregationBuilder;
import org.elasticsearch.search.aggregations.support.ValuesSourceAggregatorFactory;
import org.elasticsearch.search.aggregations.support.ValuesSourceConfig;
import org.elasticsearch.search.aggregations.support.ValuesSourceType;
import org.elasticsearch.search.builder.SearchSourceBuilder;
import java.io.IOException;
@ -52,14 +50,14 @@ import java.util.Objects;
* {@linkplain AggregationBuilder#rewrite(QueryRewriteContext)}, or
* {@linkplain AbstractAggregationBuilder#build(QueryShardContext, AggregatorFactory)}.
*/
public class StringStatsAggregationBuilder extends ValuesSourceAggregationBuilder<ValuesSource.Bytes, StringStatsAggregationBuilder> {
public class StringStatsAggregationBuilder extends ValuesSourceAggregationBuilder<StringStatsAggregationBuilder> {
public static final String NAME = "string_stats";
private static final ParseField SHOW_DISTRIBUTION_FIELD = new ParseField("show_distribution");
private boolean showDistribution = false;
public StringStatsAggregationBuilder(String name) {
super(name, CoreValuesSourceType.BYTES, ValueType.STRING);
super(name);
}
/**
@ -71,6 +69,11 @@ public class StringStatsAggregationBuilder extends ValuesSourceAggregationBuilde
return this;
}
@Override
protected ValuesSourceType defaultValueSourceType() {
return CoreValuesSourceType.BYTES;
}
@Override
public String getType() {
return NAME;
@ -92,7 +95,7 @@ public class StringStatsAggregationBuilder extends ValuesSourceAggregationBuilde
}
@Override
protected ValuesSourceAggregatorFactory<Bytes> innerBuild(QueryShardContext queryShardContext, ValuesSourceConfig<Bytes> config,
protected ValuesSourceAggregatorFactory innerBuild(QueryShardContext queryShardContext, ValuesSourceConfig config,
AggregatorFactory parent, Builder subFactoriesBuilder) throws IOException {
throw new UnsupportedOperationException();
}

3
client/rest-high-level/src/test/java/org/elasticsearch/client/RequestConvertersTests.java
View File

@ -1229,7 +1229,8 @@ public class RequestConvertersTests extends ESTestCase {
searchSourceBuilder.query(new TermQueryBuilder(randomAlphaOfLengthBetween(3, 10), randomAlphaOfLengthBetween(3, 10)));
}
if (randomBoolean()) {
searchSourceBuilder.aggregation(new TermsAggregationBuilder(randomAlphaOfLengthBetween(3, 10), ValueType.STRING)
searchSourceBuilder.aggregation(new TermsAggregationBuilder(randomAlphaOfLengthBetween(3, 10))
.userValueTypeHint(ValueType.STRING)
.field(randomAlphaOfLengthBetween(3, 10)));
}
if (randomBoolean()) {

22
client/rest-high-level/src/test/java/org/elasticsearch/client/SearchIT.java
View File

@ -266,7 +266,8 @@ public class SearchIT extends ESRestHighLevelClientTestCase {
public void testSearchWithTermsAgg() throws IOException {
SearchRequest searchRequest = new SearchRequest();
SearchSourceBuilder searchSourceBuilder = new SearchSourceBuilder();
searchSourceBuilder.aggregation(new TermsAggregationBuilder("agg1", ValueType.STRING).field("type.keyword"));
searchSourceBuilder.aggregation(new TermsAggregationBuilder("agg1").userValueTypeHint(ValueType.STRING)
.field("type.keyword"));
searchSourceBuilder.size(0);
searchRequest.source(searchSourceBuilder);
SearchResponse searchResponse = execute(searchRequest, highLevelClient()::search, highLevelClient()::searchAsync);
@ -358,7 +359,7 @@ public class SearchIT extends ESRestHighLevelClientTestCase {
public void testSearchWithTermsAndRangeAgg() throws IOException {
SearchRequest searchRequest = new SearchRequest("index");
SearchSourceBuilder searchSourceBuilder = new SearchSourceBuilder();
TermsAggregationBuilder agg = new TermsAggregationBuilder("agg1", ValueType.STRING).field("type.keyword");
TermsAggregationBuilder agg = new TermsAggregationBuilder("agg1").userValueTypeHint(ValueType.STRING).field("type.keyword");
agg.subAggregation(new RangeAggregationBuilder("subagg").field("num")
.addRange("first", 0, 30).addRange("second", 31, 200));
searchSourceBuilder.aggregation(agg);
@ -412,7 +413,7 @@ public class SearchIT extends ESRestHighLevelClientTestCase {
public void testSearchWithTermsAndWeightedAvg() throws IOException {
SearchRequest searchRequest = new SearchRequest("index");
SearchSourceBuilder searchSourceBuilder = new SearchSourceBuilder();
TermsAggregationBuilder agg = new TermsAggregationBuilder("agg1", ValueType.STRING).field("type.keyword");
TermsAggregationBuilder agg = new TermsAggregationBuilder("agg1").userValueTypeHint(ValueType.STRING).field("type.keyword");
agg.subAggregation(new WeightedAvgAggregationBuilder("subagg")
.value(new MultiValuesSourceFieldConfig.Builder().setFieldName("num").build())
.weight(new MultiValuesSourceFieldConfig.Builder().setFieldName("num2").build())
@ -538,10 +539,12 @@ public class SearchIT extends ESRestHighLevelClientTestCase {
client().performRequest(answerDoc2);
client().performRequest(new Request(HttpPost.METHOD_NAME, "/_refresh"));
TermsAggregationBuilder leafTermAgg = new TermsAggregationBuilder("top-names", ValueType.STRING)
TermsAggregationBuilder leafTermAgg = new TermsAggregationBuilder("top-names")
.userValueTypeHint(ValueType.STRING)
.field("owner.display_name.keyword").size(10);
ChildrenAggregationBuilder childrenAgg = new ChildrenAggregationBuilder("to-answers", "answer").subAggregation(leafTermAgg);
TermsAggregationBuilder termsAgg = new TermsAggregationBuilder("top-tags", ValueType.STRING).field("tags.keyword")
TermsAggregationBuilder termsAgg = new TermsAggregationBuilder("top-tags").userValueTypeHint(ValueType.STRING)
.field("tags.keyword")
.size(10).subAggregation(childrenAgg);
SearchSourceBuilder searchSourceBuilder = new SearchSourceBuilder();
searchSourceBuilder.size(0).aggregation(termsAgg);
@ -753,15 +756,18 @@ public class SearchIT extends ESRestHighLevelClientTestCase {
public void testMultiSearch_withAgg() throws Exception {
MultiSearchRequest multiSearchRequest = new MultiSearchRequest();
SearchRequest searchRequest1 = new SearchRequest("index1");
searchRequest1.source().size(0).aggregation(new TermsAggregationBuilder("name", ValueType.STRING).field("field.keyword")
searchRequest1.source().size(0).aggregation(new TermsAggregationBuilder("name").userValueTypeHint(ValueType.STRING)
.field("field.keyword")
.order(BucketOrder.key(true)));
multiSearchRequest.add(searchRequest1);
SearchRequest searchRequest2 = new SearchRequest("index2");
searchRequest2.source().size(0).aggregation(new TermsAggregationBuilder("name", ValueType.STRING).field("field.keyword")
searchRequest2.source().size(0).aggregation(new TermsAggregationBuilder("name").userValueTypeHint(ValueType.STRING)
.field("field.keyword")
.order(BucketOrder.key(true)));
multiSearchRequest.add(searchRequest2);
SearchRequest searchRequest3 = new SearchRequest("index3");
searchRequest3.source().size(0).aggregation(new TermsAggregationBuilder("name", ValueType.STRING).field("field.keyword")
searchRequest3.source().size(0).aggregation(new TermsAggregationBuilder("name").userValueTypeHint(ValueType.STRING)
.field("field.keyword")
.order(BucketOrder.key(true)));
multiSearchRequest.add(searchRequest3);

13
modules/aggs-matrix-stats/src/main/java/org/elasticsearch/search/aggregations/matrix/stats/MatrixStatsAggregationBuilder.java
View File

@ -28,23 +28,18 @@ import org.elasticsearch.search.aggregations.AggregationBuilder;
import org.elasticsearch.search.aggregations.AggregatorFactories;
import org.elasticsearch.search.aggregations.AggregatorFactory;
import org.elasticsearch.search.aggregations.support.ArrayValuesSourceAggregationBuilder;
import org.elasticsearch.search.aggregations.support.CoreValuesSourceType;
import org.elasticsearch.search.aggregations.support.ValueType;
import org.elasticsearch.search.aggregations.support.ValuesSource;
import org.elasticsearch.search.aggregations.support.ValuesSource.Numeric;
import org.elasticsearch.search.aggregations.support.ValuesSourceConfig;
import java.io.IOException;
import java.util.Map;
public class MatrixStatsAggregationBuilder
extends ArrayValuesSourceAggregationBuilder.LeafOnly<ValuesSource.Numeric, MatrixStatsAggregationBuilder> {
public class MatrixStatsAggregationBuilder extends ArrayValuesSourceAggregationBuilder.LeafOnly<MatrixStatsAggregationBuilder> {
public static final String NAME = "matrix_stats";
private MultiValueMode multiValueMode = MultiValueMode.AVG;
public MatrixStatsAggregationBuilder(String name) {
super(name, CoreValuesSourceType.NUMERIC, ValueType.NUMERIC);
super(name);
}
protected MatrixStatsAggregationBuilder(MatrixStatsAggregationBuilder clone,
@ -62,7 +57,7 @@ public class MatrixStatsAggregationBuilder
* Read from a stream.
*/
public MatrixStatsAggregationBuilder(StreamInput in) throws IOException {
super(in, CoreValuesSourceType.NUMERIC, ValueType.NUMERIC);
super(in);
}
@Override
@ -81,7 +76,7 @@ public class MatrixStatsAggregationBuilder
@Override
protected MatrixStatsAggregatorFactory innerBuild(QueryShardContext queryShardContext,
Map<String, ValuesSourceConfig<Numeric>> configs,
Map<String, ValuesSourceConfig> configs,
AggregatorFactory parent,
AggregatorFactories.Builder subFactoriesBuilder) throws IOException {
return new MatrixStatsAggregatorFactory(name, configs, multiValueMode, queryShardContext, parent, subFactoriesBuilder, metadata);

19
modules/aggs-matrix-stats/src/main/java/org/elasticsearch/search/aggregations/matrix/stats/MatrixStatsAggregatorFactory.java
View File

@ -20,6 +20,7 @@ package org.elasticsearch.search.aggregations.matrix.stats;
import org.elasticsearch.index.query.QueryShardContext;
import org.elasticsearch.search.MultiValueMode;
import org.elasticsearch.search.aggregations.AggregationExecutionException;
import org.elasticsearch.search.aggregations.Aggregator;
import org.elasticsearch.search.aggregations.AggregatorFactories;
import org.elasticsearch.search.aggregations.AggregatorFactory;
@ -29,14 +30,15 @@ import org.elasticsearch.search.aggregations.support.ValuesSourceConfig;
import org.elasticsearch.search.internal.SearchContext;
import java.io.IOException;
import java.util.HashMap;
import java.util.Map;
final class MatrixStatsAggregatorFactory extends ArrayValuesSourceAggregatorFactory<ValuesSource.Numeric> {
final class MatrixStatsAggregatorFactory extends ArrayValuesSourceAggregatorFactory {
private final MultiValueMode multiValueMode;
MatrixStatsAggregatorFactory(String name,
Map<String, ValuesSourceConfig<ValuesSource.Numeric>> configs,
Map<String, ValuesSourceConfig> configs,
MultiValueMode multiValueMode,
QueryShardContext queryShardContext,
AggregatorFactory parent,
@ -55,11 +57,20 @@ final class MatrixStatsAggregatorFactory extends ArrayValuesSourceAggregatorFact
}
@Override
protected Aggregator doCreateInternal(Map<String, ValuesSource.Numeric> valuesSources,
protected Aggregator doCreateInternal(Map<String, ValuesSource> valuesSources,
SearchContext searchContext,
Aggregator parent,
boolean collectsFromSingleBucket,
Map<String, Object> metadata) throws IOException {
return new MatrixStatsAggregator(name, valuesSources, searchContext, parent, multiValueMode, metadata);
Map<String, ValuesSource.Numeric> typedValuesSources = new HashMap<>(valuesSources.size());
for (Map.Entry<String, ValuesSource> entry : valuesSources.entrySet()) {
if (entry.getValue() instanceof ValuesSource.Numeric == false) {
throw new AggregationExecutionException("ValuesSource type " + entry.getValue().toString() +
"is not supported for aggregation " + this.name());
}
// TODO: There must be a better option than this.
typedValuesSources.put(entry.getKey(), (ValuesSource.Numeric) entry.getValue());
}
return new MatrixStatsAggregator(name, typedValuesSources, searchContext, parent, multiValueMode, metadata);
}
}

197
modules/aggs-matrix-stats/src/main/java/org/elasticsearch/search/aggregations/support/ArrayValuesSourceAggregationBuilder.java
View File

@ -18,18 +18,11 @@
*/
package org.elasticsearch.search.aggregations.support;
import org.elasticsearch.common.Nullable;
import org.elasticsearch.common.ParseField;
import org.elasticsearch.common.io.stream.StreamInput;
import org.elasticsearch.common.io.stream.StreamOutput;
import org.elasticsearch.common.xcontent.XContentBuilder;
import org.elasticsearch.index.fielddata.IndexFieldData;
import org.elasticsearch.index.fielddata.IndexGeoPointFieldData;
import org.elasticsearch.index.fielddata.IndexNumericFieldData;
import org.elasticsearch.index.mapper.MappedFieldType;
import org.elasticsearch.index.query.QueryShardContext;
import org.elasticsearch.script.Script;
import org.elasticsearch.search.DocValueFormat;
import org.elasticsearch.search.aggregations.AbstractAggregationBuilder;
import org.elasticsearch.search.aggregations.AggregationInitializationException;
import org.elasticsearch.search.aggregations.AggregatorFactories;
@ -44,19 +37,19 @@ import java.util.List;
import java.util.Map;
import java.util.Objects;
public abstract class ArrayValuesSourceAggregationBuilder<VS extends ValuesSource, AB extends ArrayValuesSourceAggregationBuilder<VS, AB>>
public abstract class ArrayValuesSourceAggregationBuilder<AB extends ArrayValuesSourceAggregationBuilder<AB>>
extends AbstractAggregationBuilder<AB> {
public static final ParseField MULTIVALUE_MODE_FIELD = new ParseField("mode");
public abstract static class LeafOnly<VS extends ValuesSource, AB extends ArrayValuesSourceAggregationBuilder<VS, AB>>
extends ArrayValuesSourceAggregationBuilder<VS, AB> {
public abstract static class LeafOnly<AB extends ArrayValuesSourceAggregationBuilder<AB>>
extends ArrayValuesSourceAggregationBuilder<AB> {
protected LeafOnly(String name, ValuesSourceType valuesSourceType, ValueType targetValueType) {
super(name, valuesSourceType, targetValueType);
protected LeafOnly(String name) {
super(name);
}
protected LeafOnly(LeafOnly<VS, AB> clone, Builder factoriesBuilder, Map<String, Object> metadata) {
protected LeafOnly(LeafOnly<AB> clone, Builder factoriesBuilder, Map<String, Object> metadata) {
super(clone, factoriesBuilder, metadata);
if (factoriesBuilder.count() > 0) {
throw new AggregationInitializationException("Aggregator [" + name + "] of type ["
@ -65,18 +58,10 @@ public abstract class ArrayValuesSourceAggregationBuilder<VS extends ValuesSourc
}
/**
* Read from a stream that does not serialize its targetValueType. This should be used by most subclasses.
* Read from a stream
*/
protected LeafOnly(StreamInput in, ValuesSourceType valuesSourceType, ValueType targetValueType) throws IOException {
super(in, valuesSourceType, targetValueType);
}
/**
* Read an aggregation from a stream that serializes its targetValueType. This should only be used by subclasses that override
* {@link #serializeTargetValueType()} to return true.
*/
protected LeafOnly(StreamInput in, ValuesSourceType valuesSourceType) throws IOException {
super(in, valuesSourceType);
protected LeafOnly(StreamInput in) throws IOException {
super(in);
}
@Override
@ -91,49 +76,32 @@ public abstract class ArrayValuesSourceAggregationBuilder<VS extends ValuesSourc
}
}
private final ValuesSourceType valuesSourceType;
private final ValueType targetValueType;
private List<String> fields = Collections.emptyList();
private ValueType valueType = null;
/* The parser doesn't support setting userValueTypeHint (aka valueType), but we do serialize and deserialize it, so keeping it around
for now so as to not break BWC. Future refactors should feel free to remove this field. --Tozzi 2020-01-16
*/
private ValueType userValueTypeHint = null;
private String format = null;
private Object missing = null;
private Map<String, Object> missingMap = Collections.emptyMap();
protected ArrayValuesSourceAggregationBuilder(String name, ValuesSourceType valuesSourceType, ValueType targetValueType) {
protected ArrayValuesSourceAggregationBuilder(String name) {
super(name);
if (valuesSourceType == null) {
throw new IllegalArgumentException("[valuesSourceType] must not be null: [" + name + "]");
}
this.valuesSourceType = valuesSourceType;
this.targetValueType = targetValueType;
}
protected ArrayValuesSourceAggregationBuilder(ArrayValuesSourceAggregationBuilder<VS, AB> clone,
protected ArrayValuesSourceAggregationBuilder(ArrayValuesSourceAggregationBuilder<AB> clone,
Builder factoriesBuilder, Map<String, Object> metadata) {
super(clone, factoriesBuilder, metadata);
this.valuesSourceType = clone.valuesSourceType;
this.targetValueType = clone.targetValueType;
this.fields = new ArrayList<>(clone.fields);
this.valueType = clone.valueType;
this.userValueTypeHint = clone.userValueTypeHint;
this.format = clone.format;
this.missingMap = new HashMap<>(clone.missingMap);
this.missing = clone.missing;
}
protected ArrayValuesSourceAggregationBuilder(StreamInput in, ValuesSourceType valuesSourceType, ValueType targetValueType)
protected ArrayValuesSourceAggregationBuilder(StreamInput in)
throws IOException {
super(in);
assert false == serializeTargetValueType() : "Wrong read constructor called for subclass that provides its targetValueType";
this.valuesSourceType = valuesSourceType;
this.targetValueType = targetValueType;
read(in);
}
protected ArrayValuesSourceAggregationBuilder(StreamInput in, ValuesSourceType valuesSourceType) throws IOException {
super(in);
assert serializeTargetValueType() : "Wrong read constructor called for subclass that serializes its targetValueType";
this.valuesSourceType = valuesSourceType;
this.targetValueType = in.readOptionalWriteable(ValueType::readFromStream);
read(in);
}
@ -143,18 +111,15 @@ public abstract class ArrayValuesSourceAggregationBuilder<VS extends ValuesSourc
@SuppressWarnings("unchecked")
private void read(StreamInput in) throws IOException {
fields = (ArrayList<String>)in.readGenericValue();
valueType = in.readOptionalWriteable(ValueType::readFromStream);
userValueTypeHint = in.readOptionalWriteable(ValueType::readFromStream);
format = in.readOptionalString();
missingMap = in.readMap();
}
@Override
protected final void doWriteTo(StreamOutput out) throws IOException {
if (serializeTargetValueType()) {
out.writeOptionalWriteable(targetValueType);
}
out.writeGenericValue(fields);
out.writeOptionalWriteable(valueType);
out.writeOptionalWriteable(userValueTypeHint);
out.writeOptionalString(format);
out.writeMap(missingMap);
innerWriteTo(out);
@ -184,25 +149,6 @@ public abstract class ArrayValuesSourceAggregationBuilder<VS extends ValuesSourc
return fields;
}
/**
* Sets the {@link ValueType} for the value produced by this aggregation
*/
@SuppressWarnings("unchecked")
public AB valueType(ValueType valueType) {
if (valueType == null) {
throw new IllegalArgumentException("[valueType] must not be null: [" + name + "]");
}
this.valueType = valueType;
return (AB) this;
}
/**
* Gets the {@link ValueType} for the value produced by this aggregation
*/
public ValueType valueType() {
return valueType;
}
/**
* Sets the format to use for the output of the aggregation.
*/
@ -244,96 +190,27 @@ public abstract class ArrayValuesSourceAggregationBuilder<VS extends ValuesSourc
}
@Override
protected final ArrayValuesSourceAggregatorFactory<VS> doBuild(QueryShardContext queryShardContext, AggregatorFactory parent,
Builder subFactoriesBuilder) throws IOException {
Map<String, ValuesSourceConfig<VS>> configs = resolveConfig(queryShardContext);
ArrayValuesSourceAggregatorFactory<VS> factory = innerBuild(queryShardContext, configs, parent, subFactoriesBuilder);
protected final ArrayValuesSourceAggregatorFactory doBuild(QueryShardContext queryShardContext, AggregatorFactory parent,
Builder subFactoriesBuilder) throws IOException {
Map<String, ValuesSourceConfig> configs = resolveConfig(queryShardContext);
ArrayValuesSourceAggregatorFactory factory = innerBuild(queryShardContext, configs, parent, subFactoriesBuilder);
return factory;
}
protected Map<String, ValuesSourceConfig<VS>> resolveConfig(QueryShardContext queryShardContext) {
HashMap<String, ValuesSourceConfig<VS>> configs = new HashMap<>();
protected Map<String, ValuesSourceConfig> resolveConfig(QueryShardContext queryShardContext) {
HashMap<String, ValuesSourceConfig> configs = new HashMap<>();
for (String field : fields) {
ValuesSourceConfig<VS> config = config(queryShardContext, field, null);
ValuesSourceConfig config = ValuesSourceConfig.resolveUnregistered(queryShardContext, userValueTypeHint, field, null,
missingMap.get(field), null, format, CoreValuesSourceType.BYTES);
configs.put(field, config);
}
return configs;
}
protected abstract ArrayValuesSourceAggregatorFactory<VS> innerBuild(QueryShardContext queryShardContext,
Map<String, ValuesSourceConfig<VS>> configs,
AggregatorFactory parent,
AggregatorFactories.Builder subFactoriesBuilder) throws IOException;
public ValuesSourceConfig<VS> config(QueryShardContext queryShardContext, String field, Script script) {
ValueType valueType = this.valueType != null ? this.valueType : targetValueType;
if (field == null) {
if (script == null) {
ValuesSourceConfig<VS> config = new ValuesSourceConfig<>(CoreValuesSourceType.ANY);
return config.format(resolveFormat(null, valueType));
}
ValuesSourceType valuesSourceType = valueType != null ? valueType.getValuesSourceType() : this.valuesSourceType;
if (valuesSourceType == null || valuesSourceType == CoreValuesSourceType.ANY) {
// the specific value source type is undefined, but for scripts,
// we need to have a specific value source
// type to know how to handle the script values, so we fallback
// on Bytes
valuesSourceType = CoreValuesSourceType.BYTES;
}
ValuesSourceConfig<VS> config = new ValuesSourceConfig<>(valuesSourceType);
config.missing(missingMap.get(field));
return config.format(resolveFormat(format, valueType));
}
MappedFieldType fieldType = queryShardContext.getMapperService().fieldType(field);
if (fieldType == null) {
ValuesSourceType valuesSourceType = valueType != null ? valueType.getValuesSourceType() : this.valuesSourceType;
ValuesSourceConfig<VS> config = new ValuesSourceConfig<>(valuesSourceType);
config.missing(missingMap.get(field));
config.format(resolveFormat(format, valueType));
return config.unmapped(true);
}
IndexFieldData<?> indexFieldData = queryShardContext.getForField(fieldType);
ValuesSourceConfig<VS> config;
if (valuesSourceType == CoreValuesSourceType.ANY) {
if (indexFieldData instanceof IndexNumericFieldData) {
config = new ValuesSourceConfig<>(CoreValuesSourceType.NUMERIC);
} else if (indexFieldData instanceof IndexGeoPointFieldData) {
config = new ValuesSourceConfig<>(CoreValuesSourceType.GEOPOINT);
} else {
config = new ValuesSourceConfig<>(CoreValuesSourceType.BYTES);
}
} else {
config = new ValuesSourceConfig<>(valuesSourceType);
}
config.fieldContext(new FieldContext(field, indexFieldData, fieldType));
config.missing(missingMap.get(field));
return config.format(fieldType.docValueFormat(format, null));
}
private static DocValueFormat resolveFormat(@Nullable String format, @Nullable ValueType valueType) {
if (valueType == null) {
return DocValueFormat.RAW; // we can't figure it out
}
DocValueFormat valueFormat = valueType.defaultFormat();
if (valueFormat instanceof DocValueFormat.Decimal && format != null) {
valueFormat = new DocValueFormat.Decimal(format);
}
return valueFormat;
}
/**
* Should this builder serialize its targetValueType? Defaults to false. All subclasses that override this to true
* should use the three argument read constructor rather than the four argument version.
*/
protected boolean serializeTargetValueType() {
return false;
}
protected abstract ArrayValuesSourceAggregatorFactory innerBuild(QueryShardContext queryShardContext,
Map<String, ValuesSourceConfig> configs,
AggregatorFactory parent,
AggregatorFactories.Builder subFactoriesBuilder) throws IOException;
@Override
public final XContentBuilder internalXContent(XContentBuilder builder, Params params) throws IOException {
@ -348,8 +225,8 @@ public abstract class ArrayValuesSourceAggregationBuilder<VS extends ValuesSourc
if (format != null) {
builder.field(CommonFields.FORMAT.getPreferredName(), format);
}
if (valueType != null) {
builder.field(CommonFields.VALUE_TYPE.getPreferredName(), valueType.getPreferredName());
if (userValueTypeHint != null) {
builder.field(CommonFields.VALUE_TYPE.getPreferredName(), userValueTypeHint.getPreferredName());
}
doXContentBody(builder, params);
builder.endObject();
@ -360,7 +237,7 @@ public abstract class ArrayValuesSourceAggregationBuilder<VS extends ValuesSourc
@Override
public int hashCode() {
return Objects.hash(super.hashCode(), fields, format, missing, targetValueType, valueType, valuesSourceType);
return Objects.hash(super.hashCode(), fields, format, missing, userValueTypeHint);
}
@Override
@ -368,12 +245,10 @@ public abstract class ArrayValuesSourceAggregationBuilder<VS extends ValuesSourc
if (this == obj) return true;
if (obj == null || getClass() != obj.getClass()) return false;
if (super.equals(obj) == false) return false;
ArrayValuesSourceAggregationBuilder<?, ?> other = (ArrayValuesSourceAggregationBuilder<?, ?>) obj;
ArrayValuesSourceAggregationBuilder<?> other = (ArrayValuesSourceAggregationBuilder<?>) obj;
return Objects.equals(fields, other.fields)
&& Objects.equals(format, other.format)
&& Objects.equals(missing, other.missing)
&& Objects.equals(targetValueType, other.targetValueType)
&& Objects.equals(valueType, other.valueType)
&& Objects.equals(valuesSourceType, other.valuesSourceType);
&& Objects.equals(userValueTypeHint, other.userValueTypeHint);
}
}

14
modules/aggs-matrix-stats/src/main/java/org/elasticsearch/search/aggregations/support/ArrayValuesSourceAggregatorFactory.java
View File

@ -29,12 +29,12 @@ import java.io.IOException;
import java.util.HashMap;
import java.util.Map;
public abstract class ArrayValuesSourceAggregatorFactory<VS extends ValuesSource>
public abstract class ArrayValuesSourceAggregatorFactory
extends AggregatorFactory {
protected Map<String, ValuesSourceConfig<VS>> configs;
protected Map<String, ValuesSourceConfig> configs;
public ArrayValuesSourceAggregatorFactory(String name, Map<String, ValuesSourceConfig<VS>> configs,
public ArrayValuesSourceAggregatorFactory(String name, Map<String, ValuesSourceConfig> configs,
QueryShardContext queryShardContext, AggregatorFactory parent,
AggregatorFactories.Builder subFactoriesBuilder,
Map<String, Object> metadata) throws IOException {
@ -47,10 +47,10 @@ public abstract class ArrayValuesSourceAggregatorFactory<VS extends ValuesSource
Aggregator parent,
boolean collectsFromSingleBucket,
Map<String, Object> metadata) throws IOException {
HashMap<String, VS> valuesSources = new HashMap<>();
HashMap<String, ValuesSource> valuesSources = new HashMap<>();
for (Map.Entry<String, ValuesSourceConfig<VS>> config : configs.entrySet()) {
VS vs = config.getValue().toValuesSource(queryShardContext);
for (Map.Entry<String, ValuesSourceConfig> config : configs.entrySet()) {
ValuesSource vs = config.getValue().toValuesSource();
if (vs != null) {
valuesSources.put(config.getKey(), vs);
}
@ -65,7 +65,7 @@ public abstract class ArrayValuesSourceAggregatorFactory<VS extends ValuesSource
Aggregator parent,
Map<String, Object> metadata) throws IOException;
protected abstract Aggregator doCreateInternal(Map<String, VS> valuesSources,
protected abstract Aggregator doCreateInternal(Map<String, ValuesSource> valuesSources,
SearchContext searchContext,
Aggregator parent,
boolean collectsFromSingleBucket,

19
modules/aggs-matrix-stats/src/main/java/org/elasticsearch/search/aggregations/support/ArrayValuesSourceParser.java
View File

@ -35,13 +35,6 @@ import java.util.Map;
public abstract class ArrayValuesSourceParser<VS extends ValuesSource> implements Aggregator.Parser {
public abstract static class AnyValuesSourceParser extends ArrayValuesSourceParser<ValuesSource> {
protected AnyValuesSourceParser(boolean formattable) {
super(formattable, CoreValuesSourceType.ANY, null);
}
}
public abstract static class NumericValuesSourceParser extends ArrayValuesSourceParser<ValuesSource.Numeric> {
protected NumericValuesSourceParser(boolean formattable) {
@ -74,7 +67,7 @@ public abstract class ArrayValuesSourceParser<VS extends ValuesSource> implement
}
@Override
public final ArrayValuesSourceAggregationBuilder<VS, ?> parse(String aggregationName, XContentParser parser)
public final ArrayValuesSourceAggregationBuilder<?> parse(String aggregationName, XContentParser parser)
throws IOException {
List<String> fields = null;
@ -139,7 +132,7 @@ public abstract class ArrayValuesSourceParser<VS extends ValuesSource> implement
}
}
ArrayValuesSourceAggregationBuilder<VS, ?> factory = createFactory(aggregationName, this.valuesSourceType, this.targetValueType,
ArrayValuesSourceAggregationBuilder<?> factory = createFactory(aggregationName, this.valuesSourceType, this.targetValueType,
otherOptions);
if (fields != null) {
factory.fields(fields);
@ -193,10 +186,10 @@ public abstract class ArrayValuesSourceParser<VS extends ValuesSource> implement
* method
* @return the created factory
*/
protected abstract ArrayValuesSourceAggregationBuilder<VS, ?> createFactory(String aggregationName,
ValuesSourceType valuesSourceType,
ValueType targetValueType,
Map<ParseField, Object> otherOptions);
protected abstract ArrayValuesSourceAggregationBuilder<?> createFactory(String aggregationName,
ValuesSourceType valuesSourceType,
ValueType targetValueType,
Map<ParseField, Object> otherOptions);
/**
* Allows subclasses of {@link ArrayValuesSourceParser} to parse extra

2
modules/mapper-extras/src/main/java/org/elasticsearch/index/mapper/ScaledFloatFieldMapper.java
View File

@ -60,6 +60,8 @@ import org.elasticsearch.index.query.QueryShardContext;
import org.elasticsearch.indices.breaker.CircuitBreakerService;
import org.elasticsearch.search.DocValueFormat;
import org.elasticsearch.search.MultiValueMode;
import org.elasticsearch.search.aggregations.support.CoreValuesSourceType;
import org.elasticsearch.search.aggregations.support.ValuesSourceType;
import org.elasticsearch.search.sort.BucketedSort;
import org.elasticsearch.search.sort.SortOrder;
import org.elasticsearch.search.aggregations.support.CoreValuesSourceType;

40
modules/parent-join/src/main/java/org/elasticsearch/join/aggregations/ChildrenAggregationBuilder.java
View File

@ -25,7 +25,6 @@ import org.elasticsearch.common.io.stream.StreamInput;
import org.elasticsearch.common.io.stream.StreamOutput;
import org.elasticsearch.common.xcontent.XContentBuilder;
import org.elasticsearch.common.xcontent.XContentParser;
import org.elasticsearch.index.fielddata.plain.SortedSetDVOrdinalsIndexFieldData;
import org.elasticsearch.index.mapper.MappedFieldType;
import org.elasticsearch.index.query.QueryShardContext;
import org.elasticsearch.join.mapper.ParentIdFieldMapper;
@ -34,19 +33,17 @@ import org.elasticsearch.search.aggregations.AggregationBuilder;
import org.elasticsearch.search.aggregations.AggregatorFactories.Builder;
import org.elasticsearch.search.aggregations.AggregatorFactory;
import org.elasticsearch.search.aggregations.support.CoreValuesSourceType;
import org.elasticsearch.search.aggregations.support.FieldContext;
import org.elasticsearch.search.aggregations.support.ValueType;
import org.elasticsearch.search.aggregations.support.ValuesSource.Bytes.WithOrdinals;
import org.elasticsearch.search.aggregations.support.ValuesSourceAggregationBuilder;
import org.elasticsearch.search.aggregations.support.ValuesSourceAggregatorFactory;
import org.elasticsearch.search.aggregations.support.ValuesSourceConfig;
import org.elasticsearch.search.aggregations.support.ValuesSourceType;
import java.io.IOException;
import java.util.Map;
import java.util.Objects;
public class ChildrenAggregationBuilder
extends ValuesSourceAggregationBuilder<WithOrdinals, ChildrenAggregationBuilder> {
extends ValuesSourceAggregationBuilder<ChildrenAggregationBuilder> {
public static final String NAME = "children";
@ -61,7 +58,7 @@ public class ChildrenAggregationBuilder
* the type of children documents
*/
public ChildrenAggregationBuilder(String name, String childType) {
super(name, CoreValuesSourceType.BYTES, ValueType.STRING);
super(name);
if (childType == null) {
throw new IllegalArgumentException("[childType] must not be null: [" + name + "]");
}
@ -76,6 +73,11 @@ public class ChildrenAggregationBuilder
this.parentFilter = clone.parentFilter;
}
@Override
protected ValuesSourceType defaultValueSourceType() {
return CoreValuesSourceType.BYTES;
}
@Override
protected AggregationBuilder shallowCopy(Builder factoriesBuilder, Map<String, Object> metadata) {
return new ChildrenAggregationBuilder(this, factoriesBuilder, metadata);
@ -85,7 +87,7 @@ public class ChildrenAggregationBuilder
* Read from a stream.
*/
public ChildrenAggregationBuilder(StreamInput in) throws IOException {
super(in, CoreValuesSourceType.BYTES, ValueType.STRING);
super(in);
childType = in.readString();
}
@ -100,33 +102,29 @@ public class ChildrenAggregationBuilder
}
@Override
protected ValuesSourceAggregatorFactory<WithOrdinals> innerBuild(QueryShardContext queryShardContext,
ValuesSourceConfig<WithOrdinals> config,
AggregatorFactory parent,
Builder subFactoriesBuilder) throws IOException {
protected ValuesSourceAggregatorFactory innerBuild(QueryShardContext queryShardContext,
ValuesSourceConfig config,
AggregatorFactory parent,
Builder subFactoriesBuilder) throws IOException {
return new ChildrenAggregatorFactory(name, config, childFilter, parentFilter, queryShardContext, parent,
subFactoriesBuilder, metadata);
}
@Override
protected ValuesSourceConfig<WithOrdinals> resolveConfig(QueryShardContext queryShardContext) {
ValuesSourceConfig<WithOrdinals> config = new ValuesSourceConfig<>(CoreValuesSourceType.BYTES);
joinFieldResolveConfig(queryShardContext, config);
return config;
}
private void joinFieldResolveConfig(QueryShardContext queryShardContext, ValuesSourceConfig<WithOrdinals> config) {
protected ValuesSourceConfig resolveConfig(QueryShardContext queryShardContext) {
ValuesSourceConfig config;
ParentJoinFieldMapper parentJoinFieldMapper = ParentJoinFieldMapper.getMapper(queryShardContext.getMapperService());
ParentIdFieldMapper parentIdFieldMapper = parentJoinFieldMapper.getParentIdFieldMapper(childType, false);
if (parentIdFieldMapper != null) {
parentFilter = parentIdFieldMapper.getParentFilter();
childFilter = parentIdFieldMapper.getChildFilter(childType);
MappedFieldType fieldType = parentIdFieldMapper.fieldType();
final SortedSetDVOrdinalsIndexFieldData fieldData = queryShardContext.getForField(fieldType);
config.fieldContext(new FieldContext(fieldType.name(), fieldData, fieldType));
config = ValuesSourceConfig.resolveFieldOnly(fieldType, queryShardContext);
} else {
config.unmapped(true);
// Unmapped field case
config = ValuesSourceConfig.resolveUnmapped(defaultValueSourceType(), queryShardContext);
}
return config;
}
@Override

13
modules/parent-join/src/main/java/org/elasticsearch/join/aggregations/ChildrenAggregatorFactory.java
View File

@ -21,11 +21,13 @@ package org.elasticsearch.join.aggregations;
import org.apache.lucene.search.Query;
import org.elasticsearch.index.query.QueryShardContext;
import org.elasticsearch.search.aggregations.AggregationExecutionException;
import org.elasticsearch.search.aggregations.Aggregator;
import org.elasticsearch.search.aggregations.AggregatorFactories;
import org.elasticsearch.search.aggregations.AggregatorFactory;
import org.elasticsearch.search.aggregations.InternalAggregation;
import org.elasticsearch.search.aggregations.NonCollectingAggregator;
import org.elasticsearch.search.aggregations.support.ValuesSource;
import org.elasticsearch.search.aggregations.support.ValuesSource.Bytes.WithOrdinals;
import org.elasticsearch.search.aggregations.support.ValuesSourceAggregatorFactory;
import org.elasticsearch.search.aggregations.support.ValuesSourceConfig;
@ -34,13 +36,13 @@ import org.elasticsearch.search.internal.SearchContext;
import java.io.IOException;
import java.util.Map;
public class ChildrenAggregatorFactory extends ValuesSourceAggregatorFactory<WithOrdinals> {
public class ChildrenAggregatorFactory extends ValuesSourceAggregatorFactory {
private final Query parentFilter;
private final Query childFilter;
public ChildrenAggregatorFactory(String name,
ValuesSourceConfig<WithOrdinals> config,
ValuesSourceConfig config,
Query childFilter,
Query parentFilter,
QueryShardContext context,
@ -64,11 +66,16 @@ public class ChildrenAggregatorFactory extends ValuesSourceAggregatorFactory<Wit
}
@Override
protected Aggregator doCreateInternal(WithOrdinals valuesSource,
protected Aggregator doCreateInternal(ValuesSource rawValuesSource,
SearchContext searchContext, Aggregator parent,
boolean collectsFromSingleBucket,
Map<String, Object> metadata) throws IOException {
if (rawValuesSource instanceof WithOrdinals == false) {
throw new AggregationExecutionException("ValuesSource type " + rawValuesSource.toString() +
"is not supported for aggregation " + this.name());
}
WithOrdinals valuesSource = (WithOrdinals) rawValuesSource;
long maxOrd = valuesSource.globalMaxOrd(searchContext.searcher());
if (collectsFromSingleBucket) {
return new ParentToChildrenAggregator(name, factories, searchContext, parent, childFilter,

40
modules/parent-join/src/main/java/org/elasticsearch/join/aggregations/ParentAggregationBuilder.java
View File

@ -25,7 +25,6 @@ import org.elasticsearch.common.io.stream.StreamInput;
import org.elasticsearch.common.io.stream.StreamOutput;
import org.elasticsearch.common.xcontent.XContentBuilder;
import org.elasticsearch.common.xcontent.XContentParser;
import org.elasticsearch.index.fielddata.plain.SortedSetDVOrdinalsIndexFieldData;
import org.elasticsearch.index.mapper.MappedFieldType;
import org.elasticsearch.index.query.QueryShardContext;
import org.elasticsearch.join.mapper.ParentIdFieldMapper;
@ -34,19 +33,17 @@ import org.elasticsearch.search.aggregations.AggregationBuilder;
import org.elasticsearch.search.aggregations.AggregatorFactories.Builder;
import org.elasticsearch.search.aggregations.AggregatorFactory;
import org.elasticsearch.search.aggregations.support.CoreValuesSourceType;
import org.elasticsearch.search.aggregations.support.FieldContext;
import org.elasticsearch.search.aggregations.support.ValueType;
import org.elasticsearch.search.aggregations.support.ValuesSource.Bytes.WithOrdinals;
import org.elasticsearch.search.aggregations.support.ValuesSourceAggregationBuilder;
import org.elasticsearch.search.aggregations.support.ValuesSourceAggregatorFactory;
import org.elasticsearch.search.aggregations.support.ValuesSourceConfig;
import org.elasticsearch.search.aggregations.support.ValuesSourceType;
import java.io.IOException;
import java.util.Map;
import java.util.Objects;
public class ParentAggregationBuilder
extends ValuesSourceAggregationBuilder<WithOrdinals, ParentAggregationBuilder> {
extends ValuesSourceAggregationBuilder<ParentAggregationBuilder> {
public static final String NAME = "parent";
@ -61,7 +58,7 @@ public class ParentAggregationBuilder
* the type of children documents
*/
public ParentAggregationBuilder(String name, String childType) {
super(name, CoreValuesSourceType.BYTES, ValueType.STRING);
super(name);
if (childType == null) {
throw new IllegalArgumentException("[childType] must not be null: [" + name + "]");
}
@ -76,6 +73,11 @@ public class ParentAggregationBuilder
this.parentFilter = clone.parentFilter;
}
@Override
protected ValuesSourceType defaultValueSourceType() {
return CoreValuesSourceType.BYTES;
}
@Override
protected AggregationBuilder shallowCopy(Builder factoriesBuilder, Map<String, Object> metadata) {
return new ParentAggregationBuilder(this, factoriesBuilder, metadata);
@ -85,7 +87,7 @@ public class ParentAggregationBuilder
* Read from a stream.
*/
public ParentAggregationBuilder(StreamInput in) throws IOException {
super(in, CoreValuesSourceType.BYTES, ValueType.STRING);
super(in);
childType = in.readString();
}
@ -100,33 +102,29 @@ public class ParentAggregationBuilder
}
@Override
protected ValuesSourceAggregatorFactory<WithOrdinals> innerBuild(QueryShardContext queryShardContext,
ValuesSourceConfig<WithOrdinals> config,
AggregatorFactory parent,
Builder subFactoriesBuilder) throws IOException {
protected ValuesSourceAggregatorFactory innerBuild(QueryShardContext queryShardContext,
ValuesSourceConfig config,
AggregatorFactory parent,
Builder subFactoriesBuilder) throws IOException {
return new ParentAggregatorFactory(name, config, childFilter, parentFilter, queryShardContext, parent,
subFactoriesBuilder, metadata);
}
@Override
protected ValuesSourceConfig<WithOrdinals> resolveConfig(QueryShardContext queryShardContext) {
ValuesSourceConfig<WithOrdinals> config = new ValuesSourceConfig<>(CoreValuesSourceType.BYTES);
joinFieldResolveConfig(queryShardContext, config);
return config;
}
private void joinFieldResolveConfig(QueryShardContext queryShardContext, ValuesSourceConfig<WithOrdinals> config) {
protected ValuesSourceConfig resolveConfig(QueryShardContext queryShardContext) {
ValuesSourceConfig config;
ParentJoinFieldMapper parentJoinFieldMapper = ParentJoinFieldMapper.getMapper(queryShardContext.getMapperService());
ParentIdFieldMapper parentIdFieldMapper = parentJoinFieldMapper.getParentIdFieldMapper(childType, false);
if (parentIdFieldMapper != null) {
parentFilter = parentIdFieldMapper.getParentFilter();
childFilter = parentIdFieldMapper.getChildFilter(childType);
MappedFieldType fieldType = parentIdFieldMapper.fieldType();
final SortedSetDVOrdinalsIndexFieldData fieldData = queryShardContext.getForField(fieldType);
config.fieldContext(new FieldContext(fieldType.name(), fieldData, fieldType));
config = ValuesSourceConfig.resolveFieldOnly(fieldType, queryShardContext);
} else {
config.unmapped(true);
// unmapped case
config = ValuesSourceConfig.resolveUnmapped(defaultValueSourceType(), queryShardContext);
}
return config;
}
@Override

13
modules/parent-join/src/main/java/org/elasticsearch/join/aggregations/ParentAggregatorFactory.java
View File

@ -21,11 +21,13 @@ package org.elasticsearch.join.aggregations;
import org.apache.lucene.search.Query;
import org.elasticsearch.index.query.QueryShardContext;
import org.elasticsearch.search.aggregations.AggregationExecutionException;
import org.elasticsearch.search.aggregations.Aggregator;
import org.elasticsearch.search.aggregations.AggregatorFactories;
import org.elasticsearch.search.aggregations.AggregatorFactory;
import org.elasticsearch.search.aggregations.InternalAggregation;
import org.elasticsearch.search.aggregations.NonCollectingAggregator;
import org.elasticsearch.search.aggregations.support.ValuesSource;
import org.elasticsearch.search.aggregations.support.ValuesSource.Bytes.WithOrdinals;
import org.elasticsearch.search.aggregations.support.ValuesSourceAggregatorFactory;
import org.elasticsearch.search.aggregations.support.ValuesSourceConfig;
@ -34,13 +36,13 @@ import org.elasticsearch.search.internal.SearchContext;
import java.io.IOException;
import java.util.Map;
public class ParentAggregatorFactory extends ValuesSourceAggregatorFactory<WithOrdinals> {
public class ParentAggregatorFactory extends ValuesSourceAggregatorFactory {
private final Query parentFilter;
private final Query childFilter;
public ParentAggregatorFactory(String name,
ValuesSourceConfig<WithOrdinals> config,
ValuesSourceConfig config,
Query childFilter,
Query parentFilter,
QueryShardContext queryShardContext,
@ -65,11 +67,16 @@ public class ParentAggregatorFactory extends ValuesSourceAggregatorFactory<WithO
}
@Override
protected Aggregator doCreateInternal(WithOrdinals valuesSource,
protected Aggregator doCreateInternal(ValuesSource rawValuesSource,
SearchContext searchContext, Aggregator children,
boolean collectsFromSingleBucket,
Map<String, Object> metadata) throws IOException {
if (rawValuesSource instanceof WithOrdinals == false) {
throw new AggregationExecutionException("ValuesSource type " + rawValuesSource.toString() +
"is not supported for aggregation " + this.name());
}
WithOrdinals valuesSource = (WithOrdinals) rawValuesSource;
long maxOrd = valuesSource.globalMaxOrd(searchContext.searcher());
if (collectsFromSingleBucket) {
return new ChildrenToParentAggregator(name, factories, searchContext, children, childFilter,

7
modules/parent-join/src/test/java/org/elasticsearch/join/aggregations/ChildrenToParentAggregatorTests.java
View File

@ -304,7 +304,8 @@ public class ChildrenToParentAggregatorTests extends AggregatorTestCase {
throws IOException {
ParentAggregationBuilder aggregationBuilder = new ParentAggregationBuilder("_name", CHILD_TYPE);
aggregationBuilder.subAggregation(new TermsAggregationBuilder("value_terms", ValueType.LONG).field("number"));
aggregationBuilder.subAggregation(new TermsAggregationBuilder("value_terms").userValueTypeHint(ValueType.LONG)
.field("number"));
MappedFieldType fieldType = new NumberFieldMapper.NumberFieldType(NumberFieldMapper.NumberType.LONG);
fieldType.setName("number");
@ -316,9 +317,9 @@ public class ChildrenToParentAggregatorTests extends AggregatorTestCase {
private void testCaseTermsParentTerms(Query query, IndexSearcher indexSearcher, Consumer<LongTerms> verify)
throws IOException {
AggregationBuilder aggregationBuilder =
new TermsAggregationBuilder("subvalue_terms", ValueType.LONG).field("subNumber").
new TermsAggregationBuilder("subvalue_terms").userValueTypeHint(ValueType.LONG).field("subNumber").
subAggregation(new ParentAggregationBuilder("to_parent", CHILD_TYPE).
subAggregation(new TermsAggregationBuilder("value_terms", ValueType.LONG).field("number")));
subAggregation(new TermsAggregationBuilder("value_terms").userValueTypeHint(ValueType.LONG).field("number")));
MappedFieldType fieldType = new NumberFieldMapper.NumberFieldType(NumberFieldMapper.NumberType.LONG);
fieldType.setName("number");

19
qa/multi-cluster-search/src/test/java/org/elasticsearch/search/CCSDuelIT.java
View File

@ -524,31 +524,32 @@ public class CCSDuelIT extends ESRestTestCase {
private static SearchSourceBuilder buildTermsAggsSource() {
SearchSourceBuilder sourceBuilder = new SearchSourceBuilder();
sourceBuilder.size(0);
TermsAggregationBuilder cluster = new TermsAggregationBuilder("cluster123", ValueType.STRING);
TermsAggregationBuilder cluster = new TermsAggregationBuilder("cluster123").userValueTypeHint(ValueType.STRING);
cluster.field("_index");
TermsAggregationBuilder type = new TermsAggregationBuilder("type", ValueType.STRING);
TermsAggregationBuilder type = new TermsAggregationBuilder("type").userValueTypeHint(ValueType.STRING);
type.field("type.keyword");
type.showTermDocCountError(true);
type.order(BucketOrder.key(true));
cluster.subAggregation(type);
sourceBuilder.aggregation(cluster);
TermsAggregationBuilder tags = new TermsAggregationBuilder("tags", ValueType.STRING);
TermsAggregationBuilder tags = new TermsAggregationBuilder("tags").userValueTypeHint(ValueType.STRING);
tags.field("tags.keyword");
tags.showTermDocCountError(true);
tags.size(100);
sourceBuilder.aggregation(tags);
TermsAggregationBuilder tags2 = new TermsAggregationBuilder("tags", ValueType.STRING);
TermsAggregationBuilder tags2 = new TermsAggregationBuilder("tags").userValueTypeHint(ValueType.STRING);
tags2.field("tags.keyword");
tags.subAggregation(tags2);
FilterAggregationBuilder answers = new FilterAggregationBuilder("answers", new TermQueryBuilder("type", "answer"));
TermsAggregationBuilder answerPerQuestion = new TermsAggregationBuilder("answer_per_question", ValueType.STRING);
TermsAggregationBuilder answerPerQuestion = new TermsAggregationBuilder("answer_per_question")
.userValueTypeHint(ValueType.STRING);
answerPerQuestion.showTermDocCountError(true);
answerPerQuestion.field("questionId.keyword");
answers.subAggregation(answerPerQuestion);
TermsAggregationBuilder answerPerUser = new TermsAggregationBuilder("answer_per_user", ValueType.STRING);
TermsAggregationBuilder answerPerUser = new TermsAggregationBuilder("answer_per_user").userValueTypeHint(ValueType.STRING);
answerPerUser.field("user.keyword");
answerPerUser.size(30);
answerPerUser.showTermDocCountError(true);
@ -563,7 +564,7 @@ public class CCSDuelIT extends ESRestTestCase {
SearchSourceBuilder sourceBuilder = new SearchSourceBuilder();
sourceBuilder.size(0);
searchRequest.source(sourceBuilder);
TermsAggregationBuilder tags = new TermsAggregationBuilder("tags", ValueType.STRING);
TermsAggregationBuilder tags = new TermsAggregationBuilder("tags").userValueTypeHint(ValueType.STRING);
tags.field("tags.keyword");
tags.showTermDocCountError(true);
DateHistogramAggregationBuilder creation = new DateHistogramAggregationBuilder("creation");
@ -580,7 +581,7 @@ public class CCSDuelIT extends ESRestTestCase {
SearchSourceBuilder sourceBuilder = new SearchSourceBuilder();
sourceBuilder.size(0);
searchRequest.source(sourceBuilder);
CardinalityAggregationBuilder tags = new CardinalityAggregationBuilder("tags", ValueType.STRING);
CardinalityAggregationBuilder tags = new CardinalityAggregationBuilder("tags").userValueTypeHint(ValueType.STRING);
tags.field("tags.keyword");
sourceBuilder.aggregation(tags);
duelSearch(searchRequest, CCSDuelIT::assertAggs);
@ -619,7 +620,7 @@ public class CCSDuelIT extends ESRestTestCase {
topHits.size(10);
topHits.sort("creationDate", SortOrder.DESC);
topHits.sort("id", SortOrder.ASC);
TermsAggregationBuilder tags = new TermsAggregationBuilder("tags", ValueType.STRING);
TermsAggregationBuilder tags = new TermsAggregationBuilder("tags").userValueTypeHint(ValueType.STRING);
tags.field("tags.keyword");
tags.size(10);
tags.subAggregation(topHits);

2
rest-api-spec/src/main/resources/rest-api-spec/test/search.aggregation/280_rare_terms.yml
View File

@ -112,7 +112,7 @@ setup:
- length: { aggregations.ip_terms.buckets: 0 }
- do:
catch: request
catch: /Aggregation \[ip_terms\] cannot support regular expression style include\/exclude settings as they can only be applied to string fields\. Use an array of values for include\/exclude clauses/
search:
index: test_1
body: { "size" : 0, "aggs" : { "ip_terms" : { "rare_terms" : { "field" : "ip", "exclude" : "127.*" } } } }

2
rest-api-spec/src/main/resources/rest-api-spec/test/search.aggregation/30_sig_terms.yml
View File

@ -133,7 +133,7 @@
- length: { aggregations.ip_terms.buckets: 0 }
- do:
catch: request
catch: /Aggregation \[ip_terms\] cannot support regular expression style include\/exclude settings as they can only be applied to string fields\. Use an array of values for include\/exclude clauses/
search:
rest_total_hits_as_int: true
body: { "size" : 0, "aggs" : { "ip_terms" : { "significant_terms" : { "field" : "ip", "exclude" : "127.*" } } } }

37
server/src/main/java/org/elasticsearch/index/IndexModule.java
View File

@ -61,6 +61,7 @@ import org.elasticsearch.indices.fielddata.cache.IndicesFieldDataCache;
import org.elasticsearch.indices.mapper.MapperRegistry;
import org.elasticsearch.plugins.IndexStorePlugin;
import org.elasticsearch.script.ScriptService;
import org.elasticsearch.search.aggregations.support.ValuesSourceRegistry;
import org.elasticsearch.threadpool.ThreadPool;
import java.io.IOException;
@ -388,23 +389,22 @@ public final class IndexModule {
}
}
public IndexService newIndexService(
IndexService.IndexCreationContext indexCreationContext,
NodeEnvironment environment,
NamedXContentRegistry xContentRegistry,
IndexService.ShardStoreDeleter shardStoreDeleter,
CircuitBreakerService circuitBreakerService,
BigArrays bigArrays,
ThreadPool threadPool,
ScriptService scriptService,
ClusterService clusterService,
Client client,
IndicesQueryCache indicesQueryCache,
MapperRegistry mapperRegistry,
IndicesFieldDataCache indicesFieldDataCache,
NamedWriteableRegistry namedWriteableRegistry,
BooleanSupplier idFieldDataEnabled)
throws IOException {
public IndexService newIndexService(IndexService.IndexCreationContext indexCreationContext,
NodeEnvironment environment,
NamedXContentRegistry xContentRegistry,
IndexService.ShardStoreDeleter shardStoreDeleter,
CircuitBreakerService circuitBreakerService,
BigArrays bigArrays,
ThreadPool threadPool,
ScriptService scriptService,
ClusterService clusterService,
Client client,
IndicesQueryCache indicesQueryCache,
MapperRegistry mapperRegistry,
IndicesFieldDataCache indicesFieldDataCache,
NamedWriteableRegistry namedWriteableRegistry,
BooleanSupplier idFieldDataEnabled,
ValuesSourceRegistry valuesSourceRegistry) throws IOException {
final IndexEventListener eventListener = freeze();
Function<IndexService, CheckedFunction<DirectoryReader, DirectoryReader, IOException>> readerWrapperFactory =
indexReaderWrapper.get() == null ? (shard) -> null : indexReaderWrapper.get();
@ -431,7 +431,8 @@ public final class IndexModule {
new SimilarityService(indexSettings, scriptService, similarities), shardStoreDeleter, indexAnalyzers,
engineFactory, circuitBreakerService, bigArrays, threadPool, scriptService, clusterService, client, queryCache,
directoryFactory, eventListener, readerWrapperFactory, mapperRegistry, indicesFieldDataCache, searchOperationListeners,
indexOperationListeners, namedWriteableRegistry, idFieldDataEnabled, allowExpensiveQueries, expressionResolver);
indexOperationListeners, namedWriteableRegistry, idFieldDataEnabled, allowExpensiveQueries, expressionResolver,
valuesSourceRegistry);
success = true;
return indexService;
} finally {

8
server/src/main/java/org/elasticsearch/index/IndexService.java
View File

@ -81,6 +81,7 @@ import org.elasticsearch.indices.fielddata.cache.IndicesFieldDataCache;
import org.elasticsearch.indices.mapper.MapperRegistry;
import org.elasticsearch.plugins.IndexStorePlugin;
import org.elasticsearch.script.ScriptService;
import org.elasticsearch.search.aggregations.support.ValuesSourceRegistry;
import org.elasticsearch.threadpool.ThreadPool;
import java.io.Closeable;
@ -146,6 +147,7 @@ public class IndexService extends AbstractIndexComponent implements IndicesClust
private final CircuitBreakerService circuitBreakerService;
private final IndexNameExpressionResolver expressionResolver;
private Supplier<Sort> indexSortSupplier;
private ValuesSourceRegistry valuesSourceRegistry;
public IndexService(
IndexSettings indexSettings,
@ -172,7 +174,8 @@ public class IndexService extends AbstractIndexComponent implements IndicesClust
NamedWriteableRegistry namedWriteableRegistry,
BooleanSupplier idFieldDataEnabled,
BooleanSupplier allowExpensiveQueries,
IndexNameExpressionResolver expressionResolver) {
IndexNameExpressionResolver expressionResolver,
ValuesSourceRegistry valuesSourceRegistry) {
super(indexSettings);
this.allowExpensiveQueries = allowExpensiveQueries;
this.indexSettings = indexSettings;
@ -181,6 +184,7 @@ public class IndexService extends AbstractIndexComponent implements IndicesClust
this.namedWriteableRegistry = namedWriteableRegistry;
this.circuitBreakerService = circuitBreakerService;
this.expressionResolver = expressionResolver;
this.valuesSourceRegistry = valuesSourceRegistry;
if (needsMapperService(indexSettings, indexCreationContext)) {
assert indexAnalyzers != null;
this.mapperService = new MapperService(indexSettings, indexAnalyzers, xContentRegistry, similarityService, mapperRegistry,
@ -578,7 +582,7 @@ public class IndexService extends AbstractIndexComponent implements IndicesClust
return new QueryShardContext(
shardId, indexSettings, bigArrays, indexCache.bitsetFilterCache(), indexFieldData::getForField, mapperService(),
similarityService(), scriptService, xContentRegistry, namedWriteableRegistry, client, searcher, nowInMillis, clusterAlias,
indexNameMatcher, allowExpensiveQueries);
indexNameMatcher, allowExpensiveQueries, valuesSourceRegistry);
}
/**

2
server/src/main/java/org/elasticsearch/index/mapper/BooleanFieldMapper.java
View File

@ -193,7 +193,7 @@ public class BooleanFieldMapper extends FieldMapper {
@Override
public ValuesSourceType getValuesSourceType() {
return CoreValuesSourceType.NUMERIC;
return CoreValuesSourceType.BOOLEAN;
}
@Override

2
server/src/main/java/org/elasticsearch/index/mapper/DateFieldMapper.java
View File

@ -545,7 +545,7 @@ public final class DateFieldMapper extends FieldMapper {
@Override
public ValuesSourceType getValuesSourceType() {
return CoreValuesSourceType.NUMERIC;
return CoreValuesSourceType.DATE;
}
@Override

2
server/src/main/java/org/elasticsearch/index/mapper/IdFieldMapper.java
View File

@ -48,6 +48,8 @@ import org.elasticsearch.indices.IndicesService;
import org.elasticsearch.indices.breaker.CircuitBreakerService;
import org.elasticsearch.search.DocValueFormat;
import org.elasticsearch.search.MultiValueMode;
import org.elasticsearch.search.aggregations.support.CoreValuesSourceType;
import org.elasticsearch.search.aggregations.support.ValuesSourceType;
import org.elasticsearch.search.sort.BucketedSort;
import org.elasticsearch.search.sort.SortOrder;
import org.elasticsearch.search.aggregations.support.CoreValuesSourceType;

2
server/src/main/java/org/elasticsearch/index/mapper/IpFieldMapper.java
View File

@ -298,7 +298,7 @@ public class IpFieldMapper extends FieldMapper {
@Override
public ValuesSourceType getValuesSourceType() {
return CoreValuesSourceType.BYTES;
return CoreValuesSourceType.IP;
}
@Override

17
server/src/main/java/org/elasticsearch/index/query/QueryShardContext.java
View File

@ -58,6 +58,7 @@ import org.elasticsearch.script.Script;
import org.elasticsearch.script.ScriptContext;
import org.elasticsearch.script.ScriptFactory;
import org.elasticsearch.script.ScriptService;
import org.elasticsearch.search.aggregations.support.ValuesSourceRegistry;
import org.elasticsearch.search.lookup.SearchLookup;
import org.elasticsearch.transport.RemoteClusterAware;
@ -115,6 +116,7 @@ public class QueryShardContext extends QueryRewriteContext {
private boolean allowUnmappedFields;
private boolean mapUnmappedFieldAsString;
private NestedScope nestedScope;
private ValuesSourceRegistry valuesSourceRegistry;
public QueryShardContext(int shardId,
IndexSettings indexSettings,
@ -131,18 +133,19 @@ public class QueryShardContext extends QueryRewriteContext {
LongSupplier nowInMillis,
String clusterAlias,
Predicate<String> indexNameMatcher,
BooleanSupplier allowExpensiveQueries) {
BooleanSupplier allowExpensiveQueries,
ValuesSourceRegistry valuesSourceRegistry) {
this(shardId, indexSettings, bigArrays, bitsetFilterCache, indexFieldDataLookup, mapperService, similarityService,
scriptService, xContentRegistry, namedWriteableRegistry, client, searcher, nowInMillis, indexNameMatcher,
new Index(RemoteClusterAware.buildRemoteIndexName(clusterAlias, indexSettings.getIndex().getName()),
indexSettings.getIndex().getUUID()), allowExpensiveQueries);
indexSettings.getIndex().getUUID()), allowExpensiveQueries, valuesSourceRegistry);
}
public QueryShardContext(QueryShardContext source) {
this(source.shardId, source.indexSettings, source.bigArrays, source.bitsetFilterCache, source.indexFieldDataService,
source.mapperService, source.similarityService, source.scriptService, source.getXContentRegistry(),
source.getWriteableRegistry(), source.client, source.searcher, source.nowInMillis, source.indexNameMatcher,
source.fullyQualifiedIndex, source.allowExpensiveQueries);
source.fullyQualifiedIndex, source.allowExpensiveQueries, source.valuesSourceRegistry);
}
private QueryShardContext(int shardId,
@ -160,7 +163,8 @@ public class QueryShardContext extends QueryRewriteContext {
LongSupplier nowInMillis,
Predicate<String> indexNameMatcher,
Index fullyQualifiedIndex,
BooleanSupplier allowExpensiveQueries) {
BooleanSupplier allowExpensiveQueries,
ValuesSourceRegistry valuesSourceRegistry) {
super(xContentRegistry, namedWriteableRegistry, client, nowInMillis);
this.shardId = shardId;
this.similarityService = similarityService;
@ -176,6 +180,7 @@ public class QueryShardContext extends QueryRewriteContext {
this.indexNameMatcher = indexNameMatcher;
this.fullyQualifiedIndex = fullyQualifiedIndex;
this.allowExpensiveQueries = allowExpensiveQueries;
this.valuesSourceRegistry = valuesSourceRegistry;
}
private void reset() {
@ -281,6 +286,10 @@ public class QueryShardContext extends QueryRewriteContext {
return getMapperService().searchQuoteAnalyzer();
}
public ValuesSourceRegistry getValuesSourceRegistry() {
return valuesSourceRegistry;
}
public void setAllowUnmappedFields(boolean allowUnmappedFields) {
this.allowUnmappedFields = allowUnmappedFields;
}

8
server/src/main/java/org/elasticsearch/indices/IndicesService.java
View File

@ -128,6 +128,7 @@ import org.elasticsearch.plugins.IndexStorePlugin;
import org.elasticsearch.plugins.PluginsService;
import org.elasticsearch.repositories.RepositoriesService;
import org.elasticsearch.script.ScriptService;
import org.elasticsearch.search.aggregations.support.ValuesSourceRegistry;
import org.elasticsearch.search.internal.AliasFilter;
import org.elasticsearch.search.internal.SearchContext;
import org.elasticsearch.search.internal.ShardSearchRequest;
@ -231,6 +232,7 @@ public class IndicesService extends AbstractLifecycleComponent
private final EsThreadPoolExecutor danglingIndicesThreadPoolExecutor;
private final Set<Index> danglingIndicesToWrite = Sets.newConcurrentHashSet();
private final boolean nodeWriteDanglingIndicesInfo;
private ValuesSourceRegistry valuesSourceRegistry;
@Override
@ -245,12 +247,13 @@ public class IndicesService extends AbstractLifecycleComponent
IndexScopedSettings indexScopedSettings, CircuitBreakerService circuitBreakerService, BigArrays bigArrays,
ScriptService scriptService, ClusterService clusterService, Client client, MetaStateService metaStateService,
Collection<Function<IndexSettings, Optional<EngineFactory>>> engineFactoryProviders,
Map<String, IndexStorePlugin.DirectoryFactory> directoryFactories) {
Map<String, IndexStorePlugin.DirectoryFactory> directoryFactories, ValuesSourceRegistry valuesSourceRegistry) {
this.settings = settings;
this.threadPool = threadPool;
this.pluginsService = pluginsService;
this.nodeEnv = nodeEnv;
this.xContentRegistry = xContentRegistry;
this.valuesSourceRegistry = valuesSourceRegistry;
this.shardsClosedTimeout = settings.getAsTime(INDICES_SHARDS_CLOSED_TIMEOUT, new TimeValue(1, TimeUnit.DAYS));
this.analysisRegistry = analysisRegistry;
this.indexNameExpressionResolver = indexNameExpressionResolver;
@ -659,7 +662,8 @@ public class IndicesService extends AbstractLifecycleComponent
mapperRegistry,
indicesFieldDataCache,
namedWriteableRegistry,
this::isIdFieldDataEnabled
this::isIdFieldDataEnabled,
valuesSourceRegistry
);
}

3
server/src/main/java/org/elasticsearch/node/Node.java
View File

@ -460,7 +460,8 @@ public class Node implements Closeable {
new IndicesService(settings, pluginsService, nodeEnvironment, xContentRegistry, analysisModule.getAnalysisRegistry(),
clusterModule.getIndexNameExpressionResolver(), indicesModule.getMapperRegistry(), namedWriteableRegistry,
threadPool, settingsModule.getIndexScopedSettings(), circuitBreakerService, bigArrays, scriptService,
clusterService, client, metaStateService, engineFactoryProviders, indexStoreFactories);
clusterService, client, metaStateService, engineFactoryProviders, indexStoreFactories,
searchModule.getValuesSourceRegistry());
final AliasValidator aliasValidator = new AliasValidator();

27
server/src/main/java/org/elasticsearch/plugins/SearchPlugin.java
View File

@ -44,6 +44,7 @@ import org.elasticsearch.search.aggregations.bucket.significant.heuristics.Signi
import org.elasticsearch.search.aggregations.pipeline.MovAvgModel;
import org.elasticsearch.search.aggregations.pipeline.MovAvgPipelineAggregator;
import org.elasticsearch.search.aggregations.pipeline.PipelineAggregator;
import org.elasticsearch.search.aggregations.support.ValuesSourceRegistry;
import org.elasticsearch.search.fetch.FetchSubPhase;
import org.elasticsearch.search.fetch.subphase.highlight.Highlighter;
import org.elasticsearch.search.rescore.Rescorer;
@ -57,6 +58,7 @@ import java.util.List;
import java.util.Map;
import java.util.TreeMap;
import java.util.function.BiFunction;
import java.util.function.Consumer;
import static java.util.Collections.emptyList;
import static java.util.Collections.emptyMap;
@ -121,6 +123,13 @@ public interface SearchPlugin {
default List<AggregationSpec> getAggregations() {
return emptyList();
}
/**
* Allows plugins to register new aggregations using aggregation names that are already defined
* in Core, as long as the new aggregations target different ValuesSourceTypes
*/
default List<Consumer<ValuesSourceRegistry>> getBareAggregatorRegistrar() {
return emptyList();
}
/**
* The new {@link PipelineAggregator}s added by this plugin.
*/
@ -259,6 +268,7 @@ public interface SearchPlugin {
*/
class AggregationSpec extends SearchExtensionSpec<AggregationBuilder, ContextParser<String, ? extends AggregationBuilder>> {
private final Map<String, Writeable.Reader<? extends InternalAggregation>> resultReaders = new TreeMap<>();
private Consumer<ValuesSourceRegistry> aggregatorRegistrar;
/**
* Specification for an {@link Aggregation}.
@ -341,6 +351,23 @@ public interface SearchPlugin {
public Map<String, Writeable.Reader<? extends InternalAggregation>> getResultReaders() {
return resultReaders;
}
/**
* Get the function to register the {@link org.elasticsearch.search.aggregations.support.ValuesSource} to aggregator mappings for
* this aggregation
*/
public Consumer<ValuesSourceRegistry> getAggregatorRegistrar() {
return aggregatorRegistrar;
}
/**
* Set the function to register the {@link org.elasticsearch.search.aggregations.support.ValuesSource} to aggregator mappings for
* this aggregation
*/
public AggregationSpec setAggregatorRegistrar(Consumer<ValuesSourceRegistry> aggregatorRegistrar) {
this.aggregatorRegistrar = aggregatorRegistrar;
return this;
}
}
/**

94
server/src/main/java/org/elasticsearch/search/SearchModule.java
View File

@ -241,6 +241,7 @@ import org.elasticsearch.search.aggregations.pipeline.StatsBucketPipelineAggrega
import org.elasticsearch.search.aggregations.pipeline.StatsBucketPipelineAggregator;
import org.elasticsearch.search.aggregations.pipeline.SumBucketPipelineAggregationBuilder;
import org.elasticsearch.search.aggregations.pipeline.SumBucketPipelineAggregator;
import org.elasticsearch.search.aggregations.support.ValuesSourceRegistry;
import org.elasticsearch.search.fetch.FetchPhase;
import org.elasticsearch.search.fetch.FetchSubPhase;
import org.elasticsearch.search.fetch.subphase.FetchDocValuesPhase;
@ -306,6 +307,7 @@ public class SearchModule {
private final Settings settings;
private final List<NamedWriteableRegistry.Entry> namedWriteables = new ArrayList<>();
private final List<NamedXContentRegistry.Entry> namedXContents = new ArrayList<>();
private ValuesSourceRegistry valuesSourceRegistry;
/**
* Constructs a new SearchModule object
@ -320,6 +322,7 @@ public class SearchModule {
public SearchModule(Settings settings, boolean transportClient, List<SearchPlugin> plugins) {
this.settings = settings;
this.transportClient = transportClient;
this.valuesSourceRegistry = new ValuesSourceRegistry();
registerSuggesters(plugins);
highlighters = setupHighlighters(settings, plugins);
registerScoreFunctions(plugins);
@ -346,6 +349,10 @@ public class SearchModule {
return namedXContents;
}
public ValuesSourceRegistry getValuesSourceRegistry() {
return valuesSourceRegistry;
}
/**
* Returns the {@link Highlighter} registry
*/
@ -362,34 +369,45 @@ public class SearchModule {
private void registerAggregations(List<SearchPlugin> plugins) {
registerAggregation(new AggregationSpec(AvgAggregationBuilder.NAME, AvgAggregationBuilder::new, AvgAggregationBuilder.PARSER)
.addResultReader(InternalAvg::new));
.addResultReader(InternalAvg::new)
.setAggregatorRegistrar(AvgAggregationBuilder::registerAggregators));
registerAggregation(new AggregationSpec(WeightedAvgAggregationBuilder.NAME, WeightedAvgAggregationBuilder::new,
WeightedAvgAggregationBuilder.PARSER).addResultReader(InternalWeightedAvg::new));
registerAggregation(new AggregationSpec(SumAggregationBuilder.NAME, SumAggregationBuilder::new, SumAggregationBuilder.PARSER)
.addResultReader(InternalSum::new));
.addResultReader(InternalSum::new)
.setAggregatorRegistrar(SumAggregationBuilder::registerAggregators));
registerAggregation(new AggregationSpec(MinAggregationBuilder.NAME, MinAggregationBuilder::new, MinAggregationBuilder.PARSER)
.addResultReader(InternalMin::new));
.addResultReader(InternalMin::new)
.setAggregatorRegistrar(MinAggregationBuilder::registerAggregators));
registerAggregation(new AggregationSpec(MaxAggregationBuilder.NAME, MaxAggregationBuilder::new, MaxAggregationBuilder.PARSER)
.addResultReader(InternalMax::new));
.addResultReader(InternalMax::new)
.setAggregatorRegistrar(MaxAggregationBuilder::registerAggregators));
registerAggregation(new AggregationSpec(StatsAggregationBuilder.NAME, StatsAggregationBuilder::new, StatsAggregationBuilder.PARSER)
.addResultReader(InternalStats::new));
.addResultReader(InternalStats::new)
.setAggregatorRegistrar(StatsAggregationBuilder::registerAggregators));
registerAggregation(new AggregationSpec(ExtendedStatsAggregationBuilder.NAME, ExtendedStatsAggregationBuilder::new,
ExtendedStatsAggregationBuilder.PARSER).addResultReader(InternalExtendedStats::new));
registerAggregation(new AggregationSpec(ValueCountAggregationBuilder.NAME, ValueCountAggregationBuilder::new,
ValueCountAggregationBuilder.PARSER).addResultReader(InternalValueCount::new));
ValueCountAggregationBuilder.PARSER)
.addResultReader(InternalValueCount::new)
.setAggregatorRegistrar(ValueCountAggregationBuilder::registerAggregators));
registerAggregation(new AggregationSpec(PercentilesAggregationBuilder.NAME, PercentilesAggregationBuilder::new,
PercentilesAggregationBuilder::parse)
.addResultReader(InternalTDigestPercentiles.NAME, InternalTDigestPercentiles::new)
.addResultReader(InternalHDRPercentiles.NAME, InternalHDRPercentiles::new));
.addResultReader(InternalHDRPercentiles.NAME, InternalHDRPercentiles::new)
.setAggregatorRegistrar(PercentilesAggregationBuilder::registerAggregators));
registerAggregation(new AggregationSpec(PercentileRanksAggregationBuilder.NAME, PercentileRanksAggregationBuilder::new,
PercentileRanksAggregationBuilder::parse)
.addResultReader(InternalTDigestPercentileRanks.NAME, InternalTDigestPercentileRanks::new)
.addResultReader(InternalHDRPercentileRanks.NAME, InternalHDRPercentileRanks::new));
.addResultReader(InternalHDRPercentileRanks.NAME, InternalHDRPercentileRanks::new)
.setAggregatorRegistrar(PercentileRanksAggregationBuilder::registerAggregators));
registerAggregation(new AggregationSpec(MedianAbsoluteDeviationAggregationBuilder.NAME,
MedianAbsoluteDeviationAggregationBuilder::new, MedianAbsoluteDeviationAggregationBuilder.PARSER)
.addResultReader(InternalMedianAbsoluteDeviation::new));
MedianAbsoluteDeviationAggregationBuilder::new, MedianAbsoluteDeviationAggregationBuilder.PARSER)
.addResultReader(InternalMedianAbsoluteDeviation::new)
.setAggregatorRegistrar(MedianAbsoluteDeviationAggregationBuilder::registerAggregators));
registerAggregation(new AggregationSpec(CardinalityAggregationBuilder.NAME, CardinalityAggregationBuilder::new,
CardinalityAggregationBuilder.PARSER).addResultReader(InternalCardinality::new));
CardinalityAggregationBuilder.PARSER).addResultReader(InternalCardinality::new)
.setAggregatorRegistrar(CardinalityAggregationBuilder::registerAggregators));
registerAggregation(new AggregationSpec(GlobalAggregationBuilder.NAME, GlobalAggregationBuilder::new,
GlobalAggregationBuilder::parse).addResultReader(InternalGlobal::new));
registerAggregation(new AggregationSpec(MissingAggregationBuilder.NAME, MissingAggregationBuilder::new,
@ -412,37 +430,52 @@ public class SearchModule {
.addResultReader(StringTerms.NAME, StringTerms::new)
.addResultReader(UnmappedTerms.NAME, UnmappedTerms::new)
.addResultReader(LongTerms.NAME, LongTerms::new)
.addResultReader(DoubleTerms.NAME, DoubleTerms::new));
.addResultReader(DoubleTerms.NAME, DoubleTerms::new)
.setAggregatorRegistrar(TermsAggregationBuilder::registerAggregators));
registerAggregation(new AggregationSpec(RareTermsAggregationBuilder.NAME, RareTermsAggregationBuilder::new,
RareTermsAggregationBuilder.PARSER)
.addResultReader(StringRareTerms.NAME, StringRareTerms::new)
.addResultReader(UnmappedRareTerms.NAME, UnmappedRareTerms::new)
.addResultReader(LongRareTerms.NAME, LongRareTerms::new));
.addResultReader(LongRareTerms.NAME, LongRareTerms::new)
.setAggregatorRegistrar(RareTermsAggregationBuilder::registerAggregators));
registerAggregation(new AggregationSpec(SignificantTermsAggregationBuilder.NAME, SignificantTermsAggregationBuilder::new,
SignificantTermsAggregationBuilder::parse)
.addResultReader(SignificantStringTerms.NAME, SignificantStringTerms::new)
.addResultReader(SignificantLongTerms.NAME, SignificantLongTerms::new)
.addResultReader(UnmappedSignificantTerms.NAME, UnmappedSignificantTerms::new));
.addResultReader(UnmappedSignificantTerms.NAME, UnmappedSignificantTerms::new)
.setAggregatorRegistrar(SignificantTermsAggregationBuilder::registerAggregators));
registerAggregation(new AggregationSpec(SignificantTextAggregationBuilder.NAME, SignificantTextAggregationBuilder::new,
SignificantTextAggregationBuilder::parse));
registerAggregation(new AggregationSpec(RangeAggregationBuilder.NAME, RangeAggregationBuilder::new,
RangeAggregationBuilder.PARSER).addResultReader(InternalRange::new));
RangeAggregationBuilder.PARSER)
.addResultReader(InternalRange::new)
.setAggregatorRegistrar(RangeAggregationBuilder::registerAggregators));
registerAggregation(new AggregationSpec(DateRangeAggregationBuilder.NAME, DateRangeAggregationBuilder::new,
DateRangeAggregationBuilder.PARSER).addResultReader(InternalDateRange::new));
DateRangeAggregationBuilder.PARSER)
.addResultReader(InternalDateRange::new)
.setAggregatorRegistrar(DateRangeAggregationBuilder::registerAggregators));
registerAggregation(new AggregationSpec(IpRangeAggregationBuilder.NAME, IpRangeAggregationBuilder::new,
IpRangeAggregationBuilder.PARSER).addResultReader(InternalBinaryRange::new));
registerAggregation(new AggregationSpec(HistogramAggregationBuilder.NAME, HistogramAggregationBuilder::new,
HistogramAggregationBuilder.PARSER).addResultReader(InternalHistogram::new));
HistogramAggregationBuilder.PARSER)
.addResultReader(InternalHistogram::new)
.setAggregatorRegistrar(HistogramAggregationBuilder::registerAggregators));
registerAggregation(new AggregationSpec(DateHistogramAggregationBuilder.NAME, DateHistogramAggregationBuilder::new,
DateHistogramAggregationBuilder.PARSER).addResultReader(InternalDateHistogram::new));
DateHistogramAggregationBuilder.PARSER)
.addResultReader(InternalDateHistogram::new)
.setAggregatorRegistrar(DateHistogramAggregationBuilder::registerAggregators));
registerAggregation(new AggregationSpec(AutoDateHistogramAggregationBuilder.NAME, AutoDateHistogramAggregationBuilder::new,
AutoDateHistogramAggregationBuilder.PARSER).addResultReader(InternalAutoDateHistogram::new));
registerAggregation(new AggregationSpec(GeoDistanceAggregationBuilder.NAME, GeoDistanceAggregationBuilder::new,
GeoDistanceAggregationBuilder::parse).addResultReader(InternalGeoDistance::new));
registerAggregation(new AggregationSpec(GeoHashGridAggregationBuilder.NAME, GeoHashGridAggregationBuilder::new,
GeoHashGridAggregationBuilder.PARSER).addResultReader(InternalGeoHashGrid::new));
GeoHashGridAggregationBuilder.PARSER)
.addResultReader(InternalGeoHashGrid::new)
.setAggregatorRegistrar(GeoHashGridAggregationBuilder::registerAggregators));
registerAggregation(new AggregationSpec(GeoTileGridAggregationBuilder.NAME, GeoTileGridAggregationBuilder::new,
GeoTileGridAggregationBuilder.PARSER).addResultReader(InternalGeoTileGrid::new));
GeoTileGridAggregationBuilder.PARSER)
.addResultReader(InternalGeoTileGrid::new)
.setAggregatorRegistrar(GeoTileGridAggregationBuilder::registerAggregators));
registerAggregation(new AggregationSpec(NestedAggregationBuilder.NAME, NestedAggregationBuilder::new,
NestedAggregationBuilder::parse).addResultReader(InternalNested::new));
registerAggregation(new AggregationSpec(ReverseNestedAggregationBuilder.NAME, ReverseNestedAggregationBuilder::new,
@ -450,14 +483,21 @@ public class SearchModule {
registerAggregation(new AggregationSpec(TopHitsAggregationBuilder.NAME, TopHitsAggregationBuilder::new,
TopHitsAggregationBuilder::parse).addResultReader(InternalTopHits::new));
registerAggregation(new AggregationSpec(GeoBoundsAggregationBuilder.NAME, GeoBoundsAggregationBuilder::new,
GeoBoundsAggregationBuilder.PARSER).addResultReader(InternalGeoBounds::new));
GeoBoundsAggregationBuilder.PARSER)
.addResultReader(InternalGeoBounds::new)
.setAggregatorRegistrar(GeoBoundsAggregationBuilder::registerAggregators));
registerAggregation(new AggregationSpec(GeoCentroidAggregationBuilder.NAME, GeoCentroidAggregationBuilder::new,
GeoCentroidAggregationBuilder.PARSER).addResultReader(InternalGeoCentroid::new));
GeoCentroidAggregationBuilder.PARSER)
.addResultReader(InternalGeoCentroid::new)
.setAggregatorRegistrar(GeoCentroidAggregationBuilder::registerAggregators));
registerAggregation(new AggregationSpec(ScriptedMetricAggregationBuilder.NAME, ScriptedMetricAggregationBuilder::new,
ScriptedMetricAggregationBuilder.PARSER).addResultReader(InternalScriptedMetric::new));
registerAggregation((new AggregationSpec(CompositeAggregationBuilder.NAME, CompositeAggregationBuilder::new,
CompositeAggregationBuilder.PARSER).addResultReader(InternalComposite::new)));
registerFromPlugin(plugins, SearchPlugin::getAggregations, this::registerAggregation);
// after aggs have been registered, see if there are any new VSTypes that need to be linked to core fields
registerFromPlugin(plugins, SearchPlugin::getBareAggregatorRegistrar, this::registerBareAggregatorRegistrar);
}
private void registerAggregation(AggregationSpec spec) {
@ -474,6 +514,16 @@ public class SearchModule {
Writeable.Reader<? extends InternalAggregation> internalReader = t.getValue();
namedWriteables.add(new NamedWriteableRegistry.Entry(InternalAggregation.class, writeableName, internalReader));
}
Consumer<ValuesSourceRegistry> register = spec.getAggregatorRegistrar();
if (register != null) {
register.accept(this.valuesSourceRegistry);
}
}
private void registerBareAggregatorRegistrar(Consumer<ValuesSourceRegistry> registrar) {
if (registrar != null) {
registrar.accept(this.valuesSourceRegistry);
}
}
private void registerPipelineAggregations(List<SearchPlugin> plugins) {

10
server/src/main/java/org/elasticsearch/search/aggregations/AggregationBuilders.java
View File

@ -105,7 +105,7 @@ public class AggregationBuilders {
* Create a new {@link ValueCount} aggregation with the given name.
*/
public static ValueCountAggregationBuilder count(String name) {
return new ValueCountAggregationBuilder(name, null);
return new ValueCountAggregationBuilder(name);
}
/**
@ -217,7 +217,7 @@ public class AggregationBuilders {
* Create a new {@link Missing} aggregation with the given name.
*/
public static MissingAggregationBuilder missing(String name) {
return new MissingAggregationBuilder(name, null);
return new MissingAggregationBuilder(name);
}
/**
@ -266,7 +266,7 @@ public class AggregationBuilders {
* Create a new {@link SignificantTerms} aggregation with the given name.
*/
public static SignificantTermsAggregationBuilder significantTerms(String name) {
return new SignificantTermsAggregationBuilder(name, null);
return new SignificantTermsAggregationBuilder(name);
}
@ -313,7 +313,7 @@ public class AggregationBuilders {
* Create a new {@link Terms} aggregation with the given name.
*/
public static TermsAggregationBuilder terms(String name) {
return new TermsAggregationBuilder(name, null);
return new TermsAggregationBuilder(name);
}
/**
@ -341,7 +341,7 @@ public class AggregationBuilders {
* Create a new {@link Cardinality} aggregation with the given name.
*/
public static CardinalityAggregationBuilder cardinality(String name) {
return new CardinalityAggregationBuilder(name, null);
return new CardinalityAggregationBuilder(name);
}
/**

7
server/src/main/java/org/elasticsearch/search/aggregations/bucket/composite/CompositeValuesSourceBuilder.java
View File

@ -27,6 +27,7 @@ import org.elasticsearch.common.xcontent.ToXContentFragment;
import org.elasticsearch.common.xcontent.XContentBuilder;
import org.elasticsearch.index.query.QueryShardContext;
import org.elasticsearch.script.Script;
import org.elasticsearch.search.aggregations.support.CoreValuesSourceType;
import org.elasticsearch.search.aggregations.support.ValueType;
import org.elasticsearch.search.aggregations.support.ValuesSource;
import org.elasticsearch.search.aggregations.support.ValuesSourceConfig;
@ -294,11 +295,11 @@ public abstract class CompositeValuesSourceBuilder<AB extends CompositeValuesSou
* @param config The {@link ValuesSourceConfig} for this source.
*/
protected abstract CompositeValuesSourceConfig innerBuild(QueryShardContext queryShardContext,
ValuesSourceConfig<?> config) throws IOException;
ValuesSourceConfig config) throws IOException;
public final CompositeValuesSourceConfig build(QueryShardContext queryShardContext) throws IOException {
ValuesSourceConfig<?> config = ValuesSourceConfig.resolve(queryShardContext,
valueType, field, script, null, timeZone(), format);
ValuesSourceConfig config = ValuesSourceConfig.resolveUnregistered(queryShardContext,
valueType, field, script, null, timeZone(), format, CoreValuesSourceType.BYTES);
return innerBuild(queryShardContext, config);
}

13
server/src/main/java/org/elasticsearch/search/aggregations/bucket/composite/CompositeValuesSourceParserHelper.java
View File

@ -26,9 +26,9 @@ import org.elasticsearch.common.io.stream.StreamInput;
import org.elasticsearch.common.io.stream.StreamOutput;
import org.elasticsearch.common.xcontent.AbstractObjectParser;
import org.elasticsearch.common.xcontent.ObjectParser;
import org.elasticsearch.common.xcontent.ToXContent.Params;
import org.elasticsearch.common.xcontent.XContentBuilder;
import org.elasticsearch.common.xcontent.XContentParser;
import org.elasticsearch.common.xcontent.ToXContent.Params;
import org.elasticsearch.script.Script;
import org.elasticsearch.search.aggregations.support.ValueType;
@ -37,19 +37,20 @@ import java.io.IOException;
import static org.elasticsearch.common.xcontent.XContentParserUtils.ensureExpectedToken;
public class CompositeValuesSourceParserHelper {
static <VB extends CompositeValuesSourceBuilder<VB>, T> void declareValuesSourceFields(AbstractObjectParser<VB, T> objectParser,
ValueType targetValueType) {
ValueType expectedValueType) {
objectParser.declareField(VB::field, XContentParser::text,
new ParseField("field"), ObjectParser.ValueType.STRING);
objectParser.declareBoolean(VB::missingBucket, new ParseField("missing_bucket"));
objectParser.declareField(VB::valueType, p -> {
ValueType valueType = ValueType.resolveForScript(p.text());
if (targetValueType != null && valueType.isNotA(targetValueType)) {
ValueType valueType = ValueType.lenientParse(p.text());
if (expectedValueType != null && valueType.isNotA(expectedValueType)) {
throw new ParsingException(p.getTokenLocation(),
"Aggregation [" + objectParser.getName() + "] was configured with an incompatible value type ["
+ valueType + "]. It can only work on value of type ["
+ targetValueType + "]");
+ valueType + "]. It can only work on value of type ["
+ expectedValueType + "]");
}
return valueType;
}, new ParseField("value_type"), ObjectParser.ValueType.STRING);

4
server/src/main/java/org/elasticsearch/search/aggregations/bucket/composite/DateHistogramValuesSourceBuilder.java
View File

@ -247,9 +247,9 @@ public class DateHistogramValuesSourceBuilder
}
@Override
protected CompositeValuesSourceConfig innerBuild(QueryShardContext queryShardContext, ValuesSourceConfig<?> config) throws IOException {
protected CompositeValuesSourceConfig innerBuild(QueryShardContext queryShardContext, ValuesSourceConfig config) throws IOException {
Rounding rounding = dateHistogramInterval.createRounding(timeZone(), offset);
ValuesSource orig = config.toValuesSource(queryShardContext);
ValuesSource orig = config.toValuesSource();
if (orig == null) {
orig = ValuesSource.Numeric.EMPTY;
}

4
server/src/main/java/org/elasticsearch/search/aggregations/bucket/composite/GeoTileGridValuesSourceBuilder.java
View File

@ -128,8 +128,8 @@ public class GeoTileGridValuesSourceBuilder extends CompositeValuesSourceBuilder
}
@Override
protected CompositeValuesSourceConfig innerBuild(QueryShardContext queryShardContext, ValuesSourceConfig<?> config) throws IOException {
ValuesSource orig = config.toValuesSource(queryShardContext);
protected CompositeValuesSourceConfig innerBuild(QueryShardContext queryShardContext, ValuesSourceConfig config) throws IOException {
ValuesSource orig = config.toValuesSource();
if (orig == null) {
orig = ValuesSource.GeoPoint.EMPTY;
}

4
server/src/main/java/org/elasticsearch/search/aggregations/bucket/composite/HistogramValuesSourceBuilder.java
View File

@ -110,8 +110,8 @@ public class HistogramValuesSourceBuilder extends CompositeValuesSourceBuilder<H
}
@Override
protected CompositeValuesSourceConfig innerBuild(QueryShardContext queryShardContext, ValuesSourceConfig<?> config) throws IOException {
ValuesSource orig = config.toValuesSource(queryShardContext);
protected CompositeValuesSourceConfig innerBuild(QueryShardContext queryShardContext, ValuesSourceConfig config) throws IOException {
ValuesSource orig = config.toValuesSource();
if (orig == null) {
orig = ValuesSource.Numeric.EMPTY;
}

4
server/src/main/java/org/elasticsearch/search/aggregations/bucket/composite/TermsValuesSourceBuilder.java
View File

@ -70,8 +70,8 @@ public class TermsValuesSourceBuilder extends CompositeValuesSourceBuilder<Terms
}
@Override
protected CompositeValuesSourceConfig innerBuild(QueryShardContext queryShardContext, ValuesSourceConfig<?> config) throws IOException {
ValuesSource vs = config.toValuesSource(queryShardContext);
protected CompositeValuesSourceConfig innerBuild(QueryShardContext queryShardContext, ValuesSourceConfig config) throws IOException {
ValuesSource vs = config.toValuesSource();
if (vs == null) {
// The field is unmapped so we use a value source that can parse any type of values.
// This is needed because the after values are parsed even when there are no values to process.

28
server/src/main/java/org/elasticsearch/search/aggregations/bucket/geogrid/GeoGridAggregationBuilder.java
View File

@ -34,19 +34,17 @@ import org.elasticsearch.search.aggregations.AggregatorFactories.Builder;
import org.elasticsearch.search.aggregations.AggregatorFactory;
import org.elasticsearch.search.aggregations.bucket.BucketUtils;
import org.elasticsearch.search.aggregations.support.CoreValuesSourceType;
import org.elasticsearch.search.aggregations.support.ValueType;
import org.elasticsearch.search.aggregations.support.ValuesSource;
import org.elasticsearch.search.aggregations.support.ValuesSourceAggregationBuilder;
import org.elasticsearch.search.aggregations.support.ValuesSourceAggregatorFactory;
import org.elasticsearch.search.aggregations.support.ValuesSourceConfig;
import org.elasticsearch.search.aggregations.support.ValuesSourceParserHelper;
import org.elasticsearch.search.aggregations.support.ValuesSourceType;
import java.io.IOException;
import java.util.Map;
import java.util.Objects;
import java.util.function.Function;
public abstract class GeoGridAggregationBuilder extends ValuesSourceAggregationBuilder<ValuesSource.GeoPoint, GeoGridAggregationBuilder> {
public abstract class GeoGridAggregationBuilder extends ValuesSourceAggregationBuilder<GeoGridAggregationBuilder> {
/* recognized field names in JSON */
static final ParseField FIELD_PRECISION = new ParseField("precision");
static final ParseField FIELD_SIZE = new ParseField("size");
@ -66,7 +64,7 @@ public abstract class GeoGridAggregationBuilder extends ValuesSourceAggregationB
public static <T extends GeoGridAggregationBuilder> ObjectParser<T, String> createParser(
String name, PrecisionParser precisionParser, Function<String, T> ctor) {
ObjectParser<T, String> parser = ObjectParser.fromBuilder(name, ctor);
ValuesSourceParserHelper.declareGeoFields(parser, false, false);
ValuesSourceAggregationBuilder.declareFields(parser, false, false, false);
parser.declareField((p, builder, context) -> builder.precision(precisionParser.parse(p)), FIELD_PRECISION,
org.elasticsearch.common.xcontent.ObjectParser.ValueType.INT);
parser.declareInt(GeoGridAggregationBuilder::size, FIELD_SIZE);
@ -79,7 +77,7 @@ public abstract class GeoGridAggregationBuilder extends ValuesSourceAggregationB
}
public GeoGridAggregationBuilder(String name) {
super(name, CoreValuesSourceType.GEOPOINT, ValueType.GEOPOINT);
super(name);
}
protected GeoGridAggregationBuilder(GeoGridAggregationBuilder clone, Builder factoriesBuilder, Map<String, Object> metadata) {
@ -94,7 +92,7 @@ public abstract class GeoGridAggregationBuilder extends ValuesSourceAggregationB
* Read from a stream.
*/
public GeoGridAggregationBuilder(StreamInput in) throws IOException {
super(in, CoreValuesSourceType.GEOPOINT, ValueType.GEOPOINT);
super(in);
precision = in.readVInt();
requiredSize = in.readVInt();
shardSize = in.readVInt();
@ -103,6 +101,11 @@ public abstract class GeoGridAggregationBuilder extends ValuesSourceAggregationB
}
}
@Override
protected ValuesSourceType defaultValueSourceType() {
return CoreValuesSourceType.GEOPOINT;
}
@Override
protected void innerWriteTo(StreamOutput out) throws IOException {
out.writeVInt(precision);
@ -123,8 +126,8 @@ public abstract class GeoGridAggregationBuilder extends ValuesSourceAggregationB
/**
* Creates a new instance of the {@link ValuesSourceAggregatorFactory}-derived class specific to the geo aggregation.
*/
protected abstract ValuesSourceAggregatorFactory<ValuesSource.GeoPoint> createFactory(
String name, ValuesSourceConfig<ValuesSource.GeoPoint> config, int precision, int requiredSize, int shardSize,
protected abstract ValuesSourceAggregatorFactory createFactory(
String name, ValuesSourceConfig config, int precision, int requiredSize, int shardSize,
GeoBoundingBox geoBoundingBox, QueryShardContext queryShardContext, AggregatorFactory parent,
Builder subFactoriesBuilder, Map<String, Object> metadata
) throws IOException;
@ -175,9 +178,10 @@ public abstract class GeoGridAggregationBuilder extends ValuesSourceAggregationB
}
@Override
protected ValuesSourceAggregatorFactory<ValuesSource.GeoPoint> innerBuild(QueryShardContext queryShardContext,
ValuesSourceConfig<ValuesSource.GeoPoint> config,
AggregatorFactory parent, Builder subFactoriesBuilder) throws IOException {
protected ValuesSourceAggregatorFactory innerBuild(QueryShardContext queryShardContext,
ValuesSourceConfig config,
AggregatorFactory parent, Builder subFactoriesBuilder)
throws IOException {
int shardSize = this.shardSize;
int requiredSize = this.requiredSize;

10
server/src/main/java/org/elasticsearch/search/aggregations/bucket/geogrid/GeoHashGridAggregationBuilder.java
View File

@ -27,9 +27,9 @@ import org.elasticsearch.index.query.QueryShardContext;
import org.elasticsearch.search.aggregations.AggregationBuilder;
import org.elasticsearch.search.aggregations.AggregatorFactories;
import org.elasticsearch.search.aggregations.AggregatorFactory;
import org.elasticsearch.search.aggregations.support.ValuesSource;
import org.elasticsearch.search.aggregations.support.ValuesSourceAggregatorFactory;
import org.elasticsearch.search.aggregations.support.ValuesSourceConfig;
import org.elasticsearch.search.aggregations.support.ValuesSourceRegistry;
import java.io.IOException;
import java.util.Map;
@ -53,6 +53,10 @@ public class GeoHashGridAggregationBuilder extends GeoGridAggregationBuilder {
super(in);
}
public static void registerAggregators(ValuesSourceRegistry valuesSourceRegistry) {
GeoHashGridAggregatorFactory.registerAggregators(valuesSourceRegistry);
}
@Override
public GeoGridAggregationBuilder precision(int precision) {
this.precision = GeoUtils.checkPrecisionRange(precision);
@ -60,8 +64,8 @@ public class GeoHashGridAggregationBuilder extends GeoGridAggregationBuilder {
}
@Override
protected ValuesSourceAggregatorFactory<ValuesSource.GeoPoint> createFactory(
String name, ValuesSourceConfig<ValuesSource.GeoPoint> config, int precision, int requiredSize, int shardSize,
protected ValuesSourceAggregatorFactory createFactory(
String name, ValuesSourceConfig config, int precision, int requiredSize, int shardSize,
GeoBoundingBox geoBoundingBox, QueryShardContext queryShardContext,
AggregatorFactory parent, AggregatorFactories.Builder subFactoriesBuilder,
Map<String, Object> metadata) throws IOException {

33
server/src/main/java/org/elasticsearch/search/aggregations/bucket/geogrid/GeoHashGridAggregatorFactory.java
View File

@ -22,14 +22,19 @@ package org.elasticsearch.search.aggregations.bucket.geogrid;
import org.elasticsearch.common.geo.GeoBoundingBox;
import org.elasticsearch.geometry.utils.Geohash;
import org.elasticsearch.index.query.QueryShardContext;
import org.elasticsearch.search.aggregations.AggregationExecutionException;
import org.elasticsearch.search.aggregations.Aggregator;
import org.elasticsearch.search.aggregations.AggregatorFactories;
import org.elasticsearch.search.aggregations.AggregatorFactory;
import org.elasticsearch.search.aggregations.InternalAggregation;
import org.elasticsearch.search.aggregations.NonCollectingAggregator;
import org.elasticsearch.search.aggregations.metrics.GeoGridAggregatorSupplier;
import org.elasticsearch.search.aggregations.support.AggregatorSupplier;
import org.elasticsearch.search.aggregations.support.CoreValuesSourceType;
import org.elasticsearch.search.aggregations.support.ValuesSource;
import org.elasticsearch.search.aggregations.support.ValuesSourceAggregatorFactory;
import org.elasticsearch.search.aggregations.support.ValuesSourceConfig;
import org.elasticsearch.search.aggregations.support.ValuesSourceRegistry;
import org.elasticsearch.search.internal.SearchContext;
import java.io.IOException;
@ -37,14 +42,14 @@ import java.util.Map;
import static java.util.Collections.emptyList;
public class GeoHashGridAggregatorFactory extends ValuesSourceAggregatorFactory<ValuesSource.GeoPoint> {
public class GeoHashGridAggregatorFactory extends ValuesSourceAggregatorFactory {
private final int precision;
private final int requiredSize;
private final int shardSize;
private final GeoBoundingBox geoBoundingBox;
GeoHashGridAggregatorFactory(String name, ValuesSourceConfig<ValuesSource.GeoPoint> config, int precision, int requiredSize,
GeoHashGridAggregatorFactory(String name, ValuesSourceConfig config, int precision, int requiredSize,
int shardSize, GeoBoundingBox geoBoundingBox, QueryShardContext queryShardContext,
AggregatorFactory parent, AggregatorFactories.Builder subFactoriesBuilder,
Map<String, Object> metadata) throws IOException {
@ -69,16 +74,32 @@ public class GeoHashGridAggregatorFactory extends ValuesSourceAggregatorFactory<
}
@Override
protected Aggregator doCreateInternal(final ValuesSource.GeoPoint valuesSource,
protected Aggregator doCreateInternal(final ValuesSource valuesSource,
SearchContext searchContext,
Aggregator parent,
boolean collectsFromSingleBucket,
Map<String, Object> metadata) throws IOException {
AggregatorSupplier aggregatorSupplier = queryShardContext.getValuesSourceRegistry()
.getAggregator(config.valueSourceType(), GeoHashGridAggregationBuilder.NAME);
if (aggregatorSupplier instanceof GeoGridAggregatorSupplier == false) {
throw new AggregationExecutionException("Registry miss-match - expected "
+ GeoGridAggregatorSupplier.class.getName() + ", found [" + aggregatorSupplier.getClass().toString() + "]");
}
if (collectsFromSingleBucket == false) {
return asMultiBucketAggregator(this, searchContext, parent);
}
CellIdSource cellIdSource = new CellIdSource(valuesSource, precision, geoBoundingBox, Geohash::longEncode);
return new GeoHashGridAggregator(name, factories, cellIdSource, requiredSize, shardSize,
searchContext, parent, metadata);
return ((GeoGridAggregatorSupplier) aggregatorSupplier).build(name, factories, valuesSource, precision, geoBoundingBox,
requiredSize, shardSize, searchContext, parent, metadata);
}
static void registerAggregators(ValuesSourceRegistry valuesSourceRegistry) {
valuesSourceRegistry.register(GeoHashGridAggregationBuilder.NAME, CoreValuesSourceType.GEOPOINT,
(GeoGridAggregatorSupplier) (name, factories, valuesSource, precision, geoBoundingBox, requiredSize, shardSize,
aggregationContext, parent, metadata) -> {
CellIdSource cellIdSource = new CellIdSource((ValuesSource.GeoPoint) valuesSource, precision, geoBoundingBox,
Geohash::longEncode);
return new GeoHashGridAggregator(name, factories, cellIdSource, requiredSize, shardSize, aggregationContext,
parent, metadata);
});
}
}

12
server/src/main/java/org/elasticsearch/search/aggregations/bucket/geogrid/GeoTileGridAggregationBuilder.java
View File

@ -29,9 +29,9 @@ import org.elasticsearch.index.query.QueryShardContext;
import org.elasticsearch.search.aggregations.AggregationBuilder;
import org.elasticsearch.search.aggregations.AggregatorFactories;
import org.elasticsearch.search.aggregations.AggregatorFactory;
import org.elasticsearch.search.aggregations.support.ValuesSource;
import org.elasticsearch.search.aggregations.support.ValuesSourceAggregatorFactory;
import org.elasticsearch.search.aggregations.support.ValuesSourceConfig;
import org.elasticsearch.search.aggregations.support.ValuesSourceRegistry;
public class GeoTileGridAggregationBuilder extends GeoGridAggregationBuilder {
public static final String NAME = "geotile_grid";
@ -52,6 +52,10 @@ public class GeoTileGridAggregationBuilder extends GeoGridAggregationBuilder {
super(in);
}
public static void registerAggregators(ValuesSourceRegistry valuesSourceRegistry) {
GeoTileGridAggregatorFactory.registerAggregators(valuesSourceRegistry);
}
@Override
public GeoGridAggregationBuilder precision(int precision) {
this.precision = GeoTileUtils.checkPrecisionRange(precision);
@ -59,10 +63,10 @@ public class GeoTileGridAggregationBuilder extends GeoGridAggregationBuilder {
}
@Override
protected ValuesSourceAggregatorFactory<ValuesSource.GeoPoint> createFactory(
String name, ValuesSourceConfig<ValuesSource.GeoPoint> config, int precision, int requiredSize, int shardSize,
protected ValuesSourceAggregatorFactory createFactory(
String name, ValuesSourceConfig config, int precision, int requiredSize, int shardSize,
GeoBoundingBox geoBoundingBox, QueryShardContext queryShardContext, AggregatorFactory parent,
AggregatorFactories.Builder subFactoriesBuilder, Map<String, Object> metadata ) throws IOException {
AggregatorFactories.Builder subFactoriesBuilder, Map<String, Object> metadata) throws IOException {
return new GeoTileGridAggregatorFactory(name, config, precision, requiredSize, shardSize, geoBoundingBox,
queryShardContext, parent, subFactoriesBuilder, metadata);
}

33
server/src/main/java/org/elasticsearch/search/aggregations/bucket/geogrid/GeoTileGridAggregatorFactory.java
View File

@ -21,28 +21,33 @@ package org.elasticsearch.search.aggregations.bucket.geogrid;
import org.elasticsearch.common.geo.GeoBoundingBox;
import org.elasticsearch.index.query.QueryShardContext;
import org.elasticsearch.search.aggregations.AggregationExecutionException;
import org.elasticsearch.search.aggregations.Aggregator;
import org.elasticsearch.search.aggregations.AggregatorFactories;
import org.elasticsearch.search.aggregations.AggregatorFactory;
import org.elasticsearch.search.aggregations.InternalAggregation;
import org.elasticsearch.search.aggregations.NonCollectingAggregator;
import org.elasticsearch.search.aggregations.metrics.GeoGridAggregatorSupplier;
import org.elasticsearch.search.aggregations.support.AggregatorSupplier;
import org.elasticsearch.search.aggregations.support.CoreValuesSourceType;
import org.elasticsearch.search.aggregations.support.ValuesSource;
import org.elasticsearch.search.aggregations.support.ValuesSourceAggregatorFactory;
import org.elasticsearch.search.aggregations.support.ValuesSourceConfig;
import org.elasticsearch.search.aggregations.support.ValuesSourceRegistry;
import org.elasticsearch.search.internal.SearchContext;
import java.io.IOException;
import java.util.Collections;
import java.util.Map;
public class GeoTileGridAggregatorFactory extends ValuesSourceAggregatorFactory<ValuesSource.GeoPoint> {
public class GeoTileGridAggregatorFactory extends ValuesSourceAggregatorFactory {
private final int precision;
private final int requiredSize;
private final int shardSize;
private final GeoBoundingBox geoBoundingBox;
GeoTileGridAggregatorFactory(String name, ValuesSourceConfig<ValuesSource.GeoPoint> config, int precision, int requiredSize,
GeoTileGridAggregatorFactory(String name, ValuesSourceConfig config, int precision, int requiredSize,
int shardSize, GeoBoundingBox geoBoundingBox, QueryShardContext queryShardContext,
AggregatorFactory parent, AggregatorFactories.Builder subFactoriesBuilder,
Map<String, Object> metadata) throws IOException {
@ -67,16 +72,32 @@ public class GeoTileGridAggregatorFactory extends ValuesSourceAggregatorFactory<
}
@Override
protected Aggregator doCreateInternal(final ValuesSource.GeoPoint valuesSource,
protected Aggregator doCreateInternal(final ValuesSource valuesSource,
SearchContext searchContext,
Aggregator parent,
boolean collectsFromSingleBucket,
Map<String, Object> metadata) throws IOException {
AggregatorSupplier aggregatorSupplier = queryShardContext.getValuesSourceRegistry()
.getAggregator(config.valueSourceType(), GeoTileGridAggregationBuilder.NAME);
if (aggregatorSupplier instanceof GeoGridAggregatorSupplier == false) {
throw new AggregationExecutionException("Registry miss-match - expected "
+ GeoGridAggregatorSupplier.class.getName() + ", found [" + aggregatorSupplier.getClass().toString() + "]");
}
if (collectsFromSingleBucket == false) {
return asMultiBucketAggregator(this, searchContext, parent);
}
CellIdSource cellIdSource = new CellIdSource(valuesSource, precision, geoBoundingBox, GeoTileUtils::longEncode);
return new GeoTileGridAggregator(name, factories, cellIdSource, requiredSize, shardSize,
searchContext, parent, metadata);
return ((GeoGridAggregatorSupplier) aggregatorSupplier).build(name, factories, valuesSource, precision, geoBoundingBox,
requiredSize, shardSize, searchContext, parent, metadata);
}
static void registerAggregators(ValuesSourceRegistry valuesSourceRegistry) {
valuesSourceRegistry.register(GeoTileGridAggregationBuilder.NAME, CoreValuesSourceType.GEOPOINT,
(GeoGridAggregatorSupplier) (name, factories, valuesSource, precision, geoBoundingBox, requiredSize, shardSize,
aggregationContext, parent, metadata) -> {
CellIdSource cellIdSource = new CellIdSource((ValuesSource.GeoPoint) valuesSource, precision, geoBoundingBox,
GeoTileUtils::longEncode);
return new GeoTileGridAggregator(name, factories, cellIdSource, requiredSize, shardSize, aggregationContext,
parent, metadata);
});
}
}

23
server/src/main/java/org/elasticsearch/search/aggregations/bucket/histogram/AutoDateHistogramAggregationBuilder.java
View File

@ -34,13 +34,10 @@ import org.elasticsearch.search.aggregations.AggregatorFactories.Builder;
import org.elasticsearch.search.aggregations.AggregatorFactory;
import org.elasticsearch.search.aggregations.MultiBucketConsumerService;
import org.elasticsearch.search.aggregations.support.CoreValuesSourceType;
import org.elasticsearch.search.aggregations.support.ValueType;
import org.elasticsearch.search.aggregations.support.ValuesSource;
import org.elasticsearch.search.aggregations.support.ValuesSource.Numeric;
import org.elasticsearch.search.aggregations.support.ValuesSourceAggregationBuilder;
import org.elasticsearch.search.aggregations.support.ValuesSourceAggregatorFactory;
import org.elasticsearch.search.aggregations.support.ValuesSourceConfig;
import org.elasticsearch.search.aggregations.support.ValuesSourceParserHelper;
import org.elasticsearch.search.aggregations.support.ValuesSourceType;
import java.io.IOException;
import java.time.ZoneId;
@ -50,7 +47,7 @@ import java.util.Map;
import java.util.Objects;
public class AutoDateHistogramAggregationBuilder
extends ValuesSourceAggregationBuilder<ValuesSource.Numeric, AutoDateHistogramAggregationBuilder> {
extends ValuesSourceAggregationBuilder<AutoDateHistogramAggregationBuilder> {
public static final String NAME = "auto_date_histogram";
@ -60,7 +57,7 @@ public class AutoDateHistogramAggregationBuilder
public static final ObjectParser<AutoDateHistogramAggregationBuilder, String> PARSER =
ObjectParser.fromBuilder(NAME, AutoDateHistogramAggregationBuilder::new);
static {
ValuesSourceParserHelper.declareNumericFields(PARSER, true, true, true);
ValuesSourceAggregationBuilder.declareFields(PARSER, true, true, true);
PARSER.declareInt(AutoDateHistogramAggregationBuilder::setNumBuckets, NUM_BUCKETS_FIELD);
PARSER.declareStringOrNull(AutoDateHistogramAggregationBuilder::setMinimumIntervalExpression, MINIMUM_INTERVAL_FIELD);
}
@ -115,12 +112,12 @@ public class AutoDateHistogramAggregationBuilder
/** Create a new builder with the given name. */
public AutoDateHistogramAggregationBuilder(String name) {
super(name, CoreValuesSourceType.NUMERIC, ValueType.DATE);
super(name);
}
/** Read from a stream, for internal use only. */
public AutoDateHistogramAggregationBuilder(StreamInput in) throws IOException {
super(in, CoreValuesSourceType.NUMERIC, ValueType.DATE);
super(in);
numBuckets = in.readVInt();
if (in.getVersion().onOrAfter(Version.V_7_3_0)) {
minimumIntervalExpression = in.readOptionalString();
@ -142,6 +139,12 @@ public class AutoDateHistogramAggregationBuilder
this.minimumIntervalExpression = clone.minimumIntervalExpression;
}
@Override
protected ValuesSourceType defaultValueSourceType() {
// TODO: This should probably be DATE, but we're not failing tests with BYTES, so needs more tests?
return CoreValuesSourceType.BYTES;
}
@Override
protected AggregationBuilder shallowCopy(Builder factoriesBuilder, Map<String, Object> metadata) {
return new AutoDateHistogramAggregationBuilder(this, factoriesBuilder, metadata);
@ -183,8 +186,8 @@ public class AutoDateHistogramAggregationBuilder
}
@Override
protected ValuesSourceAggregatorFactory<Numeric> innerBuild(QueryShardContext queryShardContext, ValuesSourceConfig<Numeric> config,
AggregatorFactory parent, Builder subFactoriesBuilder) throws IOException {
protected ValuesSourceAggregatorFactory innerBuild(QueryShardContext queryShardContext, ValuesSourceConfig config,
AggregatorFactory parent, Builder subFactoriesBuilder) throws IOException {
RoundingInfo[] roundings = buildRoundings(timeZone(), getMinimumIntervalExpression());
int maxRoundingInterval = Arrays.stream(roundings,0, roundings.length-1)
.map(rounding -> rounding.innerIntervals)

13
server/src/main/java/org/elasticsearch/search/aggregations/bucket/histogram/AutoDateHistogramAggregatorFactory.java
View File

@ -20,6 +20,7 @@
package org.elasticsearch.search.aggregations.bucket.histogram;
import org.elasticsearch.index.query.QueryShardContext;
import org.elasticsearch.search.aggregations.AggregationExecutionException;
import org.elasticsearch.search.aggregations.Aggregator;
import org.elasticsearch.search.aggregations.AggregatorFactories;
import org.elasticsearch.search.aggregations.AggregatorFactory;
@ -34,13 +35,13 @@ import java.io.IOException;
import java.util.Map;
public final class AutoDateHistogramAggregatorFactory
extends ValuesSourceAggregatorFactory<ValuesSource.Numeric> {
extends ValuesSourceAggregatorFactory {
private final int numBuckets;
private RoundingInfo[] roundingInfos;
public AutoDateHistogramAggregatorFactory(String name,
ValuesSourceConfig<Numeric> config,
ValuesSourceConfig config,
int numBuckets,
RoundingInfo[] roundingInfos,
QueryShardContext queryShardContext,
@ -53,15 +54,19 @@ public final class AutoDateHistogramAggregatorFactory
}
@Override
protected Aggregator doCreateInternal(Numeric valuesSource,
protected Aggregator doCreateInternal(ValuesSource valuesSource,
SearchContext searchContext,
Aggregator parent,
boolean collectsFromSingleBucket,
Map<String, Object> metadata) throws IOException {
if (valuesSource instanceof Numeric == false) {
throw new AggregationExecutionException("ValuesSource type " + valuesSource.toString() + "is not supported for aggregation " +
this.name());
}
if (collectsFromSingleBucket == false) {
return asMultiBucketAggregator(this, searchContext, parent);
}
return createAggregator(valuesSource, searchContext, parent, metadata);
return createAggregator((Numeric) valuesSource, searchContext, parent, metadata);
}
private Aggregator createAggregator(ValuesSource.Numeric valuesSource,

31
server/src/main/java/org/elasticsearch/search/aggregations/bucket/histogram/DateHistogramAggregationBuilder.java
View File

@ -36,7 +36,6 @@ import org.elasticsearch.index.mapper.DateFieldMapper;
import org.elasticsearch.index.mapper.MappedFieldType;
import org.elasticsearch.index.mapper.MappedFieldType.Relation;
import org.elasticsearch.index.query.QueryShardContext;
import org.elasticsearch.script.Script;
import org.elasticsearch.search.aggregations.AggregationBuilder;
import org.elasticsearch.search.aggregations.AggregatorFactories.Builder;
import org.elasticsearch.search.aggregations.AggregatorFactory;
@ -44,12 +43,10 @@ import org.elasticsearch.search.aggregations.BucketOrder;
import org.elasticsearch.search.aggregations.InternalOrder;
import org.elasticsearch.search.aggregations.InternalOrder.CompoundOrder;
import org.elasticsearch.search.aggregations.support.CoreValuesSourceType;
import org.elasticsearch.search.aggregations.support.ValueType;
import org.elasticsearch.search.aggregations.support.ValuesSource;
import org.elasticsearch.search.aggregations.support.ValuesSourceAggregationBuilder;
import org.elasticsearch.search.aggregations.support.ValuesSourceAggregatorFactory;
import org.elasticsearch.search.aggregations.support.ValuesSourceConfig;
import org.elasticsearch.search.aggregations.support.ValuesSourceParserHelper;
import org.elasticsearch.search.aggregations.support.ValuesSourceRegistry;
import org.elasticsearch.search.aggregations.support.ValuesSourceType;
import java.io.IOException;
@ -67,8 +64,9 @@ import static java.util.Collections.unmodifiableMap;
/**
* A builder for histograms on date fields.
*/
public class DateHistogramAggregationBuilder extends ValuesSourceAggregationBuilder<ValuesSource, DateHistogramAggregationBuilder>
public class DateHistogramAggregationBuilder extends ValuesSourceAggregationBuilder<DateHistogramAggregationBuilder>
implements DateIntervalConsumer {
public static final String NAME = "date_histogram";
public static final Map<String, Rounding.DateTimeUnit> DATE_FIELD_UNITS;
@ -97,7 +95,7 @@ public class DateHistogramAggregationBuilder extends ValuesSourceAggregationBuil
public static final ObjectParser<DateHistogramAggregationBuilder, String> PARSER =
ObjectParser.fromBuilder(NAME, DateHistogramAggregationBuilder::new);
static {
ValuesSourceParserHelper.declareAnyFields(PARSER, true, true, true);
ValuesSourceAggregationBuilder.declareFields(PARSER, true, true, true);
DateIntervalWrapper.declareIntervalFields(PARSER);
PARSER.declareField(DateHistogramAggregationBuilder::offset, p -> {
@ -119,6 +117,10 @@ public class DateHistogramAggregationBuilder extends ValuesSourceAggregationBuil
Histogram.ORDER_FIELD);
}
public static void registerAggregators(ValuesSourceRegistry valuesSourceRegistry) {
DateHistogramAggregatorFactory.registerAggregators(valuesSourceRegistry);
}
private DateIntervalWrapper dateHistogramInterval = new DateIntervalWrapper();
private long offset = 0;
private ExtendedBounds extendedBounds;
@ -128,7 +130,7 @@ public class DateHistogramAggregationBuilder extends ValuesSourceAggregationBuil
/** Create a new builder with the given name. */
public DateHistogramAggregationBuilder(String name) {
super(name, CoreValuesSourceType.ANY, ValueType.DATE);
super(name);
}
protected DateHistogramAggregationBuilder(DateHistogramAggregationBuilder clone,
@ -149,7 +151,7 @@ public class DateHistogramAggregationBuilder extends ValuesSourceAggregationBuil
/** Read from a stream, for internal use only. */
public DateHistogramAggregationBuilder(StreamInput in) throws IOException {
super(in, CoreValuesSourceType.ANY, ValueType.DATE);
super(in);
order = InternalOrder.Streams.readHistogramOrder(in, true);
keyed = in.readBoolean();
minDocCount = in.readVLong();
@ -159,9 +161,8 @@ public class DateHistogramAggregationBuilder extends ValuesSourceAggregationBuil
}
@Override
protected ValuesSourceType resolveScriptAny(Script script) {
// TODO: No idea how we'd support Range scripts here.
return CoreValuesSourceType.NUMERIC;
protected ValuesSourceType defaultValueSourceType() {
return CoreValuesSourceType.DATE;
}
@ -522,10 +523,10 @@ public class DateHistogramAggregationBuilder extends ValuesSourceAggregationBuil
}
@Override
protected ValuesSourceAggregatorFactory<ValuesSource> innerBuild(QueryShardContext queryShardContext,
ValuesSourceConfig<ValuesSource> config,
AggregatorFactory parent,
Builder subFactoriesBuilder) throws IOException {
protected ValuesSourceAggregatorFactory innerBuild(QueryShardContext queryShardContext,
ValuesSourceConfig config,
AggregatorFactory parent,
Builder subFactoriesBuilder) throws IOException {
final ZoneId tz = timeZone();
final Rounding rounding = dateHistogramInterval.createRounding(tz, offset);
final ZoneId rewrittenTimeZone = rewriteTimeZone(queryShardContext);

50
server/src/main/java/org/elasticsearch/search/aggregations/bucket/histogram/DateHistogramAggregationSupplier.java Normal file
View File

@ -0,0 +1,50 @@
/*
* Licensed to Elasticsearch under one or more contributor
* license agreements. See the NOTICE file distributed with
* this work for additional information regarding copyright
* ownership. Elasticsearch licenses this file to you under
* the Apache License, Version 2.0 (the "License"); you may
* not use this file except in compliance with the License.
* You may obtain a copy of the License at
*
* http://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing,
* software distributed under the License is distributed on an
* "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
* KIND, either express or implied. See the License for the
* specific language governing permissions and limitations
* under the License.
*/
package org.elasticsearch.search.aggregations.bucket.histogram;
import org.elasticsearch.common.Nullable;
import org.elasticsearch.common.Rounding;
import org.elasticsearch.search.DocValueFormat;
import org.elasticsearch.search.aggregations.Aggregator;
import org.elasticsearch.search.aggregations.AggregatorFactories;
import org.elasticsearch.search.aggregations.BucketOrder;
import org.elasticsearch.search.aggregations.support.AggregatorSupplier;
import org.elasticsearch.search.aggregations.support.ValuesSource;
import org.elasticsearch.search.internal.SearchContext;
import java.io.IOException;
import java.util.Map;
@FunctionalInterface
public interface DateHistogramAggregationSupplier extends AggregatorSupplier {
Aggregator build(String name,
AggregatorFactories factories,
Rounding rounding,
Rounding shardRounding,
BucketOrder order,
boolean keyed,
long minDocCount,
@Nullable ExtendedBounds extendedBounds,
@Nullable ValuesSource valuesSource,
DocValueFormat formatter,
SearchContext aggregationContext,
Aggregator parent,
Map<String, Object> metadata) throws IOException;
}

102
server/src/main/java/org/elasticsearch/search/aggregations/bucket/histogram/DateHistogramAggregatorFactory.java
View File

@ -19,22 +19,74 @@
package org.elasticsearch.search.aggregations.bucket.histogram;
import org.elasticsearch.common.Nullable;
import org.elasticsearch.common.Rounding;
import org.elasticsearch.index.mapper.RangeType;
import org.elasticsearch.index.query.QueryShardContext;
import org.elasticsearch.search.DocValueFormat;
import org.elasticsearch.search.aggregations.AggregationExecutionException;
import org.elasticsearch.search.aggregations.Aggregator;
import org.elasticsearch.search.aggregations.AggregatorFactories;
import org.elasticsearch.search.aggregations.AggregatorFactory;
import org.elasticsearch.search.aggregations.BucketOrder;
import org.elasticsearch.search.aggregations.support.AggregatorSupplier;
import org.elasticsearch.search.aggregations.support.CoreValuesSourceType;
import org.elasticsearch.search.aggregations.support.ValuesSource;
import org.elasticsearch.search.aggregations.support.ValuesSourceAggregatorFactory;
import org.elasticsearch.search.aggregations.support.ValuesSourceConfig;
import org.elasticsearch.search.aggregations.support.ValuesSourceRegistry;
import org.elasticsearch.search.internal.SearchContext;
import java.io.IOException;
import java.util.Arrays;
import java.util.Map;
public final class DateHistogramAggregatorFactory extends ValuesSourceAggregatorFactory<ValuesSource> {
public final class DateHistogramAggregatorFactory extends ValuesSourceAggregatorFactory {
public static void registerAggregators(ValuesSourceRegistry valuesSourceRegistry) {
valuesSourceRegistry.register(DateHistogramAggregationBuilder.NAME,
Arrays.asList(CoreValuesSourceType.DATE, CoreValuesSourceType.NUMERIC, CoreValuesSourceType.BOOLEAN),
(DateHistogramAggregationSupplier) (String name,
AggregatorFactories factories,
Rounding rounding,
Rounding shardRounding,
BucketOrder order,
boolean keyed,
long minDocCount,
@Nullable ExtendedBounds extendedBounds,
@Nullable ValuesSource valuesSource,
DocValueFormat formatter,
SearchContext aggregationContext,
Aggregator parent,
Map<String, Object> metadata) -> new DateHistogramAggregator(name,
factories, rounding, shardRounding, order, keyed, minDocCount, extendedBounds, (ValuesSource.Numeric) valuesSource,
formatter, aggregationContext, parent, metadata));
valuesSourceRegistry.register(DateHistogramAggregationBuilder.NAME,
CoreValuesSourceType.RANGE,
(DateHistogramAggregationSupplier) (String name,
AggregatorFactories factories,
Rounding rounding,
Rounding shardRounding,
BucketOrder order,
boolean keyed,
long minDocCount,
@Nullable ExtendedBounds extendedBounds,
@Nullable ValuesSource valuesSource,
DocValueFormat formatter,
SearchContext aggregationContext,
Aggregator parent,
Map<String, Object> metadata) -> {
ValuesSource.Range rangeValueSource = (ValuesSource.Range) valuesSource;
if (rangeValueSource.rangeType() != RangeType.DATE) {
throw new IllegalArgumentException("Expected date range type but found range type [" + rangeValueSource.rangeType().name
+ "]");
}
return new DateRangeHistogramAggregator(name,
factories, rounding, shardRounding, order, keyed, minDocCount, extendedBounds, rangeValueSource, formatter,
aggregationContext, parent, metadata); });
}
private final BucketOrder order;
private final boolean keyed;
@ -43,7 +95,7 @@ public final class DateHistogramAggregatorFactory extends ValuesSourceAggregator
private final Rounding rounding;
private final Rounding shardRounding;
public DateHistogramAggregatorFactory(String name, ValuesSourceConfig<ValuesSource> config,
public DateHistogramAggregatorFactory(String name, ValuesSourceConfig config,
BucketOrder order, boolean keyed, long minDocCount,
Rounding rounding, Rounding shardRounding, ExtendedBounds extendedBounds, QueryShardContext queryShardContext,
AggregatorFactory parent, AggregatorFactories.Builder subFactoriesBuilder,
@ -61,15 +113,6 @@ public final class DateHistogramAggregatorFactory extends ValuesSourceAggregator
return minDocCount;
}
@Override
protected ValuesSource resolveMissingAny(Object missing) {
if (missing instanceof Number) {
return ValuesSource.Numeric.EMPTY;
}
throw new IllegalArgumentException("Only numeric missing values are supported for date histogram aggregation, found ["
+ missing + "]");
}
@Override
protected Aggregator doCreateInternal(ValuesSource valuesSource,
SearchContext searchContext,
@ -79,40 +122,21 @@ public final class DateHistogramAggregatorFactory extends ValuesSourceAggregator
if (collectsFromSingleBucket == false) {
return asMultiBucketAggregator(this, searchContext, parent);
}
if (valuesSource instanceof ValuesSource.Numeric) {
return createAggregator((ValuesSource.Numeric) valuesSource, searchContext, parent, metadata);
} else if (valuesSource instanceof ValuesSource.Range) {
ValuesSource.Range rangeValueSource = (ValuesSource.Range) valuesSource;
if (rangeValueSource.rangeType() != RangeType.DATE) {
throw new IllegalArgumentException("Expected date range type but found range type [" + rangeValueSource.rangeType().name
+ "]");
}
return createRangeAggregator((ValuesSource.Range) valuesSource, searchContext, parent, metadata);
AggregatorSupplier aggregatorSupplier = queryShardContext.getValuesSourceRegistry().getAggregator(config.valueSourceType(),
DateHistogramAggregationBuilder.NAME);
if (aggregatorSupplier instanceof DateHistogramAggregationSupplier == false) {
throw new AggregationExecutionException("Registry miss-match - expected DateHistogramAggregationSupplier, found [" +
aggregatorSupplier.getClass().toString() + "]");
}
else {
throw new IllegalArgumentException("Expected one of [Date, Range] values source, found ["
+ valuesSource.toString() + "]");
}
}
private Aggregator createAggregator(ValuesSource.Numeric valuesSource, SearchContext searchContext,
Aggregator parent, Map<String, Object> metadata) throws IOException {
return new DateHistogramAggregator(name, factories, rounding, shardRounding, order, keyed, minDocCount, extendedBounds,
valuesSource, config.format(), searchContext, parent, metadata);
}
private Aggregator createRangeAggregator(ValuesSource.Range valuesSource,
SearchContext searchContext,
Aggregator parent,
Map<String, Object> metadata) throws IOException {
return new DateRangeHistogramAggregator(name, factories, rounding, shardRounding, order, keyed, minDocCount, extendedBounds,
valuesSource, config.format(), searchContext, parent, metadata);
return ((DateHistogramAggregationSupplier) aggregatorSupplier).build(name, factories, rounding, shardRounding, order, keyed,
minDocCount, extendedBounds, valuesSource, config.format(), searchContext, parent, metadata);
}
@Override
protected Aggregator createUnmapped(SearchContext searchContext,
Aggregator parent,
Map<String, Object> metadata) throws IOException {
return createAggregator(null, searchContext, parent, metadata);
return new DateHistogramAggregator(name, factories, rounding, shardRounding, order, keyed, minDocCount, extendedBounds,
null, config.format(), searchContext, parent, metadata);
}
}

28
server/src/main/java/org/elasticsearch/search/aggregations/bucket/histogram/HistogramAggregationBuilder.java
View File

@ -25,7 +25,6 @@ import org.elasticsearch.common.io.stream.StreamOutput;
import org.elasticsearch.common.xcontent.ObjectParser;
import org.elasticsearch.common.xcontent.XContentBuilder;
import org.elasticsearch.index.query.QueryShardContext;
import org.elasticsearch.script.Script;
import org.elasticsearch.search.aggregations.AggregationBuilder;
import org.elasticsearch.search.aggregations.AggregatorFactories.Builder;
import org.elasticsearch.search.aggregations.AggregatorFactory;
@ -33,12 +32,10 @@ import org.elasticsearch.search.aggregations.BucketOrder;
import org.elasticsearch.search.aggregations.InternalOrder;
import org.elasticsearch.search.aggregations.InternalOrder.CompoundOrder;
import org.elasticsearch.search.aggregations.support.CoreValuesSourceType;
import org.elasticsearch.search.aggregations.support.ValueType;
import org.elasticsearch.search.aggregations.support.ValuesSource;
import org.elasticsearch.search.aggregations.support.ValuesSourceAggregationBuilder;
import org.elasticsearch.search.aggregations.support.ValuesSourceAggregatorFactory;
import org.elasticsearch.search.aggregations.support.ValuesSourceConfig;
import org.elasticsearch.search.aggregations.support.ValuesSourceParserHelper;
import org.elasticsearch.search.aggregations.support.ValuesSourceRegistry;
import org.elasticsearch.search.aggregations.support.ValuesSourceType;
import java.io.IOException;
@ -50,7 +47,7 @@ import java.util.Objects;
* A builder for histograms on numeric fields. This builder can operate on either base numeric fields, or numeric range fields. IP range
* fields are unsupported, and will throw at the factory layer.
*/
public class HistogramAggregationBuilder extends ValuesSourceAggregationBuilder<ValuesSource, HistogramAggregationBuilder> {
public class HistogramAggregationBuilder extends ValuesSourceAggregationBuilder<HistogramAggregationBuilder> {
public static final String NAME = "histogram";
private static final ObjectParser<double[], Void> EXTENDED_BOUNDS_PARSER = new ObjectParser<>(
@ -64,7 +61,7 @@ public class HistogramAggregationBuilder extends ValuesSourceAggregationBuilder<
public static final ObjectParser<HistogramAggregationBuilder, String> PARSER =
ObjectParser.fromBuilder(NAME, HistogramAggregationBuilder::new);
static {
ValuesSourceParserHelper.declareAnyFields(PARSER, true, true);
ValuesSourceAggregationBuilder.declareFields(PARSER, true, true, false);
PARSER.declareDouble(HistogramAggregationBuilder::interval, Histogram.INTERVAL_FIELD);
@ -82,6 +79,10 @@ public class HistogramAggregationBuilder extends ValuesSourceAggregationBuilder<
Histogram.ORDER_FIELD);
}
public static void registerAggregators(ValuesSourceRegistry valuesSourceRegistry) {
HistogramAggregatorFactory.registerAggregators(valuesSourceRegistry);
}
private double interval;
private double offset = 0;
private double minBound = Double.POSITIVE_INFINITY;
@ -91,14 +92,13 @@ public class HistogramAggregationBuilder extends ValuesSourceAggregationBuilder<
private long minDocCount = 0;
@Override
protected ValuesSourceType resolveScriptAny(Script script) {
// TODO: No idea how we'd support Range scripts here.
protected ValuesSourceType defaultValueSourceType() {
return CoreValuesSourceType.NUMERIC;
}
/** Create a new builder with the given name. */
public HistogramAggregationBuilder(String name) {
super(name, CoreValuesSourceType.ANY, ValueType.NUMERIC);
super(name);
}
protected HistogramAggregationBuilder(HistogramAggregationBuilder clone, Builder factoriesBuilder, Map<String, Object> metadata) {
@ -119,7 +119,7 @@ public class HistogramAggregationBuilder extends ValuesSourceAggregationBuilder<
/** Read from a stream, for internal use only. */
public HistogramAggregationBuilder(StreamInput in) throws IOException {
super(in, CoreValuesSourceType.ANY, ValueType.NUMERIC);
super(in);
order = InternalOrder.Streams.readHistogramOrder(in, true);
keyed = in.readBoolean();
minDocCount = in.readVLong();
@ -301,10 +301,10 @@ public class HistogramAggregationBuilder extends ValuesSourceAggregationBuilder<
}
@Override
protected ValuesSourceAggregatorFactory<ValuesSource> innerBuild(QueryShardContext queryShardContext,
ValuesSourceConfig<ValuesSource> config,
AggregatorFactory parent,
Builder subFactoriesBuilder) throws IOException {
protected ValuesSourceAggregatorFactory innerBuild(QueryShardContext queryShardContext,
ValuesSourceConfig config,
AggregatorFactory parent,
Builder subFactoriesBuilder) throws IOException {
return new HistogramAggregatorFactory(name, config, interval, offset, order, keyed, minDocCount, minBound, maxBound,
queryShardContext, parent, subFactoriesBuilder, metadata);
}

80
server/src/main/java/org/elasticsearch/search/aggregations/bucket/histogram/HistogramAggregatorFactory.java
View File

@ -20,23 +20,31 @@
package org.elasticsearch.search.aggregations.bucket.histogram;
import org.elasticsearch.index.query.QueryShardContext;
import org.elasticsearch.search.DocValueFormat;
import org.elasticsearch.search.aggregations.AggregationExecutionException;
import org.elasticsearch.search.aggregations.Aggregator;
import org.elasticsearch.search.aggregations.AggregatorFactories;
import org.elasticsearch.search.aggregations.AggregatorFactory;
import org.elasticsearch.search.aggregations.BucketOrder;
import org.elasticsearch.search.aggregations.support.AggregatorSupplier;
import org.elasticsearch.search.aggregations.support.CoreValuesSourceType;
import org.elasticsearch.search.aggregations.support.HistogramAggregatorSupplier;
import org.elasticsearch.search.aggregations.support.ValuesSource;
import org.elasticsearch.search.aggregations.support.ValuesSourceAggregatorFactory;
import org.elasticsearch.search.aggregations.support.ValuesSourceConfig;
import org.elasticsearch.search.aggregations.support.ValuesSourceRegistry;
import org.elasticsearch.search.internal.SearchContext;
import java.io.IOException;
import java.util.Arrays;
import java.util.Collections;
import java.util.Map;
/**
* Constructs the per-shard aggregator instance for histogram aggregation. Selects the numeric or range field implementation based on the
* field type.
*/
public final class HistogramAggregatorFactory extends ValuesSourceAggregatorFactory<ValuesSource> {
public final class HistogramAggregatorFactory extends ValuesSourceAggregatorFactory {
private final double interval, offset;
private final BucketOrder order;
@ -44,17 +52,47 @@ public final class HistogramAggregatorFactory extends ValuesSourceAggregatorFact
private final long minDocCount;
private final double minBound, maxBound;
@Override
protected ValuesSource resolveMissingAny(Object missing) {
if (missing instanceof Number) {
return ValuesSource.Numeric.EMPTY;
}
throw new IllegalArgumentException("Only numeric missing values are supported for histogram aggregation, found ["
+ missing + "]");
// TODO: Registration should happen on the actual aggregator classes, but I don't want to set up the whole dynamic loading thing yet
static void registerAggregators(ValuesSourceRegistry valuesSourceRegistry) {
valuesSourceRegistry.register(HistogramAggregationBuilder.NAME, CoreValuesSourceType.RANGE,
new HistogramAggregatorSupplier() {
@Override
public Aggregator build(String name, AggregatorFactories factories, double interval, double offset,
BucketOrder order, boolean keyed, long minDocCount, double minBound, double maxBound,
ValuesSource valuesSource, DocValueFormat formatter, SearchContext context,
Aggregator parent,
Map<String, Object> metadata) throws IOException {
ValuesSource.Range rangeValueSource = (ValuesSource.Range) valuesSource;
if (rangeValueSource.rangeType().isNumeric() == false) {
throw new IllegalArgumentException("Expected numeric range type but found non-numeric range ["
+ rangeValueSource.rangeType().name + "]");
}
return new RangeHistogramAggregator(name, factories, interval, offset, order, keyed, minDocCount, minBound,
maxBound, rangeValueSource, formatter, context, parent, metadata);
}
}
);
valuesSourceRegistry.register(HistogramAggregationBuilder.NAME,
Collections.unmodifiableList(Arrays.asList(CoreValuesSourceType.NUMERIC,
CoreValuesSourceType.DATE,
CoreValuesSourceType.BOOLEAN)),
new HistogramAggregatorSupplier() {
@Override
public Aggregator build(String name, AggregatorFactories factories, double interval, double offset,
BucketOrder order, boolean keyed, long minDocCount, double minBound, double maxBound,
ValuesSource valuesSource, DocValueFormat formatter, SearchContext context,
Aggregator parent,
Map<String, Object> metadata) throws IOException {
return new NumericHistogramAggregator(name, factories, interval, offset, order, keyed, minDocCount, minBound,
maxBound, (ValuesSource.Numeric) valuesSource, formatter, context, parent, metadata);
}
}
);
}
public HistogramAggregatorFactory(String name,
ValuesSourceConfig<ValuesSource> config,
ValuesSourceConfig config,
double interval,
double offset,
BucketOrder order,
@ -89,22 +127,16 @@ public final class HistogramAggregatorFactory extends ValuesSourceAggregatorFact
if (collectsFromSingleBucket == false) {
return asMultiBucketAggregator(this, searchContext, parent);
}
if (valuesSource instanceof ValuesSource.Numeric) {
return new NumericHistogramAggregator(name, factories, interval, offset, order, keyed, minDocCount, minBound, maxBound,
(ValuesSource.Numeric) valuesSource, config.format(), searchContext, parent, metadata);
} else if (valuesSource instanceof ValuesSource.Range) {
ValuesSource.Range rangeValueSource = (ValuesSource.Range) valuesSource;
if (rangeValueSource.rangeType().isNumeric() == false) {
throw new IllegalArgumentException("Expected numeric range type but found non-numeric range ["
+ rangeValueSource.rangeType().name + "]");
}
return new RangeHistogramAggregator(name, factories, interval, offset, order, keyed, minDocCount, minBound, maxBound,
(ValuesSource.Range) valuesSource, config.format(), searchContext, parent, metadata);
}
else {
throw new IllegalArgumentException("Expected one of [Numeric, Range] values source, found ["
+ valuesSource.toString() + "]");
AggregatorSupplier aggregatorSupplier = queryShardContext.getValuesSourceRegistry().getAggregator(config.valueSourceType(),
HistogramAggregationBuilder.NAME);
if (aggregatorSupplier instanceof HistogramAggregatorSupplier == false) {
throw new AggregationExecutionException("Registry miss-match - expected HistogramAggregatorSupplier, found [" +
aggregatorSupplier.getClass().toString() + "]");
}
HistogramAggregatorSupplier histogramAggregatorSupplier = (HistogramAggregatorSupplier) aggregatorSupplier;
return histogramAggregatorSupplier.build(name, factories, interval, offset, order, keyed, minDocCount, minBound, maxBound,
valuesSource, config.format(), searchContext, parent, metadata);
}
@Override

4
server/src/main/java/org/elasticsearch/search/aggregations/bucket/histogram/NumericHistogramAggregator.java
View File

@ -50,7 +50,7 @@ import java.util.Map;
* written as {@code interval * x + offset} and yet is less than or equal to
* {@code value}.
*/
class NumericHistogramAggregator extends BucketsAggregator {
public class NumericHistogramAggregator extends BucketsAggregator {
private final ValuesSource.Numeric valuesSource;
private final DocValueFormat formatter;
@ -62,7 +62,7 @@ class NumericHistogramAggregator extends BucketsAggregator {
private final LongHash bucketOrds;
NumericHistogramAggregator(String name, AggregatorFactories factories, double interval, double offset,
public NumericHistogramAggregator(String name, AggregatorFactories factories, double interval, double offset,
BucketOrder order, boolean keyed, long minDocCount, double minBound, double maxBound,
@Nullable ValuesSource.Numeric valuesSource, DocValueFormat formatter,
SearchContext context, Aggregator parent, Map<String, Object> metadata) throws IOException {

2
server/src/main/java/org/elasticsearch/search/aggregations/bucket/histogram/RangeHistogramAggregator.java
View File

@ -56,7 +56,7 @@ public class RangeHistogramAggregator extends BucketsAggregator {
private final LongHash bucketOrds;
RangeHistogramAggregator(String name, AggregatorFactories factories, double interval, double offset,
public RangeHistogramAggregator(String name, AggregatorFactories factories, double interval, double offset,
BucketOrder order, boolean keyed, long minDocCount, double minBound, double maxBound,
@Nullable ValuesSource.Range valuesSource, DocValueFormat formatter,
SearchContext context, Aggregator parent, Map<String, Object> metadata) throws IOException {

29
server/src/main/java/org/elasticsearch/search/aggregations/bucket/missing/MissingAggregationBuilder.java
View File

@ -29,33 +29,36 @@ import org.elasticsearch.search.aggregations.AggregationBuilder;
import org.elasticsearch.search.aggregations.AggregatorFactories.Builder;
import org.elasticsearch.search.aggregations.AggregatorFactory;
import org.elasticsearch.search.aggregations.support.CoreValuesSourceType;
import org.elasticsearch.search.aggregations.support.ValueType;
import org.elasticsearch.search.aggregations.support.ValuesSource;
import org.elasticsearch.search.aggregations.support.ValuesSourceAggregationBuilder;
import org.elasticsearch.search.aggregations.support.ValuesSourceAggregatorFactory;
import org.elasticsearch.search.aggregations.support.ValuesSourceConfig;
import org.elasticsearch.search.aggregations.support.ValuesSourceParserHelper;
import org.elasticsearch.search.aggregations.support.ValuesSourceType;
import java.io.IOException;
import java.util.Map;
public class MissingAggregationBuilder extends ValuesSourceAggregationBuilder<ValuesSource, MissingAggregationBuilder> {
public class MissingAggregationBuilder extends ValuesSourceAggregationBuilder<MissingAggregationBuilder> {
public static final String NAME = "missing";
public static final ObjectParser<MissingAggregationBuilder, String> PARSER =
ObjectParser.fromBuilder(NAME, name -> new MissingAggregationBuilder(name, null));
ObjectParser.fromBuilder(NAME, MissingAggregationBuilder::new);
static {
ValuesSourceParserHelper.declareAnyFields(PARSER, true, true);
ValuesSourceAggregationBuilder.declareFields(PARSER, true, true, false);
}
public MissingAggregationBuilder(String name, ValueType targetValueType) {
super(name, CoreValuesSourceType.ANY, targetValueType);
public MissingAggregationBuilder(String name) {
super(name);
}
protected MissingAggregationBuilder(MissingAggregationBuilder clone, Builder factoriesBuilder, Map<String, Object> metadata) {
super(clone, factoriesBuilder, metadata);
}
@Override
protected ValuesSourceType defaultValueSourceType() {
return CoreValuesSourceType.BYTES;
}
@Override
protected AggregationBuilder shallowCopy(Builder factoriesBuilder, Map<String, Object> metadata) {
return new MissingAggregationBuilder(this, factoriesBuilder, metadata);
@ -65,7 +68,7 @@ public class MissingAggregationBuilder extends ValuesSourceAggregationBuilder<Va
* Read from a stream.
*/
public MissingAggregationBuilder(StreamInput in) throws IOException {
super(in, CoreValuesSourceType.ANY);
super(in);
}
@Override
@ -84,10 +87,10 @@ public class MissingAggregationBuilder extends ValuesSourceAggregationBuilder<Va
}
@Override
protected ValuesSourceAggregatorFactory<ValuesSource> innerBuild(QueryShardContext queryShardContext,
ValuesSourceConfig<ValuesSource> config,
AggregatorFactory parent,
Builder subFactoriesBuilder) throws IOException {
protected ValuesSourceAggregatorFactory innerBuild(QueryShardContext queryShardContext,
ValuesSourceConfig config,
AggregatorFactory parent,
Builder subFactoriesBuilder) throws IOException {
return new MissingAggregatorFactory(name, config, queryShardContext, parent, subFactoriesBuilder, metadata);
}

7
server/src/main/java/org/elasticsearch/search/aggregations/bucket/missing/MissingAggregatorFactory.java
View File

@ -31,10 +31,11 @@ import org.elasticsearch.search.internal.SearchContext;
import java.io.IOException;
import java.util.Map;
public class MissingAggregatorFactory extends ValuesSourceAggregatorFactory<ValuesSource> {
public class MissingAggregatorFactory extends ValuesSourceAggregatorFactory {
public MissingAggregatorFactory(String name, ValuesSourceConfig<ValuesSource> config, QueryShardContext queryShardContext,
AggregatorFactory parent, AggregatorFactories.Builder subFactoriesBuilder, Map<String, Object> metadata) throws IOException {
public MissingAggregatorFactory(String name, ValuesSourceConfig config, QueryShardContext queryShardContext,
AggregatorFactory parent, AggregatorFactories.Builder subFactoriesBuilder,
Map<String, Object> metadata) throws IOException {
super(name, config, queryShardContext, parent, subFactoriesBuilder, metadata);
}

61
server/src/main/java/org/elasticsearch/search/aggregations/bucket/range/AbstractRangeAggregatorFactory.java
View File

@ -20,39 +20,68 @@
package org.elasticsearch.search.aggregations.bucket.range;
import org.elasticsearch.index.query.QueryShardContext;
import org.elasticsearch.search.DocValueFormat;
import org.elasticsearch.search.aggregations.AggregationExecutionException;
import org.elasticsearch.search.aggregations.Aggregator;
import org.elasticsearch.search.aggregations.AggregatorFactories;
import org.elasticsearch.search.aggregations.AggregatorFactory;
import org.elasticsearch.search.aggregations.bucket.range.RangeAggregator.Range;
import org.elasticsearch.search.aggregations.bucket.range.RangeAggregator.Unmapped;
import org.elasticsearch.search.aggregations.support.AggregatorSupplier;
import org.elasticsearch.search.aggregations.support.CoreValuesSourceType;
import org.elasticsearch.search.aggregations.support.ValuesSource;
import org.elasticsearch.search.aggregations.support.ValuesSource.Numeric;
import org.elasticsearch.search.aggregations.support.ValuesSourceAggregatorFactory;
import org.elasticsearch.search.aggregations.support.ValuesSourceConfig;
import org.elasticsearch.search.aggregations.support.ValuesSourceRegistry;
import org.elasticsearch.search.internal.SearchContext;
import java.io.IOException;
import java.util.Arrays;
import java.util.Map;
public class AbstractRangeAggregatorFactory<R extends Range> extends ValuesSourceAggregatorFactory<ValuesSource.Numeric> {
public class AbstractRangeAggregatorFactory<R extends Range> extends ValuesSourceAggregatorFactory {
private final InternalRange.Factory<?, ?> rangeFactory;
private final R[] ranges;
private final boolean keyed;
private final String aggregationTypeName;
public static void registerAggregators(ValuesSourceRegistry valuesSourceRegistry, String aggregationName) {
valuesSourceRegistry.register(aggregationName,
Arrays.asList(CoreValuesSourceType.NUMERIC, CoreValuesSourceType.DATE, CoreValuesSourceType.BOOLEAN),
new RangeAggregatorSupplier() {
@Override
public Aggregator build(String name,
AggregatorFactories factories,
Numeric valuesSource,
DocValueFormat format,
InternalRange.Factory rangeFactory,
Range[] ranges,
boolean keyed,
SearchContext context,
Aggregator parent,
Map<String, Object> metadata) throws IOException {
return new RangeAggregator(name, factories, valuesSource, format, rangeFactory, ranges, keyed, context, parent,
metadata);
}
});
}
public AbstractRangeAggregatorFactory(String name,
ValuesSourceConfig<Numeric> config,
R[] ranges,
boolean keyed,
InternalRange.Factory<?, ?> rangeFactory,
QueryShardContext queryShardContext,
AggregatorFactory parent,
AggregatorFactories.Builder subFactoriesBuilder,
Map<String, Object> metadata) throws IOException {
String aggregationTypeName,
ValuesSourceConfig config,
R[] ranges,
boolean keyed,
InternalRange.Factory<?, ?> rangeFactory,
QueryShardContext queryShardContext,
AggregatorFactory parent,
AggregatorFactories.Builder subFactoriesBuilder,
Map<String, Object> metadata) throws IOException {
super(name, config, queryShardContext, parent, subFactoriesBuilder, metadata);
this.ranges = ranges;
this.keyed = keyed;
this.rangeFactory = rangeFactory;
this.aggregationTypeName = aggregationTypeName;
}
@Override
@ -63,14 +92,18 @@ public class AbstractRangeAggregatorFactory<R extends Range> extends ValuesSourc
}
@Override
protected Aggregator doCreateInternal(Numeric valuesSource,
protected Aggregator doCreateInternal(ValuesSource valuesSource,
SearchContext searchContext,
Aggregator parent,
boolean collectsFromSingleBucket,
Map<String, Object> metadata) throws IOException {
return new RangeAggregator(name, factories, valuesSource, config.format(), rangeFactory, ranges, keyed, searchContext, parent,
metadata);
AggregatorSupplier aggregatorSupplier = queryShardContext.getValuesSourceRegistry().getAggregator(config.valueSourceType(),
aggregationTypeName);
if (aggregatorSupplier instanceof RangeAggregatorSupplier == false) {
throw new AggregationExecutionException("Registry miss-match - expected RangeAggregatorSupplier, found [" +
aggregatorSupplier.getClass().toString() + "]");
}
return ((RangeAggregatorSupplier)aggregatorSupplier).build(name, factories, (Numeric) valuesSource, config.format(), rangeFactory,
ranges, keyed, searchContext, parent, metadata);
}
}

15
server/src/main/java/org/elasticsearch/search/aggregations/bucket/range/AbstractRangeBuilder.java
View File

@ -26,8 +26,8 @@ import org.elasticsearch.common.io.stream.Writeable;
import org.elasticsearch.common.xcontent.XContentBuilder;
import org.elasticsearch.search.aggregations.AggregatorFactories;
import org.elasticsearch.search.aggregations.bucket.range.RangeAggregator.Range;
import org.elasticsearch.search.aggregations.support.ValuesSource;
import org.elasticsearch.search.aggregations.support.ValuesSourceAggregationBuilder;
import org.elasticsearch.search.aggregations.support.ValuesSourceType;
import java.io.IOException;
import java.util.ArrayList;
@ -37,13 +37,14 @@ import java.util.Objects;
import java.util.function.Function;
public abstract class AbstractRangeBuilder<AB extends AbstractRangeBuilder<AB, R>, R extends Range>
extends ValuesSourceAggregationBuilder<ValuesSource.Numeric, AB> {
extends ValuesSourceAggregationBuilder<AB> {
protected final InternalRange.Factory<?, ?> rangeFactory;
protected List<R> ranges = new ArrayList<>();
protected boolean keyed = false;
protected AbstractRangeBuilder(String name, InternalRange.Factory<?, ?> rangeFactory) {
super(name, rangeFactory.getValueSourceType(), rangeFactory.getValueType());
super(name);
this.rangeFactory = rangeFactory;
}
@ -60,12 +61,18 @@ public abstract class AbstractRangeBuilder<AB extends AbstractRangeBuilder<AB, R
*/
protected AbstractRangeBuilder(StreamInput in, InternalRange.Factory<?, ?> rangeFactory, Writeable.Reader<R> rangeReader)
throws IOException {
super(in, rangeFactory.getValueSourceType(), rangeFactory.getValueType());
super(in);
this.rangeFactory = rangeFactory;
ranges = in.readList(rangeReader);
keyed = in.readBoolean();
}
@Override
protected ValuesSourceType defaultValueSourceType() {
// Copied over from the old targetValueType setting. Not sure what cases this is still relevant for. --Tozzi 2020-01-13
return rangeFactory.getValueSourceType();
}
/**
* Resolve any strings in the ranges so we have a number value for the from
* and to of each range. The ranges are also sorted before being returned.

13
server/src/main/java/org/elasticsearch/search/aggregations/bucket/range/BinaryRangeAggregatorFactory.java
View File

@ -19,6 +19,7 @@
package org.elasticsearch.search.aggregations.bucket.range;
import org.elasticsearch.index.query.QueryShardContext;
import org.elasticsearch.search.aggregations.AggregationExecutionException;
import org.elasticsearch.search.aggregations.Aggregator;
import org.elasticsearch.search.aggregations.AggregatorFactories.Builder;
import org.elasticsearch.search.aggregations.AggregatorFactory;
@ -32,13 +33,13 @@ import java.util.List;
import java.util.Map;
public class BinaryRangeAggregatorFactory
extends ValuesSourceAggregatorFactory<ValuesSource.Bytes> {
extends ValuesSourceAggregatorFactory {
private final List<BinaryRangeAggregator.Range> ranges;
private final boolean keyed;
public BinaryRangeAggregatorFactory(String name,
ValuesSourceConfig<ValuesSource.Bytes> config,
ValuesSourceConfig config,
List<BinaryRangeAggregator.Range> ranges, boolean keyed,
QueryShardContext queryShardContext,
AggregatorFactory parent, Builder subFactoriesBuilder,
@ -55,11 +56,15 @@ public class BinaryRangeAggregatorFactory
}
@Override
protected Aggregator doCreateInternal(ValuesSource.Bytes valuesSource,
protected Aggregator doCreateInternal(ValuesSource valuesSource,
SearchContext searchContext, Aggregator parent,
boolean collectsFromSingleBucket,
Map<String, Object> metadata) throws IOException {
return new BinaryRangeAggregator(name, factories, valuesSource, config.format(),
if (valuesSource instanceof ValuesSource.Bytes == false) {
throw new AggregationExecutionException("ValuesSource type " + valuesSource.toString() + "is not supported for aggregation " +
this.name());
}
return new BinaryRangeAggregator(name, factories, (ValuesSource.Bytes) valuesSource, config.format(),
ranges, keyed, searchContext, parent, metadata);
}

19
server/src/main/java/org/elasticsearch/search/aggregations/bucket/range/DateRangeAggregationBuilder.java
View File

@ -27,9 +27,11 @@ import org.elasticsearch.search.DocValueFormat;
import org.elasticsearch.search.aggregations.AggregationBuilder;
import org.elasticsearch.search.aggregations.AggregatorFactories.Builder;
import org.elasticsearch.search.aggregations.AggregatorFactory;
import org.elasticsearch.search.aggregations.support.ValuesSource.Numeric;
import org.elasticsearch.search.aggregations.support.CoreValuesSourceType;
import org.elasticsearch.search.aggregations.support.ValuesSourceAggregationBuilder;
import org.elasticsearch.search.aggregations.support.ValuesSourceConfig;
import org.elasticsearch.search.aggregations.support.ValuesSourceParserHelper;
import org.elasticsearch.search.aggregations.support.ValuesSourceRegistry;
import org.elasticsearch.search.aggregations.support.ValuesSourceType;
import java.io.IOException;
import java.time.ZonedDateTime;
@ -41,7 +43,7 @@ public class DateRangeAggregationBuilder extends AbstractRangeBuilder<DateRangeA
public static final ObjectParser<DateRangeAggregationBuilder, String> PARSER =
ObjectParser.fromBuilder(NAME, DateRangeAggregationBuilder::new);
static {
ValuesSourceParserHelper.declareNumericFields(PARSER, true, true, true);
ValuesSourceAggregationBuilder.declareFields(PARSER, true, true, true);
PARSER.declareBoolean(DateRangeAggregationBuilder::keyed, RangeAggregator.KEYED_FIELD);
PARSER.declareObjectArray((agg, ranges) -> {
@ -56,6 +58,10 @@ public class DateRangeAggregationBuilder extends AbstractRangeBuilder<DateRangeA
return RangeAggregator.Range.fromXContent(parser);
}
public static void registerAggregators(ValuesSourceRegistry valuesSourceRegistry) {
AbstractRangeAggregatorFactory.registerAggregators(valuesSourceRegistry, NAME);
}
public DateRangeAggregationBuilder(String name) {
super(name, InternalDateRange.FACTORY);
}
@ -81,6 +87,11 @@ public class DateRangeAggregationBuilder extends AbstractRangeBuilder<DateRangeA
return NAME;
}
@Override
protected ValuesSourceType defaultValueSourceType() {
return CoreValuesSourceType.DATE;
}
/**
* Add a new range to this aggregation.
*
@ -285,7 +296,7 @@ public class DateRangeAggregationBuilder extends AbstractRangeBuilder<DateRangeA
}
@Override
protected DateRangeAggregatorFactory innerBuild(QueryShardContext queryShardContext, ValuesSourceConfig<Numeric> config,
protected DateRangeAggregatorFactory innerBuild(QueryShardContext queryShardContext, ValuesSourceConfig config,
AggregatorFactory parent, Builder subFactoriesBuilder) throws IOException {
// We need to call processRanges here so they are parsed and we know whether `now` has been used before we make
// the decision of whether to cache the request

20
server/src/main/java/org/elasticsearch/search/aggregations/bucket/range/DateRangeAggregatorFactory.java
View File

@ -22,7 +22,6 @@ package org.elasticsearch.search.aggregations.bucket.range;
import org.elasticsearch.index.query.QueryShardContext;
import org.elasticsearch.search.aggregations.AggregatorFactories;
import org.elasticsearch.search.aggregations.AggregatorFactory;
import org.elasticsearch.search.aggregations.support.ValuesSource.Numeric;
import org.elasticsearch.search.aggregations.support.ValuesSourceConfig;
import java.io.IOException;
@ -31,15 +30,16 @@ import java.util.Map;
public class DateRangeAggregatorFactory extends AbstractRangeAggregatorFactory<RangeAggregator.Range> {
public DateRangeAggregatorFactory(String name,
ValuesSourceConfig<Numeric> config,
RangeAggregator.Range[] ranges,
boolean keyed,
InternalRange.Factory<?, ?> rangeFactory,
QueryShardContext queryShardContext,
AggregatorFactory parent,
AggregatorFactories.Builder subFactoriesBuilder,
Map<String, Object> metadata) throws IOException {
super(name, config, ranges, keyed, rangeFactory, queryShardContext, parent, subFactoriesBuilder, metadata);
ValuesSourceConfig config,
RangeAggregator.Range[] ranges,
boolean keyed,
InternalRange.Factory<?, ?> rangeFactory,
QueryShardContext queryShardContext,
AggregatorFactory parent,
AggregatorFactories.Builder subFactoriesBuilder,
Map<String, Object> metadata) throws IOException {
super(name, DateRangeAggregationBuilder.NAME, config, ranges, keyed, rangeFactory, queryShardContext, parent, subFactoriesBuilder,
metadata);
}
}

28
server/src/main/java/org/elasticsearch/search/aggregations/bucket/range/GeoDistanceAggregationBuilder.java
View File

@ -35,11 +35,11 @@ import org.elasticsearch.index.query.QueryShardContext;
import org.elasticsearch.search.aggregations.AggregationBuilder;
import org.elasticsearch.search.aggregations.AggregatorFactories.Builder;
import org.elasticsearch.search.aggregations.AggregatorFactory;
import org.elasticsearch.search.aggregations.support.ValuesSource;
import org.elasticsearch.search.aggregations.support.CoreValuesSourceType;
import org.elasticsearch.search.aggregations.support.ValuesSourceAggregationBuilder;
import org.elasticsearch.search.aggregations.support.ValuesSourceAggregatorFactory;
import org.elasticsearch.search.aggregations.support.ValuesSourceConfig;
import org.elasticsearch.search.aggregations.support.ValuesSourceParserHelper;
import org.elasticsearch.search.aggregations.support.ValuesSourceType;
import java.io.IOException;
import java.util.ArrayList;
@ -51,7 +51,7 @@ import static org.elasticsearch.search.aggregations.bucket.range.RangeAggregator
import static org.elasticsearch.search.aggregations.bucket.range.RangeAggregator.Range.KEY_FIELD;
import static org.elasticsearch.search.aggregations.bucket.range.RangeAggregator.Range.TO_FIELD;
public class GeoDistanceAggregationBuilder extends ValuesSourceAggregationBuilder<ValuesSource.GeoPoint, GeoDistanceAggregationBuilder> {
public class GeoDistanceAggregationBuilder extends ValuesSourceAggregationBuilder<GeoDistanceAggregationBuilder> {
public static final String NAME = "geo_distance";
static final ParseField ORIGIN_FIELD = new ParseField("origin", "center", "point", "por");
static final ParseField UNIT_FIELD = new ParseField("unit");
@ -60,7 +60,7 @@ public class GeoDistanceAggregationBuilder extends ValuesSourceAggregationBuilde
private static final ObjectParser<GeoDistanceAggregationBuilder, Void> PARSER;
static {
PARSER = new ObjectParser<>(GeoDistanceAggregationBuilder.NAME);
ValuesSourceParserHelper.declareGeoFields(PARSER, true, false);
ValuesSourceAggregationBuilder.declareFields(PARSER, true, false, false);
PARSER.declareBoolean(GeoDistanceAggregationBuilder::keyed, RangeAggregator.KEYED_FIELD);
@ -230,15 +230,16 @@ public class GeoDistanceAggregationBuilder extends ValuesSourceAggregationBuilde
private GeoDistanceAggregationBuilder(String name, GeoPoint origin,
InternalRange.Factory<InternalGeoDistance.Bucket, InternalGeoDistance> rangeFactory) {
super(name, rangeFactory.getValueSourceType(), rangeFactory.getValueType());
super(name);
this.origin = origin;
}
/**
* Read from a stream.
*/
public GeoDistanceAggregationBuilder(StreamInput in) throws IOException {
super(in, InternalGeoDistance.FACTORY.getValueSourceType(), InternalGeoDistance.FACTORY.getValueType());
super(in);
origin = new GeoPoint(in.readDouble(), in.readDouble());
int size = in.readVInt();
ranges = new ArrayList<>(size);
@ -264,6 +265,13 @@ public class GeoDistanceAggregationBuilder extends ValuesSourceAggregationBuilde
this.ranges = new ArrayList<>(clone.ranges);
}
@Override
protected ValuesSourceType defaultValueSourceType() {
// TODO: This should probably not be BYTES, but we're not failing tests with BYTES, so needs more tests?
// TODO: this should set defaultValuesSourceType to GEOPOINT
return CoreValuesSourceType.BYTES;
}
@Override
protected AggregationBuilder shallowCopy(Builder factoriesBuilder, Map<String, Object> metadata) {
return new GeoDistanceAggregationBuilder(this, factoriesBuilder, metadata);
@ -416,10 +424,10 @@ public class GeoDistanceAggregationBuilder extends ValuesSourceAggregationBuilde
}
@Override
protected ValuesSourceAggregatorFactory<ValuesSource.GeoPoint> innerBuild(QueryShardContext queryShardContext,
ValuesSourceConfig<ValuesSource.GeoPoint> config,
AggregatorFactory parent,
Builder subFactoriesBuilder) throws IOException {
protected ValuesSourceAggregatorFactory innerBuild(QueryShardContext queryShardContext,
ValuesSourceConfig config,
AggregatorFactory parent,
Builder subFactoriesBuilder) throws IOException {
Range[] ranges = this.ranges.toArray(new Range[this.range().size()]);
if (ranges.length == 0) {
throw new IllegalArgumentException("No [ranges] specified for the [" + this.getName() + "] aggregation");

19
server/src/main/java/org/elasticsearch/search/aggregations/bucket/range/GeoDistanceRangeAggregatorFactory.java
View File

@ -29,6 +29,7 @@ import org.elasticsearch.index.fielddata.MultiGeoPointValues;
import org.elasticsearch.index.fielddata.SortedBinaryDocValues;
import org.elasticsearch.index.fielddata.SortedNumericDoubleValues;
import org.elasticsearch.index.query.QueryShardContext;
import org.elasticsearch.search.aggregations.AggregationExecutionException;
import org.elasticsearch.search.aggregations.Aggregator;
import org.elasticsearch.search.aggregations.AggregatorFactories;
import org.elasticsearch.search.aggregations.AggregatorFactory;
@ -42,7 +43,7 @@ import java.io.IOException;
import java.util.Map;
public class GeoDistanceRangeAggregatorFactory
extends ValuesSourceAggregatorFactory<ValuesSource.GeoPoint> {
extends ValuesSourceAggregatorFactory {
private final InternalRange.Factory<InternalGeoDistance.Bucket, InternalGeoDistance> rangeFactory = InternalGeoDistance.FACTORY;
private final GeoPoint origin;
@ -51,9 +52,11 @@ public class GeoDistanceRangeAggregatorFactory
private final GeoDistance distanceType;
private final boolean keyed;
public GeoDistanceRangeAggregatorFactory(String name, ValuesSourceConfig<ValuesSource.GeoPoint> config, GeoPoint origin,
Range[] ranges, DistanceUnit unit, GeoDistance distanceType, boolean keyed, QueryShardContext queryShardContext,
AggregatorFactory parent, AggregatorFactories.Builder subFactoriesBuilder, Map<String, Object> metadata) throws IOException {
public GeoDistanceRangeAggregatorFactory(String name, ValuesSourceConfig config, GeoPoint origin,
Range[] ranges, DistanceUnit unit, GeoDistance distanceType, boolean keyed,
QueryShardContext queryShardContext, AggregatorFactory parent,
AggregatorFactories.Builder subFactoriesBuilder,
Map<String, Object> metadata) throws IOException {
super(name, config, queryShardContext, parent, subFactoriesBuilder, metadata);
this.origin = origin;
this.ranges = ranges;
@ -71,12 +74,16 @@ public class GeoDistanceRangeAggregatorFactory
}
@Override
protected Aggregator doCreateInternal(final ValuesSource.GeoPoint valuesSource,
protected Aggregator doCreateInternal(final ValuesSource valuesSource,
SearchContext searchContext,
Aggregator parent,
boolean collectsFromSingleBucket,
Map<String, Object> metadata) throws IOException {
DistanceSource distanceSource = new DistanceSource(valuesSource, distanceType, origin, unit);
if (valuesSource instanceof ValuesSource.GeoPoint == false) {
throw new AggregationExecutionException("ValuesSource type " + valuesSource.toString() + "is not supported for aggregation " +
this.name());
}
DistanceSource distanceSource = new DistanceSource((ValuesSource.GeoPoint) valuesSource, distanceType, origin, unit);
return new RangeAggregator(name, factories, distanceSource, config.format(), rangeFactory, ranges, keyed, searchContext,
parent, metadata);
}

24
server/src/main/java/org/elasticsearch/search/aggregations/bucket/range/IpRangeAggregationBuilder.java
View File

@ -37,12 +37,10 @@ import org.elasticsearch.search.aggregations.AggregationBuilder;
import org.elasticsearch.search.aggregations.AggregatorFactories.Builder;
import org.elasticsearch.search.aggregations.AggregatorFactory;
import org.elasticsearch.search.aggregations.support.CoreValuesSourceType;
import org.elasticsearch.search.aggregations.support.ValueType;
import org.elasticsearch.search.aggregations.support.ValuesSource;
import org.elasticsearch.search.aggregations.support.ValuesSourceAggregationBuilder;
import org.elasticsearch.search.aggregations.support.ValuesSourceAggregatorFactory;
import org.elasticsearch.search.aggregations.support.ValuesSourceConfig;
import org.elasticsearch.search.aggregations.support.ValuesSourceParserHelper;
import org.elasticsearch.search.aggregations.support.ValuesSourceType;
import java.io.IOException;
import java.net.InetAddress;
@ -55,14 +53,14 @@ import java.util.Objects;
public final class IpRangeAggregationBuilder
extends ValuesSourceAggregationBuilder<ValuesSource.Bytes, IpRangeAggregationBuilder> {
extends ValuesSourceAggregationBuilder<IpRangeAggregationBuilder> {
public static final String NAME = "ip_range";
private static final ParseField MASK_FIELD = new ParseField("mask");
public static final ObjectParser<IpRangeAggregationBuilder, String> PARSER =
ObjectParser.fromBuilder(NAME, IpRangeAggregationBuilder::new);
static {
ValuesSourceParserHelper.declareBytesFields(PARSER, false, false);
ValuesSourceAggregationBuilder.declareFields(PARSER, false, false, false);
PARSER.declareBoolean(IpRangeAggregationBuilder::keyed, RangeAggregator.KEYED_FIELD);
@ -216,7 +214,7 @@ public final class IpRangeAggregationBuilder
private List<Range> ranges = new ArrayList<>();
public IpRangeAggregationBuilder(String name) {
super(name, CoreValuesSourceType.BYTES, ValueType.IP);
super(name);
}
protected IpRangeAggregationBuilder(IpRangeAggregationBuilder clone, Builder factoriesBuilder, Map<String, Object> metadata) {
@ -333,7 +331,7 @@ public final class IpRangeAggregationBuilder
}
public IpRangeAggregationBuilder(StreamInput in) throws IOException {
super(in, CoreValuesSourceType.BYTES, ValueType.IP);
super(in);
final int numRanges = in.readVInt();
for (int i = 0; i < numRanges; ++i) {
addRange(new Range(in));
@ -341,6 +339,11 @@ public final class IpRangeAggregationBuilder
keyed = in.readBoolean();
}
@Override
protected ValuesSourceType defaultValueSourceType() {
return CoreValuesSourceType.IP;
}
@Override
protected void innerWriteTo(StreamOutput out) throws IOException {
out.writeVInt(ranges.size());
@ -365,9 +368,10 @@ public final class IpRangeAggregationBuilder
}
@Override
protected ValuesSourceAggregatorFactory<ValuesSource.Bytes> innerBuild(
QueryShardContext queryShardContext, ValuesSourceConfig<ValuesSource.Bytes> config,
AggregatorFactory parent, Builder subFactoriesBuilder) throws IOException {
protected ValuesSourceAggregatorFactory innerBuild(
QueryShardContext queryShardContext, ValuesSourceConfig config,
AggregatorFactory parent, Builder subFactoriesBuilder)
throws IOException {
List<BinaryRangeAggregator.Range> ranges = new ArrayList<>();
if(this.ranges.size() == 0){
throw new IllegalArgumentException("No [ranges] specified for the [" + this.getName() + "] aggregation");

12
server/src/main/java/org/elasticsearch/search/aggregations/bucket/range/RangeAggregationBuilder.java
View File

@ -28,9 +28,9 @@ import org.elasticsearch.search.aggregations.AggregationBuilder;
import org.elasticsearch.search.aggregations.AggregatorFactories.Builder;
import org.elasticsearch.search.aggregations.AggregatorFactory;
import org.elasticsearch.search.aggregations.bucket.range.RangeAggregator.Range;
import org.elasticsearch.search.aggregations.support.ValuesSource.Numeric;
import org.elasticsearch.search.aggregations.support.ValuesSourceAggregationBuilder;
import org.elasticsearch.search.aggregations.support.ValuesSourceConfig;
import org.elasticsearch.search.aggregations.support.ValuesSourceParserHelper;
import org.elasticsearch.search.aggregations.support.ValuesSourceRegistry;
import java.io.IOException;
import java.util.Map;
@ -41,7 +41,7 @@ public class RangeAggregationBuilder extends AbstractRangeBuilder<RangeAggregati
public static final ObjectParser<RangeAggregationBuilder, String> PARSER =
ObjectParser.fromBuilder(NAME, RangeAggregationBuilder::new);
static {
ValuesSourceParserHelper.declareNumericFields(PARSER, true, true, false);
ValuesSourceAggregationBuilder.declareFields(PARSER, true, true, false);
PARSER.declareBoolean(RangeAggregationBuilder::keyed, RangeAggregator.KEYED_FIELD);
PARSER.declareObjectArray((agg, ranges) -> {
@ -55,6 +55,10 @@ public class RangeAggregationBuilder extends AbstractRangeBuilder<RangeAggregati
return Range.fromXContent(parser);
}
public static void registerAggregators(ValuesSourceRegistry valuesSourceRegistry) {
AbstractRangeAggregatorFactory.registerAggregators(valuesSourceRegistry, NAME);
}
public RangeAggregationBuilder(String name) {
super(name, InternalRange.FACTORY);
}
@ -142,7 +146,7 @@ public class RangeAggregationBuilder extends AbstractRangeBuilder<RangeAggregati
}
@Override
protected RangeAggregatorFactory innerBuild(QueryShardContext queryShardContext, ValuesSourceConfig<Numeric> config,
protected RangeAggregatorFactory innerBuild(QueryShardContext queryShardContext, ValuesSourceConfig config,
AggregatorFactory parent, Builder subFactoriesBuilder) throws IOException {
// We need to call processRanges here so they are parsed before we make the decision of whether to cache the request
Range[] ranges = processRanges(range -> {

16
server/src/main/java/org/elasticsearch/search/aggregations/bucket/range/RangeAggregatorFactory.java
View File

@ -24,7 +24,6 @@ import org.elasticsearch.search.aggregations.AggregatorFactories;
import org.elasticsearch.search.aggregations.AggregatorFactory;
import org.elasticsearch.search.aggregations.bucket.range.InternalRange.Factory;
import org.elasticsearch.search.aggregations.bucket.range.RangeAggregator.Range;
import org.elasticsearch.search.aggregations.support.ValuesSource.Numeric;
import org.elasticsearch.search.aggregations.support.ValuesSourceConfig;
import java.io.IOException;
@ -32,10 +31,17 @@ import java.util.Map;
public class RangeAggregatorFactory extends AbstractRangeAggregatorFactory<RangeAggregator.Range> {
public RangeAggregatorFactory(String name, ValuesSourceConfig<Numeric> config, Range[] ranges, boolean keyed,
Factory<?, ?> rangeFactory, QueryShardContext queryShardContext, AggregatorFactory parent,
AggregatorFactories.Builder subFactoriesBuilder, Map<String, Object> metadata) throws IOException {
super(name, config, ranges, keyed, rangeFactory, queryShardContext, parent, subFactoriesBuilder, metadata);
public RangeAggregatorFactory(String name,
ValuesSourceConfig config,
Range[] ranges,
boolean keyed,
Factory<?, ?> rangeFactory,
QueryShardContext queryShardContext,
AggregatorFactory parent,
AggregatorFactories.Builder subFactoriesBuilder,
Map<String, Object> metadata) throws IOException {
super(name, RangeAggregationBuilder.NAME, config, ranges, keyed, rangeFactory, queryShardContext, parent, subFactoriesBuilder,
metadata);
}
}

42
server/src/main/java/org/elasticsearch/search/aggregations/bucket/range/RangeAggregatorSupplier.java Normal file
View File

@ -0,0 +1,42 @@
/*
* Licensed to Elasticsearch under one or more contributor
* license agreements. See the NOTICE file distributed with
* this work for additional information regarding copyright
* ownership. Elasticsearch licenses this file to you under
* the Apache License, Version 2.0 (the "License"); you may
* not use this file except in compliance with the License.
* You may obtain a copy of the License at
*
* http://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing,
* software distributed under the License is distributed on an
* "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
* KIND, either express or implied. See the License for the
* specific language governing permissions and limitations
* under the License.
*/
package org.elasticsearch.search.aggregations.bucket.range;
import org.elasticsearch.search.DocValueFormat;
import org.elasticsearch.search.aggregations.Aggregator;
import org.elasticsearch.search.aggregations.AggregatorFactories;
import org.elasticsearch.search.aggregations.support.AggregatorSupplier;
import org.elasticsearch.search.aggregations.support.ValuesSource;
import org.elasticsearch.search.internal.SearchContext;
import java.io.IOException;
import java.util.Map;
public interface RangeAggregatorSupplier extends AggregatorSupplier {
Aggregator build(String name,
AggregatorFactories factories,
ValuesSource.Numeric valuesSource,
DocValueFormat format,
InternalRange.Factory rangeFactory,
RangeAggregator.Range[] ranges,
boolean keyed,
SearchContext context,
Aggregator parent,
Map<String, Object> metadata) throws IOException;
}

24
server/src/main/java/org/elasticsearch/search/aggregations/bucket/sampler/DiversifiedAggregationBuilder.java
View File

@ -28,17 +28,16 @@ import org.elasticsearch.search.aggregations.AggregationBuilder;
import org.elasticsearch.search.aggregations.AggregatorFactories.Builder;
import org.elasticsearch.search.aggregations.AggregatorFactory;
import org.elasticsearch.search.aggregations.support.CoreValuesSourceType;
import org.elasticsearch.search.aggregations.support.ValuesSource;
import org.elasticsearch.search.aggregations.support.ValuesSourceAggregationBuilder;
import org.elasticsearch.search.aggregations.support.ValuesSourceAggregatorFactory;
import org.elasticsearch.search.aggregations.support.ValuesSourceConfig;
import org.elasticsearch.search.aggregations.support.ValuesSourceParserHelper;
import org.elasticsearch.search.aggregations.support.ValuesSourceType;
import java.io.IOException;
import java.util.Map;
import java.util.Objects;
public class DiversifiedAggregationBuilder extends ValuesSourceAggregationBuilder<ValuesSource, DiversifiedAggregationBuilder> {
public class DiversifiedAggregationBuilder extends ValuesSourceAggregationBuilder<DiversifiedAggregationBuilder> {
public static final String NAME = "diversified_sampler";
public static final int MAX_DOCS_PER_VALUE_DEFAULT = 1;
@ -46,7 +45,7 @@ public class DiversifiedAggregationBuilder extends ValuesSourceAggregationBuilde
public static final ObjectParser<DiversifiedAggregationBuilder, String> PARSER =
ObjectParser.fromBuilder(NAME, DiversifiedAggregationBuilder::new);
static {
ValuesSourceParserHelper.declareAnyFields(PARSER, true, false);
ValuesSourceAggregationBuilder.declareFields(PARSER, true, false, false);
PARSER.declareInt(DiversifiedAggregationBuilder::shardSize, SamplerAggregator.SHARD_SIZE_FIELD);
PARSER.declareInt(DiversifiedAggregationBuilder::maxDocsPerValue, SamplerAggregator.MAX_DOCS_PER_VALUE_FIELD);
PARSER.declareString(DiversifiedAggregationBuilder::executionHint, SamplerAggregator.EXECUTION_HINT_FIELD);
@ -57,7 +56,7 @@ public class DiversifiedAggregationBuilder extends ValuesSourceAggregationBuilde
private String executionHint = null;
public DiversifiedAggregationBuilder(String name) {
super(name, CoreValuesSourceType.ANY, null);
super(name);
}
protected DiversifiedAggregationBuilder(DiversifiedAggregationBuilder clone, Builder factoriesBuilder, Map<String, Object> metadata) {
@ -67,6 +66,11 @@ public class DiversifiedAggregationBuilder extends ValuesSourceAggregationBuilde
this.executionHint = clone.executionHint;
}
@Override
protected ValuesSourceType defaultValueSourceType() {
return CoreValuesSourceType.BYTES;
}
@Override
protected AggregationBuilder shallowCopy(Builder factoriesBuilder, Map<String, Object> metadata) {
return new DiversifiedAggregationBuilder(this, factoriesBuilder, metadata);
@ -76,7 +80,7 @@ public class DiversifiedAggregationBuilder extends ValuesSourceAggregationBuilde
* Read from a stream.
*/
public DiversifiedAggregationBuilder(StreamInput in) throws IOException {
super(in, CoreValuesSourceType.ANY, null);
super(in);
shardSize = in.readVInt();
maxDocsPerValue = in.readVInt();
executionHint = in.readOptionalString();
@ -148,10 +152,10 @@ public class DiversifiedAggregationBuilder extends ValuesSourceAggregationBuilde
}
@Override
protected ValuesSourceAggregatorFactory<ValuesSource> innerBuild(QueryShardContext queryShardContext,
ValuesSourceConfig<ValuesSource> config,
AggregatorFactory parent,
Builder subFactoriesBuilder) throws IOException {
protected ValuesSourceAggregatorFactory innerBuild(QueryShardContext queryShardContext,
ValuesSourceConfig config,
AggregatorFactory parent,
Builder subFactoriesBuilder) throws IOException {
return new DiversifiedAggregatorFactory(name, config, shardSize, maxDocsPerValue, executionHint, queryShardContext, parent,
subFactoriesBuilder, metadata);
}

4
server/src/main/java/org/elasticsearch/search/aggregations/bucket/sampler/DiversifiedAggregatorFactory.java
View File

@ -36,13 +36,13 @@ import org.elasticsearch.search.internal.SearchContext;
import java.io.IOException;
import java.util.Map;
public class DiversifiedAggregatorFactory extends ValuesSourceAggregatorFactory<ValuesSource> {
public class DiversifiedAggregatorFactory extends ValuesSourceAggregatorFactory {
private final int shardSize;
private final int maxDocsPerValue;
private final String executionHint;
DiversifiedAggregatorFactory(String name, ValuesSourceConfig<ValuesSource> config, int shardSize, int maxDocsPerValue,
DiversifiedAggregatorFactory(String name, ValuesSourceConfig config, int shardSize, int maxDocsPerValue,
String executionHint, QueryShardContext queryShardContext, AggregatorFactory parent,
AggregatorFactories.Builder subFactoriesBuilder, Map<String, Object> metadata) throws IOException {
super(name, config, queryShardContext, parent, subFactoriesBuilder, metadata);

35
server/src/main/java/org/elasticsearch/search/aggregations/bucket/significant/SignificantTermsAggregationBuilder.java
View File

@ -38,12 +38,11 @@ import org.elasticsearch.search.aggregations.bucket.terms.TermsAggregationBuilde
import org.elasticsearch.search.aggregations.bucket.terms.TermsAggregator;
import org.elasticsearch.search.aggregations.bucket.terms.TermsAggregator.BucketCountThresholds;
import org.elasticsearch.search.aggregations.support.CoreValuesSourceType;
import org.elasticsearch.search.aggregations.support.ValueType;
import org.elasticsearch.search.aggregations.support.ValuesSource;
import org.elasticsearch.search.aggregations.support.ValuesSourceAggregationBuilder;
import org.elasticsearch.search.aggregations.support.ValuesSourceAggregatorFactory;
import org.elasticsearch.search.aggregations.support.ValuesSourceConfig;
import org.elasticsearch.search.aggregations.support.ValuesSourceParserHelper;
import org.elasticsearch.search.aggregations.support.ValuesSourceRegistry;
import org.elasticsearch.search.aggregations.support.ValuesSourceType;
import java.io.IOException;
import java.util.Map;
@ -51,7 +50,7 @@ import java.util.Objects;
import static org.elasticsearch.index.query.AbstractQueryBuilder.parseInnerQueryBuilder;
public class SignificantTermsAggregationBuilder extends ValuesSourceAggregationBuilder<ValuesSource, SignificantTermsAggregationBuilder> {
public class SignificantTermsAggregationBuilder extends ValuesSourceAggregationBuilder<SignificantTermsAggregationBuilder> {
public static final String NAME = "significant_terms";
static final ParseField BACKGROUND_FILTER = new ParseField("background_filter");
@ -65,7 +64,7 @@ public class SignificantTermsAggregationBuilder extends ValuesSourceAggregationB
SignificantTermsAggregationBuilder.NAME,
SignificanceHeuristic.class, SignificantTermsAggregationBuilder::significanceHeuristic, null);
static {
ValuesSourceParserHelper.declareAnyFields(PARSER, true, true);
ValuesSourceAggregationBuilder.declareFields(PARSER, true, true, false);
PARSER.declareInt(SignificantTermsAggregationBuilder::shardSize, TermsAggregationBuilder.SHARD_SIZE_FIELD_NAME);
@ -90,7 +89,11 @@ public class SignificantTermsAggregationBuilder extends ValuesSourceAggregationB
IncludeExclude::parseExclude, IncludeExclude.EXCLUDE_FIELD, ObjectParser.ValueType.STRING_ARRAY);
}
public static SignificantTermsAggregationBuilder parse(String aggregationName, XContentParser parser) throws IOException {
return PARSER.parse(parser, new SignificantTermsAggregationBuilder(aggregationName, null), null);
return PARSER.parse(parser, new SignificantTermsAggregationBuilder(aggregationName), null);
}
public static void registerAggregators(ValuesSourceRegistry valuesSourceRegistry) {
SignificantTermsAggregatorFactory.registerAggregators(valuesSourceRegistry);
}
private IncludeExclude includeExclude = null;
@ -99,15 +102,15 @@ public class SignificantTermsAggregationBuilder extends ValuesSourceAggregationB
private TermsAggregator.BucketCountThresholds bucketCountThresholds = new BucketCountThresholds(DEFAULT_BUCKET_COUNT_THRESHOLDS);
private SignificanceHeuristic significanceHeuristic = DEFAULT_SIGNIFICANCE_HEURISTIC;
public SignificantTermsAggregationBuilder(String name, ValueType valueType) {
super(name, CoreValuesSourceType.ANY, valueType);
public SignificantTermsAggregationBuilder(String name) {
super(name);
}
/**
* Read from a Stream.
*/
public SignificantTermsAggregationBuilder(StreamInput in) throws IOException {
super(in, CoreValuesSourceType.ANY);
super(in);
bucketCountThresholds = new BucketCountThresholds(in);
executionHint = in.readOptionalString();
filterBuilder = in.readOptionalNamedWriteable(QueryBuilder.class);
@ -125,12 +128,16 @@ public class SignificantTermsAggregationBuilder extends ValuesSourceAggregationB
this.significanceHeuristic = clone.significanceHeuristic;
}
@Override
protected ValuesSourceType defaultValueSourceType() {
return CoreValuesSourceType.BYTES;
}
@Override
protected SignificantTermsAggregationBuilder shallowCopy(Builder factoriesBuilder, Map<String, Object> metadata) {
return new SignificantTermsAggregationBuilder(this, factoriesBuilder, metadata);
}
@Override
protected AggregationBuilder doRewrite(QueryRewriteContext queryShardContext) throws IOException {
if (filterBuilder != null) {
QueryBuilder rewrittenFilter = filterBuilder.rewrite(queryShardContext);
@ -286,10 +293,10 @@ public class SignificantTermsAggregationBuilder extends ValuesSourceAggregationB
}
@Override
protected ValuesSourceAggregatorFactory<ValuesSource> innerBuild(QueryShardContext queryShardContext,
ValuesSourceConfig<ValuesSource> config,
AggregatorFactory parent,
Builder subFactoriesBuilder) throws IOException {
protected ValuesSourceAggregatorFactory innerBuild(QueryShardContext queryShardContext,
ValuesSourceConfig config,
AggregatorFactory parent,
Builder subFactoriesBuilder) throws IOException {
SignificanceHeuristic executionHeuristic = this.significanceHeuristic.rewrite(queryShardContext);
return new SignificantTermsAggregatorFactory(name, config, includeExclude, executionHint, filterBuilder,
bucketCountThresholds, executionHeuristic, queryShardContext, parent, subFactoriesBuilder, metadata);

196
server/src/main/java/org/elasticsearch/search/aggregations/bucket/significant/SignificantTermsAggregatorFactory.java
View File

@ -49,16 +49,20 @@ import org.elasticsearch.search.aggregations.bucket.significant.heuristics.Signi
import org.elasticsearch.search.aggregations.bucket.terms.IncludeExclude;
import org.elasticsearch.search.aggregations.bucket.terms.TermsAggregator;
import org.elasticsearch.search.aggregations.bucket.terms.TermsAggregator.BucketCountThresholds;
import org.elasticsearch.search.aggregations.support.AggregatorSupplier;
import org.elasticsearch.search.aggregations.support.CoreValuesSourceType;
import org.elasticsearch.search.aggregations.support.ValuesSource;
import org.elasticsearch.search.aggregations.support.ValuesSourceAggregatorFactory;
import org.elasticsearch.search.aggregations.support.ValuesSourceConfig;
import org.elasticsearch.search.aggregations.support.ValuesSourceRegistry;
import org.elasticsearch.search.internal.SearchContext;
import java.io.IOException;
import java.util.Arrays;
import java.util.Map;
public class SignificantTermsAggregatorFactory extends ValuesSourceAggregatorFactory<ValuesSource>
implements Releasable {
public class SignificantTermsAggregatorFactory extends ValuesSourceAggregatorFactory
implements Releasable {
private static final DeprecationLogger deprecationLogger = new DeprecationLogger(
LogManager.getLogger(SignificantTermsAggregatorFactory.class));
@ -73,19 +77,123 @@ public class SignificantTermsAggregatorFactory extends ValuesSourceAggregatorFac
private final TermsAggregator.BucketCountThresholds bucketCountThresholds;
private final SignificanceHeuristic significanceHeuristic;
public SignificantTermsAggregatorFactory(String name,
ValuesSourceConfig<ValuesSource> config,
IncludeExclude includeExclude,
String executionHint,
QueryBuilder filterBuilder,
TermsAggregator.BucketCountThresholds bucketCountThresholds,
SignificanceHeuristic significanceHeuristic,
QueryShardContext queryShardContext,
AggregatorFactory parent,
AggregatorFactories.Builder subFactoriesBuilder,
Map<String, Object> metadata) throws IOException {
static void registerAggregators(ValuesSourceRegistry valuesSourceRegistry) {
valuesSourceRegistry.register(SignificantTermsAggregationBuilder.NAME,
Arrays.asList(CoreValuesSourceType.BYTES, CoreValuesSourceType.IP),
SignificantTermsAggregatorFactory.bytesSupplier());
valuesSourceRegistry.register(SignificantTermsAggregationBuilder.NAME,
Arrays.asList(CoreValuesSourceType.DATE, CoreValuesSourceType.BOOLEAN, CoreValuesSourceType.NUMERIC),
SignificantTermsAggregatorFactory.numericSupplier());
}
/**
* This supplier is used for all the field types that should be aggregated as bytes/strings,
* including those that need global ordinals
*/
private static SignificantTermsAggregatorSupplier bytesSupplier() {
return new SignificantTermsAggregatorSupplier() {
@Override
public Aggregator build(String name,
AggregatorFactories factories,
ValuesSource valuesSource,
DocValueFormat format,
TermsAggregator.BucketCountThresholds bucketCountThresholds,
IncludeExclude includeExclude,
String executionHint,
SearchContext context,
Aggregator parent,
SignificanceHeuristic significanceHeuristic,
SignificantTermsAggregatorFactory sigTermsFactory,
Map<String, Object> metadata) throws IOException {
ExecutionMode execution = null;
if (executionHint != null) {
execution = ExecutionMode.fromString(executionHint, deprecationLogger);
}
if (valuesSource instanceof ValuesSource.Bytes.WithOrdinals == false) {
execution = ExecutionMode.MAP;
}
if (execution == null) {
execution = ExecutionMode.GLOBAL_ORDINALS;
}
if ((includeExclude != null) && (includeExclude.isRegexBased()) && format != DocValueFormat.RAW) {
throw new IllegalArgumentException("Aggregation [" + name + "] cannot support regular expression style "
+ "include/exclude settings as they can only be applied to string fields. Use an array of values for "
+ "include/exclude clauses");
}
return execution.create(name, factories, valuesSource, format, bucketCountThresholds, includeExclude, context, parent,
significanceHeuristic, sigTermsFactory, metadata);
}
};
}
/**
* This supplier is used for all fields that expect to be aggregated as a numeric value.
* This includes floating points, and formatted types that use numerics internally for storage (date, boolean, etc)
*/
private static SignificantTermsAggregatorSupplier numericSupplier() {
return new SignificantTermsAggregatorSupplier() {
@Override
public Aggregator build(String name,
AggregatorFactories factories,
ValuesSource valuesSource,
DocValueFormat format,
TermsAggregator.BucketCountThresholds bucketCountThresholds,
IncludeExclude includeExclude,
String executionHint,
SearchContext context,
Aggregator parent,
SignificanceHeuristic significanceHeuristic,
SignificantTermsAggregatorFactory sigTermsFactory,
Map<String, Object> metadata) throws IOException {
if ((includeExclude != null) && (includeExclude.isRegexBased())) {
throw new IllegalArgumentException("Aggregation [" + name + "] cannot support regular expression style include/exclude "
+ "settings as they can only be applied to string fields. Use an array of numeric " +
"values for include/exclude clauses used to filter numeric fields");
}
if (((ValuesSource.Numeric) valuesSource).isFloatingPoint()) {
throw new UnsupportedOperationException("No support for examining floating point numerics");
}
IncludeExclude.LongFilter longFilter = null;
if (includeExclude != null) {
longFilter = includeExclude.convertToLongFilter(format);
}
return new SignificantLongTermsAggregator(name, factories, (ValuesSource.Numeric) valuesSource, format,
bucketCountThresholds, context, parent, significanceHeuristic, sigTermsFactory, longFilter,
metadata);
}
};
}
SignificantTermsAggregatorFactory(String name,
ValuesSourceConfig config,
IncludeExclude includeExclude,
String executionHint,
QueryBuilder filterBuilder,
TermsAggregator.BucketCountThresholds bucketCountThresholds,
SignificanceHeuristic significanceHeuristic,
QueryShardContext queryShardContext,
AggregatorFactory parent,
AggregatorFactories.Builder subFactoriesBuilder,
Map<String, Object> metadata) throws IOException {
super(name, config, queryShardContext, parent, subFactoriesBuilder, metadata);
if (config.unmapped() == false) {
if (config.fieldContext().fieldType().isSearchable() == false) {
throw new IllegalArgumentException("SignificantText aggregation requires fields to be searchable, but ["
+ config.fieldContext().fieldType().name() + "] is not");
}
}
if (!config.unmapped()) {
this.fieldType = config.fieldContext().fieldType();
this.indexedFieldName = fieldType.name();
@ -109,7 +217,7 @@ public class SignificantTermsAggregatorFactory extends ValuesSourceAggregatorFac
/**
* Get the number of docs in the superset.
*/
public long getSupersetNumDocs() {
long getSupersetNumDocs() {
return supersetNumDocs;
}
@ -149,12 +257,12 @@ public class SignificantTermsAggregatorFactory extends ValuesSourceAggregatorFac
return queryShardContext.searcher().count(query);
}
public long getBackgroundFrequency(BytesRef termBytes) throws IOException {
long getBackgroundFrequency(BytesRef termBytes) throws IOException {
String value = config.format().format(termBytes).toString();
return getBackgroundFrequency(value);
}
public long getBackgroundFrequency(long termNum) throws IOException {
long getBackgroundFrequency(long termNum) throws IOException {
String value = config.format().format(termNum).toString();
return getBackgroundFrequency(value);
}
@ -183,6 +291,13 @@ public class SignificantTermsAggregatorFactory extends ValuesSourceAggregatorFac
return asMultiBucketAggregator(this, searchContext, parent);
}
AggregatorSupplier aggregatorSupplier = queryShardContext.getValuesSourceRegistry().getAggregator(config.valueSourceType(),
SignificantTermsAggregationBuilder.NAME);
if (aggregatorSupplier instanceof SignificantTermsAggregatorSupplier == false) {
throw new AggregationExecutionException("Registry miss-match - expected SignificantTermsAggregatorSupplier, found [" +
aggregatorSupplier.getClass().toString() + "]");
}
numberOfAggregatorsCreated++;
BucketCountThresholds bucketCountThresholds = new BucketCountThresholds(this.bucketCountThresholds);
if (bucketCountThresholds.getShardSize() == SignificantTermsAggregationBuilder.DEFAULT_BUCKET_COUNT_THRESHOLDS.getShardSize()) {
@ -201,51 +316,12 @@ public class SignificantTermsAggregatorFactory extends ValuesSourceAggregatorFac
bucketCountThresholds.setShardSize(2 * BucketUtils.suggestShardSideQueueSize(bucketCountThresholds.getRequiredSize()));
}
if (valuesSource instanceof ValuesSource.Bytes) {
ExecutionMode execution = null;
if (executionHint != null) {
execution = ExecutionMode.fromString(executionHint, deprecationLogger);
}
if (valuesSource instanceof ValuesSource.Bytes.WithOrdinals == false) {
execution = ExecutionMode.MAP;
}
if (execution == null) {
execution = ExecutionMode.GLOBAL_ORDINALS;
}
assert execution != null;
SignificantTermsAggregatorSupplier sigTermsAggregatorSupplier = (SignificantTermsAggregatorSupplier) aggregatorSupplier;
DocValueFormat format = config.format();
if ((includeExclude != null) && (includeExclude.isRegexBased()) && format != DocValueFormat.RAW) {
throw new AggregationExecutionException("Aggregation [" + name + "] cannot support regular expression style "
+ "include/exclude settings as they can only be applied to string fields. Use an array of values for "
+ "include/exclude clauses");
}
return execution.create(name, factories, valuesSource, format, bucketCountThresholds, includeExclude, searchContext, parent,
significanceHeuristic, this, metadata);
}
if ((includeExclude != null) && (includeExclude.isRegexBased())) {
throw new AggregationExecutionException("Aggregation [" + name + "] cannot support regular expression style include/exclude "
+ "settings as they can only be applied to string fields. Use an array of numeric values for include/exclude clauses "
+ "used to filter numeric fields");
}
if (valuesSource instanceof ValuesSource.Numeric) {
if (((ValuesSource.Numeric) valuesSource).isFloatingPoint()) {
throw new UnsupportedOperationException("No support for examining floating point numerics");
}
IncludeExclude.LongFilter longFilter = null;
if (includeExclude != null) {
longFilter = includeExclude.convertToLongFilter(config.format());
}
return new SignificantLongTermsAggregator(name, factories, (ValuesSource.Numeric) valuesSource, config.format(),
bucketCountThresholds, searchContext, parent, significanceHeuristic, this, longFilter, metadata);
}
throw new AggregationExecutionException("significant_terms aggregation cannot be applied to field ["
+ config.fieldContext().field() + "]. It can only be applied to numeric or string fields.");
// TODO we should refactor so that we don't need to use this Factory as a singleton (e.g. stop passing `this` to the aggregators)
return sigTermsAggregatorSupplier.build(name, factories, valuesSource, config.format(),
bucketCountThresholds, includeExclude, executionHint, searchContext, parent,
significanceHeuristic, this, metadata);
}
public enum ExecutionMode {

47
server/src/main/java/org/elasticsearch/search/aggregations/bucket/significant/SignificantTermsAggregatorSupplier.java Normal file
View File

@ -0,0 +1,47 @@
/*
* Licensed to Elasticsearch under one or more contributor
* license agreements. See the NOTICE file distributed with
* this work for additional information regarding copyright
* ownership. Elasticsearch licenses this file to you under
* the Apache License, Version 2.0 (the "License"); you may
* not use this file except in compliance with the License.
* You may obtain a copy of the License at
*
* http://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing,
* software distributed under the License is distributed on an
* "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
* KIND, either express or implied. See the License for the
* specific language governing permissions and limitations
* under the License.
*/
package org.elasticsearch.search.aggregations.bucket.significant;
import org.elasticsearch.search.DocValueFormat;
import org.elasticsearch.search.aggregations.Aggregator;
import org.elasticsearch.search.aggregations.AggregatorFactories;
import org.elasticsearch.search.aggregations.bucket.significant.heuristics.SignificanceHeuristic;
import org.elasticsearch.search.aggregations.bucket.terms.IncludeExclude;
import org.elasticsearch.search.aggregations.bucket.terms.TermsAggregator;
import org.elasticsearch.search.aggregations.support.AggregatorSupplier;
import org.elasticsearch.search.aggregations.support.ValuesSource;
import org.elasticsearch.search.internal.SearchContext;
import java.io.IOException;
import java.util.Map;
interface SignificantTermsAggregatorSupplier extends AggregatorSupplier {
Aggregator build(String name,
AggregatorFactories factories,
ValuesSource valuesSource,
DocValueFormat format,
TermsAggregator.BucketCountThresholds bucketCountThresholds,
IncludeExclude includeExclude,
String executionHint,
SearchContext context,
Aggregator parent,
SignificanceHeuristic significanceHeuristic,
SignificantTermsAggregatorFactory sigTermsFactory,
Map<String, Object> metadata) throws IOException;
}

34
server/src/main/java/org/elasticsearch/search/aggregations/bucket/terms/RareTermsAggregationBuilder.java
View File

@ -29,18 +29,17 @@ import org.elasticsearch.search.aggregations.AggregationBuilder;
import org.elasticsearch.search.aggregations.AggregatorFactories.Builder;
import org.elasticsearch.search.aggregations.AggregatorFactory;
import org.elasticsearch.search.aggregations.support.CoreValuesSourceType;
import org.elasticsearch.search.aggregations.support.ValueType;
import org.elasticsearch.search.aggregations.support.ValuesSource;
import org.elasticsearch.search.aggregations.support.ValuesSourceAggregationBuilder;
import org.elasticsearch.search.aggregations.support.ValuesSourceAggregatorFactory;
import org.elasticsearch.search.aggregations.support.ValuesSourceConfig;
import org.elasticsearch.search.aggregations.support.ValuesSourceParserHelper;
import org.elasticsearch.search.aggregations.support.ValuesSourceRegistry;
import org.elasticsearch.search.aggregations.support.ValuesSourceType;
import java.io.IOException;
import java.util.Map;
import java.util.Objects;
public class RareTermsAggregationBuilder extends ValuesSourceAggregationBuilder<ValuesSource, RareTermsAggregationBuilder> {
public class RareTermsAggregationBuilder extends ValuesSourceAggregationBuilder<RareTermsAggregationBuilder> {
public static final String NAME = "rare_terms";
private static final ParseField MAX_DOC_COUNT_FIELD_NAME = new ParseField("max_doc_count");
@ -48,9 +47,9 @@ public class RareTermsAggregationBuilder extends ValuesSourceAggregationBuilder<
private static final int MAX_MAX_DOC_COUNT = 100;
public static final ObjectParser<RareTermsAggregationBuilder, String> PARSER =
ObjectParser.fromBuilder(NAME, name -> new RareTermsAggregationBuilder(name, null));
ObjectParser.fromBuilder(NAME, RareTermsAggregationBuilder::new);
static {
ValuesSourceParserHelper.declareAnyFields(PARSER, true, true);
ValuesSourceAggregationBuilder.declareFields(PARSER, true, true, false);
PARSER.declareLong(RareTermsAggregationBuilder::maxDocCount, MAX_DOC_COUNT_FIELD_NAME);
PARSER.declareField((b, v) -> b.includeExclude(IncludeExclude.merge(v, b.includeExclude())),
@ -62,12 +61,16 @@ public class RareTermsAggregationBuilder extends ValuesSourceAggregationBuilder<
PARSER.declareDouble(RareTermsAggregationBuilder::setPrecision, PRECISION);
}
public static void registerAggregators(ValuesSourceRegistry valuesSourceRegistry) {
RareTermsAggregatorFactory.registerAggregators(valuesSourceRegistry);
}
private IncludeExclude includeExclude = null;
private int maxDocCount = 1;
private double precision = 0.001;
public RareTermsAggregationBuilder(String name, ValueType valueType) {
super(name, CoreValuesSourceType.ANY, valueType);
public RareTermsAggregationBuilder(String name) {
super(name);
}
private RareTermsAggregationBuilder(RareTermsAggregationBuilder clone, Builder factoriesBuilder, Map<String, Object> metadata) {
@ -75,6 +78,11 @@ public class RareTermsAggregationBuilder extends ValuesSourceAggregationBuilder<
this.includeExclude = clone.includeExclude;
}
@Override
protected ValuesSourceType defaultValueSourceType() {
return CoreValuesSourceType.BYTES;
}
@Override
protected AggregationBuilder shallowCopy(Builder factoriesBuilder, Map<String, Object> metadata) {
return new RareTermsAggregationBuilder(this, factoriesBuilder, metadata);
@ -84,7 +92,7 @@ public class RareTermsAggregationBuilder extends ValuesSourceAggregationBuilder<
* Read from a stream.
*/
public RareTermsAggregationBuilder(StreamInput in) throws IOException {
super(in, CoreValuesSourceType.ANY);
super(in);
includeExclude = in.readOptionalWriteable(IncludeExclude::new);
maxDocCount = in.readVInt();
}
@ -162,10 +170,10 @@ public class RareTermsAggregationBuilder extends ValuesSourceAggregationBuilder<
}
@Override
protected ValuesSourceAggregatorFactory<ValuesSource> innerBuild(QueryShardContext queryShardContext,
ValuesSourceConfig<ValuesSource> config,
AggregatorFactory parent,
Builder subFactoriesBuilder) throws IOException {
protected ValuesSourceAggregatorFactory innerBuild(QueryShardContext queryShardContext,
ValuesSourceConfig config,
AggregatorFactory parent,
Builder subFactoriesBuilder) throws IOException {
return new RareTermsAggregatorFactory(name, config, includeExclude,
queryShardContext, parent, subFactoriesBuilder, metadata, maxDocCount, precision);
}

126
server/src/main/java/org/elasticsearch/search/aggregations/bucket/terms/RareTermsAggregatorFactory.java
View File

@ -29,20 +29,104 @@ import org.elasticsearch.search.aggregations.AggregatorFactories;
import org.elasticsearch.search.aggregations.AggregatorFactory;
import org.elasticsearch.search.aggregations.InternalAggregation;
import org.elasticsearch.search.aggregations.NonCollectingAggregator;
import org.elasticsearch.search.aggregations.support.AggregatorSupplier;
import org.elasticsearch.search.aggregations.support.CoreValuesSourceType;
import org.elasticsearch.search.aggregations.support.ValuesSource;
import org.elasticsearch.search.aggregations.support.ValuesSourceAggregatorFactory;
import org.elasticsearch.search.aggregations.support.ValuesSourceConfig;
import org.elasticsearch.search.aggregations.support.ValuesSourceRegistry;
import org.elasticsearch.search.internal.SearchContext;
import java.io.IOException;
import java.util.Arrays;
import java.util.Map;
public class RareTermsAggregatorFactory extends ValuesSourceAggregatorFactory<ValuesSource> {
public class RareTermsAggregatorFactory extends ValuesSourceAggregatorFactory {
private final IncludeExclude includeExclude;
private final int maxDocCount;
private final double precision;
RareTermsAggregatorFactory(String name, ValuesSourceConfig<ValuesSource> config,
static void registerAggregators(ValuesSourceRegistry valuesSourceRegistry) {
valuesSourceRegistry.register(RareTermsAggregationBuilder.NAME,
Arrays.asList(CoreValuesSourceType.BYTES, CoreValuesSourceType.IP),
RareTermsAggregatorFactory.bytesSupplier());
valuesSourceRegistry.register(RareTermsAggregationBuilder.NAME,
Arrays.asList(CoreValuesSourceType.DATE, CoreValuesSourceType.BOOLEAN, CoreValuesSourceType.NUMERIC),
RareTermsAggregatorFactory.numericSupplier());
}
/**
* This supplier is used for all the field types that should be aggregated as bytes/strings,
* including those that need global ordinals
*/
private static RareTermsAggregatorSupplier bytesSupplier() {
return new RareTermsAggregatorSupplier() {
@Override
public Aggregator build(String name,
AggregatorFactories factories,
ValuesSource valuesSource,
DocValueFormat format,
int maxDocCount,
double precision,
IncludeExclude includeExclude,
SearchContext context,
Aggregator parent,
Map<String, Object> metadata) throws IOException {
ExecutionMode execution = ExecutionMode.MAP; //TODO global ords not implemented yet, only supports "map"
if ((includeExclude != null) && (includeExclude.isRegexBased()) && format != DocValueFormat.RAW) {
throw new IllegalArgumentException("Aggregation [" + name + "] cannot support " +
"regular expression style include/exclude settings as they can only be applied to string fields. " +
"Use an array of values for include/exclude clauses");
}
return execution.create(name, factories, valuesSource, format,
includeExclude, context, parent, metadata, maxDocCount, precision);
}
};
}
/**
* This supplier is used for all fields that expect to be aggregated as a numeric value.
* This includes floating points, and formatted types that use numerics internally for storage (date, boolean, etc)
*/
private static RareTermsAggregatorSupplier numericSupplier() {
return new RareTermsAggregatorSupplier() {
@Override
public Aggregator build(String name,
AggregatorFactories factories,
ValuesSource valuesSource,
DocValueFormat format,
int maxDocCount,
double precision,
IncludeExclude includeExclude,
SearchContext context,
Aggregator parent,
Map<String, Object> metadata) throws IOException {
if ((includeExclude != null) && (includeExclude.isRegexBased())) {
throw new IllegalArgumentException("Aggregation [" + name + "] cannot support regular expression " +
"style include/exclude settings as they can only be applied to string fields. Use an array of numeric " +
"values for include/exclude clauses used to filter numeric fields");
}
IncludeExclude.LongFilter longFilter = null;
if (((ValuesSource.Numeric) valuesSource).isFloatingPoint()) {
throw new IllegalArgumentException("RareTerms aggregation does not support floating point fields.");
}
if (includeExclude != null) {
longFilter = includeExclude.convertToLongFilter(format);
}
return new LongRareTermsAggregator(name, factories, (ValuesSource.Numeric) valuesSource, format,
context, parent, longFilter, maxDocCount, precision, metadata);
}
};
}
RareTermsAggregatorFactory(String name, ValuesSourceConfig config,
IncludeExclude includeExclude,
QueryShardContext queryShardContext,
AggregatorFactory parent, AggregatorFactories.Builder subFactoriesBuilder,
@ -75,40 +159,16 @@ public class RareTermsAggregatorFactory extends ValuesSourceAggregatorFactory<Va
if (collectsFromSingleBucket == false) {
return asMultiBucketAggregator(this, searchContext, parent);
}
if (valuesSource instanceof ValuesSource.Bytes) {
ExecutionMode execution = ExecutionMode.MAP; //TODO global ords not implemented yet, only supports "map"
DocValueFormat format = config.format();
if ((includeExclude != null) && (includeExclude.isRegexBased()) && format != DocValueFormat.RAW) {
throw new AggregationExecutionException("Aggregation [" + name + "] cannot support " +
"regular expression style include/exclude settings as they can only be applied to string fields. " +
"Use an array of values for include/exclude clauses");
}
return execution.create(name, factories, valuesSource, format,
includeExclude, searchContext, parent, metadata, maxDocCount, precision);
AggregatorSupplier aggregatorSupplier = queryShardContext.getValuesSourceRegistry().getAggregator(config.valueSourceType(),
RareTermsAggregationBuilder.NAME);
if (aggregatorSupplier instanceof RareTermsAggregatorSupplier == false) {
throw new AggregationExecutionException("Registry miss-match - expected RareTermsAggregatorSupplier, found [" +
aggregatorSupplier.getClass().toString() + "]");
}
if ((includeExclude != null) && (includeExclude.isRegexBased())) {
throw new AggregationExecutionException("Aggregation [" + name + "] cannot support regular expression style include/exclude "
+ "settings as they can only be applied to string fields. Use an array of numeric values for include/exclude clauses " +
"used to filter numeric fields");
}
if (valuesSource instanceof ValuesSource.Numeric) {
IncludeExclude.LongFilter longFilter = null;
if (((ValuesSource.Numeric) valuesSource).isFloatingPoint()) {
throw new AggregationExecutionException("RareTerms aggregation does not support floating point fields.");
}
if (includeExclude != null) {
longFilter = includeExclude.convertToLongFilter(config.format());
}
return new LongRareTermsAggregator(name, factories, (ValuesSource.Numeric) valuesSource, config.format(),
searchContext, parent, longFilter, maxDocCount, precision, metadata);
}
throw new AggregationExecutionException("RareTerms aggregation cannot be applied to field [" + config.fieldContext().field()
+ "]. It can only be applied to numeric or string fields.");
return ((RareTermsAggregatorSupplier) aggregatorSupplier).build(name, factories, valuesSource, config.format(),
maxDocCount, precision, includeExclude, searchContext, parent, metadata);
}
public enum ExecutionMode {

42
server/src/main/java/org/elasticsearch/search/aggregations/bucket/terms/RareTermsAggregatorSupplier.java Normal file
View File

@ -0,0 +1,42 @@
/*
* Licensed to Elasticsearch under one or more contributor
* license agreements. See the NOTICE file distributed with
* this work for additional information regarding copyright
* ownership. Elasticsearch licenses this file to you under
* the Apache License, Version 2.0 (the "License"); you may
* not use this file except in compliance with the License.
* You may obtain a copy of the License at
*
* http://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing,
* software distributed under the License is distributed on an
* "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
* KIND, either express or implied. See the License for the
* specific language governing permissions and limitations
* under the License.
*/
package org.elasticsearch.search.aggregations.bucket.terms;
import org.elasticsearch.search.DocValueFormat;
import org.elasticsearch.search.aggregations.Aggregator;
import org.elasticsearch.search.aggregations.AggregatorFactories;
import org.elasticsearch.search.aggregations.support.AggregatorSupplier;
import org.elasticsearch.search.aggregations.support.ValuesSource;
import org.elasticsearch.search.internal.SearchContext;
import java.io.IOException;
import java.util.Map;
interface RareTermsAggregatorSupplier extends AggregatorSupplier {
Aggregator build(String name,
AggregatorFactories factories,
ValuesSource valuesSource,
DocValueFormat format,
int maxDocCount,
double precision,
IncludeExclude includeExclude,
SearchContext context,
Aggregator parent,
Map<String, Object> metadata) throws IOException;
}

34
server/src/main/java/org/elasticsearch/search/aggregations/bucket/terms/TermsAggregationBuilder.java
View File

@ -36,19 +36,18 @@ import org.elasticsearch.search.aggregations.InternalOrder;
import org.elasticsearch.search.aggregations.InternalOrder.CompoundOrder;
import org.elasticsearch.search.aggregations.bucket.terms.TermsAggregator.BucketCountThresholds;
import org.elasticsearch.search.aggregations.support.CoreValuesSourceType;
import org.elasticsearch.search.aggregations.support.ValueType;
import org.elasticsearch.search.aggregations.support.ValuesSource;
import org.elasticsearch.search.aggregations.support.ValuesSourceAggregationBuilder;
import org.elasticsearch.search.aggregations.support.ValuesSourceAggregatorFactory;
import org.elasticsearch.search.aggregations.support.ValuesSourceConfig;
import org.elasticsearch.search.aggregations.support.ValuesSourceParserHelper;
import org.elasticsearch.search.aggregations.support.ValuesSourceRegistry;
import org.elasticsearch.search.aggregations.support.ValuesSourceType;
import java.io.IOException;
import java.util.List;
import java.util.Map;
import java.util.Objects;
public class TermsAggregationBuilder extends ValuesSourceAggregationBuilder<ValuesSource, TermsAggregationBuilder> {
public class TermsAggregationBuilder extends ValuesSourceAggregationBuilder<TermsAggregationBuilder> {
public static final String NAME = "terms";
public static final ParseField EXECUTION_HINT_FIELD_NAME = new ParseField("execution_hint");
@ -63,9 +62,9 @@ public class TermsAggregationBuilder extends ValuesSourceAggregationBuilder<Valu
public static final ParseField ORDER_FIELD = new ParseField("order");
public static final ObjectParser<TermsAggregationBuilder, String> PARSER =
ObjectParser.fromBuilder(NAME, name -> new TermsAggregationBuilder(name, null));
ObjectParser.fromBuilder(NAME, TermsAggregationBuilder::new);
static {
ValuesSourceParserHelper.declareAnyFields(PARSER, true, true);
ValuesSourceAggregationBuilder.declareFields(PARSER, true, true, false);
PARSER.declareBoolean(TermsAggregationBuilder::showTermDocCountError,
TermsAggregationBuilder.SHOW_TERM_DOC_COUNT_ERROR);
@ -94,6 +93,10 @@ public class TermsAggregationBuilder extends ValuesSourceAggregationBuilder<Valu
IncludeExclude::parseExclude, IncludeExclude.EXCLUDE_FIELD, ObjectParser.ValueType.STRING_ARRAY);
}
public static void registerAggregators(ValuesSourceRegistry valuesSourceRegistry) {
TermsAggregatorFactory.registerAggregators(valuesSourceRegistry);
}
private BucketOrder order = BucketOrder.compound(BucketOrder.count(false)); // automatically adds tie-breaker key asc order
private IncludeExclude includeExclude = null;
private String executionHint = null;
@ -102,8 +105,8 @@ public class TermsAggregationBuilder extends ValuesSourceAggregationBuilder<Valu
DEFAULT_BUCKET_COUNT_THRESHOLDS);
private boolean showTermDocCountError = false;
public TermsAggregationBuilder(String name, ValueType valueType) {
super(name, CoreValuesSourceType.ANY, valueType);
public TermsAggregationBuilder(String name) {
super(name);
}
protected TermsAggregationBuilder(TermsAggregationBuilder clone, Builder factoriesBuilder, Map<String, Object> metadata) {
@ -116,6 +119,11 @@ public class TermsAggregationBuilder extends ValuesSourceAggregationBuilder<Valu
this.showTermDocCountError = clone.showTermDocCountError;
}
@Override
protected ValuesSourceType defaultValueSourceType() {
return CoreValuesSourceType.BYTES;
}
@Override
protected AggregationBuilder shallowCopy(Builder factoriesBuilder, Map<String, Object> metadata) {
return new TermsAggregationBuilder(this, factoriesBuilder, metadata);
@ -125,7 +133,7 @@ public class TermsAggregationBuilder extends ValuesSourceAggregationBuilder<Valu
* Read from a stream.
*/
public TermsAggregationBuilder(StreamInput in) throws IOException {
super(in, CoreValuesSourceType.ANY);
super(in);
bucketCountThresholds = new BucketCountThresholds(in);
collectMode = in.readOptionalWriteable(SubAggCollectionMode::readFromStream);
executionHint = in.readOptionalString();
@ -333,10 +341,10 @@ public class TermsAggregationBuilder extends ValuesSourceAggregationBuilder<Valu
}
@Override
protected ValuesSourceAggregatorFactory<ValuesSource> innerBuild(QueryShardContext queryShardContext,
ValuesSourceConfig<ValuesSource> config,
AggregatorFactory parent,
Builder subFactoriesBuilder) throws IOException {
protected ValuesSourceAggregatorFactory innerBuild(QueryShardContext queryShardContext,
ValuesSourceConfig config,
AggregatorFactory parent,
Builder subFactoriesBuilder) throws IOException {
return new TermsAggregatorFactory(name, config, order, includeExclude, executionHint, collectMode,
bucketCountThresholds, showTermDocCountError, queryShardContext, parent, subFactoriesBuilder, metadata);
}

235
server/src/main/java/org/elasticsearch/search/aggregations/bucket/terms/TermsAggregatorFactory.java
View File

@ -37,15 +37,19 @@ import org.elasticsearch.search.aggregations.InternalOrder.CompoundOrder;
import org.elasticsearch.search.aggregations.NonCollectingAggregator;
import org.elasticsearch.search.aggregations.bucket.BucketUtils;
import org.elasticsearch.search.aggregations.bucket.terms.TermsAggregator.BucketCountThresholds;
import org.elasticsearch.search.aggregations.support.AggregatorSupplier;
import org.elasticsearch.search.aggregations.support.CoreValuesSourceType;
import org.elasticsearch.search.aggregations.support.ValuesSource;
import org.elasticsearch.search.aggregations.support.ValuesSourceAggregatorFactory;
import org.elasticsearch.search.aggregations.support.ValuesSourceConfig;
import org.elasticsearch.search.aggregations.support.ValuesSourceRegistry;
import org.elasticsearch.search.internal.SearchContext;
import java.io.IOException;
import java.util.Arrays;
import java.util.Map;
public class TermsAggregatorFactory extends ValuesSourceAggregatorFactory<ValuesSource> {
public class TermsAggregatorFactory extends ValuesSourceAggregatorFactory {
private static final DeprecationLogger deprecationLogger = new DeprecationLogger(LogManager.getLogger(TermsAggregatorFactory.class));
static Boolean REMAP_GLOBAL_ORDS, COLLECT_SEGMENT_ORDS;
@ -57,19 +61,139 @@ public class TermsAggregatorFactory extends ValuesSourceAggregatorFactory<Values
private final TermsAggregator.BucketCountThresholds bucketCountThresholds;
private final boolean showTermDocCountError;
// TODO: Registration should happen on the actual aggregator classes
static void registerAggregators(ValuesSourceRegistry valuesSourceRegistry) {
valuesSourceRegistry.register(TermsAggregationBuilder.NAME,
Arrays.asList(CoreValuesSourceType.BYTES, CoreValuesSourceType.IP),
TermsAggregatorFactory.bytesSupplier());
valuesSourceRegistry.register(TermsAggregationBuilder.NAME,
Arrays.asList(CoreValuesSourceType.DATE, CoreValuesSourceType.BOOLEAN, CoreValuesSourceType.NUMERIC),
TermsAggregatorFactory.numericSupplier());
}
/**
* This supplier is used for all the field types that should be aggregated as bytes/strings,
* including those that need global ordinals
*/
private static TermsAggregatorSupplier bytesSupplier() {
return new TermsAggregatorSupplier() {
@Override
public Aggregator build(String name,
AggregatorFactories factories,
ValuesSource valuesSource,
BucketOrder order,
DocValueFormat format,
TermsAggregator.BucketCountThresholds bucketCountThresholds,
IncludeExclude includeExclude,
String executionHint,
SearchContext context,
Aggregator parent,
SubAggCollectionMode subAggCollectMode,
boolean showTermDocCountError,
Map<String, Object> metadata) throws IOException {
ExecutionMode execution = null;
if (executionHint != null) {
execution = ExecutionMode.fromString(executionHint, deprecationLogger);
}
// In some cases, using ordinals is just not supported: override it
if (valuesSource instanceof ValuesSource.Bytes.WithOrdinals == false) {
execution = ExecutionMode.MAP;
}
if (execution == null) {
execution = ExecutionMode.GLOBAL_ORDINALS;
}
final long maxOrd = execution == ExecutionMode.GLOBAL_ORDINALS ? getMaxOrd(valuesSource, context.searcher()) : -1;
if (subAggCollectMode == null) {
subAggCollectMode = SubAggCollectionMode.DEPTH_FIRST;
// TODO can we remove concept of AggregatorFactories.EMPTY?
if (factories != AggregatorFactories.EMPTY) {
subAggCollectMode = subAggCollectionMode(bucketCountThresholds.getShardSize(), maxOrd);
}
}
if ((includeExclude != null) && (includeExclude.isRegexBased()) && format != DocValueFormat.RAW) {
// TODO this exception message is not really accurate for the string case. It's really disallowing regex + formatter
throw new AggregationExecutionException("Aggregation [" + name + "] cannot support regular expression style "
+ "include/exclude settings as they can only be applied to string fields. Use an array of values for "
+ "include/exclude clauses");
}
// TODO: [Zach] we might want refactor and remove ExecutionMode#create(), moving that logic outside the enum
return execution.create(name, factories, valuesSource, order, format, bucketCountThresholds, includeExclude,
context, parent, subAggCollectMode, showTermDocCountError, metadata);
}
};
}
/**
* This supplier is used for all fields that expect to be aggregated as a numeric value.
* This includes floating points, and formatted types that use numerics internally for storage (date, boolean, etc)
*/
private static TermsAggregatorSupplier numericSupplier() {
return new TermsAggregatorSupplier() {
@Override
public Aggregator build(String name,
AggregatorFactories factories,
ValuesSource valuesSource,
BucketOrder order,
DocValueFormat format,
TermsAggregator.BucketCountThresholds bucketCountThresholds,
IncludeExclude includeExclude,
String executionHint,
SearchContext context,
Aggregator parent,
SubAggCollectionMode subAggCollectMode,
boolean showTermDocCountError,
Map<String, Object> metadata) throws IOException {
if ((includeExclude != null) && (includeExclude.isRegexBased())) {
throw new AggregationExecutionException("Aggregation [" + name + "] cannot support regular expression style "
+ "include/exclude settings as they can only be applied to string fields. Use an array of numeric values for "
+ "include/exclude clauses used to filter numeric fields");
}
IncludeExclude.LongFilter longFilter = null;
if (subAggCollectMode == null) {
// TODO can we remove concept of AggregatorFactories.EMPTY?
if (factories != AggregatorFactories.EMPTY) {
subAggCollectMode = subAggCollectionMode(bucketCountThresholds.getShardSize(), -1);
} else {
subAggCollectMode = SubAggCollectionMode.DEPTH_FIRST;
}
}
if (((ValuesSource.Numeric) valuesSource).isFloatingPoint()) {
if (includeExclude != null) {
longFilter = includeExclude.convertToDoubleFilter();
}
return new DoubleTermsAggregator(name, factories, (ValuesSource.Numeric) valuesSource, format, order,
bucketCountThresholds, context, parent, subAggCollectMode, showTermDocCountError, longFilter,
metadata);
}
if (includeExclude != null) {
longFilter = includeExclude.convertToLongFilter(format);
}
return new LongTermsAggregator(name, factories, (ValuesSource.Numeric) valuesSource, format, order,
bucketCountThresholds, context, parent, subAggCollectMode, showTermDocCountError, longFilter,
metadata);
}
};
}
TermsAggregatorFactory(String name,
ValuesSourceConfig<ValuesSource> config,
BucketOrder order,
IncludeExclude includeExclude,
String executionHint,
SubAggCollectionMode collectMode,
TermsAggregator.BucketCountThresholds bucketCountThresholds,
boolean showTermDocCountError,
QueryShardContext queryShardContext,
AggregatorFactory parent,
AggregatorFactories.Builder subFactoriesBuilder,
Map<String, Object> metadata) throws IOException {
ValuesSourceConfig config,
BucketOrder order,
IncludeExclude includeExclude,
String executionHint,
SubAggCollectionMode collectMode,
TermsAggregator.BucketCountThresholds bucketCountThresholds,
boolean showTermDocCountError,
QueryShardContext queryShardContext,
AggregatorFactory parent,
AggregatorFactories.Builder subFactoriesBuilder,
Map<String, Object> metadata) throws IOException {
super(name, config, queryShardContext, parent, subFactoriesBuilder, metadata);
this.order = order;
this.includeExclude = includeExclude;
@ -109,86 +233,35 @@ public class TermsAggregatorFactory extends ValuesSourceAggregatorFactory<Values
@Override
protected Aggregator doCreateInternal(ValuesSource valuesSource,
SearchContext searchContext,
Aggregator parent,
boolean collectsFromSingleBucket,
Map<String, Object> metadata) throws IOException {
SearchContext searchContext,
Aggregator parent,
boolean collectsFromSingleBucket,
Map<String, Object> metadata) throws IOException {
if (collectsFromSingleBucket == false) {
return asMultiBucketAggregator(this, searchContext, parent);
}
AggregatorSupplier aggregatorSupplier = queryShardContext.getValuesSourceRegistry().getAggregator(config.valueSourceType(),
TermsAggregationBuilder.NAME);
if (aggregatorSupplier instanceof TermsAggregatorSupplier == false) {
throw new AggregationExecutionException("Registry miss-match - expected TermsAggregatorSupplier, found [" +
aggregatorSupplier.getClass().toString() + "]");
}
TermsAggregatorSupplier termsAggregatorSupplier = (TermsAggregatorSupplier) aggregatorSupplier;
BucketCountThresholds bucketCountThresholds = new BucketCountThresholds(this.bucketCountThresholds);
if (InternalOrder.isKeyOrder(order) == false
&& bucketCountThresholds.getShardSize() == TermsAggregationBuilder.DEFAULT_BUCKET_COUNT_THRESHOLDS.getShardSize()) {
&& bucketCountThresholds.getShardSize() == TermsAggregationBuilder.DEFAULT_BUCKET_COUNT_THRESHOLDS.getShardSize()) {
// The user has not made a shardSize selection. Use default
// heuristic to avoid any wrong-ranking caused by distributed
// counting
bucketCountThresholds.setShardSize(BucketUtils.suggestShardSideQueueSize(bucketCountThresholds.getRequiredSize()));
}
bucketCountThresholds.ensureValidity();
if (valuesSource instanceof ValuesSource.Bytes) {
ExecutionMode execution = null;
if (executionHint != null) {
execution = ExecutionMode.fromString(executionHint, deprecationLogger);
}
// In some cases, using ordinals is just not supported: override it
if (valuesSource instanceof ValuesSource.Bytes.WithOrdinals == false) {
execution = ExecutionMode.MAP;
}
if (execution == null) {
execution = ExecutionMode.GLOBAL_ORDINALS;
}
final long maxOrd = execution == ExecutionMode.GLOBAL_ORDINALS ? getMaxOrd(valuesSource, searchContext.searcher()) : -1;
SubAggCollectionMode cm = collectMode;
if (cm == null) {
cm = SubAggCollectionMode.DEPTH_FIRST;
if (factories != AggregatorFactories.EMPTY) {
cm = subAggCollectionMode(bucketCountThresholds.getShardSize(), maxOrd);
}
}
DocValueFormat format = config.format();
if ((includeExclude != null) && (includeExclude.isRegexBased()) && format != DocValueFormat.RAW) {
throw new AggregationExecutionException("Aggregation [" + name + "] cannot support regular expression style "
+ "include/exclude settings as they can only be applied to string fields. Use an array of values for "
+ "include/exclude clauses");
}
return execution.create(name, factories, valuesSource, order, format,
bucketCountThresholds, includeExclude, searchContext, parent, cm, showTermDocCountError, metadata);
}
if ((includeExclude != null) && (includeExclude.isRegexBased())) {
throw new AggregationExecutionException("Aggregation [" + name + "] cannot support regular expression style "
+ "include/exclude settings as they can only be applied to string fields. Use an array of numeric values for "
+ "include/exclude clauses used to filter numeric fields");
}
if (valuesSource instanceof ValuesSource.Numeric) {
IncludeExclude.LongFilter longFilter = null;
SubAggCollectionMode cm = collectMode;
if (cm == null) {
if (factories != AggregatorFactories.EMPTY) {
cm = subAggCollectionMode(bucketCountThresholds.getShardSize(), -1);
} else {
cm = SubAggCollectionMode.DEPTH_FIRST;
}
}
if (((ValuesSource.Numeric) valuesSource).isFloatingPoint()) {
if (includeExclude != null) {
longFilter = includeExclude.convertToDoubleFilter();
}
return new DoubleTermsAggregator(name, factories, (ValuesSource.Numeric) valuesSource, config.format(), order,
bucketCountThresholds, searchContext, parent, cm, showTermDocCountError, longFilter, metadata);
}
if (includeExclude != null) {
longFilter = includeExclude.convertToLongFilter(config.format());
}
return new LongTermsAggregator(name, factories, (ValuesSource.Numeric) valuesSource, config.format(), order,
bucketCountThresholds, searchContext, parent, cm, showTermDocCountError, longFilter, metadata);
}
throw new AggregationExecutionException("terms aggregation cannot be applied to field [" + config.fieldContext().field()
+ "]. It can only be applied to numeric or string fields.");
return termsAggregatorSupplier.build(name, factories, valuesSource, order, config.format(),
bucketCountThresholds, includeExclude, executionHint, searchContext, parent, collectMode,
showTermDocCountError, metadata);
}
// return the SubAggCollectionMode that this aggregation should use based on the expected size
@ -209,7 +282,7 @@ public class TermsAggregatorFactory extends ValuesSourceAggregatorFactory<Values
* Get the maximum global ordinal value for the provided {@link ValuesSource} or -1
* if the values source is not an instance of {@link ValuesSource.Bytes.WithOrdinals}.
*/
static long getMaxOrd(ValuesSource source, IndexSearcher searcher) throws IOException {
private static long getMaxOrd(ValuesSource source, IndexSearcher searcher) throws IOException {
if (source instanceof ValuesSource.Bytes.WithOrdinals) {
ValuesSource.Bytes.WithOrdinals valueSourceWithOrdinals = (ValuesSource.Bytes.WithOrdinals) source;
return valueSourceWithOrdinals.globalMaxOrd(searcher);

46
server/src/main/java/org/elasticsearch/search/aggregations/bucket/terms/TermsAggregatorSupplier.java Normal file
View File

@ -0,0 +1,46 @@
/*
* Licensed to Elasticsearch under one or more contributor
* license agreements. See the NOTICE file distributed with
* this work for additional information regarding copyright
* ownership. Elasticsearch licenses this file to you under
* the Apache License, Version 2.0 (the "License"); you may
* not use this file except in compliance with the License.
* You may obtain a copy of the License at
*
* http://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing,
* software distributed under the License is distributed on an
* "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
* KIND, either express or implied. See the License for the
* specific language governing permissions and limitations
* under the License.
*/
package org.elasticsearch.search.aggregations.bucket.terms;
import org.elasticsearch.search.DocValueFormat;
import org.elasticsearch.search.aggregations.Aggregator;
import org.elasticsearch.search.aggregations.AggregatorFactories;
import org.elasticsearch.search.aggregations.BucketOrder;
import org.elasticsearch.search.aggregations.support.AggregatorSupplier;
import org.elasticsearch.search.aggregations.support.ValuesSource;
import org.elasticsearch.search.internal.SearchContext;
import java.io.IOException;
import java.util.Map;
interface TermsAggregatorSupplier extends AggregatorSupplier {
Aggregator build(String name,
AggregatorFactories factories,
ValuesSource valuesSource,
BucketOrder order,
DocValueFormat format,
TermsAggregator.BucketCountThresholds bucketCountThresholds,
IncludeExclude includeExclude,
String executionHint,
SearchContext context,
Aggregator parent,
Aggregator.SubAggCollectionMode subAggCollectMode,
boolean showTermDocCountError,
Map<String, Object> metadata) throws IOException;
}

42
server/src/main/java/org/elasticsearch/search/aggregations/metrics/AbstractHDRPercentilesAggregator.java
View File

@ -26,8 +26,6 @@ import org.elasticsearch.common.lease.Releasables;
import org.elasticsearch.common.util.ArrayUtils;
import org.elasticsearch.common.util.BigArrays;
import org.elasticsearch.common.util.ObjectArray;
import org.elasticsearch.index.fielddata.HistogramValue;
import org.elasticsearch.index.fielddata.HistogramValues;
import org.elasticsearch.index.fielddata.SortedNumericDoubleValues;
import org.elasticsearch.search.DocValueFormat;
import org.elasticsearch.search.aggregations.Aggregator;
@ -76,18 +74,8 @@ abstract class AbstractHDRPercentilesAggregator extends NumericMetricsAggregator
return LeafBucketCollector.NO_OP_COLLECTOR;
}
final BigArrays bigArrays = context.bigArrays();
if (valuesSource instanceof ValuesSource.Histogram) {
final HistogramValues values = ((ValuesSource.Histogram)valuesSource).getHistogramValues(ctx);
return collectHistogramValues(values, bigArrays, sub);
} else {
final SortedNumericDoubleValues values = ((ValuesSource.Numeric)valuesSource).doubleValues(ctx);
return collectNumeric(values, bigArrays, sub);
}
}
private LeafBucketCollector collectNumeric(final SortedNumericDoubleValues values,
final BigArrays bigArrays, final LeafBucketCollector sub) {
final SortedNumericDoubleValues values = ((ValuesSource.Numeric)valuesSource).doubleValues(ctx);
return new LeafBucketCollectorBase(sub, values) {
@Override
public void collect(int doc, long bucket) throws IOException {
@ -100,22 +88,7 @@ abstract class AbstractHDRPercentilesAggregator extends NumericMetricsAggregator
}
}
};
}
private LeafBucketCollector collectHistogramValues(final HistogramValues values,
final BigArrays bigArrays, final LeafBucketCollector sub) {
return new LeafBucketCollectorBase(sub, values) {
@Override
public void collect(int doc, long bucket) throws IOException {
DoubleHistogram state = getExistingOrNewHistogram(bigArrays, bucket);
if (values.advanceExact(doc)) {
final HistogramValue sketch = values.histogram();
while (sketch.next()) {
state.recordValueWithCount(sketch.value(), sketch.count());
}
}
}
};
}
private DoubleHistogram getExistingOrNewHistogram(final BigArrays bigArrays, long bucket) {
@ -123,12 +96,13 @@ abstract class AbstractHDRPercentilesAggregator extends NumericMetricsAggregator
DoubleHistogram state = states.get(bucket);
if (state == null) {
state = new DoubleHistogram(numberOfSignificantValueDigits);
// Set the histogram to autosize so it can resize itself as
// the data range increases. Resize operations should be
// rare as the histogram buckets are exponential (on the top
// level). In the future we could expose the range as an
// option on the request so the histogram can be fixed at
// initialisation and doesn't need resizing.
/* Set the histogram to autosize so it can resize itself as
the data range increases. Resize operations should be
rare as the histogram buckets are exponential (on the top
level). In the future we could expose the range as an
option on the request so the histogram can be fixed at
initialisation and doesn't need resizing.
*/
state.setAutoResize(true);
states.set(bucket, state);
}

9
server/src/main/java/org/elasticsearch/search/aggregations/metrics/AbstractPercentilesAggregationBuilder.java
View File

@ -28,11 +28,8 @@ import org.elasticsearch.common.io.stream.Writeable;
import org.elasticsearch.common.xcontent.ConstructingObjectParser;
import org.elasticsearch.common.xcontent.XContentBuilder;
import org.elasticsearch.search.aggregations.AggregatorFactories;
import org.elasticsearch.search.aggregations.support.CoreValuesSourceType;
import org.elasticsearch.search.aggregations.support.ValueType;
import org.elasticsearch.search.aggregations.support.ValuesSource;
import org.elasticsearch.search.aggregations.support.ValuesSourceAggregationBuilder;
import org.elasticsearch.search.aggregations.support.ValuesSourceParserHelper;
import java.io.IOException;
import java.util.Arrays;
@ -107,7 +104,7 @@ public abstract class AbstractPercentilesAggregationBuilder<T extends AbstractPe
return ctor.apply(name, values, percentilesConfig);
});
ValuesSourceParserHelper.declareAnyFields(parser, true, true);
ValuesSourceAggregationBuilder.declareFields(parser, true, true, false);
parser.declareDoubleArray(ConstructingObjectParser.optionalConstructorArg(), valuesField);
parser.declareBoolean(T::keyed, KEYED_FIELD);
parser.declareObject(ConstructingObjectParser.optionalConstructorArg(), PercentilesMethod.TDIGEST_PARSER,
@ -120,7 +117,7 @@ public abstract class AbstractPercentilesAggregationBuilder<T extends AbstractPe
AbstractPercentilesAggregationBuilder(String name, double[] values, PercentilesConfig percentilesConfig,
ParseField valuesField) {
super(name, CoreValuesSourceType.NUMERIC, ValueType.NUMERIC);
super(name);
if (values == null) {
throw new IllegalArgumentException("[" + valuesField.getPreferredName() + "] must not be null: [" + name + "]");
}
@ -144,7 +141,7 @@ public abstract class AbstractPercentilesAggregationBuilder<T extends AbstractPe
}
AbstractPercentilesAggregationBuilder(StreamInput in) throws IOException {
super(in, CoreValuesSourceType.NUMERIC, ValueType.NUMERIC);
super(in);
values = in.readDoubleArray();
keyed = in.readBoolean();
if (in.getVersion().onOrAfter(Version.V_7_8_0)) {

31
server/src/main/java/org/elasticsearch/search/aggregations/metrics/AbstractTDigestPercentilesAggregator.java
View File

@ -25,8 +25,6 @@ import org.elasticsearch.common.lease.Releasables;
import org.elasticsearch.common.util.ArrayUtils;
import org.elasticsearch.common.util.BigArrays;
import org.elasticsearch.common.util.ObjectArray;
import org.elasticsearch.index.fielddata.HistogramValue;
import org.elasticsearch.index.fielddata.HistogramValues;
import org.elasticsearch.index.fielddata.SortedNumericDoubleValues;
import org.elasticsearch.search.DocValueFormat;
import org.elasticsearch.search.aggregations.Aggregator;
@ -75,18 +73,7 @@ abstract class AbstractTDigestPercentilesAggregator extends NumericMetricsAggreg
return LeafBucketCollector.NO_OP_COLLECTOR;
}
final BigArrays bigArrays = context.bigArrays();
if (valuesSource instanceof ValuesSource.Histogram) {
final HistogramValues values = ((ValuesSource.Histogram)valuesSource).getHistogramValues(ctx);
return collectHistogramValues(values, bigArrays, sub);
} else {
final SortedNumericDoubleValues values = ((ValuesSource.Numeric)valuesSource).doubleValues(ctx);
return collectNumeric(values, bigArrays, sub);
}
}
private LeafBucketCollector collectNumeric(final SortedNumericDoubleValues values,
final BigArrays bigArrays, final LeafBucketCollector sub) {
final SortedNumericDoubleValues values = ((ValuesSource.Numeric)valuesSource).doubleValues(ctx);
return new LeafBucketCollectorBase(sub, values) {
@Override
public void collect(int doc, long bucket) throws IOException {
@ -101,22 +88,6 @@ abstract class AbstractTDigestPercentilesAggregator extends NumericMetricsAggreg
};
}
private LeafBucketCollector collectHistogramValues(final HistogramValues values,
final BigArrays bigArrays, final LeafBucketCollector sub) {
return new LeafBucketCollectorBase(sub, values) {
@Override
public void collect(int doc, long bucket) throws IOException {
TDigestState state = getExistingOrNewHistogram(bigArrays, bucket);
if (values.advanceExact(doc)) {
final HistogramValue sketch = values.histogram();
while(sketch.next()) {
state.add(sketch.value(), sketch.count());
}
}
}
};
}
private TDigestState getExistingOrNewHistogram(final BigArrays bigArrays, long bucket) {
states = bigArrays.grow(states, bucket + 1);
TDigestState state = states.get(bucket);

22
server/src/main/java/org/elasticsearch/search/aggregations/metrics/AvgAggregationBuilder.java
View File

@ -28,12 +28,11 @@ import org.elasticsearch.search.aggregations.AggregationBuilder;
import org.elasticsearch.search.aggregations.AggregatorFactories.Builder;
import org.elasticsearch.search.aggregations.AggregatorFactory;
import org.elasticsearch.search.aggregations.support.CoreValuesSourceType;
import org.elasticsearch.search.aggregations.support.ValueType;
import org.elasticsearch.search.aggregations.support.ValuesSource;
import org.elasticsearch.search.aggregations.support.ValuesSource.Numeric;
import org.elasticsearch.search.aggregations.support.ValuesSourceAggregationBuilder;
import org.elasticsearch.search.aggregations.support.ValuesSourceConfig;
import org.elasticsearch.search.aggregations.support.ValuesSourceParserHelper;
import org.elasticsearch.search.aggregations.support.ValuesSourceRegistry;
import org.elasticsearch.search.aggregations.support.ValuesSourceType;
import java.io.IOException;
import java.util.Map;
@ -43,22 +42,31 @@ public class AvgAggregationBuilder extends ValuesSourceAggregationBuilder.LeafOn
public static final ObjectParser<AvgAggregationBuilder, String> PARSER = ObjectParser.fromBuilder(NAME, AvgAggregationBuilder::new);
static {
ValuesSourceParserHelper.declareNumericFields(PARSER, true, true, false);
ValuesSourceAggregationBuilder.declareFields(PARSER, true, true, false);
}
public static void registerAggregators(ValuesSourceRegistry valuesSourceRegistry) {
AvgAggregatorFactory.registerAggregators(valuesSourceRegistry);
}
public AvgAggregationBuilder(String name) {
super(name, CoreValuesSourceType.NUMERIC, ValueType.NUMERIC);
super(name);
}
public AvgAggregationBuilder(AvgAggregationBuilder clone, Builder factoriesBuilder, Map<String, Object> metadata) {
super(clone, factoriesBuilder, metadata);
}
@Override
protected ValuesSourceType defaultValueSourceType() {
return CoreValuesSourceType.NUMERIC;
}
/**
* Read from a stream.
*/
public AvgAggregationBuilder(StreamInput in) throws IOException {
super(in, CoreValuesSourceType.NUMERIC, ValueType.NUMERIC);
super(in);
}
@Override
@ -72,7 +80,7 @@ public class AvgAggregationBuilder extends ValuesSourceAggregationBuilder.LeafOn
}
@Override
protected AvgAggregatorFactory innerBuild(QueryShardContext queryShardContext, ValuesSourceConfig<Numeric> config,
protected AvgAggregatorFactory innerBuild(QueryShardContext queryShardContext, ValuesSourceConfig config,
AggregatorFactory parent, Builder subFactoriesBuilder) throws IOException {
return new AvgAggregatorFactory(name, config, queryShardContext, parent, subFactoriesBuilder, metadata);
}

40
server/src/main/java/org/elasticsearch/search/aggregations/metrics/AvgAggregatorFactory.java
View File

@ -20,25 +20,48 @@
package org.elasticsearch.search.aggregations.metrics;
import org.elasticsearch.index.query.QueryShardContext;
import org.elasticsearch.search.DocValueFormat;
import org.elasticsearch.search.aggregations.AggregationExecutionException;
import org.elasticsearch.search.aggregations.Aggregator;
import org.elasticsearch.search.aggregations.AggregatorFactories;
import org.elasticsearch.search.aggregations.AggregatorFactory;
import org.elasticsearch.search.aggregations.support.AggregatorSupplier;
import org.elasticsearch.search.aggregations.support.CoreValuesSourceType;
import org.elasticsearch.search.aggregations.support.ValuesSource;
import org.elasticsearch.search.aggregations.support.ValuesSource.Numeric;
import org.elasticsearch.search.aggregations.support.ValuesSourceAggregatorFactory;
import org.elasticsearch.search.aggregations.support.ValuesSourceConfig;
import org.elasticsearch.search.aggregations.support.ValuesSourceRegistry;
import org.elasticsearch.search.internal.SearchContext;
import java.io.IOException;
import java.util.Arrays;
import java.util.Map;
class AvgAggregatorFactory extends ValuesSourceAggregatorFactory<ValuesSource.Numeric> {
class AvgAggregatorFactory extends ValuesSourceAggregatorFactory {
AvgAggregatorFactory(String name, ValuesSourceConfig<Numeric> config, QueryShardContext queryShardContext,
AggregatorFactory parent, AggregatorFactories.Builder subFactoriesBuilder, Map<String, Object> metadata) throws IOException {
AvgAggregatorFactory(String name, ValuesSourceConfig config, QueryShardContext queryShardContext,
AggregatorFactory parent, AggregatorFactories.Builder subFactoriesBuilder,
Map<String, Object> metadata) throws IOException {
super(name, config, queryShardContext, parent, subFactoriesBuilder, metadata);
}
static void registerAggregators(ValuesSourceRegistry valuesSourceRegistry) {
valuesSourceRegistry.register(AvgAggregationBuilder.NAME,
Arrays.asList(CoreValuesSourceType.NUMERIC, CoreValuesSourceType.DATE, CoreValuesSourceType.BOOLEAN),
new MetricAggregatorSupplier() {
@Override
public Aggregator build(String name,
ValuesSource valuesSource,
DocValueFormat formatter,
SearchContext context,
Aggregator parent,
Map<String, Object> metadata) throws IOException {
return new AvgAggregator(name, (Numeric) valuesSource, formatter, context, parent, metadata);
}
});
}
@Override
protected Aggregator createUnmapped(SearchContext searchContext,
Aggregator parent,
@ -47,11 +70,18 @@ class AvgAggregatorFactory extends ValuesSourceAggregatorFactory<ValuesSource.Nu
}
@Override
protected Aggregator doCreateInternal(Numeric valuesSource,
protected Aggregator doCreateInternal(ValuesSource valuesSource,
SearchContext searchContext,
Aggregator parent,
boolean collectsFromSingleBucket,
Map<String, Object> metadata) throws IOException {
return new AvgAggregator(name, valuesSource, config.format(), searchContext, parent, metadata);
AggregatorSupplier aggregatorSupplier = queryShardContext.getValuesSourceRegistry().getAggregator(config.valueSourceType(),
AvgAggregationBuilder.NAME);
if (aggregatorSupplier instanceof MetricAggregatorSupplier == false) {
throw new AggregationExecutionException("Registry miss-match - expected MetricAggregatorSupplier, found [" +
aggregatorSupplier.getClass().toString() + "]");
}
return ((MetricAggregatorSupplier) aggregatorSupplier).build(name, valuesSource, config.format(), searchContext, parent, metadata);
}
}

25
server/src/main/java/org/elasticsearch/search/aggregations/metrics/CardinalityAggregationBuilder.java
View File

@ -30,11 +30,11 @@ import org.elasticsearch.search.aggregations.AggregationBuilder;
import org.elasticsearch.search.aggregations.AggregatorFactories.Builder;
import org.elasticsearch.search.aggregations.AggregatorFactory;
import org.elasticsearch.search.aggregations.support.CoreValuesSourceType;
import org.elasticsearch.search.aggregations.support.ValueType;
import org.elasticsearch.search.aggregations.support.ValuesSource;
import org.elasticsearch.search.aggregations.support.ValuesSourceAggregationBuilder;
import org.elasticsearch.search.aggregations.support.ValuesSourceConfig;
import org.elasticsearch.search.aggregations.support.ValuesSourceParserHelper;
import org.elasticsearch.search.aggregations.support.ValuesSourceRegistry;
import org.elasticsearch.search.aggregations.support.ValuesSourceType;
import java.io.IOException;
import java.util.Map;
@ -49,17 +49,21 @@ public final class CardinalityAggregationBuilder
public static final ParseField PRECISION_THRESHOLD_FIELD = new ParseField("precision_threshold");
public static final ObjectParser<CardinalityAggregationBuilder, String> PARSER =
ObjectParser.fromBuilder(NAME, name -> new CardinalityAggregationBuilder(name, null));
ObjectParser.fromBuilder(NAME, CardinalityAggregationBuilder::new);
static {
ValuesSourceParserHelper.declareAnyFields(PARSER, true, false);
ValuesSourceAggregationBuilder.declareFields(PARSER, true, false, false);
PARSER.declareLong(CardinalityAggregationBuilder::precisionThreshold, CardinalityAggregationBuilder.PRECISION_THRESHOLD_FIELD);
PARSER.declareLong((b, v) -> {/*ignore*/}, REHASH);
}
public static void registerAggregators(ValuesSourceRegistry valuesSourceRegistry) {
CardinalityAggregatorFactory.registerAggregators(valuesSourceRegistry);
}
private Long precisionThreshold = null;
public CardinalityAggregationBuilder(String name, ValueType targetValueType) {
super(name, CoreValuesSourceType.ANY, targetValueType);
public CardinalityAggregationBuilder(String name) {
super(name);
}
public CardinalityAggregationBuilder(CardinalityAggregationBuilder clone, Builder factoriesBuilder, Map<String, Object> metadata) {
@ -67,11 +71,16 @@ public final class CardinalityAggregationBuilder
this.precisionThreshold = clone.precisionThreshold;
}
@Override
protected ValuesSourceType defaultValueSourceType() {
return CoreValuesSourceType.BYTES;
}
/**
* Read from a stream.
*/
public CardinalityAggregationBuilder(StreamInput in) throws IOException {
super(in, CoreValuesSourceType.ANY);
super(in);
if (in.readBoolean()) {
precisionThreshold = in.readLong();
}
@ -119,7 +128,7 @@ public final class CardinalityAggregationBuilder
}
@Override
protected CardinalityAggregatorFactory innerBuild(QueryShardContext queryShardContext, ValuesSourceConfig<ValuesSource> config,
protected CardinalityAggregatorFactory innerBuild(QueryShardContext queryShardContext, ValuesSourceConfig config,
AggregatorFactory parent, Builder subFactoriesBuilder) throws IOException {
return new CardinalityAggregatorFactory(name, config, precisionThreshold, queryShardContext, parent, subFactoriesBuilder, metadata);
}

34
server/src/main/java/org/elasticsearch/search/aggregations/metrics/CardinalityAggregatorFactory.java
View File

@ -20,22 +20,25 @@
package org.elasticsearch.search.aggregations.metrics;
import org.elasticsearch.index.query.QueryShardContext;
import org.elasticsearch.search.aggregations.AggregationExecutionException;
import org.elasticsearch.search.aggregations.Aggregator;
import org.elasticsearch.search.aggregations.AggregatorFactories;
import org.elasticsearch.search.aggregations.AggregatorFactory;
import org.elasticsearch.search.aggregations.support.AggregatorSupplier;
import org.elasticsearch.search.aggregations.support.ValuesSource;
import org.elasticsearch.search.aggregations.support.ValuesSourceAggregatorFactory;
import org.elasticsearch.search.aggregations.support.ValuesSourceConfig;
import org.elasticsearch.search.aggregations.support.ValuesSourceRegistry;
import org.elasticsearch.search.internal.SearchContext;
import java.io.IOException;
import java.util.Map;
class CardinalityAggregatorFactory extends ValuesSourceAggregatorFactory<ValuesSource> {
class CardinalityAggregatorFactory extends ValuesSourceAggregatorFactory {
private final Long precisionThreshold;
CardinalityAggregatorFactory(String name, ValuesSourceConfig<ValuesSource> config,
CardinalityAggregatorFactory(String name, ValuesSourceConfig config,
Long precisionThreshold,
QueryShardContext queryShardContext,
AggregatorFactory parent,
@ -45,6 +48,24 @@ class CardinalityAggregatorFactory extends ValuesSourceAggregatorFactory<ValuesS
this.precisionThreshold = precisionThreshold;
}
static void registerAggregators(ValuesSourceRegistry valuesSourceRegistry) {
valuesSourceRegistry.registerAny(CardinalityAggregationBuilder.NAME, cardinalityAggregatorSupplier());
}
private static CardinalityAggregatorSupplier cardinalityAggregatorSupplier(){
return new CardinalityAggregatorSupplier() {
@Override
public Aggregator build(String name,
ValuesSource valuesSource,
int precision,
SearchContext context,
Aggregator parent,
Map<String, Object> metadata) throws IOException {
return new CardinalityAggregator(name, valuesSource, precision, context, parent, metadata);
}
};
}
@Override
protected Aggregator createUnmapped(SearchContext searchContext,
Aggregator parent,
@ -58,7 +79,14 @@ class CardinalityAggregatorFactory extends ValuesSourceAggregatorFactory<ValuesS
Aggregator parent,
boolean collectsFromSingleBucket,
Map<String, Object> metadata) throws IOException {
return new CardinalityAggregator(name, valuesSource, precision(), searchContext, parent, metadata);
AggregatorSupplier aggregatorSupplier = queryShardContext.getValuesSourceRegistry().getAggregator(config.valueSourceType(),
CardinalityAggregationBuilder.NAME);
if (aggregatorSupplier instanceof CardinalityAggregatorSupplier == false) {
throw new AggregationExecutionException("Registry miss-match - expected CardinalityAggregatorSupplier, found [" +
aggregatorSupplier.getClass().toString() + "]");
}
CardinalityAggregatorSupplier cardinalityAggregatorSupplier = (CardinalityAggregatorSupplier) aggregatorSupplier;
return cardinalityAggregatorSupplier.build(name, valuesSource, precision(), searchContext, parent, metadata);
}
private int precision() {

37
server/src/main/java/org/elasticsearch/search/aggregations/metrics/CardinalityAggregatorSupplier.java Normal file
View File

@ -0,0 +1,37 @@
/*
* Licensed to Elasticsearch under one or more contributor
* license agreements. See the NOTICE file distributed with
* this work for additional information regarding copyright
* ownership. Elasticsearch licenses this file to you under
* the Apache License, Version 2.0 (the "License"); you may
* not use this file except in compliance with the License.
* You may obtain a copy of the License at
*
* http://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing,
* software distributed under the License is distributed on an
* "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
* KIND, either express or implied. See the License for the
* specific language governing permissions and limitations
* under the License.
*/
package org.elasticsearch.search.aggregations.metrics;
import org.elasticsearch.search.aggregations.Aggregator;
import org.elasticsearch.search.aggregations.support.AggregatorSupplier;
import org.elasticsearch.search.aggregations.support.ValuesSource;
import org.elasticsearch.search.internal.SearchContext;
import java.io.IOException;
import java.util.Map;
public interface CardinalityAggregatorSupplier extends AggregatorSupplier {
Aggregator build(String name,
ValuesSource valuesSource,
int precision,
SearchContext context,
Aggregator parent,
Map<String, Object> metadata) throws IOException;
}

17
server/src/main/java/org/elasticsearch/search/aggregations/metrics/ExtendedStatsAggregationBuilder.java
View File

@ -28,12 +28,10 @@ import org.elasticsearch.search.aggregations.AggregationBuilder;
import org.elasticsearch.search.aggregations.AggregatorFactories.Builder;
import org.elasticsearch.search.aggregations.AggregatorFactory;
import org.elasticsearch.search.aggregations.support.CoreValuesSourceType;
import org.elasticsearch.search.aggregations.support.ValueType;
import org.elasticsearch.search.aggregations.support.ValuesSource;
import org.elasticsearch.search.aggregations.support.ValuesSource.Numeric;
import org.elasticsearch.search.aggregations.support.ValuesSourceAggregationBuilder;
import org.elasticsearch.search.aggregations.support.ValuesSourceConfig;
import org.elasticsearch.search.aggregations.support.ValuesSourceParserHelper;
import org.elasticsearch.search.aggregations.support.ValuesSourceType;
import java.io.IOException;
import java.util.Map;
@ -46,14 +44,14 @@ public class ExtendedStatsAggregationBuilder
public static final ObjectParser<ExtendedStatsAggregationBuilder, String> PARSER =
ObjectParser.fromBuilder(NAME, ExtendedStatsAggregationBuilder::new);
static {
ValuesSourceParserHelper.declareNumericFields(PARSER, true, true, false);
ValuesSourceAggregationBuilder.declareFields(PARSER, true, true, false);
PARSER.declareDouble(ExtendedStatsAggregationBuilder::sigma, ExtendedStatsAggregator.SIGMA_FIELD);
}
private double sigma = 2.0;
public ExtendedStatsAggregationBuilder(String name) {
super(name, CoreValuesSourceType.NUMERIC, ValueType.NUMERIC);
super(name);
}
protected ExtendedStatsAggregationBuilder(ExtendedStatsAggregationBuilder clone,
@ -71,10 +69,15 @@ public class ExtendedStatsAggregationBuilder
* Read from a stream.
*/
public ExtendedStatsAggregationBuilder(StreamInput in) throws IOException {
super(in, CoreValuesSourceType.NUMERIC, ValueType.NUMERIC);
super(in);
sigma = in.readDouble();
}
@Override
protected ValuesSourceType defaultValueSourceType() {
return CoreValuesSourceType.NUMERIC;
}
@Override
protected void innerWriteTo(StreamOutput out) throws IOException {
out.writeDouble(sigma);
@ -93,7 +96,7 @@ public class ExtendedStatsAggregationBuilder
}
@Override
protected ExtendedStatsAggregatorFactory innerBuild(QueryShardContext queryShardContext, ValuesSourceConfig<Numeric> config,
protected ExtendedStatsAggregatorFactory innerBuild(QueryShardContext queryShardContext, ValuesSourceConfig config,
AggregatorFactory parent, Builder subFactoriesBuilder) throws IOException {
return new ExtendedStatsAggregatorFactory(name, config, sigma, queryShardContext, parent, subFactoriesBuilder, metadata);
}

13
server/src/main/java/org/elasticsearch/search/aggregations/metrics/ExtendedStatsAggregatorFactory.java
View File

@ -20,6 +20,7 @@
package org.elasticsearch.search.aggregations.metrics;
import org.elasticsearch.index.query.QueryShardContext;
import org.elasticsearch.search.aggregations.AggregationExecutionException;
import org.elasticsearch.search.aggregations.Aggregator;
import org.elasticsearch.search.aggregations.AggregatorFactories;
import org.elasticsearch.search.aggregations.AggregatorFactory;
@ -32,12 +33,12 @@ import org.elasticsearch.search.internal.SearchContext;
import java.io.IOException;
import java.util.Map;
class ExtendedStatsAggregatorFactory extends ValuesSourceAggregatorFactory<ValuesSource.Numeric> {
class ExtendedStatsAggregatorFactory extends ValuesSourceAggregatorFactory {
private final double sigma;
ExtendedStatsAggregatorFactory(String name,
ValuesSourceConfig<Numeric> config,
ValuesSourceConfig config,
double sigma,
QueryShardContext queryShardContext,
AggregatorFactory parent,
@ -55,12 +56,16 @@ class ExtendedStatsAggregatorFactory extends ValuesSourceAggregatorFactory<Value
}
@Override
protected Aggregator doCreateInternal(Numeric valuesSource,
protected Aggregator doCreateInternal(ValuesSource valuesSource,
SearchContext searchContext,
Aggregator parent,
boolean collectsFromSingleBucket,
Map<String, Object> metadata) throws IOException {
return new ExtendedStatsAggregator(name, valuesSource, config.format(), searchContext,
if (valuesSource instanceof Numeric == false) {
throw new AggregationExecutionException("ValuesSource type " + valuesSource.toString() + "is not supported for aggregation " +
this.name());
}
return new ExtendedStatsAggregator(name, (Numeric) valuesSource, config.format(), searchContext,
parent, sigma, metadata);
}
}

24
server/src/main/java/org/elasticsearch/search/aggregations/metrics/GeoBoundsAggregationBuilder.java
View File

@ -28,30 +28,33 @@ import org.elasticsearch.search.aggregations.AggregationBuilder;
import org.elasticsearch.search.aggregations.AggregatorFactories.Builder;
import org.elasticsearch.search.aggregations.AggregatorFactory;
import org.elasticsearch.search.aggregations.support.CoreValuesSourceType;
import org.elasticsearch.search.aggregations.support.ValueType;
import org.elasticsearch.search.aggregations.support.ValuesSource;
import org.elasticsearch.search.aggregations.support.ValuesSourceAggregationBuilder;
import org.elasticsearch.search.aggregations.support.ValuesSourceConfig;
import org.elasticsearch.search.aggregations.support.ValuesSourceParserHelper;
import org.elasticsearch.search.aggregations.support.ValuesSourceRegistry;
import org.elasticsearch.search.aggregations.support.ValuesSourceType;
import java.io.IOException;
import java.util.Map;
import java.util.Objects;
public class GeoBoundsAggregationBuilder extends ValuesSourceAggregationBuilder<ValuesSource.GeoPoint, GeoBoundsAggregationBuilder> {
public class GeoBoundsAggregationBuilder extends ValuesSourceAggregationBuilder<GeoBoundsAggregationBuilder> {
public static final String NAME = "geo_bounds";
public static final ObjectParser<GeoBoundsAggregationBuilder, String> PARSER =
ObjectParser.fromBuilder(NAME, GeoBoundsAggregationBuilder::new);
static {
ValuesSourceParserHelper.declareGeoFields(PARSER, false, false);
ValuesSourceAggregationBuilder.declareFields(PARSER, false, false, false);
PARSER.declareBoolean(GeoBoundsAggregationBuilder::wrapLongitude, GeoBoundsAggregator.WRAP_LONGITUDE_FIELD);
}
public static void registerAggregators(ValuesSourceRegistry valuesSourceRegistry) {
GeoBoundsAggregatorFactory.registerAggregators(valuesSourceRegistry);
}
private boolean wrapLongitude = true;
public GeoBoundsAggregationBuilder(String name) {
super(name, CoreValuesSourceType.GEOPOINT, ValueType.GEOPOINT);
super(name);
}
protected GeoBoundsAggregationBuilder(GeoBoundsAggregationBuilder clone, Builder factoriesBuilder, Map<String, Object> metadata) {
@ -68,7 +71,7 @@ public class GeoBoundsAggregationBuilder extends ValuesSourceAggregationBuilder<
* Read from a stream.
*/
public GeoBoundsAggregationBuilder(StreamInput in) throws IOException {
super(in, CoreValuesSourceType.GEOPOINT, ValueType.GEOPOINT);
super(in);
wrapLongitude = in.readBoolean();
}
@ -77,6 +80,11 @@ public class GeoBoundsAggregationBuilder extends ValuesSourceAggregationBuilder<
out.writeBoolean(wrapLongitude);
}
@Override
protected ValuesSourceType defaultValueSourceType() {
return CoreValuesSourceType.GEOPOINT;
}
/**
* Set whether to wrap longitudes. Defaults to true.
*/
@ -98,7 +106,7 @@ public class GeoBoundsAggregationBuilder extends ValuesSourceAggregationBuilder<
}
@Override
protected GeoBoundsAggregatorFactory innerBuild(QueryShardContext queryShardContext, ValuesSourceConfig<ValuesSource.GeoPoint> config,
protected GeoBoundsAggregatorFactory innerBuild(QueryShardContext queryShardContext, ValuesSourceConfig config,
AggregatorFactory parent, Builder subFactoriesBuilder) throws IOException {
return new GeoBoundsAggregatorFactory(name, config, wrapLongitude, queryShardContext, parent, subFactoriesBuilder, metadata);
}

27
server/src/main/java/org/elasticsearch/search/aggregations/metrics/GeoBoundsAggregatorFactory.java
View File

@ -20,23 +20,27 @@
package org.elasticsearch.search.aggregations.metrics;
import org.elasticsearch.index.query.QueryShardContext;
import org.elasticsearch.search.aggregations.AggregationExecutionException;
import org.elasticsearch.search.aggregations.Aggregator;
import org.elasticsearch.search.aggregations.AggregatorFactories;
import org.elasticsearch.search.aggregations.AggregatorFactory;
import org.elasticsearch.search.aggregations.support.AggregatorSupplier;
import org.elasticsearch.search.aggregations.support.CoreValuesSourceType;
import org.elasticsearch.search.aggregations.support.ValuesSource;
import org.elasticsearch.search.aggregations.support.ValuesSourceAggregatorFactory;
import org.elasticsearch.search.aggregations.support.ValuesSourceConfig;
import org.elasticsearch.search.aggregations.support.ValuesSourceRegistry;
import org.elasticsearch.search.internal.SearchContext;
import java.io.IOException;
import java.util.Map;
class GeoBoundsAggregatorFactory extends ValuesSourceAggregatorFactory<ValuesSource.GeoPoint> {
class GeoBoundsAggregatorFactory extends ValuesSourceAggregatorFactory {
private final boolean wrapLongitude;
GeoBoundsAggregatorFactory(String name,
ValuesSourceConfig<ValuesSource.GeoPoint> config,
ValuesSourceConfig config,
boolean wrapLongitude,
QueryShardContext queryShardContext,
AggregatorFactory parent,
@ -54,11 +58,26 @@ class GeoBoundsAggregatorFactory extends ValuesSourceAggregatorFactory<ValuesSou
}
@Override
protected Aggregator doCreateInternal(ValuesSource.GeoPoint valuesSource,
protected Aggregator doCreateInternal(ValuesSource valuesSource,
SearchContext searchContext,
Aggregator parent,
boolean collectsFromSingleBucket,
Map<String, Object> metadata) throws IOException {
return new GeoBoundsAggregator(name, searchContext, parent, valuesSource, wrapLongitude, metadata);
AggregatorSupplier aggregatorSupplier = queryShardContext.getValuesSourceRegistry()
.getAggregator(config.valueSourceType(), GeoBoundsAggregationBuilder.NAME);
if (aggregatorSupplier instanceof GeoBoundsAggregatorSupplier == false) {
throw new AggregationExecutionException("Registry miss-match - expected "
+ GeoBoundsAggregatorSupplier.class.getName() + ", found [" + aggregatorSupplier.getClass().toString() + "]");
}
return ((GeoBoundsAggregatorSupplier) aggregatorSupplier).build(name, searchContext, parent, valuesSource, wrapLongitude, metadata);
}
static void registerAggregators(ValuesSourceRegistry valuesSourceRegistry) {
valuesSourceRegistry.register(GeoBoundsAggregationBuilder.NAME, CoreValuesSourceType.GEOPOINT,
(GeoBoundsAggregatorSupplier) (name, aggregationContext, parent, valuesSource, wrapLongitude, metadata)
-> new GeoBoundsAggregator(name, aggregationContext, parent, (ValuesSource.GeoPoint) valuesSource,
wrapLongitude, metadata));
}
}

36
server/src/main/java/org/elasticsearch/search/aggregations/metrics/GeoBoundsAggregatorSupplier.java Normal file
View File

@ -0,0 +1,36 @@
/*
* Licensed to Elasticsearch under one or more contributor
* license agreements. See the NOTICE file distributed with
* this work for additional information regarding copyright
* ownership. Elasticsearch licenses this file to you under
* the Apache License, Version 2.0 (the "License"); you may
* not use this file except in compliance with the License.
* You may obtain a copy of the License at
*
* http://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing,
* software distributed under the License is distributed on an
* "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
* KIND, either express or implied. See the License for the
* specific language governing permissions and limitations
* under the License.
*/
package org.elasticsearch.search.aggregations.metrics;
import org.elasticsearch.search.aggregations.Aggregator;
import org.elasticsearch.search.aggregations.support.AggregatorSupplier;
import org.elasticsearch.search.aggregations.support.ValuesSource;
import org.elasticsearch.search.internal.SearchContext;
import java.io.IOException;
import java.util.Map;
@FunctionalInterface
public interface GeoBoundsAggregatorSupplier extends AggregatorSupplier {
GeoBoundsAggregator build(String name, SearchContext aggregationContext, Aggregator parent,
ValuesSource valuesSource, boolean wrapLongitude,
Map<String, Object> metadata) throws IOException;
}

27
server/src/main/java/org/elasticsearch/search/aggregations/metrics/GeoCentroidAggregationBuilder.java
View File

@ -19,9 +19,6 @@
package org.elasticsearch.search.aggregations.metrics;
import java.io.IOException;
import java.util.Map;
import org.elasticsearch.common.io.stream.StreamInput;
import org.elasticsearch.common.io.stream.StreamOutput;
import org.elasticsearch.common.xcontent.ObjectParser;
@ -31,11 +28,14 @@ import org.elasticsearch.search.aggregations.AggregationBuilder;
import org.elasticsearch.search.aggregations.AggregatorFactories.Builder;
import org.elasticsearch.search.aggregations.AggregatorFactory;
import org.elasticsearch.search.aggregations.support.CoreValuesSourceType;
import org.elasticsearch.search.aggregations.support.ValueType;
import org.elasticsearch.search.aggregations.support.ValuesSource;
import org.elasticsearch.search.aggregations.support.ValuesSourceAggregationBuilder;
import org.elasticsearch.search.aggregations.support.ValuesSourceConfig;
import org.elasticsearch.search.aggregations.support.ValuesSourceParserHelper;
import org.elasticsearch.search.aggregations.support.ValuesSourceRegistry;
import org.elasticsearch.search.aggregations.support.ValuesSourceType;
import java.io.IOException;
import java.util.Map;
public class GeoCentroidAggregationBuilder
extends ValuesSourceAggregationBuilder.LeafOnly<ValuesSource.GeoPoint, GeoCentroidAggregationBuilder> {
@ -44,17 +44,26 @@ public class GeoCentroidAggregationBuilder
public static final ObjectParser<GeoCentroidAggregationBuilder, String> PARSER =
ObjectParser.fromBuilder(NAME, GeoCentroidAggregationBuilder::new);
static {
ValuesSourceParserHelper.declareGeoFields(PARSER, true, false);
ValuesSourceAggregationBuilder.declareFields(PARSER, true, false, false);
}
public static void registerAggregators(ValuesSourceRegistry valuesSourceRegistry) {
GeoCentroidAggregatorFactory.registerAggregators(valuesSourceRegistry);
}
public GeoCentroidAggregationBuilder(String name) {
super(name, CoreValuesSourceType.GEOPOINT, ValueType.GEOPOINT);
super(name);
}
protected GeoCentroidAggregationBuilder(GeoCentroidAggregationBuilder clone, Builder factoriesBuilder, Map<String, Object> metadata) {
super(clone, factoriesBuilder, metadata);
}
@Override
protected ValuesSourceType defaultValueSourceType() {
return CoreValuesSourceType.GEOPOINT;
}
@Override
protected AggregationBuilder shallowCopy(Builder factoriesBuilder, Map<String, Object> metadata) {
return new GeoCentroidAggregationBuilder(this, factoriesBuilder, metadata);
@ -64,7 +73,7 @@ public class GeoCentroidAggregationBuilder
* Read from a stream.
*/
public GeoCentroidAggregationBuilder(StreamInput in) throws IOException {
super(in, CoreValuesSourceType.GEOPOINT, ValueType.GEOPOINT);
super(in);
}
@Override
@ -73,7 +82,7 @@ public class GeoCentroidAggregationBuilder
}
@Override
protected GeoCentroidAggregatorFactory innerBuild(QueryShardContext queryShardContext, ValuesSourceConfig<ValuesSource.GeoPoint> config,
protected GeoCentroidAggregatorFactory innerBuild(QueryShardContext queryShardContext, ValuesSourceConfig config,
AggregatorFactory parent, Builder subFactoriesBuilder) throws IOException {
return new GeoCentroidAggregatorFactory(name, config, queryShardContext, parent, subFactoriesBuilder, metadata);
}

25
server/src/main/java/org/elasticsearch/search/aggregations/metrics/GeoCentroidAggregatorFactory.java
View File

@ -20,21 +20,25 @@
package org.elasticsearch.search.aggregations.metrics;
import org.elasticsearch.index.query.QueryShardContext;
import org.elasticsearch.search.aggregations.AggregationExecutionException;
import org.elasticsearch.search.aggregations.Aggregator;
import org.elasticsearch.search.aggregations.AggregatorFactories;
import org.elasticsearch.search.aggregations.AggregatorFactory;
import org.elasticsearch.search.aggregations.support.AggregatorSupplier;
import org.elasticsearch.search.aggregations.support.CoreValuesSourceType;
import org.elasticsearch.search.aggregations.support.ValuesSource;
import org.elasticsearch.search.aggregations.support.ValuesSourceAggregatorFactory;
import org.elasticsearch.search.aggregations.support.ValuesSourceConfig;
import org.elasticsearch.search.aggregations.support.ValuesSourceRegistry;
import org.elasticsearch.search.internal.SearchContext;
import java.io.IOException;
import java.util.Map;
class GeoCentroidAggregatorFactory extends ValuesSourceAggregatorFactory<ValuesSource.GeoPoint> {
class GeoCentroidAggregatorFactory extends ValuesSourceAggregatorFactory {
GeoCentroidAggregatorFactory(String name,
ValuesSourceConfig<ValuesSource.GeoPoint> config,
ValuesSourceConfig config,
QueryShardContext queryShardContext,
AggregatorFactory parent,
AggregatorFactories.Builder subFactoriesBuilder,
@ -50,11 +54,24 @@ class GeoCentroidAggregatorFactory extends ValuesSourceAggregatorFactory<ValuesS
}
@Override
protected Aggregator doCreateInternal(ValuesSource.GeoPoint valuesSource,
protected Aggregator doCreateInternal(ValuesSource valuesSource,
SearchContext searchContext,
Aggregator parent,
boolean collectsFromSingleBucket,
Map<String, Object> metadata) throws IOException {
return new GeoCentroidAggregator(name, searchContext, parent, valuesSource, metadata);
AggregatorSupplier aggregatorSupplier = queryShardContext.getValuesSourceRegistry().getAggregator(config.valueSourceType(),
GeoCentroidAggregationBuilder.NAME);
if (aggregatorSupplier instanceof GeoCentroidAggregatorSupplier == false) {
throw new AggregationExecutionException("Registry miss-match - expected "
+ GeoCentroidAggregatorSupplier.class.getName() + ", found [" + aggregatorSupplier.getClass().toString() + "]");
}
return ((GeoCentroidAggregatorSupplier) aggregatorSupplier).build(name, searchContext, parent, valuesSource, metadata);
}
static void registerAggregators(ValuesSourceRegistry valuesSourceRegistry) {
valuesSourceRegistry.register(GeoCentroidAggregationBuilder.NAME, CoreValuesSourceType.GEOPOINT,
(GeoCentroidAggregatorSupplier) (name, context, parent, valuesSource, metadata) ->
new GeoCentroidAggregator(name, context, parent, (ValuesSource.GeoPoint) valuesSource, metadata));
}
}

36
server/src/main/java/org/elasticsearch/search/aggregations/metrics/GeoCentroidAggregatorSupplier.java Normal file
View File

@ -0,0 +1,36 @@
/*
* Licensed to Elasticsearch under one or more contributor
* license agreements. See the NOTICE file distributed with
* this work for additional information regarding copyright
* ownership. Elasticsearch licenses this file to you under
* the Apache License, Version 2.0 (the "License"); you may
* not use this file except in compliance with the License.
* You may obtain a copy of the License at
*
* http://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing,
* software distributed under the License is distributed on an
* "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
* KIND, either express or implied. See the License for the
* specific language governing permissions and limitations
* under the License.
*/
package org.elasticsearch.search.aggregations.metrics;
import org.elasticsearch.search.aggregations.Aggregator;
import org.elasticsearch.search.aggregations.support.AggregatorSupplier;
import org.elasticsearch.search.aggregations.support.ValuesSource;
import org.elasticsearch.search.internal.SearchContext;
import java.io.IOException;
import java.util.Map;
@FunctionalInterface
public interface GeoCentroidAggregatorSupplier extends AggregatorSupplier {
GeoCentroidAggregator build(String name, SearchContext context, Aggregator parent,
ValuesSource valuesSource,
Map<String, Object> metadata) throws IOException;
}

40
server/src/main/java/org/elasticsearch/search/aggregations/metrics/GeoGridAggregatorSupplier.java Normal file
View File

@ -0,0 +1,40 @@
/*
* Licensed to Elasticsearch under one or more contributor
* license agreements. See the NOTICE file distributed with
* this work for additional information regarding copyright
* ownership. Elasticsearch licenses this file to you under
* the Apache License, Version 2.0 (the "License"); you may
* not use this file except in compliance with the License.
* You may obtain a copy of the License at
*
* http://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing,
* software distributed under the License is distributed on an
* "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
* KIND, either express or implied. See the License for the
* specific language governing permissions and limitations
* under the License.
*/
package org.elasticsearch.search.aggregations.metrics;
import org.elasticsearch.common.geo.GeoBoundingBox;
import org.elasticsearch.search.aggregations.Aggregator;
import org.elasticsearch.search.aggregations.AggregatorFactories;
import org.elasticsearch.search.aggregations.bucket.geogrid.GeoGridAggregator;
import org.elasticsearch.search.aggregations.support.AggregatorSupplier;
import org.elasticsearch.search.aggregations.support.ValuesSource;
import org.elasticsearch.search.internal.SearchContext;
import java.io.IOException;
import java.util.Map;
@FunctionalInterface
public interface GeoGridAggregatorSupplier extends AggregatorSupplier {
GeoGridAggregator build(String name, AggregatorFactories factories, ValuesSource valuesSource,
int precision, GeoBoundingBox geoBoundingBox, int requiredSize, int shardSize,
SearchContext aggregationContext, Aggregator parent,
Map<String, Object> metadata) throws IOException;
}

8
server/src/main/java/org/elasticsearch/search/aggregations/metrics/InternalHDRPercentileRanks.java
View File

@ -29,9 +29,9 @@ import java.util.Map;
public class InternalHDRPercentileRanks extends AbstractInternalHDRPercentiles implements PercentileRanks {
public static final String NAME = "hdr_percentile_ranks";
InternalHDRPercentileRanks(String name, double[] cdfValues, DoubleHistogram state, boolean keyed, DocValueFormat formatter,
Map<String, Object> metadata) {
super(name, cdfValues, state, keyed, formatter, metadata);
public InternalHDRPercentileRanks(String name, double[] cdfValues, DoubleHistogram state, boolean keyed, DocValueFormat formatter,
Map<String, Object> metadata) {
super(name, cdfValues, state, keyed, formatter, metadata);
}
/**
@ -72,7 +72,7 @@ public class InternalHDRPercentileRanks extends AbstractInternalHDRPercentiles i
return new InternalHDRPercentileRanks(name, keys, merged, keyed, format, metadata);
}
static double percentileRank(DoubleHistogram state, double value) {
public static double percentileRank(DoubleHistogram state, double value) {
if (state.getTotalCount() == 0) {
return Double.NaN;
}

6
server/src/main/java/org/elasticsearch/search/aggregations/metrics/InternalHDRPercentiles.java
View File

@ -29,9 +29,9 @@ import java.util.Map;
public class InternalHDRPercentiles extends AbstractInternalHDRPercentiles implements Percentiles {
public static final String NAME = "hdr_percentiles";
InternalHDRPercentiles(String name, double[] percents, DoubleHistogram state, boolean keyed, DocValueFormat formatter,
Map<String, Object> metadata) {
super(name, percents, state, keyed, formatter, metadata);
public InternalHDRPercentiles(String name, double[] percents, DoubleHistogram state, boolean keyed, DocValueFormat formatter,
Map<String, Object> metadata) {
super(name, percents, state, keyed, formatter, metadata);
}
/**

6
server/src/main/java/org/elasticsearch/search/aggregations/metrics/InternalTDigestPercentileRanks.java
View File

@ -28,8 +28,8 @@ import java.util.Map;
public class InternalTDigestPercentileRanks extends AbstractInternalTDigestPercentiles implements PercentileRanks {
public static final String NAME = "tdigest_percentile_ranks";
InternalTDigestPercentileRanks(String name, double[] cdfValues, TDigestState state, boolean keyed, DocValueFormat formatter,
Map<String, Object> metadata) {
public InternalTDigestPercentileRanks(String name, double[] cdfValues, TDigestState state, boolean keyed, DocValueFormat formatter,
Map<String, Object> metadata) {
super(name, cdfValues, state, keyed, formatter, metadata);
}
@ -71,7 +71,7 @@ public class InternalTDigestPercentileRanks extends AbstractInternalTDigestPerce
return new InternalTDigestPercentileRanks(name, keys, merged, keyed, format, metadata);
}
static double percentileRank(TDigestState state, double value) {
public static double percentileRank(TDigestState state, double value) {
double percentileRank = state.cdf(value);
if (percentileRank < 0) {
percentileRank = 0;

4
server/src/main/java/org/elasticsearch/search/aggregations/metrics/InternalTDigestPercentiles.java
View File

@ -28,8 +28,8 @@ import java.util.Map;
public class InternalTDigestPercentiles extends AbstractInternalTDigestPercentiles implements Percentiles {
public static final String NAME = "tdigest_percentiles";
InternalTDigestPercentiles(String name, double[] percents, TDigestState state, boolean keyed, DocValueFormat formatter,
Map<String, Object> metadata) {
public InternalTDigestPercentiles(String name, double[] percents, TDigestState state, boolean keyed, DocValueFormat formatter,
Map<String, Object> metadata) {
super(name, percents, state, keyed, formatter, metadata);
}

22
server/src/main/java/org/elasticsearch/search/aggregations/metrics/MaxAggregationBuilder.java
View File

@ -28,12 +28,11 @@ import org.elasticsearch.search.aggregations.AggregationBuilder;
import org.elasticsearch.search.aggregations.AggregatorFactories.Builder;
import org.elasticsearch.search.aggregations.AggregatorFactory;
import org.elasticsearch.search.aggregations.support.CoreValuesSourceType;
import org.elasticsearch.search.aggregations.support.ValueType;
import org.elasticsearch.search.aggregations.support.ValuesSource;
import org.elasticsearch.search.aggregations.support.ValuesSource.Numeric;
import org.elasticsearch.search.aggregations.support.ValuesSourceAggregationBuilder;
import org.elasticsearch.search.aggregations.support.ValuesSourceConfig;
import org.elasticsearch.search.aggregations.support.ValuesSourceParserHelper;
import org.elasticsearch.search.aggregations.support.ValuesSourceRegistry;
import org.elasticsearch.search.aggregations.support.ValuesSourceType;
import java.io.IOException;
import java.util.Map;
@ -43,17 +42,26 @@ public class MaxAggregationBuilder extends ValuesSourceAggregationBuilder.LeafOn
public static final ObjectParser<MaxAggregationBuilder, String> PARSER = ObjectParser.fromBuilder(NAME, MaxAggregationBuilder::new);
static {
ValuesSourceParserHelper.declareNumericFields(PARSER, true, true, false);
ValuesSourceAggregationBuilder.declareFields(PARSER, true, true, false);
}
public static void registerAggregators(ValuesSourceRegistry valuesSourceRegistry) {
MaxAggregatorFactory.registerAggregators(valuesSourceRegistry);
}
public MaxAggregationBuilder(String name) {
super(name, CoreValuesSourceType.NUMERIC, ValueType.NUMERIC);
super(name);
}
protected MaxAggregationBuilder(MaxAggregationBuilder clone, Builder factoriesBuilder, Map<String, Object> metadata) {
super(clone, factoriesBuilder, metadata);
}
@Override
protected ValuesSourceType defaultValueSourceType() {
return CoreValuesSourceType.NUMERIC;
}
@Override
protected AggregationBuilder shallowCopy(Builder factoriesBuilder, Map<String, Object> metadata) {
return new MaxAggregationBuilder(this, factoriesBuilder, metadata);
@ -63,7 +71,7 @@ public class MaxAggregationBuilder extends ValuesSourceAggregationBuilder.LeafOn
* Read from a stream.
*/
public MaxAggregationBuilder(StreamInput in) throws IOException {
super(in, CoreValuesSourceType.NUMERIC, ValueType.NUMERIC);
super(in);
}
@Override
@ -72,7 +80,7 @@ public class MaxAggregationBuilder extends ValuesSourceAggregationBuilder.LeafOn
}
@Override
protected MaxAggregatorFactory innerBuild(QueryShardContext queryShardContext, ValuesSourceConfig<Numeric> config,
protected MaxAggregatorFactory innerBuild(QueryShardContext queryShardContext, ValuesSourceConfig config,
AggregatorFactory parent, Builder subFactoriesBuilder) throws IOException {
return new MaxAggregatorFactory(name, config, queryShardContext, parent, subFactoriesBuilder, metadata);
}

2
server/src/main/java/org/elasticsearch/search/aggregations/metrics/MaxAggregator.java
View File

@ -57,7 +57,7 @@ class MaxAggregator extends NumericMetricsAggregator.SingleValue {
DoubleArray maxes;
MaxAggregator(String name,
ValuesSourceConfig<ValuesSource.Numeric> config,
ValuesSourceConfig config,
ValuesSource.Numeric valuesSource,
SearchContext context,
Aggregator parent, Map<String, Object> metadata) throws IOException {

40
server/src/main/java/org/elasticsearch/search/aggregations/metrics/MaxAggregatorFactory.java
View File

@ -20,22 +20,44 @@
package org.elasticsearch.search.aggregations.metrics;
import org.elasticsearch.index.query.QueryShardContext;
import org.elasticsearch.search.aggregations.AggregationExecutionException;
import org.elasticsearch.search.aggregations.Aggregator;
import org.elasticsearch.search.aggregations.AggregatorFactories;
import org.elasticsearch.search.aggregations.AggregatorFactory;
import org.elasticsearch.search.aggregations.support.AggregatorSupplier;
import org.elasticsearch.search.aggregations.support.CoreValuesSourceType;
import org.elasticsearch.search.aggregations.support.ValuesSource;
import org.elasticsearch.search.aggregations.support.ValuesSource.Numeric;
import org.elasticsearch.search.aggregations.support.ValuesSourceAggregatorFactory;
import org.elasticsearch.search.aggregations.support.ValuesSourceConfig;
import org.elasticsearch.search.aggregations.support.ValuesSourceRegistry;
import org.elasticsearch.search.internal.SearchContext;
import java.io.IOException;
import java.util.Arrays;
import java.util.Map;
class MaxAggregatorFactory extends ValuesSourceAggregatorFactory<ValuesSource.Numeric> {
class MaxAggregatorFactory extends ValuesSourceAggregatorFactory {
MaxAggregatorFactory(String name, ValuesSourceConfig<Numeric> config, QueryShardContext queryShardContext,
AggregatorFactory parent, AggregatorFactories.Builder subFactoriesBuilder, Map<String, Object> metadata) throws IOException {
static void registerAggregators(ValuesSourceRegistry valuesSourceRegistry) {
valuesSourceRegistry.register(MaxAggregationBuilder.NAME,
Arrays.asList(CoreValuesSourceType.NUMERIC, CoreValuesSourceType.DATE, CoreValuesSourceType.BOOLEAN),
new MinMaxAggregatorSupplier() {
@Override
public Aggregator build(String name,
ValuesSourceConfig config,
ValuesSource valuesSource,
SearchContext context,
Aggregator parent,
Map<String, Object> metadata) throws IOException {
return new MaxAggregator(name, config, (Numeric) valuesSource, context, parent, metadata);
}
});
}
MaxAggregatorFactory(String name, ValuesSourceConfig config, QueryShardContext queryShardContext,
AggregatorFactory parent, AggregatorFactories.Builder subFactoriesBuilder,
Map<String, Object> metadata) throws IOException {
super(name, config, queryShardContext, parent, subFactoriesBuilder, metadata);
}
@ -47,11 +69,19 @@ class MaxAggregatorFactory extends ValuesSourceAggregatorFactory<ValuesSource.Nu
}
@Override
protected Aggregator doCreateInternal(Numeric valuesSource,
protected Aggregator doCreateInternal(ValuesSource valuesSource,
SearchContext searchContext,
Aggregator parent,
boolean collectsFromSingleBucket,
Map<String, Object> metadata) throws IOException {
return new MaxAggregator(name, config, valuesSource, searchContext, parent, metadata);
AggregatorSupplier aggregatorSupplier = queryShardContext.getValuesSourceRegistry().getAggregator(config.valueSourceType(),
MaxAggregationBuilder.NAME);
if (aggregatorSupplier instanceof MinMaxAggregatorSupplier == false) {
throw new AggregationExecutionException("Registry miss-match - expected MinMaxAggregatorSupplier, found [" +
aggregatorSupplier.getClass().toString() + "]");
}
return ((MinMaxAggregatorSupplier) aggregatorSupplier).build(name, config, valuesSource, searchContext, parent,
metadata);
}
}

28
server/src/main/java/org/elasticsearch/search/aggregations/metrics/MedianAbsoluteDeviationAggregationBuilder.java
View File

@ -29,12 +29,13 @@ import org.elasticsearch.search.aggregations.AggregationBuilder;
import org.elasticsearch.search.aggregations.AggregatorFactories;
import org.elasticsearch.search.aggregations.AggregatorFactory;
import org.elasticsearch.search.aggregations.support.CoreValuesSourceType;
import org.elasticsearch.search.aggregations.support.ValueType;
import org.elasticsearch.search.aggregations.support.ValuesSource;
import org.elasticsearch.search.aggregations.support.ValuesSourceAggregationBuilder;
import org.elasticsearch.search.aggregations.support.ValuesSourceAggregationBuilder.LeafOnly;
import org.elasticsearch.search.aggregations.support.ValuesSourceAggregatorFactory;
import org.elasticsearch.search.aggregations.support.ValuesSourceConfig;
import org.elasticsearch.search.aggregations.support.ValuesSourceParserHelper;
import org.elasticsearch.search.aggregations.support.ValuesSourceRegistry;
import org.elasticsearch.search.aggregations.support.ValuesSourceType;
import java.io.IOException;
import java.util.Map;
@ -49,18 +50,22 @@ public class MedianAbsoluteDeviationAggregationBuilder extends LeafOnly<ValuesSo
public static final ObjectParser<MedianAbsoluteDeviationAggregationBuilder, String> PARSER =
ObjectParser.fromBuilder(NAME, MedianAbsoluteDeviationAggregationBuilder::new);
static {
ValuesSourceParserHelper.declareNumericFields(PARSER, true, true, false);
ValuesSourceAggregationBuilder.declareFields(PARSER, true, true, false);
PARSER.declareDouble(MedianAbsoluteDeviationAggregationBuilder::compression, COMPRESSION_FIELD);
}
public static void registerAggregators(ValuesSourceRegistry valuesSourceRegistry) {
MedianAbsoluteDeviationAggregatorFactory.registerAggregators(valuesSourceRegistry);
}
private double compression = 1000d;
public MedianAbsoluteDeviationAggregationBuilder(String name) {
super(name, CoreValuesSourceType.NUMERIC, ValueType.NUMERIC);
super(name);
}
public MedianAbsoluteDeviationAggregationBuilder(StreamInput in) throws IOException {
super(in, CoreValuesSourceType.NUMERIC, ValueType.NUMERIC);
super(in);
compression = in.readDouble();
}
@ -95,16 +100,21 @@ public class MedianAbsoluteDeviationAggregationBuilder extends LeafOnly<ValuesSo
return new MedianAbsoluteDeviationAggregationBuilder(this, factoriesBuilder, metadata);
}
@Override
protected ValuesSourceType defaultValueSourceType() {
return CoreValuesSourceType.NUMERIC;
}
@Override
protected void innerWriteTo(StreamOutput out) throws IOException {
out.writeDouble(compression);
}
@Override
protected ValuesSourceAggregatorFactory<ValuesSource.Numeric> innerBuild(QueryShardContext queryShardContext,
ValuesSourceConfig<ValuesSource.Numeric> config,
AggregatorFactory parent,
AggregatorFactories.Builder subFactoriesBuilder)
protected ValuesSourceAggregatorFactory innerBuild(QueryShardContext queryShardContext,
ValuesSourceConfig config,
AggregatorFactory parent,
AggregatorFactories.Builder subFactoriesBuilder)
throws IOException {
return new MedianAbsoluteDeviationAggregatorFactory(name, config, queryShardContext,
parent, subFactoriesBuilder, metadata, compression);

69
server/src/main/java/org/elasticsearch/search/aggregations/metrics/MedianAbsoluteDeviationAggregatorFactory.java
View File

@ -20,37 +20,68 @@
package org.elasticsearch.search.aggregations.metrics;
import org.elasticsearch.index.query.QueryShardContext;
import org.elasticsearch.search.DocValueFormat;
import org.elasticsearch.search.aggregations.AggregationExecutionException;
import org.elasticsearch.search.aggregations.Aggregator;
import org.elasticsearch.search.aggregations.AggregatorFactories;
import org.elasticsearch.search.aggregations.AggregatorFactory;
import org.elasticsearch.search.aggregations.support.AggregatorSupplier;
import org.elasticsearch.search.aggregations.support.CoreValuesSourceType;
import org.elasticsearch.search.aggregations.support.ValuesSource;
import org.elasticsearch.search.aggregations.support.ValuesSourceAggregatorFactory;
import org.elasticsearch.search.aggregations.support.ValuesSourceConfig;
import org.elasticsearch.search.aggregations.support.ValuesSourceRegistry;
import org.elasticsearch.search.internal.SearchContext;
import java.io.IOException;
import java.util.Map;
public class MedianAbsoluteDeviationAggregatorFactory extends ValuesSourceAggregatorFactory<ValuesSource.Numeric> {
public class MedianAbsoluteDeviationAggregatorFactory extends ValuesSourceAggregatorFactory {
private final double compression;
MedianAbsoluteDeviationAggregatorFactory(String name,
ValuesSourceConfig<ValuesSource.Numeric> config,
QueryShardContext queryShardContext,
AggregatorFactory parent,
AggregatorFactories.Builder subFactoriesBuilder,
Map<String, Object> metadata,
double compression) throws IOException {
ValuesSourceConfig config,
QueryShardContext queryShardContext,
AggregatorFactory parent,
AggregatorFactories.Builder subFactoriesBuilder,
Map<String, Object> metadata,
double compression) throws IOException {
super(name, config, queryShardContext, parent, subFactoriesBuilder, metadata);
this.compression = compression;
}
static void registerAggregators(ValuesSourceRegistry valuesSourceRegistry) {
valuesSourceRegistry.register(MedianAbsoluteDeviationAggregationBuilder.NAME,
CoreValuesSourceType.NUMERIC,
new MedianAbsoluteDeviationAggregatorSupplier() {
@Override
public Aggregator build(String name,
ValuesSource valuesSource,
DocValueFormat format,
SearchContext context,
Aggregator parent,
Map<String, Object> metadata,
double compression) throws IOException {
return new MedianAbsoluteDeviationAggregator(
name,
context,
parent,
metadata,
(ValuesSource.Numeric) valuesSource,
format,
compression
);
}
});
}
@Override
protected Aggregator createUnmapped(SearchContext searchContext,
Aggregator parent,
Map<String, Object> metadata) throws IOException {
Aggregator parent,
Map<String, Object> metadata) throws IOException {
return new MedianAbsoluteDeviationAggregator(
name,
searchContext,
@ -63,19 +94,19 @@ public class MedianAbsoluteDeviationAggregatorFactory extends ValuesSourceAggreg
}
@Override
protected Aggregator doCreateInternal(ValuesSource.Numeric valuesSource,
protected Aggregator doCreateInternal(ValuesSource valuesSource,
SearchContext searchContext,
Aggregator parent,
boolean collectsFromSingleBucket,
Map<String, Object> metadata) throws IOException {
return new MedianAbsoluteDeviationAggregator(
name,
searchContext,
parent,
metadata,
valuesSource,
config.format(),
compression
);
AggregatorSupplier aggregatorSupplier = queryShardContext.getValuesSourceRegistry().getAggregator(config.valueSourceType(),
MedianAbsoluteDeviationAggregationBuilder.NAME);
if (aggregatorSupplier instanceof MedianAbsoluteDeviationAggregatorSupplier == false) {
throw new AggregationExecutionException("Registry miss-match - expected MedianAbsoluteDeviationAggregatorSupplier, found [" +
aggregatorSupplier.getClass().toString() + "]");
}
return ((MedianAbsoluteDeviationAggregatorSupplier) aggregatorSupplier).build(name, valuesSource, config.format(),
searchContext, parent, metadata, compression);
}
}

38
server/src/main/java/org/elasticsearch/search/aggregations/metrics/MedianAbsoluteDeviationAggregatorSupplier.java Normal file
View File

@ -0,0 +1,38 @@
/*
* Licensed to Elasticsearch under one or more contributor
* license agreements. See the NOTICE file distributed with
* this work for additional information regarding copyright
* ownership. Elasticsearch licenses this file to you under
* the Apache License, Version 2.0 (the "License"); you may
* not use this file except in compliance with the License.
* You may obtain a copy of the License at
*
* http://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing,
* software distributed under the License is distributed on an
* "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
* KIND, either express or implied. See the License for the
* specific language governing permissions and limitations
* under the License.
*/
package org.elasticsearch.search.aggregations.metrics;
import org.elasticsearch.search.DocValueFormat;
import org.elasticsearch.search.aggregations.Aggregator;
import org.elasticsearch.search.aggregations.support.AggregatorSupplier;
import org.elasticsearch.search.aggregations.support.ValuesSource;
import org.elasticsearch.search.internal.SearchContext;
import java.io.IOException;
import java.util.Map;
public interface MedianAbsoluteDeviationAggregatorSupplier extends AggregatorSupplier {
Aggregator build(String name,
ValuesSource valuesSource,
DocValueFormat format,
SearchContext context,
Aggregator parent,
Map<String, Object> metadata,
double compression) throws IOException;
}

Some files were not shown because too many files have changed in this diff Show More