honeymoose
/
OpenSearch
mirror of https://github.com/honeymoose/OpenSearch.git
1
0
Fork 0
Code Issues Packages Projects Releases Wiki Activity

Support geo_point fields in lucene expressions. 

Closes #18096
This commit is contained in:
Robert Muir 2016-05-02 17:49:21 -04:00
parent 0c6d6a5495
commit 693c1f6671
18 changed files with 666 additions and 269 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

91
docs/reference/modules/scripting/scripting.asciidoc
View File

@ -23,7 +23,7 @@ to specify the language of the script. Plugins are available for following langu
|groovy |no |built-in
|expression |yes |built-in
|mustache |yes |built-in
/painless /yes /built-in (module)
|painless |yes |built-in (module)
|javascript |no |{plugins}/lang-javascript.html[elasticsearch-lang-javascript]
|python |no |{plugins}/lang-python.html[elasticsearch-lang-python]
|=======================================================================
@ -455,41 +455,94 @@ for details on what operators and functions are available.
Variables in `expression` scripts are available to access:
* document fields, e.g. `doc['myfield'].value` or just `doc['myfield']`.
* whether the field is empty, e.g. `doc['myfield'].empty`
* document fields, e.g. `doc['myfield'].value`
* variables and methods that the field supports, e.g. `doc['myfield'].empty`
* Parameters passed into the script, e.g. `mymodifier`
* The current document's score, `_score` (only available when used in a `script_score`)
[float]
=== Expressions API for numeric fields
[cols="<,<",options="header",]
|=======================================================================
|Expression |Description
|`doc['field_name'].value` |The native value of the field. For example,
if its a short type, it will be short.
|`doc['field_name'].empty` |A boolean indicating if the field has no
values within the doc.
|`doc['field_name'].min()` |The minimum value of the field in this document.
|`doc['field_name'].max()` |The maximum value of the field in this document.
|`doc['field_name'].median()` |The median value of the field in this document.
|`doc['field_name'].avg()` |The average of the values in this document.
|`doc['field_name'].sum()` |The sum of the values in this document.
|`doc['field_name'].count()` |The number of values in this document.
|=======================================================================
When a document is missing the field completely, by default the value will be treated as `0`.
You can treat it as another value instead, e.g. `doc['myfield'].empty ? 100 : doc['myfield'].value`
When a document has multiple values for the field, by default the minimum value is returned.
You can choose a different value instead, e.g. `doc['myfield'].sum()`. The following methods are available
for any field:
You can choose a different value instead, e.g. `doc['myfield'].sum()`.
* min()
* max()
* avg()
* median()
* sum()
* count()
When a document is missing the field completely, by default the value will be treated as `0`.
Variables in `expression` scripts that are of type `date` may use the following member methods:
[float]
=== Additional methods for date fields
Date fields are treated as the number of milliseconds since January 1, 1970 and
support the numeric API above, with these additional methods:
* getYear()
* getMonth()
* getDayOfMonth()
* getHourOfDay()
* getMinutes()
* getSeconds()
[cols="<,<",options="header",]
|=======================================================================
|Expression |Description
|`doc['field_name'].getYear()` |Year component, e.g. `1970`.
|`doc['field_name'].getMonth()` |Month component (0-11), e.g. `0` for January.
|`doc['field_name'].getDayOfMonth()` |Day component, e.g. `1` for the first of the month.
|`doc['field_name'].getHourOfDay()` |Hour component (0-23)
|`doc['field_name'].getMinutes()` |Minutes component (0-59)
|`doc['field_name'].getSeconds()` |Seconds component (0-59)
|=======================================================================
The following example shows the difference in years between the `date` fields date0 and date1:
`doc['date1'].getYear() - doc['date0'].getYear()`
[float]
=== Expressions API for `geo_point` fields
[cols="<,<",options="header",]
|=======================================================================
|Expression |Description
|`doc['field_name'].empty` |A boolean indicating if the field has no
values within the doc.
|`doc['field_name'].lat` |The latitude of the geo point.
|`doc['field_name'].lon` |The longitude of the geo point.
|=======================================================================
The following example computes distance in kilometers from Washington, DC:
`haversin(38.9072, 77.0369, doc['field_name'].lat, doc['field_name'].lon)`
In this example the coordinates could have been passed as parameters to the script,
e.g. based on geolocation of the user.
[float]
=== Expressions limitations
There are a few limitations relative to other script languages:
* Only numeric fields may be accessed
* Only numeric, date, and geo_point fields may be accessed
* Stored fields are not available
[float]

44
modules/lang-expression/src/main/java/org/elasticsearch/script/expression/CountMethodFunctionValues.java
View File

@ -1,44 +0,0 @@
/*
* Licensed to Elasticsearch under one or more contributor
* license agreements. See the NOTICE file distributed with
* this work for additional information regarding copyright
* ownership. Elasticsearch licenses this file to you under
* the Apache License, Version 2.0 (the "License"); you may
* not use this file except in compliance with the License.
* You may obtain a copy of the License at
*
* http://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing,
* software distributed under the License is distributed on an
* "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
* KIND, either express or implied. See the License for the
* specific language governing permissions and limitations
* under the License.
*/
package org.elasticsearch.script.expression;
import org.apache.lucene.queries.function.ValueSource;
import org.apache.lucene.queries.function.docvalues.DoubleDocValues;
import org.elasticsearch.index.fielddata.AtomicNumericFieldData;
import org.elasticsearch.index.fielddata.SortedNumericDoubleValues;
/**
* FunctionValues to get the count of the number of values in a field for a document.
*/
public class CountMethodFunctionValues extends DoubleDocValues {
SortedNumericDoubleValues values;
CountMethodFunctionValues(ValueSource parent, AtomicNumericFieldData fieldData) {
super(parent);
values = fieldData.getDoubleValues();
}
@Override
public double doubleVal(int doc) {
values.setDocument(doc);
return values.count();
}
}

21
modules/lang-expression/src/main/java/org/elasticsearch/script/expression/CountMethodValueSource.java
View File

@ -26,17 +26,18 @@ import java.util.Objects;
import org.apache.lucene.index.LeafReaderContext;
import org.apache.lucene.queries.function.FunctionValues;
import org.apache.lucene.queries.function.ValueSource;
import org.elasticsearch.index.fielddata.AtomicFieldData;
import org.apache.lucene.queries.function.docvalues.DoubleDocValues;
import org.elasticsearch.index.fielddata.AtomicNumericFieldData;
import org.elasticsearch.index.fielddata.IndexFieldData;
import org.elasticsearch.index.fielddata.SortedNumericDoubleValues;
/**
* A ValueSource to create FunctionValues to get the count of the number of values in a field for a document.
*/
public class CountMethodValueSource extends ValueSource {
protected IndexFieldData<?> fieldData;
final class CountMethodValueSource extends ValueSource {
IndexFieldData<?> fieldData;
protected CountMethodValueSource(IndexFieldData<?> fieldData) {
CountMethodValueSource(IndexFieldData<?> fieldData) {
Objects.requireNonNull(fieldData);
this.fieldData = fieldData;
@ -45,10 +46,16 @@ public class CountMethodValueSource extends ValueSource {
@Override
@SuppressWarnings("rawtypes") // ValueSource uses a rawtype
public FunctionValues getValues(Map context, LeafReaderContext leaf) throws IOException {
AtomicFieldData leafData = fieldData.load(leaf);
assert(leafData instanceof AtomicNumericFieldData);
AtomicNumericFieldData leafData = (AtomicNumericFieldData) fieldData.load(leaf);
final SortedNumericDoubleValues values = leafData.getDoubleValues();
return new CountMethodFunctionValues(this, (AtomicNumericFieldData)leafData);
return new DoubleDocValues(this) {
@Override
public double doubleVal(int doc) {
values.setDocument(doc);
return values.count();
}
};
}
@Override

94
modules/lang-expression/src/main/java/org/elasticsearch/script/expression/DateField.java Normal file
View File

@ -0,0 +1,94 @@
package org.elasticsearch.script.expression;
/*
* Licensed to Elasticsearch under one or more contributor
* license agreements. See the NOTICE file distributed with
* this work for additional information regarding copyright
* ownership. Elasticsearch licenses this file to you under
* the Apache License, Version 2.0 (the "License"); you may
* not use this file except in compliance with the License.
* You may obtain a copy of the License at
*
* http://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing,
* software distributed under the License is distributed on an
* "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
* KIND, either express or implied. See the License for the
* specific language governing permissions and limitations
* under the License.
*/
import java.util.Calendar;
import org.apache.lucene.queries.function.ValueSource;
import org.elasticsearch.index.fielddata.IndexFieldData;
import org.elasticsearch.search.MultiValueMode;
/**
* Expressions API for date fields.
*/
final class DateField {
// no instance
private DateField() {}
// supported variables
static final String VALUE_VARIABLE = "value";
static final String EMPTY_VARIABLE = "empty";
// supported methods
static final String MINIMUM_METHOD = "min";
static final String MAXIMUM_METHOD = "max";
static final String AVERAGE_METHOD = "avg";
static final String MEDIAN_METHOD = "median";
static final String SUM_METHOD = "sum";
static final String COUNT_METHOD = "count";
static final String GET_YEAR_METHOD = "getYear";
static final String GET_MONTH_METHOD = "getMonth";
static final String GET_DAY_OF_MONTH_METHOD = "getDayOfMonth";
static final String GET_HOUR_OF_DAY_METHOD = "getHourOfDay";
static final String GET_MINUTES_METHOD = "getMinutes";
static final String GET_SECONDS_METHOD = "getSeconds";
static ValueSource getVariable(IndexFieldData<?> fieldData, String fieldName, String variable) {
switch (variable) {
case VALUE_VARIABLE:
return new FieldDataValueSource(fieldData, MultiValueMode.MIN);
case EMPTY_VARIABLE:
return new EmptyMemberValueSource(fieldData);
default:
throw new IllegalArgumentException("Member variable [" + variable + "] does not exist for date field [" + fieldName + "].");
}
}
static ValueSource getMethod(IndexFieldData<?> fieldData, String fieldName, String method) {
switch (method) {
case MINIMUM_METHOD:
return new FieldDataValueSource(fieldData, MultiValueMode.MIN);
case MAXIMUM_METHOD:
return new FieldDataValueSource(fieldData, MultiValueMode.MAX);
case AVERAGE_METHOD:
return new FieldDataValueSource(fieldData, MultiValueMode.AVG);
case MEDIAN_METHOD:
return new FieldDataValueSource(fieldData, MultiValueMode.MEDIAN);
case SUM_METHOD:
return new FieldDataValueSource(fieldData, MultiValueMode.SUM);
case COUNT_METHOD:
return new CountMethodValueSource(fieldData);
case GET_YEAR_METHOD:
return new DateMethodValueSource(fieldData, MultiValueMode.MIN, method, Calendar.YEAR);
case GET_MONTH_METHOD:
return new DateMethodValueSource(fieldData, MultiValueMode.MIN, method, Calendar.MONTH);
case GET_DAY_OF_MONTH_METHOD:
return new DateMethodValueSource(fieldData, MultiValueMode.MIN, method, Calendar.DAY_OF_MONTH);
case GET_HOUR_OF_DAY_METHOD:
return new DateMethodValueSource(fieldData, MultiValueMode.MIN, method, Calendar.HOUR_OF_DAY);
case GET_MINUTES_METHOD:
return new DateMethodValueSource(fieldData, MultiValueMode.MIN, method, Calendar.MINUTE);
case GET_SECONDS_METHOD:
return new DateMethodValueSource(fieldData, MultiValueMode.MIN, method, Calendar.SECOND);
default:
throw new IllegalArgumentException("Member method [" + method + "] does not exist for date field [" + fieldName + "].");
}
}
}

47
modules/lang-expression/src/main/java/org/elasticsearch/script/expression/DateMethodFunctionValues.java
View File

@ -1,47 +0,0 @@
/*
* Licensed to Elasticsearch under one or more contributor
* license agreements. See the NOTICE file distributed with
* this work for additional information regarding copyright
* ownership. Elasticsearch licenses this file to you under
* the Apache License, Version 2.0 (the "License"); you may
* not use this file except in compliance with the License.
* You may obtain a copy of the License at
*
* http://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing,
* software distributed under the License is distributed on an
* "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
* KIND, either express or implied. See the License for the
* specific language governing permissions and limitations
* under the License.
*/
package org.elasticsearch.script.expression;
import org.apache.lucene.queries.function.ValueSource;
import org.elasticsearch.index.fielddata.AtomicNumericFieldData;
import org.elasticsearch.search.MultiValueMode;
import java.util.Calendar;
import java.util.Locale;
import java.util.TimeZone;
class DateMethodFunctionValues extends FieldDataFunctionValues {
private final int calendarType;
private final Calendar calendar;
DateMethodFunctionValues(ValueSource parent, MultiValueMode multiValueMode, AtomicNumericFieldData data, int calendarType) {
super(parent, multiValueMode, data);
this.calendarType = calendarType;
calendar = Calendar.getInstance(TimeZone.getTimeZone("UTC"), Locale.ROOT);
}
@Override
public double doubleVal(int docId) {
long millis = (long)dataAccessor.get(docId);
calendar.setTimeInMillis(millis);
return calendar.get(calendarType);
}
}

26
modules/lang-expression/src/main/java/org/elasticsearch/script/expression/DateMethodValueSource.java
View File

@ -20,20 +20,25 @@
package org.elasticsearch.script.expression;
import java.io.IOException;
import java.util.Calendar;
import java.util.Locale;
import java.util.Map;
import java.util.Objects;
import java.util.TimeZone;
import org.apache.lucene.index.LeafReaderContext;
import org.apache.lucene.queries.function.FunctionValues;
import org.elasticsearch.index.fielddata.AtomicFieldData;
import org.apache.lucene.queries.function.docvalues.DoubleDocValues;
import org.elasticsearch.index.fielddata.AtomicNumericFieldData;
import org.elasticsearch.index.fielddata.IndexFieldData;
import org.elasticsearch.index.fielddata.NumericDoubleValues;
import org.elasticsearch.search.MultiValueMode;
/** Extracts a portion of a date field with {@code Calendar.get()} */
class DateMethodValueSource extends FieldDataValueSource {
protected final String methodName;
protected final int calendarType;
final String methodName;
final int calendarType;
DateMethodValueSource(IndexFieldData<?> indexFieldData, MultiValueMode multiValueMode, String methodName, int calendarType) {
super(indexFieldData, multiValueMode);
@ -47,10 +52,17 @@ class DateMethodValueSource extends FieldDataValueSource {
@Override
@SuppressWarnings("rawtypes") // ValueSource uses a rawtype
public FunctionValues getValues(Map context, LeafReaderContext leaf) throws IOException {
AtomicFieldData leafData = fieldData.load(leaf);
assert(leafData instanceof AtomicNumericFieldData);
return new DateMethodFunctionValues(this, multiValueMode, (AtomicNumericFieldData)leafData, calendarType);
AtomicNumericFieldData leafData = (AtomicNumericFieldData) fieldData.load(leaf);
final Calendar calendar = Calendar.getInstance(TimeZone.getTimeZone("UTC"), Locale.ROOT);
NumericDoubleValues docValues = multiValueMode.select(leafData.getDoubleValues(), 0d);
return new DoubleDocValues(this) {
@Override
public double doubleVal(int docId) {
long millis = (long)docValues.get(docId);
calendar.setTimeInMillis(millis);
return calendar.get(calendarType);
}
};
}
@Override

6
modules/lang-expression/src/main/java/org/elasticsearch/script/expression/EmptyMemberValueSource.java
View File

@ -36,10 +36,10 @@ import org.elasticsearch.index.fielddata.SortedNumericDoubleValues;
* <p>
* This is essentially sugar over !count()
*/
public class EmptyMemberValueSource extends ValueSource {
protected IndexFieldData<?> fieldData;
final class EmptyMemberValueSource extends ValueSource {
final IndexFieldData<?> fieldData;
protected EmptyMemberValueSource(IndexFieldData<?> fieldData) {
EmptyMemberValueSource(IndexFieldData<?> fieldData) {
this.fieldData = Objects.requireNonNull(fieldData);
}

113
modules/lang-expression/src/main/java/org/elasticsearch/script/expression/ExpressionScriptEngineService.java
View File

@ -37,20 +37,19 @@ import org.elasticsearch.index.mapper.MappedFieldType;
import org.elasticsearch.index.mapper.MapperService;
import org.elasticsearch.index.mapper.core.DateFieldMapper;
import org.elasticsearch.index.mapper.core.LegacyDateFieldMapper;
import org.elasticsearch.index.mapper.geo.BaseGeoPointFieldMapper;
import org.elasticsearch.script.ClassPermission;
import org.elasticsearch.script.CompiledScript;
import org.elasticsearch.script.ExecutableScript;
import org.elasticsearch.script.ScriptEngineService;
import org.elasticsearch.script.ScriptException;
import org.elasticsearch.script.SearchScript;
import org.elasticsearch.search.MultiValueMode;
import org.elasticsearch.search.lookup.SearchLookup;
import java.security.AccessControlContext;
import java.security.AccessController;
import java.security.PrivilegedAction;
import java.text.ParseException;
import java.util.Calendar;
import java.util.Collections;
import java.util.List;
import java.util.Map;
@ -65,26 +64,6 @@ public class ExpressionScriptEngineService extends AbstractComponent implements
public static final List<String> TYPES = Collections.singletonList(NAME);
// these methods only work on dates, e.g. doc['datefield'].getYear()
protected static final String GET_YEAR_METHOD = "getYear";
protected static final String GET_MONTH_METHOD = "getMonth";
protected static final String GET_DAY_OF_MONTH_METHOD = "getDayOfMonth";
protected static final String GET_HOUR_OF_DAY_METHOD = "getHourOfDay";
protected static final String GET_MINUTES_METHOD = "getMinutes";
protected static final String GET_SECONDS_METHOD = "getSeconds";
// these methods work on any field, e.g. doc['field'].sum()
protected static final String MINIMUM_METHOD = "min";
protected static final String MAXIMUM_METHOD = "max";
protected static final String AVERAGE_METHOD = "avg";
protected static final String MEDIAN_METHOD = "median";
protected static final String SUM_METHOD = "sum";
protected static final String COUNT_METHOD = "count";
// these variables work on any field, e.g. doc['field'].value
protected static final String VALUE_VARIABLE = "value";
protected static final String EMPTY_VARIABLE = "empty";
@Inject
public ExpressionScriptEngineService(Settings settings) {
super(settings);
@ -175,7 +154,7 @@ public class ExpressionScriptEngineService extends AbstractComponent implements
} else {
String fieldname = null;
String methodname = null;
String variablename = VALUE_VARIABLE; // .value is the default for doc['field'], its optional.
String variablename = "value"; // .value is the default for doc['field'], its optional.
VariableContext[] parts = VariableContext.parse(variable);
if (parts[0].text.equals("doc") == false) {
throw new ScriptException("Unknown variable [" + parts[0].text + "] in expression");
@ -205,15 +184,38 @@ public class ExpressionScriptEngineService extends AbstractComponent implements
}
IndexFieldData<?> fieldData = lookup.doc().fieldDataService().getForField(fieldType);
if (fieldData instanceof IndexNumericFieldData == false) {
// TODO: more context (which expression?)
throw new ScriptException("Field [" + fieldname + "] used in expression must be numeric");
}
if (methodname == null) {
bindings.add(variable, getVariableValueSource(fieldType, fieldData, fieldname, variablename));
// delegate valuesource creation based on field's type
// there are three types of "fields" to expressions, and each one has a different "api" of variables and methods.
final ValueSource valueSource;
if (fieldType instanceof BaseGeoPointFieldMapper.GeoPointFieldType) {
// geo
if (methodname == null) {
valueSource = GeoField.getVariable(fieldData, fieldname, variablename);
} else {
valueSource = GeoField.getMethod(fieldData, fieldname, methodname);
}
} else if (fieldType instanceof LegacyDateFieldMapper.DateFieldType ||
fieldType instanceof DateFieldMapper.DateFieldType) {
// date
if (methodname == null) {
valueSource = DateField.getVariable(fieldData, fieldname, variablename);
} else {
valueSource = DateField.getMethod(fieldData, fieldname, methodname);
}
} else if (fieldData instanceof IndexNumericFieldData) {
// number
if (methodname == null) {
valueSource = NumericField.getVariable(fieldData, fieldname, variablename);
} else {
valueSource = NumericField.getMethod(fieldData, fieldname, methodname);
}
} else {
bindings.add(variable, getMethodValueSource(fieldType, fieldData, fieldname, methodname));
throw new ScriptException("Field [" + fieldname + "] used in expression must be numeric, date, or geopoint");
}
bindings.add(variable, valueSource);
}
}
@ -224,57 +226,6 @@ public class ExpressionScriptEngineService extends AbstractComponent implements
}
}
protected ValueSource getMethodValueSource(MappedFieldType fieldType, IndexFieldData<?> fieldData, String fieldName, String methodName) {
switch (methodName) {
case GET_YEAR_METHOD:
return getDateMethodValueSource(fieldType, fieldData, fieldName, methodName, Calendar.YEAR);
case GET_MONTH_METHOD:
return getDateMethodValueSource(fieldType, fieldData, fieldName, methodName, Calendar.MONTH);
case GET_DAY_OF_MONTH_METHOD:
return getDateMethodValueSource(fieldType, fieldData, fieldName, methodName, Calendar.DAY_OF_MONTH);
case GET_HOUR_OF_DAY_METHOD:
return getDateMethodValueSource(fieldType, fieldData, fieldName, methodName, Calendar.HOUR_OF_DAY);
case GET_MINUTES_METHOD:
return getDateMethodValueSource(fieldType, fieldData, fieldName, methodName, Calendar.MINUTE);
case GET_SECONDS_METHOD:
return getDateMethodValueSource(fieldType, fieldData, fieldName, methodName, Calendar.SECOND);
case MINIMUM_METHOD:
return new FieldDataValueSource(fieldData, MultiValueMode.MIN);
case MAXIMUM_METHOD:
return new FieldDataValueSource(fieldData, MultiValueMode.MAX);
case AVERAGE_METHOD:
return new FieldDataValueSource(fieldData, MultiValueMode.AVG);
case MEDIAN_METHOD:
return new FieldDataValueSource(fieldData, MultiValueMode.MEDIAN);
case SUM_METHOD:
return new FieldDataValueSource(fieldData, MultiValueMode.SUM);
case COUNT_METHOD:
return new CountMethodValueSource(fieldData);
default:
throw new IllegalArgumentException("Member method [" + methodName + "] does not exist.");
}
}
protected ValueSource getVariableValueSource(MappedFieldType fieldType, IndexFieldData<?> fieldData, String fieldName, String memberName) {
switch (memberName) {
case VALUE_VARIABLE:
return new FieldDataValueSource(fieldData, MultiValueMode.MIN);
case EMPTY_VARIABLE:
return new EmptyMemberValueSource(fieldData);
default:
throw new IllegalArgumentException("Member variable [" + memberName + "] does not exist.");
}
}
protected ValueSource getDateMethodValueSource(MappedFieldType fieldType, IndexFieldData<?> fieldData, String fieldName, String methodName, int calendarType) {
if (fieldType instanceof LegacyDateFieldMapper.DateFieldType == false
&& fieldType instanceof DateFieldMapper.DateFieldType == false) {
throw new IllegalArgumentException("Member method [" + methodName + "] can only be used with a date field type, not the field [" + fieldName + "].");
}
return new DateMethodValueSource(fieldData, MultiValueMode.MIN, methodName, calendarType);
}
@Override
public ExecutableScript executable(CompiledScript compiledScript, Map<String, Object> vars) {
return new ExpressionExecutableScript(compiledScript, vars);

43
modules/lang-expression/src/main/java/org/elasticsearch/script/expression/FieldDataFunctionValues.java
View File

@ -1,43 +0,0 @@
/*
* Licensed to Elasticsearch under one or more contributor
* license agreements. See the NOTICE file distributed with
* this work for additional information regarding copyright
* ownership. Elasticsearch licenses this file to you under
* the Apache License, Version 2.0 (the "License"); you may
* not use this file except in compliance with the License.
* You may obtain a copy of the License at
*
* http://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing,
* software distributed under the License is distributed on an
* "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
* KIND, either express or implied. See the License for the
* specific language governing permissions and limitations
* under the License.
*/
package org.elasticsearch.script.expression;
import org.apache.lucene.queries.function.ValueSource;
import org.apache.lucene.queries.function.docvalues.DoubleDocValues;
import org.elasticsearch.index.fielddata.AtomicNumericFieldData;
import org.elasticsearch.index.fielddata.NumericDoubleValues;
import org.elasticsearch.search.MultiValueMode;
/**
* A {@link org.apache.lucene.queries.function.FunctionValues} which wrap field data.
*/
class FieldDataFunctionValues extends DoubleDocValues {
NumericDoubleValues dataAccessor;
FieldDataFunctionValues(ValueSource parent, MultiValueMode m, AtomicNumericFieldData d) {
super(parent);
dataAccessor = m.select(d.getDoubleValues(), 0d);
}
@Override
public double doubleVal(int i) {
return dataAccessor.get(i);
}
}

27
modules/lang-expression/src/main/java/org/elasticsearch/script/expression/FieldDataValueSource.java
View File

@ -26,9 +26,10 @@ import java.util.Objects;
import org.apache.lucene.index.LeafReaderContext;
import org.apache.lucene.queries.function.FunctionValues;
import org.apache.lucene.queries.function.ValueSource;
import org.elasticsearch.index.fielddata.AtomicFieldData;
import org.apache.lucene.queries.function.docvalues.DoubleDocValues;
import org.elasticsearch.index.fielddata.AtomicNumericFieldData;
import org.elasticsearch.index.fielddata.IndexFieldData;
import org.elasticsearch.index.fielddata.NumericDoubleValues;
import org.elasticsearch.search.MultiValueMode;
/**
@ -36,15 +37,12 @@ import org.elasticsearch.search.MultiValueMode;
*/
class FieldDataValueSource extends ValueSource {
protected IndexFieldData<?> fieldData;
protected MultiValueMode multiValueMode;
final IndexFieldData<?> fieldData;
final MultiValueMode multiValueMode;
protected FieldDataValueSource(IndexFieldData<?> d, MultiValueMode m) {
Objects.requireNonNull(d);
Objects.requireNonNull(m);
fieldData = d;
multiValueMode = m;
protected FieldDataValueSource(IndexFieldData<?> fieldData, MultiValueMode multiValueMode) {
this.fieldData = Objects.requireNonNull(fieldData);
this.multiValueMode = Objects.requireNonNull(multiValueMode);
}
@Override
@ -69,9 +67,14 @@ class FieldDataValueSource extends ValueSource {
@Override
@SuppressWarnings("rawtypes") // ValueSource uses a rawtype
public FunctionValues getValues(Map context, LeafReaderContext leaf) throws IOException {
AtomicFieldData leafData = fieldData.load(leaf);
assert(leafData instanceof AtomicNumericFieldData);
return new FieldDataFunctionValues(this, multiValueMode, (AtomicNumericFieldData)leafData);
AtomicNumericFieldData leafData = (AtomicNumericFieldData) fieldData.load(leaf);
NumericDoubleValues docValues = multiValueMode.select(leafData.getDoubleValues(), 0d);
return new DoubleDocValues(this) {
@Override
public double doubleVal(int doc) {
return docValues.get(doc);
}
};
}
@Override

81
modules/lang-expression/src/main/java/org/elasticsearch/script/expression/GeoEmptyValueSource.java Normal file
View File

@ -0,0 +1,81 @@
/*
* Licensed to Elasticsearch under one or more contributor
* license agreements. See the NOTICE file distributed with
* this work for additional information regarding copyright
* ownership. Elasticsearch licenses this file to you under
* the Apache License, Version 2.0 (the "License"); you may
* not use this file except in compliance with the License.
* You may obtain a copy of the License at
*
* http://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing,
* software distributed under the License is distributed on an
* "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
* KIND, either express or implied. See the License for the
* specific language governing permissions and limitations
* under the License.
*/
package org.elasticsearch.script.expression;
import java.io.IOException;
import java.util.Map;
import java.util.Objects;
import org.apache.lucene.index.LeafReaderContext;
import org.apache.lucene.queries.function.FunctionValues;
import org.apache.lucene.queries.function.ValueSource;
import org.apache.lucene.queries.function.docvalues.DoubleDocValues;
import org.elasticsearch.index.fielddata.AtomicGeoPointFieldData;
import org.elasticsearch.index.fielddata.IndexFieldData;
import org.elasticsearch.index.fielddata.MultiGeoPointValues;
/**
* ValueSource to return non-zero if a field is missing.
*/
final class GeoEmptyValueSource extends ValueSource {
IndexFieldData<?> fieldData;
GeoEmptyValueSource(IndexFieldData<?> fieldData) {
this.fieldData = Objects.requireNonNull(fieldData);
}
@Override
@SuppressWarnings("rawtypes") // ValueSource uses a rawtype
public FunctionValues getValues(Map context, LeafReaderContext leaf) throws IOException {
AtomicGeoPointFieldData leafData = (AtomicGeoPointFieldData) fieldData.load(leaf);
final MultiGeoPointValues values = leafData.getGeoPointValues();
return new DoubleDocValues(this) {
@Override
public double doubleVal(int doc) {
values.setDocument(doc);
if (values.count() == 0) {
return 1;
} else {
return 0;
}
}
};
}
@Override
public int hashCode() {
return 31 * getClass().hashCode() + fieldData.hashCode();
}
@Override
public boolean equals(Object obj) {
if (this == obj) return true;
if (obj == null) return false;
if (getClass() != obj.getClass()) return false;
GeoEmptyValueSource other = (GeoEmptyValueSource) obj;
if (!fieldData.equals(other.fieldData)) return false;
return true;
}
@Override
public String description() {
return "empty: field(" + fieldData.getFieldName() + ")";
}
}

53
modules/lang-expression/src/main/java/org/elasticsearch/script/expression/GeoField.java Normal file
View File

@ -0,0 +1,53 @@
package org.elasticsearch.script.expression;
/*
* Licensed to Elasticsearch under one or more contributor
* license agreements. See the NOTICE file distributed with
* this work for additional information regarding copyright
* ownership. Elasticsearch licenses this file to you under
* the Apache License, Version 2.0 (the "License"); you may
* not use this file except in compliance with the License.
* You may obtain a copy of the License at
*
* http://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing,
* software distributed under the License is distributed on an
* "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
* KIND, either express or implied. See the License for the
* specific language governing permissions and limitations
* under the License.
*/
import org.apache.lucene.queries.function.ValueSource;
import org.elasticsearch.index.fielddata.IndexFieldData;
/**
* Expressions API for geo_point fields.
*/
final class GeoField {
// no instance
private GeoField() {}
// supported variables
static final String EMPTY_VARIABLE = "empty";
static final String LAT_VARIABLE = "lat";
static final String LON_VARIABLE = "lon";
static ValueSource getVariable(IndexFieldData<?> fieldData, String fieldName, String variable) {
switch (variable) {
case EMPTY_VARIABLE:
return new GeoEmptyValueSource(fieldData);
case LAT_VARIABLE:
return new GeoLatitudeValueSource(fieldData);
case LON_VARIABLE:
return new GeoLongitudeValueSource(fieldData);
default:
throw new IllegalArgumentException("Member variable [" + variable + "] does not exist for geo field [" + fieldName + "].");
}
}
static ValueSource getMethod(IndexFieldData<?> fieldData, String fieldName, String method) {
throw new IllegalArgumentException("Member method [" + method + "] does not exist for geo field [" + fieldName + "].");
}
}

81
modules/lang-expression/src/main/java/org/elasticsearch/script/expression/GeoLatitudeValueSource.java Normal file
View File

@ -0,0 +1,81 @@
/*
* Licensed to Elasticsearch under one or more contributor
* license agreements. See the NOTICE file distributed with
* this work for additional information regarding copyright
* ownership. Elasticsearch licenses this file to you under
* the Apache License, Version 2.0 (the "License"); you may
* not use this file except in compliance with the License.
* You may obtain a copy of the License at
*
* http://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing,
* software distributed under the License is distributed on an
* "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
* KIND, either express or implied. See the License for the
* specific language governing permissions and limitations
* under the License.
*/
package org.elasticsearch.script.expression;
import java.io.IOException;
import java.util.Map;
import java.util.Objects;
import org.apache.lucene.index.LeafReaderContext;
import org.apache.lucene.queries.function.FunctionValues;
import org.apache.lucene.queries.function.ValueSource;
import org.apache.lucene.queries.function.docvalues.DoubleDocValues;
import org.elasticsearch.index.fielddata.AtomicGeoPointFieldData;
import org.elasticsearch.index.fielddata.IndexFieldData;
import org.elasticsearch.index.fielddata.MultiGeoPointValues;
/**
* ValueSource to return latitudes as a double "stream" for geopoint fields
*/
final class GeoLatitudeValueSource extends ValueSource {
final IndexFieldData<?> fieldData;
GeoLatitudeValueSource(IndexFieldData<?> fieldData) {
this.fieldData = Objects.requireNonNull(fieldData);
}
@Override
@SuppressWarnings("rawtypes") // ValueSource uses a rawtype
public FunctionValues getValues(Map context, LeafReaderContext leaf) throws IOException {
AtomicGeoPointFieldData leafData = (AtomicGeoPointFieldData) fieldData.load(leaf);
final MultiGeoPointValues values = leafData.getGeoPointValues();
return new DoubleDocValues(this) {
@Override
public double doubleVal(int doc) {
values.setDocument(doc);
if (values.count() == 0) {
return 0.0;
} else {
return values.valueAt(0).getLat();
}
}
};
}
@Override
public int hashCode() {
return 31 * getClass().hashCode() + fieldData.hashCode();
}
@Override
public boolean equals(Object obj) {
if (this == obj) return true;
if (obj == null) return false;
if (getClass() != obj.getClass()) return false;
GeoLatitudeValueSource other = (GeoLatitudeValueSource) obj;
if (!fieldData.equals(other.fieldData)) return false;
return true;
}
@Override
public String description() {
return "lat: field(" + fieldData.getFieldName() + ")";
}
}

81
modules/lang-expression/src/main/java/org/elasticsearch/script/expression/GeoLongitudeValueSource.java Normal file
View File

@ -0,0 +1,81 @@
/*
* Licensed to Elasticsearch under one or more contributor
* license agreements. See the NOTICE file distributed with
* this work for additional information regarding copyright
* ownership. Elasticsearch licenses this file to you under
* the Apache License, Version 2.0 (the "License"); you may
* not use this file except in compliance with the License.
* You may obtain a copy of the License at
*
* http://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing,
* software distributed under the License is distributed on an
* "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
* KIND, either express or implied. See the License for the
* specific language governing permissions and limitations
* under the License.
*/
package org.elasticsearch.script.expression;
import java.io.IOException;
import java.util.Map;
import java.util.Objects;
import org.apache.lucene.index.LeafReaderContext;
import org.apache.lucene.queries.function.FunctionValues;
import org.apache.lucene.queries.function.ValueSource;
import org.apache.lucene.queries.function.docvalues.DoubleDocValues;
import org.elasticsearch.index.fielddata.AtomicGeoPointFieldData;
import org.elasticsearch.index.fielddata.IndexFieldData;
import org.elasticsearch.index.fielddata.MultiGeoPointValues;
/**
* ValueSource to return longitudes as a double "stream" for geopoint fields
*/
final class GeoLongitudeValueSource extends ValueSource {
final IndexFieldData<?> fieldData;
GeoLongitudeValueSource(IndexFieldData<?> fieldData) {
this.fieldData = Objects.requireNonNull(fieldData);
}
@Override
@SuppressWarnings("rawtypes") // ValueSource uses a rawtype
public FunctionValues getValues(Map context, LeafReaderContext leaf) throws IOException {
AtomicGeoPointFieldData leafData = (AtomicGeoPointFieldData) fieldData.load(leaf);
final MultiGeoPointValues values = leafData.getGeoPointValues();
return new DoubleDocValues(this) {
@Override
public double doubleVal(int doc) {
values.setDocument(doc);
if (values.count() == 0) {
return 0.0;
} else {
return values.valueAt(0).getLon();
}
}
};
}
@Override
public int hashCode() {
return 31 * getClass().hashCode() + fieldData.hashCode();
}
@Override
public boolean equals(Object obj) {
if (this == obj) return true;
if (obj == null) return false;
if (getClass() != obj.getClass()) return false;
GeoLongitudeValueSource other = (GeoLongitudeValueSource) obj;
if (!fieldData.equals(other.fieldData)) return false;
return true;
}
@Override
public String description() {
return "lon: field(" + fieldData.getFieldName() + ")";
}
}

75
modules/lang-expression/src/main/java/org/elasticsearch/script/expression/NumericField.java Normal file
View File

@ -0,0 +1,75 @@
package org.elasticsearch.script.expression;
/*
* Licensed to Elasticsearch under one or more contributor
* license agreements. See the NOTICE file distributed with
* this work for additional information regarding copyright
* ownership. Elasticsearch licenses this file to you under
* the Apache License, Version 2.0 (the "License"); you may
* not use this file except in compliance with the License.
* You may obtain a copy of the License at
*
* http://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing,
* software distributed under the License is distributed on an
* "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
* KIND, either express or implied. See the License for the
* specific language governing permissions and limitations
* under the License.
*/
import org.apache.lucene.queries.function.ValueSource;
import org.elasticsearch.index.fielddata.IndexFieldData;
import org.elasticsearch.search.MultiValueMode;
/**
* Expressions API for numeric fields.
*/
final class NumericField {
// no instance
private NumericField() {}
// supported variables
static final String VALUE_VARIABLE = "value";
static final String EMPTY_VARIABLE = "empty";
// supported methods
static final String MINIMUM_METHOD = "min";
static final String MAXIMUM_METHOD = "max";
static final String AVERAGE_METHOD = "avg";
static final String MEDIAN_METHOD = "median";
static final String SUM_METHOD = "sum";
static final String COUNT_METHOD = "count";
static ValueSource getVariable(IndexFieldData<?> fieldData, String fieldName, String variable) {
switch (variable) {
case VALUE_VARIABLE:
return new FieldDataValueSource(fieldData, MultiValueMode.MIN);
case EMPTY_VARIABLE:
return new EmptyMemberValueSource(fieldData);
default:
throw new IllegalArgumentException("Member variable [" + variable + "] does not exist for " +
"numeric field [" + fieldName + "].");
}
}
static ValueSource getMethod(IndexFieldData<?> fieldData, String fieldName, String method) {
switch (method) {
case MINIMUM_METHOD:
return new FieldDataValueSource(fieldData, MultiValueMode.MIN);
case MAXIMUM_METHOD:
return new FieldDataValueSource(fieldData, MultiValueMode.MAX);
case AVERAGE_METHOD:
return new FieldDataValueSource(fieldData, MultiValueMode.AVG);
case MEDIAN_METHOD:
return new FieldDataValueSource(fieldData, MultiValueMode.MEDIAN);
case SUM_METHOD:
return new FieldDataValueSource(fieldData, MultiValueMode.SUM);
case COUNT_METHOD:
return new CountMethodValueSource(fieldData);
default:
throw new IllegalArgumentException("Member method [" + method + "] does not exist for numeric field [" + fieldName + "].");
}
}
}

4
modules/lang-expression/src/main/java/org/elasticsearch/script/expression/ReplaceableConstFunctionValues.java
View File

@ -25,10 +25,10 @@ import org.apache.lucene.queries.function.FunctionValues;
* A support class for an executable expression script that allows the double returned
* by a {@link FunctionValues} to be modified.
*/
public class ReplaceableConstFunctionValues extends FunctionValues {
final class ReplaceableConstFunctionValues extends FunctionValues {
private double value = 0;
public void setValue(double value) {
void setValue(double value) {
this.value = value;
}

4
modules/lang-expression/src/main/java/org/elasticsearch/script/expression/ReplaceableConstValueSource.java
View File

@ -29,10 +29,10 @@ import org.apache.lucene.queries.function.ValueSource;
/**
* A {@link ValueSource} which has a stub {@link FunctionValues} that holds a dynamically replaceable constant double.
*/
class ReplaceableConstValueSource extends ValueSource {
final class ReplaceableConstValueSource extends ValueSource {
final ReplaceableConstFunctionValues fv;
public ReplaceableConstValueSource() {
ReplaceableConstValueSource() {
fv = new ReplaceableConstFunctionValues();
}

44
modules/lang-expression/src/test/java/org/elasticsearch/script/expression/MoreExpressionTests.java
View File

@ -27,12 +27,17 @@ import java.util.Map;
import org.apache.lucene.expressions.Expression;
import org.apache.lucene.expressions.js.JavascriptCompiler;
import org.elasticsearch.Version;
import org.elasticsearch.action.search.SearchPhaseExecutionException;
import org.elasticsearch.action.search.SearchRequestBuilder;
import org.elasticsearch.action.search.SearchResponse;
import org.elasticsearch.action.search.SearchType;
import org.elasticsearch.action.update.UpdateRequestBuilder;
import org.elasticsearch.cluster.metadata.IndexMetaData;
import org.elasticsearch.common.lucene.search.function.CombineFunction;
import org.elasticsearch.common.settings.Settings;
import org.elasticsearch.common.xcontent.XContentBuilder;
import org.elasticsearch.common.xcontent.XContentFactory;
import org.elasticsearch.index.query.QueryBuilders;
import org.elasticsearch.index.query.functionscore.ScoreFunctionBuilder;
import org.elasticsearch.index.query.functionscore.ScoreFunctionBuilders;
@ -51,8 +56,10 @@ import org.elasticsearch.search.aggregations.pipeline.SimpleValue;
import org.elasticsearch.search.sort.SortBuilders;
import org.elasticsearch.search.sort.SortOrder;
import org.elasticsearch.test.ESIntegTestCase;
import org.elasticsearch.test.VersionUtils;
import org.elasticsearch.test.hamcrest.ElasticsearchAssertions;
import static org.elasticsearch.common.xcontent.XContentFactory.jsonBuilder;
import static org.elasticsearch.search.aggregations.AggregationBuilders.histogram;
import static org.elasticsearch.search.aggregations.AggregationBuilders.sum;
import static org.elasticsearch.search.aggregations.pipeline.PipelineAggregatorBuilders.bucketScript;
@ -257,8 +264,8 @@ public class MoreExpressionTests extends ESIntegTestCase {
} catch (SearchPhaseExecutionException e) {
assertThat(e.toString() + "should have contained IllegalArgumentException",
e.toString().contains("IllegalArgumentException"), equalTo(true));
assertThat(e.toString() + "should have contained can only be used with a date field type",
e.toString().contains("can only be used with a date field type"), equalTo(true));
assertThat(e.toString() + "should have contained does not exist for numeric field",
e.toString().contains("does not exist for numeric field"), equalTo(true));
}
}
@ -586,4 +593,37 @@ public class MoreExpressionTests extends ESIntegTestCase {
}
}
}
public void testGeo() throws Exception {
XContentBuilder xContentBuilder = XContentFactory.jsonBuilder().startObject().startObject("type1")
.startObject("properties").startObject("location").field("type", "geo_point");
xContentBuilder.endObject().endObject().endObject().endObject();
assertAcked(prepareCreate("test").addMapping("type1", xContentBuilder));
ensureGreen();
client().prepareIndex("test", "type1", "1").setSource(jsonBuilder().startObject()
.field("name", "test")
.startObject("location").field("lat", 61.5240).field("lon", 105.3188).endObject()
.endObject()).execute().actionGet();
refresh();
// access .lat
SearchResponse rsp = buildRequest("doc['location'].lat").get();
assertSearchResponse(rsp);
assertEquals(1, rsp.getHits().getTotalHits());
assertEquals(61.5240, rsp.getHits().getAt(0).field("foo").getValue(), 1.0D);
// access .lon
rsp = buildRequest("doc['location'].lon").get();
assertSearchResponse(rsp);
assertEquals(1, rsp.getHits().getTotalHits());
assertEquals(105.3188, rsp.getHits().getAt(0).field("foo").getValue(), 1.0D);
// access .empty
rsp = buildRequest("doc['location'].empty ? 1 : 0").get();
assertSearchResponse(rsp);
assertEquals(1, rsp.getHits().getTotalHits());
assertEquals(0, rsp.getHits().getAt(0).field("foo").getValue(), 1.0D);
// call haversin
rsp = buildRequest("haversin(38.9072, 77.0369, doc['location'].lat, doc['location'].lon)").get();
assertSearchResponse(rsp);
assertEquals(1, rsp.getHits().getTotalHits());
assertEquals(3170D, rsp.getHits().getAt(0).field("foo").getValue(), 50D);
}
}