Switch to Groovy as the default scripting language
This is a breaking change to move from MVEL -> Groovy
This commit is contained in:
Lee Hinman
parent
47856ec4cd
commit
5c6d28240f
|
|
@ -37,12 +37,12 @@ depending on the shard the current document resides in.
|
|||
|
||||
`_index.numDocs()`::
|
||||
|
||||
Number of documents in shard.
|
||||
|
||||
Number of documents in shard.
|
||||
|
||||
`_index.maxDoc()`::
|
||||
|
||||
Maximal document number in shard.
|
||||
|
||||
|
||||
`_index.numDeletedDocs()`::
|
||||
|
||||
Number of deleted documents in shard.
|
||||
|
|
@ -62,7 +62,7 @@ Field statistics can be accessed with a subscript operator like this:
|
|||
`_index['FIELD'].sumttf()`::
|
||||
|
||||
Sum of `ttf` over all terms that appear in field `FIELD` in all documents.
|
||||
|
||||
|
||||
`_index['FIELD'].sumdf()`::
|
||||
|
||||
The sum of `df` s over all terms that appear in field `FIELD` in all
|
||||
|
|
@ -77,7 +77,7 @@ The number of terms in a field cannot be accessed using the `_index` variable. S
|
|||
=== Term statistics:
|
||||
|
||||
Term statistics for a field can be accessed with a subscript operator like
|
||||
this: `_index['FIELD']['TERM']`. This will never return null, even if term or field does not exist.
|
||||
this: `_index['FIELD']['TERM']`. This will never return null, even if term or field does not exist.
|
||||
If you do not need the term frequency, call `_index['FIELD'].get('TERM', 0)`
|
||||
to avoid uneccesary initialization of the frequencies. The flag will have only
|
||||
affect is your set the `index_options` to `docs` (see <<mapping-core-types, mapping documentation>>).
|
||||
|
|
@ -162,11 +162,11 @@ Positions can be accessed with an iterator that returns an object
|
|||
|
||||
Example: sums up all payloads for the term `foo`.
|
||||
|
||||
[source,mvel]
|
||||
[source,groovy]
|
||||
---------------------------------------------------------
|
||||
termInfo = _index['my_field'].get('foo',_PAYLOADS);
|
||||
score = 0;
|
||||
for (pos : termInfo) {
|
||||
for (pos in termInfo) {
|
||||
score = score + pos.payloadAsInt(0);
|
||||
}
|
||||
return score;
|
||||
|
|
@ -181,4 +181,3 @@ The `_index` variable can only be used to gather statistics for single terms. If
|
|||
https://lucene.apache.org/core/4_0_0/core/org/apache/lucene/index/Fields.html[Fields]
|
||||
instance. This object can then be used as described in https://lucene.apache.org/core/4_0_0/core/org/apache/lucene/index/Fields.html[lucene doc] to iterate over fields and then for each field iterate over each term in the field.
|
||||
The method will return null if the term vectors were not stored.
|
||||
|
||||
|
|
|
|||
|
|
@ -6,28 +6,29 @@ expressions. For example, scripts can be used to return "script fields"
|
|||
as part of a search request, or can be used to evaluate a custom score
|
||||
for a query and so on.
|
||||
|
||||
The scripting module uses by default http://mvel.codehaus.org/[mvel] as
|
||||
the scripting language with some extensions. mvel is used since it is
|
||||
extremely fast and very simple to use, and in most cases, simple
|
||||
expressions are needed (for example, mathematical equations).
|
||||
deprecated[1.3.0,Groovy has replaced Mvel as the default scripting language]
|
||||
|
||||
The scripting module uses by default http://groovy.codehaus.org/[groovy]
|
||||
(previously http://mvel.codehaus.org/[mvel]) as the scripting language with some
|
||||
extensions. Groovy is used since it is extremely fast and very simple to use.
|
||||
|
||||
Additional `lang` plugins are provided to allow to execute scripts in
|
||||
different languages. Currently supported plugins are `lang-javascript`
|
||||
for JavaScript, `lang-groovy` for Groovy, and `lang-python` for Python.
|
||||
for JavaScript, `lang-mvel` for Mvel, and `lang-python` for Python.
|
||||
All places where a `script` parameter can be used, a `lang` parameter
|
||||
(on the same level) can be provided to define the language of the
|
||||
script. The `lang` options are `mvel`, `js`, `groovy`, `python`, and
|
||||
script. The `lang` options are `groovy`, `js`, `mvel`, `python`, and
|
||||
`native`.
|
||||
|
||||
added[1.2.0, Dynamic scripting is disabled by default since version 1.2.0]
|
||||
added[1.2.0, Dynamic scripting is disabled for non-sandboxed languages by default since version 1.2.0]
|
||||
|
||||
To increase security, Elasticsearch does not allow you to specify scripts with a
|
||||
request. Instead, scripts must be placed in the `scripts` directory inside the
|
||||
configuration directory (the directory where elasticsearch.yml is). Scripts
|
||||
placed into this directory will automatically be picked up and be available to
|
||||
be used. Once a script has been placed in this directory, it can be referenced
|
||||
by name. For example, a script called `calculate-score.mvel` can be referenced
|
||||
in a request like this:
|
||||
To increase security, Elasticsearch does not allow you to specify scripts for
|
||||
non-sandboxed languages with a request. Instead, scripts must be placed in the
|
||||
`scripts` directory inside the configuration directory (the directory where
|
||||
elasticsearch.yml is). Scripts placed into this directory will automatically be
|
||||
picked up and be available to be used. Once a script has been placed in this
|
||||
directory, it can be referenced by name. For example, a script called
|
||||
`calculate-score.groovy` can be referenced in a request like this:
|
||||
|
||||
[source,sh]
|
||||
--------------------------------------------------
|
||||
|
|
@ -36,13 +37,13 @@ config
|
|||
├── elasticsearch.yml
|
||||
├── logging.yml
|
||||
└── scripts
|
||||
└── calculate-score.mvel
|
||||
└── calculate-score.groovy
|
||||
--------------------------------------------------
|
||||
|
||||
[source,sh]
|
||||
--------------------------------------------------
|
||||
$ cat config/scripts/calculate-score.mvel
|
||||
Math.log(_score * 2) + my_modifier
|
||||
$ cat config/scripts/calculate-score.groovy
|
||||
log(_score * 2) + my_modifier
|
||||
--------------------------------------------------
|
||||
|
||||
[source,js]
|
||||
|
|
@ -75,21 +76,14 @@ exists under, and the file name without the lang extension. For example,
|
|||
a script placed under `config/scripts/group1/group2/test.py` will be
|
||||
named `group1_group2_test`.
|
||||
|
||||
[float]
|
||||
=== Default Scripting Language
|
||||
|
||||
The default scripting language (assuming no `lang` parameter is
|
||||
provided) is `mvel`. In order to change it set the `script.default_lang`
|
||||
to the appropriate language.
|
||||
|
||||
[float]
|
||||
=== Enabling dynamic scripting
|
||||
|
||||
We recommend running Elasticsearch behind an application or proxy,
|
||||
which protects Elasticsearch from the outside world. If users are
|
||||
allowed to run dynamic scripts (even in a search request), then they
|
||||
have the same access to your box as the user that Elasticsearch is
|
||||
running as. For this reason dynamic scripting is disabled by default.
|
||||
We recommend running Elasticsearch behind an application or proxy, which
|
||||
protects Elasticsearch from the outside world. If users are allowed to run
|
||||
dynamic scripts (even in a search request), then they have the same access to
|
||||
your box as the user that Elasticsearch is running as. For this reason dynamic
|
||||
scripting is allowed only for sandboxed languages by default.
|
||||
|
||||
First, you should not run Elasticsearch as the `root` user, as this would allow
|
||||
a script to access or do *anything* on your server, without limitations. Second,
|
||||
|
|
@ -109,6 +103,54 @@ _native_ Java scripts registered through plugins, it also allows users to run
|
|||
arbitrary scripts via the API. Instead of sending the name of the file as the
|
||||
script, the body of the script can be sent instead.
|
||||
|
||||
There are three possible configuration values for the `script.disable_dynamic`
|
||||
setting, the default value is `sandbox`:
|
||||
|
||||
[cols="<,<",options="header",]
|
||||
|=======================================================================
|
||||
|Value |Description
|
||||
| `true` |all dynamic scripting is disabled, scripts must be placed in the `config/scripts` directory.
|
||||
| `false` |all dynamic scripting is enabled, scripts may be sent as strings in requests.
|
||||
| `sandbox` |scripts may be sent as strings for languages that are sandboxed.
|
||||
|=======================================================================
|
||||
|
||||
[float]
|
||||
=== Default Scripting Language
|
||||
|
||||
The default scripting language (assuming no `lang` parameter is provided) is
|
||||
`groovy`. In order to change it, set the `script.default_lang` to the
|
||||
appropriate language.
|
||||
|
||||
[float]
|
||||
=== Groovy Sandboxing
|
||||
|
||||
Elasticsearch sandboxes Groovy scripts that are compiled and executed in order
|
||||
to ensure they don't perform unwanted actions. There are a number of options
|
||||
that can be used for configuring this sandbox:
|
||||
|
||||
`script.groovy.sandbox.receiver_whitelist`::
|
||||
|
||||
Comma-separated list of string classes for objects that may have methods
|
||||
invoked.
|
||||
|
||||
`script.groovy.sandbox.package_whitelist`::
|
||||
|
||||
Comma-separated list of packages under which new objects may be constructed.
|
||||
|
||||
`script.groovy.sandbox.class_whitelist`::
|
||||
|
||||
Comma-separated list of classes that are allowed to be constructed.
|
||||
|
||||
`script.groovy.sandbox.method_blacklist`::
|
||||
|
||||
Comma-separated list of methods that are never allowed to be invoked,
|
||||
regardless of target object.
|
||||
|
||||
`script.groovy.sandbox.enabled`::
|
||||
|
||||
Flag to disable the sandbox (defaults to `true` meaning the sandbox is
|
||||
enabled).
|
||||
|
||||
[float]
|
||||
=== Automatic Script Reloading
|
||||
|
||||
|
|
@ -122,7 +164,7 @@ to `false`.
|
|||
[float]
|
||||
=== Native (Java) Scripts
|
||||
|
||||
Even though `mvel` is pretty fast, this allows to register native Java based
|
||||
Even though `groovy` is pretty fast, this allows to register native Java based
|
||||
scripts for faster execution.
|
||||
|
||||
In order to allow for scripts, the `NativeScriptFactory` needs to be
|
||||
|
|
@ -267,7 +309,7 @@ loaded for other purposes.
|
|||
|
||||
|
||||
[float]
|
||||
=== mvel Built In Functions
|
||||
=== Groovy Built In Functions
|
||||
|
||||
There are several built in functions that can be used within scripts.
|
||||
They include:
|
||||
|
|
@ -275,8 +317,6 @@ They include:
|
|||
[cols="<,<",options="header",]
|
||||
|=======================================================================
|
||||
|Function |Description
|
||||
|`time()` |The current time in milliseconds.
|
||||
|
||||
|`sin(a)` |Returns the trigonometric sine of an angle.
|
||||
|
||||
|`cos(a)` |Returns the trigonometric cosine of an angle.
|
||||
|
|
@ -350,6 +390,16 @@ power of the second argument.
|
|||
or underflow.
|
||||
|=======================================================================
|
||||
|
||||
[float]
|
||||
=== Floating point numbers in Groovy
|
||||
|
||||
When using floating-point literals in Groovy scripts, Groovy will automatically
|
||||
use BigDecimal instead of a floating point equivalent to support the
|
||||
'least-surprising' approach to literal math operations. To use a floating-point
|
||||
number instead, use the specific suffix on the number (ie, instead of 1.2, use
|
||||
1.2f). See the http://groovy.codehaus.org/Groovy+Math[Groovy Math] page for more
|
||||
information.
|
||||
|
||||
[float]
|
||||
=== Arithmetic precision in MVEL
|
||||
|
||||
|
|
|
|||
|
|
@ -34,7 +34,7 @@
|
|||
},
|
||||
"lang": {
|
||||
"type": "string",
|
||||
"description": "The script language (default: mvel)"
|
||||
"description": "The script language (default: groovy)"
|
||||
},
|
||||
"parent": {
|
||||
"type": "string",
|
||||
|
|
|
|||
|
|
@ -17,7 +17,7 @@
|
|||
id: 1
|
||||
script: "1"
|
||||
body:
|
||||
lang: mvel
|
||||
lang: groovy
|
||||
script: "ctx._source.foo = bar"
|
||||
params: { bar: 'xxx' }
|
||||
|
||||
|
|
@ -40,7 +40,7 @@
|
|||
index: test_1
|
||||
type: test
|
||||
id: 1
|
||||
lang: mvel
|
||||
lang: groovy
|
||||
script: "ctx._source.foo = 'yyy'"
|
||||
|
||||
- match: { _index: test_1 }
|
||||
|
|
|
|||
|
|
@ -92,7 +92,7 @@ public class UpdateRequestBuilder extends InstanceShardOperationRequestBuilder<U
|
|||
/**
|
||||
* The language of the script to execute.
|
||||
* Valid options are: mvel, js, groovy, python, and native (Java)<br>
|
||||
* Default: mvel
|
||||
* Default: groovy
|
||||
* <p>
|
||||
* Ref: http://www.elasticsearch.org/guide/en/elasticsearch/reference/current/modules-scripting.html
|
||||
*/
|
||||
|
|
|
|||
|
|
@ -110,7 +110,7 @@ public class ScriptService extends AbstractComponent {
|
|||
TimeValue cacheExpire = componentSettings.getAsTime("cache.expire", null);
|
||||
logger.debug("using script cache with max_size [{}], expire [{}]", cacheMaxSize, cacheExpire);
|
||||
|
||||
this.defaultLang = settings.get(DEFAULT_SCRIPTING_LANGUAGE_SETTING, "mvel");
|
||||
this.defaultLang = settings.get(DEFAULT_SCRIPTING_LANGUAGE_SETTING, "groovy");
|
||||
this.dynamicScriptingDisabled = DynamicScriptDisabling.parse(settings.get(DISABLE_DYNAMIC_SCRIPTING_SETTING, DISABLE_DYNAMIC_SCRIPTING_DEFAULT));
|
||||
|
||||
CacheBuilder cacheBuilder = CacheBuilder.newBuilder();
|
||||
|
|
|
|||
|
|
@ -205,7 +205,7 @@ public class IndexLookupTests extends ElasticsearchIntegrationTest {
|
|||
initTestData();
|
||||
|
||||
String script = "term = _index['float_payload_field'].get('b'," + includeAllFlag
|
||||
+ "); payloadSum=0; for (pos : term) {payloadSum = pos.payloadAsInt(0)}; payloadSum";
|
||||
+ "); payloadSum=0; for (pos in term) {payloadSum = pos.payloadAsInt(0)}; payloadSum";
|
||||
|
||||
// non existing field: sum should be 0
|
||||
HashMap<String, Object> zeroArray = new HashMap<>();
|
||||
|
|
@ -215,7 +215,7 @@ public class IndexLookupTests extends ElasticsearchIntegrationTest {
|
|||
checkValueInEachDoc(script, zeroArray, 3);
|
||||
|
||||
script = "term = _index['int_payload_field'].get('b'," + includeAllFlag
|
||||
+ "); payloadSum=0; for (pos : term) {payloadSum = payloadSum + pos.payloadAsInt(0)}; payloadSum";
|
||||
+ "); payloadSum=0; for (pos in term) {payloadSum = payloadSum + pos.payloadAsInt(0)}; payloadSum";
|
||||
|
||||
// existing field: sums should be as here:
|
||||
zeroArray.put("1", 5);
|
||||
|
|
@ -263,26 +263,26 @@ public class IndexLookupTests extends ElasticsearchIntegrationTest {
|
|||
|
||||
private String createPositionsArrayScriptGetInfoObjectTwice(String term, String flags, String what) {
|
||||
String script = "term = _index['int_payload_field'].get('" + term + "'," + flags
|
||||
+ "); array=[]; for (pos : term) {array.add(pos." + what + ")}; _index['int_payload_field'].get('" + term + "',"
|
||||
+ flags + "); array=[]; for (pos : term) {array.add(pos." + what + ")}";
|
||||
+ "); array=[]; for (pos in term) {array.add(pos." + what + ")}; _index['int_payload_field'].get('" + term + "',"
|
||||
+ flags + "); array=[]; for (pos in term) {array.add(pos." + what + ")}";
|
||||
return script;
|
||||
}
|
||||
|
||||
private String createPositionsArrayScriptIterateTwice(String term, String flags, String what) {
|
||||
String script = "term = _index['int_payload_field'].get('" + term + "'," + flags
|
||||
+ "); array=[]; for (pos : term) {array.add(pos." + what + ")}; array=[]; for (pos : term) {array.add(pos." + what
|
||||
+ "); array=[]; for (pos in term) {array.add(pos." + what + ")}; array=[]; for (pos in term) {array.add(pos." + what
|
||||
+ ")}; array";
|
||||
return script;
|
||||
}
|
||||
|
||||
private String createPositionsArrayScript(String field, String term, String flags, String what) {
|
||||
String script = "term = _index['" + field + "'].get('" + term + "'," + flags
|
||||
+ "); array=[]; for (pos : term) {array.add(pos." + what + ")}; array";
|
||||
+ "); array=[]; for (pos in term) {array.add(pos." + what + ")}; array";
|
||||
return script;
|
||||
}
|
||||
|
||||
private String createPositionsArrayScriptDefaultGet(String field, String term, String what) {
|
||||
String script = "term = _index['" + field + "']['" + term + "']; array=[]; for (pos : term) {array.add(pos." + what
|
||||
String script = "term = _index['" + field + "']['" + term + "']; array=[]; for (pos in term) {array.add(pos." + what
|
||||
+ ")}; array";
|
||||
return script;
|
||||
}
|
||||
|
|
|
|||
|
|
@ -269,7 +269,7 @@ public class TopHitsTests extends ElasticsearchIntegrationTest {
|
|||
topHits("hits").setSize(1)
|
||||
)
|
||||
.subAggregation(
|
||||
max("max_score").script("_doc.score")
|
||||
max("max_score").script("_doc.score()")
|
||||
)
|
||||
)
|
||||
.get();
|
||||
|
|
|
|||
|
|
@ -687,7 +687,7 @@ public class SimpleSortTests extends ElasticsearchIntegrationTest {
|
|||
// test the long values
|
||||
SearchResponse searchResponse = client().prepareSearch()
|
||||
.setQuery(matchAllQuery())
|
||||
.addScriptField("min", "retval = Long.MAX_VALUE; for (v : doc['lvalue'].values){ retval = min(v, retval) }; retval")
|
||||
.addScriptField("min", "retval = Long.MAX_VALUE; for (v in doc['lvalue'].values){ retval = min(v, retval) }; retval")
|
||||
.addSort("ord", SortOrder.ASC).setSize(10)
|
||||
.execute().actionGet();
|
||||
|
||||
|
|
@ -700,7 +700,7 @@ public class SimpleSortTests extends ElasticsearchIntegrationTest {
|
|||
// test the double values
|
||||
searchResponse = client().prepareSearch()
|
||||
.setQuery(matchAllQuery())
|
||||
.addScriptField("min", "retval = Double.MAX_VALUE; for (v : doc['dvalue'].values){ retval = min(v, retval) }; retval")
|
||||
.addScriptField("min", "retval = Double.MAX_VALUE; for (v in doc['dvalue'].values){ retval = min(v, retval) }; retval")
|
||||
.addSort("ord", SortOrder.ASC).setSize(10)
|
||||
.execute().actionGet();
|
||||
|
||||
|
|
@ -714,7 +714,7 @@ public class SimpleSortTests extends ElasticsearchIntegrationTest {
|
|||
// test the string values
|
||||
searchResponse = client().prepareSearch()
|
||||
.setQuery(matchAllQuery())
|
||||
.addScriptField("min", "retval = Integer.MAX_VALUE; for (v : doc['svalue'].values){ retval = min(Integer.parseInt(v), retval) }; retval")
|
||||
.addScriptField("min", "retval = Integer.MAX_VALUE; for (v in doc['svalue'].values){ retval = min(Integer.parseInt(v), retval) }; retval")
|
||||
.addSort("ord", SortOrder.ASC).setSize(10)
|
||||
.execute().actionGet();
|
||||
|
||||
|
|
@ -728,7 +728,7 @@ public class SimpleSortTests extends ElasticsearchIntegrationTest {
|
|||
// test the geopoint values
|
||||
searchResponse = client().prepareSearch()
|
||||
.setQuery(matchAllQuery())
|
||||
.addScriptField("min", "retval = Double.MAX_VALUE; for (v : doc['gvalue'].values){ retval = min(v.lon, retval) }; retval")
|
||||
.addScriptField("min", "retval = Double.MAX_VALUE; for (v in doc['gvalue'].values){ retval = min(v.lon, retval) }; retval")
|
||||
.addSort("ord", SortOrder.ASC).setSize(10)
|
||||
.execute().actionGet();
|
||||
|
||||
|
|
|
|||
Loading…
Reference in New Issue