2016-05-04 12:17:10 -04:00
|
|
|
[[modules-scripting-using]]
|
|
|
|
=== How to use scripts
|
|
|
|
|
|
|
|
Wherever scripting is supported in the Elasticsearch API, the syntax follows
|
|
|
|
the same pattern:
|
|
|
|
|
|
|
|
[source,js]
|
|
|
|
-------------------------------------
|
|
|
|
"script": {
|
|
|
|
"lang": "...", <1>
|
2017-06-09 11:29:25 -04:00
|
|
|
"source" | "id": "...", <2>
|
2016-05-04 12:17:10 -04:00
|
|
|
"params": { ... } <3>
|
|
|
|
}
|
|
|
|
-------------------------------------
|
2017-04-03 10:14:21 -04:00
|
|
|
// NOTCONSOLE
|
2016-08-30 19:51:37 -04:00
|
|
|
<1> The language the script is written in, which defaults to `painless`.
|
2017-06-09 11:29:25 -04:00
|
|
|
<2> The script itself which may be specified as `source` for an inline script or `id` for a stored script.
|
2016-05-04 12:17:10 -04:00
|
|
|
<3> Any named parameters that should be passed into the script.
|
|
|
|
|
|
|
|
For example, the following script is used in a search request to return a
|
|
|
|
<<search-request-script-fields, scripted field>>:
|
|
|
|
|
|
|
|
[source,js]
|
|
|
|
-------------------------------------
|
|
|
|
PUT my_index/my_type/1
|
|
|
|
{
|
|
|
|
"my_field": 5
|
|
|
|
}
|
|
|
|
|
|
|
|
GET my_index/_search
|
|
|
|
{
|
|
|
|
"script_fields": {
|
|
|
|
"my_doubled_field": {
|
|
|
|
"script": {
|
|
|
|
"lang": "expression",
|
2017-06-09 11:29:25 -04:00
|
|
|
"source": "doc['my_field'] * multiplier",
|
2016-05-04 12:17:10 -04:00
|
|
|
"params": {
|
|
|
|
"multiplier": 2
|
|
|
|
}
|
|
|
|
}
|
|
|
|
}
|
|
|
|
}
|
|
|
|
}
|
|
|
|
-------------------------------------
|
2016-05-09 09:42:23 -04:00
|
|
|
// CONSOLE
|
2016-05-04 12:17:10 -04:00
|
|
|
|
|
|
|
[float]
|
|
|
|
=== Script Parameters
|
|
|
|
|
|
|
|
`lang`::
|
|
|
|
|
2017-10-16 08:04:43 -04:00
|
|
|
Specifies the language the script is written in. Defaults to `painless`.
|
2016-05-04 12:17:10 -04:00
|
|
|
|
|
|
|
|
2017-06-09 11:29:25 -04:00
|
|
|
`source`, `id`::
|
2016-05-04 12:17:10 -04:00
|
|
|
|
|
|
|
Specifies the source of the script. An `inline` script is specified
|
2017-06-09 11:29:25 -04:00
|
|
|
`source` as in the example above. A `stored` script is specified `id`
|
2017-05-17 17:42:25 -04:00
|
|
|
and is retrieved from the cluster state (see <<modules-scripting-stored-scripts,Stored Scripts>>).
|
|
|
|
|
2016-05-04 12:17:10 -04:00
|
|
|
|
|
|
|
`params`::
|
|
|
|
|
|
|
|
Specifies any named parameters that are passed into the script as
|
|
|
|
variables.
|
|
|
|
|
|
|
|
[IMPORTANT]
|
2016-07-13 10:02:21 -04:00
|
|
|
[[prefer-params]]
|
2016-05-04 12:17:10 -04:00
|
|
|
.Prefer parameters
|
|
|
|
========================================
|
|
|
|
|
|
|
|
The first time Elasticsearch sees a new script, it compiles it and stores the
|
|
|
|
compiled version in a cache. Compilation can be a heavy process.
|
|
|
|
|
|
|
|
If you need to pass variables into the script, you should pass them in as
|
|
|
|
named `params` instead of hard-coding values into the script itself. For
|
|
|
|
example, if you want to be able to multiply a field value by different
|
|
|
|
multipliers, don't hard-code the multiplier into the script:
|
|
|
|
|
|
|
|
[source,js]
|
|
|
|
----------------------
|
2017-06-09 11:29:25 -04:00
|
|
|
"source": "doc['my_field'] * 2"
|
2016-05-04 12:17:10 -04:00
|
|
|
----------------------
|
2017-04-03 10:14:21 -04:00
|
|
|
// NOTCONSOLE
|
2016-05-04 12:17:10 -04:00
|
|
|
|
|
|
|
Instead, pass it in as a named parameter:
|
|
|
|
|
|
|
|
[source,js]
|
|
|
|
----------------------
|
2017-06-09 11:29:25 -04:00
|
|
|
"source": "doc['my_field'] * multiplier",
|
2016-05-04 12:17:10 -04:00
|
|
|
"params": {
|
|
|
|
"multiplier": 2
|
|
|
|
}
|
|
|
|
----------------------
|
2017-04-03 10:14:21 -04:00
|
|
|
// NOTCONSOLE
|
2016-05-04 12:17:10 -04:00
|
|
|
|
|
|
|
The first version has to be recompiled every time the multiplier changes. The
|
|
|
|
second version is only compiled once.
|
|
|
|
|
Circuit break the number of inline scripts compiled per minute
When compiling many dynamically changing scripts, parameterized
scripts (<https://www.elastic.co/guide/en/elasticsearch/reference/master/modules-scripting-using.html#prefer-params>)
should be preferred. This enforces a limit to the number of scripts that
can be compiled within a minute. A new dynamic setting is added -
`script.max_compilations_per_minute`, which defaults to 15.
If more dynamic scripts are sent, a user will get the following
exception:
```json
{
"error" : {
"root_cause" : [
{
"type" : "circuit_breaking_exception",
"reason" : "[script] Too many dynamic script compilations within one minute, max: [15/min]; please use on-disk, indexed, or scripts with parameters instead",
"bytes_wanted" : 0,
"bytes_limit" : 0
}
],
"type" : "search_phase_execution_exception",
"reason" : "all shards failed",
"phase" : "query",
"grouped" : true,
"failed_shards" : [
{
"shard" : 0,
"index" : "i",
"node" : "a5V1eXcZRYiIk8lecjZ4Jw",
"reason" : {
"type" : "general_script_exception",
"reason" : "Failed to compile inline script [\"aaaaaaaaaaaaaaaa\"] using lang [painless]",
"caused_by" : {
"type" : "circuit_breaking_exception",
"reason" : "[script] Too many dynamic script compilations within one minute, max: [15/min]; please use on-disk, indexed, or scripts with parameters instead",
"bytes_wanted" : 0,
"bytes_limit" : 0
}
}
}
],
"caused_by" : {
"type" : "general_script_exception",
"reason" : "Failed to compile inline script [\"aaaaaaaaaaaaaaaa\"] using lang [painless]",
"caused_by" : {
"type" : "circuit_breaking_exception",
"reason" : "[script] Too many dynamic script compilations within one minute, max: [15/min]; please use on-disk, indexed, or scripts with parameters instead",
"bytes_wanted" : 0,
"bytes_limit" : 0
}
}
},
"status" : 500
}
```
This also fixes a bug in `ScriptService` where requests being executed
concurrently on a single node could cause a script to be compiled
multiple times (many in the case of a powerful node with many shards)
due to no synchronization between checking the cache and compiling the
script. There is now synchronization so that a script being compiled
will only be compiled once regardless of the number of concurrent
searches on a node.
Relates to #19396
2016-07-26 15:36:29 -04:00
|
|
|
If you compile too many unique scripts within a small amount of time,
|
|
|
|
Elasticsearch will reject the new dynamic scripts with a
|
|
|
|
`circuit_breaking_exception` error. By default, up to 15 inline scripts per
|
|
|
|
minute will be compiled. You can change this setting dynamically by setting
|
2017-09-01 04:15:27 -04:00
|
|
|
`script.max_compilations_rate`.
|
Circuit break the number of inline scripts compiled per minute
When compiling many dynamically changing scripts, parameterized
scripts (<https://www.elastic.co/guide/en/elasticsearch/reference/master/modules-scripting-using.html#prefer-params>)
should be preferred. This enforces a limit to the number of scripts that
can be compiled within a minute. A new dynamic setting is added -
`script.max_compilations_per_minute`, which defaults to 15.
If more dynamic scripts are sent, a user will get the following
exception:
```json
{
"error" : {
"root_cause" : [
{
"type" : "circuit_breaking_exception",
"reason" : "[script] Too many dynamic script compilations within one minute, max: [15/min]; please use on-disk, indexed, or scripts with parameters instead",
"bytes_wanted" : 0,
"bytes_limit" : 0
}
],
"type" : "search_phase_execution_exception",
"reason" : "all shards failed",
"phase" : "query",
"grouped" : true,
"failed_shards" : [
{
"shard" : 0,
"index" : "i",
"node" : "a5V1eXcZRYiIk8lecjZ4Jw",
"reason" : {
"type" : "general_script_exception",
"reason" : "Failed to compile inline script [\"aaaaaaaaaaaaaaaa\"] using lang [painless]",
"caused_by" : {
"type" : "circuit_breaking_exception",
"reason" : "[script] Too many dynamic script compilations within one minute, max: [15/min]; please use on-disk, indexed, or scripts with parameters instead",
"bytes_wanted" : 0,
"bytes_limit" : 0
}
}
}
],
"caused_by" : {
"type" : "general_script_exception",
"reason" : "Failed to compile inline script [\"aaaaaaaaaaaaaaaa\"] using lang [painless]",
"caused_by" : {
"type" : "circuit_breaking_exception",
"reason" : "[script] Too many dynamic script compilations within one minute, max: [15/min]; please use on-disk, indexed, or scripts with parameters instead",
"bytes_wanted" : 0,
"bytes_limit" : 0
}
}
},
"status" : 500
}
```
This also fixes a bug in `ScriptService` where requests being executed
concurrently on a single node could cause a script to be compiled
multiple times (many in the case of a powerful node with many shards)
due to no synchronization between checking the cache and compiling the
script. There is now synchronization so that a script being compiled
will only be compiled once regardless of the number of concurrent
searches on a node.
Relates to #19396
2016-07-26 15:36:29 -04:00
|
|
|
|
2016-05-04 12:17:10 -04:00
|
|
|
========================================
|
|
|
|
|
2017-10-16 08:04:43 -04:00
|
|
|
[float]
|
|
|
|
[[modules-scripting-short-script-form]]
|
|
|
|
=== Short Script Form
|
|
|
|
A short script form can be used for brevity. In the short form, `script` is represented
|
|
|
|
by a string instead of an object. This string contains the source of the script.
|
|
|
|
|
|
|
|
Short form:
|
|
|
|
|
|
|
|
[source,js]
|
|
|
|
----------------------
|
|
|
|
"script": "ctx._source.likes++"
|
|
|
|
----------------------
|
|
|
|
// NOTCONSOLE
|
|
|
|
|
|
|
|
The same script in the normal form:
|
|
|
|
|
|
|
|
[source,js]
|
|
|
|
----------------------
|
|
|
|
"script": {
|
|
|
|
"source": "ctx._source.likes++"
|
|
|
|
}
|
|
|
|
----------------------
|
|
|
|
// NOTCONSOLE
|
|
|
|
|
2016-05-04 12:17:10 -04:00
|
|
|
[float]
|
|
|
|
[[modules-scripting-stored-scripts]]
|
|
|
|
=== Stored Scripts
|
|
|
|
|
|
|
|
Scripts may be stored in and retrieved from the cluster state using the
|
Change Namespace for Stored Script to Only Use Id (#22206)
Currently, stored scripts use a namespace of (lang, id) to be put, get, deleted, and executed. This is not necessary since the lang is stored with the stored script. A user should only have to specify an id to use a stored script. This change makes that possible while keeping backwards compatibility with the previous namespace of (lang, id). Anywhere the previous namespace is used will log deprecation warnings.
The new behavior is the following:
When a user specifies a stored script, that script will be stored under both the new namespace and old namespace.
Take for example script 'A' with lang 'L0' and data 'D0'. If we add script 'A' to the empty set, the scripts map will be ["A" -- D0, "A#L0" -- D0]. If a script 'A' with lang 'L1' and data 'D1' is then added, the scripts map will be ["A" -- D1, "A#L1" -- D1, "A#L0" -- D0].
When a user deletes a stored script, that script will be deleted from both the new namespace (if it exists) and the old namespace.
Take for example a scripts map with {"A" -- D1, "A#L1" -- D1, "A#L0" -- D0}. If a script is removed specified by an id 'A' and lang null then the scripts map will be {"A#L0" -- D0}. To remove the final script, the deprecated namespace must be used, so an id 'A' and lang 'L0' would need to be specified.
When a user gets/executes a stored script, if the new namespace is used then the script will be retrieved/executed using only 'id', and if the old namespace is used then the script will be retrieved/executed using 'id' and 'lang'
2017-01-31 16:27:02 -05:00
|
|
|
`_scripts` end-point.
|
|
|
|
|
|
|
|
==== Request Examples
|
|
|
|
|
2017-04-03 10:14:21 -04:00
|
|
|
The following are examples of using a stored script that lives at
|
|
|
|
`/_scripts/{id}`.
|
2016-05-04 12:17:10 -04:00
|
|
|
|
2017-04-03 10:14:21 -04:00
|
|
|
First, create the script called `calculate-score` in the cluster state:
|
2016-05-04 12:17:10 -04:00
|
|
|
|
|
|
|
[source,js]
|
|
|
|
-----------------------------------
|
Change Namespace for Stored Script to Only Use Id (#22206)
Currently, stored scripts use a namespace of (lang, id) to be put, get, deleted, and executed. This is not necessary since the lang is stored with the stored script. A user should only have to specify an id to use a stored script. This change makes that possible while keeping backwards compatibility with the previous namespace of (lang, id). Anywhere the previous namespace is used will log deprecation warnings.
The new behavior is the following:
When a user specifies a stored script, that script will be stored under both the new namespace and old namespace.
Take for example script 'A' with lang 'L0' and data 'D0'. If we add script 'A' to the empty set, the scripts map will be ["A" -- D0, "A#L0" -- D0]. If a script 'A' with lang 'L1' and data 'D1' is then added, the scripts map will be ["A" -- D1, "A#L1" -- D1, "A#L0" -- D0].
When a user deletes a stored script, that script will be deleted from both the new namespace (if it exists) and the old namespace.
Take for example a scripts map with {"A" -- D1, "A#L1" -- D1, "A#L0" -- D0}. If a script is removed specified by an id 'A' and lang null then the scripts map will be {"A#L0" -- D0}. To remove the final script, the deprecated namespace must be used, so an id 'A' and lang 'L0' would need to be specified.
When a user gets/executes a stored script, if the new namespace is used then the script will be retrieved/executed using only 'id', and if the old namespace is used then the script will be retrieved/executed using 'id' and 'lang'
2017-01-31 16:27:02 -05:00
|
|
|
POST _scripts/calculate-score
|
2016-05-04 12:17:10 -04:00
|
|
|
{
|
Change Namespace for Stored Script to Only Use Id (#22206)
Currently, stored scripts use a namespace of (lang, id) to be put, get, deleted, and executed. This is not necessary since the lang is stored with the stored script. A user should only have to specify an id to use a stored script. This change makes that possible while keeping backwards compatibility with the previous namespace of (lang, id). Anywhere the previous namespace is used will log deprecation warnings.
The new behavior is the following:
When a user specifies a stored script, that script will be stored under both the new namespace and old namespace.
Take for example script 'A' with lang 'L0' and data 'D0'. If we add script 'A' to the empty set, the scripts map will be ["A" -- D0, "A#L0" -- D0]. If a script 'A' with lang 'L1' and data 'D1' is then added, the scripts map will be ["A" -- D1, "A#L1" -- D1, "A#L0" -- D0].
When a user deletes a stored script, that script will be deleted from both the new namespace (if it exists) and the old namespace.
Take for example a scripts map with {"A" -- D1, "A#L1" -- D1, "A#L0" -- D0}. If a script is removed specified by an id 'A' and lang null then the scripts map will be {"A#L0" -- D0}. To remove the final script, the deprecated namespace must be used, so an id 'A' and lang 'L0' would need to be specified.
When a user gets/executes a stored script, if the new namespace is used then the script will be retrieved/executed using only 'id', and if the old namespace is used then the script will be retrieved/executed using 'id' and 'lang'
2017-01-31 16:27:02 -05:00
|
|
|
"script": {
|
|
|
|
"lang": "painless",
|
2017-06-09 11:29:25 -04:00
|
|
|
"source": "Math.log(_score * 2) + params.my_modifier"
|
Change Namespace for Stored Script to Only Use Id (#22206)
Currently, stored scripts use a namespace of (lang, id) to be put, get, deleted, and executed. This is not necessary since the lang is stored with the stored script. A user should only have to specify an id to use a stored script. This change makes that possible while keeping backwards compatibility with the previous namespace of (lang, id). Anywhere the previous namespace is used will log deprecation warnings.
The new behavior is the following:
When a user specifies a stored script, that script will be stored under both the new namespace and old namespace.
Take for example script 'A' with lang 'L0' and data 'D0'. If we add script 'A' to the empty set, the scripts map will be ["A" -- D0, "A#L0" -- D0]. If a script 'A' with lang 'L1' and data 'D1' is then added, the scripts map will be ["A" -- D1, "A#L1" -- D1, "A#L0" -- D0].
When a user deletes a stored script, that script will be deleted from both the new namespace (if it exists) and the old namespace.
Take for example a scripts map with {"A" -- D1, "A#L1" -- D1, "A#L0" -- D0}. If a script is removed specified by an id 'A' and lang null then the scripts map will be {"A#L0" -- D0}. To remove the final script, the deprecated namespace must be used, so an id 'A' and lang 'L0' would need to be specified.
When a user gets/executes a stored script, if the new namespace is used then the script will be retrieved/executed using only 'id', and if the old namespace is used then the script will be retrieved/executed using 'id' and 'lang'
2017-01-31 16:27:02 -05:00
|
|
|
}
|
2016-05-04 12:17:10 -04:00
|
|
|
}
|
|
|
|
-----------------------------------
|
2016-05-09 09:42:23 -04:00
|
|
|
// CONSOLE
|
2016-05-04 12:17:10 -04:00
|
|
|
|
|
|
|
This same script can be retrieved with:
|
|
|
|
|
|
|
|
[source,js]
|
|
|
|
-----------------------------------
|
Change Namespace for Stored Script to Only Use Id (#22206)
Currently, stored scripts use a namespace of (lang, id) to be put, get, deleted, and executed. This is not necessary since the lang is stored with the stored script. A user should only have to specify an id to use a stored script. This change makes that possible while keeping backwards compatibility with the previous namespace of (lang, id). Anywhere the previous namespace is used will log deprecation warnings.
The new behavior is the following:
When a user specifies a stored script, that script will be stored under both the new namespace and old namespace.
Take for example script 'A' with lang 'L0' and data 'D0'. If we add script 'A' to the empty set, the scripts map will be ["A" -- D0, "A#L0" -- D0]. If a script 'A' with lang 'L1' and data 'D1' is then added, the scripts map will be ["A" -- D1, "A#L1" -- D1, "A#L0" -- D0].
When a user deletes a stored script, that script will be deleted from both the new namespace (if it exists) and the old namespace.
Take for example a scripts map with {"A" -- D1, "A#L1" -- D1, "A#L0" -- D0}. If a script is removed specified by an id 'A' and lang null then the scripts map will be {"A#L0" -- D0}. To remove the final script, the deprecated namespace must be used, so an id 'A' and lang 'L0' would need to be specified.
When a user gets/executes a stored script, if the new namespace is used then the script will be retrieved/executed using only 'id', and if the old namespace is used then the script will be retrieved/executed using 'id' and 'lang'
2017-01-31 16:27:02 -05:00
|
|
|
GET _scripts/calculate-score
|
2016-05-04 12:17:10 -04:00
|
|
|
-----------------------------------
|
2016-05-09 09:42:23 -04:00
|
|
|
// CONSOLE
|
2016-04-29 10:42:03 -04:00
|
|
|
// TEST[continued]
|
2016-05-04 12:17:10 -04:00
|
|
|
|
2017-06-09 11:29:25 -04:00
|
|
|
Stored scripts can be used by specifying the `id` parameters as follows:
|
2016-05-04 12:17:10 -04:00
|
|
|
|
|
|
|
[source,js]
|
|
|
|
--------------------------------------------------
|
2016-04-29 10:42:03 -04:00
|
|
|
GET _search
|
2016-05-04 12:17:10 -04:00
|
|
|
{
|
|
|
|
"query": {
|
|
|
|
"script": {
|
|
|
|
"script": {
|
2017-06-09 11:29:25 -04:00
|
|
|
"id": "calculate-score",
|
2016-05-04 12:17:10 -04:00
|
|
|
"params": {
|
|
|
|
"my_modifier": 2
|
|
|
|
}
|
|
|
|
}
|
|
|
|
}
|
|
|
|
}
|
|
|
|
}
|
|
|
|
--------------------------------------------------
|
2016-05-09 09:42:23 -04:00
|
|
|
// CONSOLE
|
2016-04-29 10:42:03 -04:00
|
|
|
// TEST[continued]
|
2016-05-04 12:17:10 -04:00
|
|
|
|
2016-04-29 10:42:03 -04:00
|
|
|
And deleted with:
|
|
|
|
|
|
|
|
[source,js]
|
|
|
|
-----------------------------------
|
Change Namespace for Stored Script to Only Use Id (#22206)
Currently, stored scripts use a namespace of (lang, id) to be put, get, deleted, and executed. This is not necessary since the lang is stored with the stored script. A user should only have to specify an id to use a stored script. This change makes that possible while keeping backwards compatibility with the previous namespace of (lang, id). Anywhere the previous namespace is used will log deprecation warnings.
The new behavior is the following:
When a user specifies a stored script, that script will be stored under both the new namespace and old namespace.
Take for example script 'A' with lang 'L0' and data 'D0'. If we add script 'A' to the empty set, the scripts map will be ["A" -- D0, "A#L0" -- D0]. If a script 'A' with lang 'L1' and data 'D1' is then added, the scripts map will be ["A" -- D1, "A#L1" -- D1, "A#L0" -- D0].
When a user deletes a stored script, that script will be deleted from both the new namespace (if it exists) and the old namespace.
Take for example a scripts map with {"A" -- D1, "A#L1" -- D1, "A#L0" -- D0}. If a script is removed specified by an id 'A' and lang null then the scripts map will be {"A#L0" -- D0}. To remove the final script, the deprecated namespace must be used, so an id 'A' and lang 'L0' would need to be specified.
When a user gets/executes a stored script, if the new namespace is used then the script will be retrieved/executed using only 'id', and if the old namespace is used then the script will be retrieved/executed using 'id' and 'lang'
2017-01-31 16:27:02 -05:00
|
|
|
DELETE _scripts/calculate-score
|
2016-04-29 10:42:03 -04:00
|
|
|
-----------------------------------
|
2016-05-09 09:42:23 -04:00
|
|
|
// CONSOLE
|
2016-04-29 10:42:03 -04:00
|
|
|
// TEST[continued]
|
2016-05-04 12:17:10 -04:00
|
|
|
|
2016-06-15 17:57:18 -04:00
|
|
|
[float]
|
|
|
|
[[modules-scripting-using-caching]]
|
|
|
|
=== Script Caching
|
|
|
|
|
|
|
|
All scripts are cached by default so that they only need to be recompiled
|
2017-09-11 14:39:29 -04:00
|
|
|
when updates occur. By default, scripts do not have a time-based expiration, but
|
2016-06-15 17:57:18 -04:00
|
|
|
you can change this behavior by using the `script.cache.expire` setting.
|
|
|
|
You can configure the size of this cache by using the `script.cache.max_size` setting.
|
|
|
|
By default, the cache size is `100`.
|
|
|
|
|
2016-05-04 12:17:10 -04:00
|
|
|
NOTE: The size of stored scripts is limited to 65,535 bytes. This can be
|
|
|
|
changed by setting `script.max_size_in_bytes` setting to increase that soft
|
2017-09-11 14:39:29 -04:00
|
|
|
limit, but if scripts are really large then a
|
|
|
|
<<modules-scripting-engine,native script engine>> should be considered.
|