OpenSearch/docs/reference/modules/scripting/using.asciidoc

287 lines
8.8 KiB
Plaintext
Raw Normal View History

[[modules-scripting-using]]
=== How to use scripts
Wherever scripting is supported in the Elasticsearch API, the syntax follows
the same pattern:
[source,js]
-------------------------------------
"script": {
"lang": "...", <1>
"inline" | "stored" | "file": "...", <2>
"params": { ... } <3>
}
-------------------------------------
2016-08-30 19:51:37 -04:00
<1> The language the script is written in, which defaults to `painless`.
<2> The script itself which may be specified as `inline`, `stored`, or `file`.
<3> Any named parameters that should be passed into the script.
For example, the following script is used in a search request to return a
<<search-request-script-fields, scripted field>>:
[source,js]
-------------------------------------
PUT my_index/my_type/1
{
"my_field": 5
}
GET my_index/_search
{
"script_fields": {
"my_doubled_field": {
"script": {
"lang": "expression",
"inline": "doc['my_field'] * multiplier",
"params": {
"multiplier": 2
}
}
}
}
}
-------------------------------------
// CONSOLE
[float]
=== Script Parameters
`lang`::
Specifies the language the script is written in. Defaults to `painless` but
may be set to any of languages listed in <<modules-scripting>>. The
default language may be changed in the `elasticsearch.yml` config file by
setting `script.default_lang` to the appropriate language.
Change Namespace for Stored Script to Only Use Id (#22206) Currently, stored scripts use a namespace of (lang, id) to be put, get, deleted, and executed. This is not necessary since the lang is stored with the stored script. A user should only have to specify an id to use a stored script. This change makes that possible while keeping backwards compatibility with the previous namespace of (lang, id). Anywhere the previous namespace is used will log deprecation warnings. The new behavior is the following: When a user specifies a stored script, that script will be stored under both the new namespace and old namespace. Take for example script 'A' with lang 'L0' and data 'D0'. If we add script 'A' to the empty set, the scripts map will be ["A" -- D0, "A#L0" -- D0]. If a script 'A' with lang 'L1' and data 'D1' is then added, the scripts map will be ["A" -- D1, "A#L1" -- D1, "A#L0" -- D0]. When a user deletes a stored script, that script will be deleted from both the new namespace (if it exists) and the old namespace. Take for example a scripts map with {"A" -- D1, "A#L1" -- D1, "A#L0" -- D0}. If a script is removed specified by an id 'A' and lang null then the scripts map will be {"A#L0" -- D0}. To remove the final script, the deprecated namespace must be used, so an id 'A' and lang 'L0' would need to be specified. When a user gets/executes a stored script, if the new namespace is used then the script will be retrieved/executed using only 'id', and if the old namespace is used then the script will be retrieved/executed using 'id' and 'lang'
2017-01-31 16:27:02 -05:00
`inline`, `stored`, `file`::
Specifies the source of the script. An `inline` script is specified
Change Namespace for Stored Script to Only Use Id (#22206) Currently, stored scripts use a namespace of (lang, id) to be put, get, deleted, and executed. This is not necessary since the lang is stored with the stored script. A user should only have to specify an id to use a stored script. This change makes that possible while keeping backwards compatibility with the previous namespace of (lang, id). Anywhere the previous namespace is used will log deprecation warnings. The new behavior is the following: When a user specifies a stored script, that script will be stored under both the new namespace and old namespace. Take for example script 'A' with lang 'L0' and data 'D0'. If we add script 'A' to the empty set, the scripts map will be ["A" -- D0, "A#L0" -- D0]. If a script 'A' with lang 'L1' and data 'D1' is then added, the scripts map will be ["A" -- D1, "A#L1" -- D1, "A#L0" -- D0]. When a user deletes a stored script, that script will be deleted from both the new namespace (if it exists) and the old namespace. Take for example a scripts map with {"A" -- D1, "A#L1" -- D1, "A#L0" -- D0}. If a script is removed specified by an id 'A' and lang null then the scripts map will be {"A#L0" -- D0}. To remove the final script, the deprecated namespace must be used, so an id 'A' and lang 'L0' would need to be specified. When a user gets/executes a stored script, if the new namespace is used then the script will be retrieved/executed using only 'id', and if the old namespace is used then the script will be retrieved/executed using 'id' and 'lang'
2017-01-31 16:27:02 -05:00
`inline` as in the example above, a `stored` script is specified `stored`
and is retrieved from the cluster state (see <<modules-scripting-stored-scripts,Stored Scripts>>),
and a `file` script is retrieved from a file in the `config/scripts`
directory (see <<modules-scripting-file-scripts, File Scripts>>).
+
While languages like `expression` and `painless` can be used out of the box as
inline or stored scripts, other languages can only be
specified as `file` unless you first adjust the default
<<modules-scripting-security,scripting security settings>>.
`params`::
Specifies any named parameters that are passed into the script as
variables.
[IMPORTANT]
[[prefer-params]]
.Prefer parameters
========================================
The first time Elasticsearch sees a new script, it compiles it and stores the
compiled version in a cache. Compilation can be a heavy process.
If you need to pass variables into the script, you should pass them in as
named `params` instead of hard-coding values into the script itself. For
example, if you want to be able to multiply a field value by different
multipliers, don't hard-code the multiplier into the script:
[source,js]
----------------------
"inline": "doc['my_field'] * 2"
----------------------
Instead, pass it in as a named parameter:
[source,js]
----------------------
"inline": "doc['my_field'] * multiplier",
"params": {
"multiplier": 2
}
----------------------
The first version has to be recompiled every time the multiplier changes. The
second version is only compiled once.
Circuit break the number of inline scripts compiled per minute When compiling many dynamically changing scripts, parameterized scripts (<https://www.elastic.co/guide/en/elasticsearch/reference/master/modules-scripting-using.html#prefer-params>) should be preferred. This enforces a limit to the number of scripts that can be compiled within a minute. A new dynamic setting is added - `script.max_compilations_per_minute`, which defaults to 15. If more dynamic scripts are sent, a user will get the following exception: ```json { "error" : { "root_cause" : [ { "type" : "circuit_breaking_exception", "reason" : "[script] Too many dynamic script compilations within one minute, max: [15/min]; please use on-disk, indexed, or scripts with parameters instead", "bytes_wanted" : 0, "bytes_limit" : 0 } ], "type" : "search_phase_execution_exception", "reason" : "all shards failed", "phase" : "query", "grouped" : true, "failed_shards" : [ { "shard" : 0, "index" : "i", "node" : "a5V1eXcZRYiIk8lecjZ4Jw", "reason" : { "type" : "general_script_exception", "reason" : "Failed to compile inline script [\"aaaaaaaaaaaaaaaa\"] using lang [painless]", "caused_by" : { "type" : "circuit_breaking_exception", "reason" : "[script] Too many dynamic script compilations within one minute, max: [15/min]; please use on-disk, indexed, or scripts with parameters instead", "bytes_wanted" : 0, "bytes_limit" : 0 } } } ], "caused_by" : { "type" : "general_script_exception", "reason" : "Failed to compile inline script [\"aaaaaaaaaaaaaaaa\"] using lang [painless]", "caused_by" : { "type" : "circuit_breaking_exception", "reason" : "[script] Too many dynamic script compilations within one minute, max: [15/min]; please use on-disk, indexed, or scripts with parameters instead", "bytes_wanted" : 0, "bytes_limit" : 0 } } }, "status" : 500 } ``` This also fixes a bug in `ScriptService` where requests being executed concurrently on a single node could cause a script to be compiled multiple times (many in the case of a powerful node with many shards) due to no synchronization between checking the cache and compiling the script. There is now synchronization so that a script being compiled will only be compiled once regardless of the number of concurrent searches on a node. Relates to #19396
2016-07-26 15:36:29 -04:00
If you compile too many unique scripts within a small amount of time,
Elasticsearch will reject the new dynamic scripts with a
`circuit_breaking_exception` error. By default, up to 15 inline scripts per
minute will be compiled. You can change this setting dynamically by setting
`script.max_compilations_per_minute`.
========================================
[float]
[[modules-scripting-file-scripts]]
=== File-based Scripts
To increase security, non-sandboxed languages can only be specified in script
files stored on every node in the cluster. File scripts must be saved in the
`scripts` directory whose default location depends on whether you use the
<<zip-targz-layout,`zip`/`tar.gz`>> (`$ES_HOME/config/scripts/`),
<<rpm-layout,RPM>>, or <<deb-layout,Debian>> package. The default may be
changed with the `path.scripts` setting.
The languages which are assumed to be safe by default are: `painless`,
`expression`, and `mustache` (used for search and query templates).
Any files placed in the `scripts` directory will be compiled automatically
when the node starts up and then <<reload-scripts,every 60 seconds thereafter>>.
The file should be named as follows: `{script-name}.{lang}`. For instance,
the following example creates a Groovy script called `calculate-score`:
[source,sh]
--------------------------------------------------
cat "Math.log(_score * 2) + my_modifier" > config/scripts/calculate-score.painless
--------------------------------------------------
This script can be used as follows:
[source,js]
--------------------------------------------------
GET my_index/_search
{
"query": {
"script": {
"script": {
"lang": "painless", <1>
"file": "calculate-score", <2>
"params": {
"my_modifier": 2
}
}
}
}
}
--------------------------------------------------
<1> The language of the script, which should correspond with the script file suffix.
<2> The name of the script, which should be the name of the file.
The `script` directory may contain sub-directories, in which case the
hierarchy of directories is flattened and concatenated with underscores. A
script in `group1/group2/my_script.painless` should use `group1_group2_myscript`
as the `file` name.
[[reload-scripts]]
[float]
==== Automatic script reloading
The `scripts` directory will be rescanned every `60s` (configurable with the
`resource.reload.interval` setting) and new, changed, or removed scripts will
be compiled, updated, or deleted from the script cache.
Script reloading can be completely disabled by setting
`script.auto_reload_enabled` to `false`.
[float]
[[modules-scripting-stored-scripts]]
=== Stored Scripts
Scripts may be stored in and retrieved from the cluster state using the
Change Namespace for Stored Script to Only Use Id (#22206) Currently, stored scripts use a namespace of (lang, id) to be put, get, deleted, and executed. This is not necessary since the lang is stored with the stored script. A user should only have to specify an id to use a stored script. This change makes that possible while keeping backwards compatibility with the previous namespace of (lang, id). Anywhere the previous namespace is used will log deprecation warnings. The new behavior is the following: When a user specifies a stored script, that script will be stored under both the new namespace and old namespace. Take for example script 'A' with lang 'L0' and data 'D0'. If we add script 'A' to the empty set, the scripts map will be ["A" -- D0, "A#L0" -- D0]. If a script 'A' with lang 'L1' and data 'D1' is then added, the scripts map will be ["A" -- D1, "A#L1" -- D1, "A#L0" -- D0]. When a user deletes a stored script, that script will be deleted from both the new namespace (if it exists) and the old namespace. Take for example a scripts map with {"A" -- D1, "A#L1" -- D1, "A#L0" -- D0}. If a script is removed specified by an id 'A' and lang null then the scripts map will be {"A#L0" -- D0}. To remove the final script, the deprecated namespace must be used, so an id 'A' and lang 'L0' would need to be specified. When a user gets/executes a stored script, if the new namespace is used then the script will be retrieved/executed using only 'id', and if the old namespace is used then the script will be retrieved/executed using 'id' and 'lang'
2017-01-31 16:27:02 -05:00
`_scripts` end-point.
==== Deprecated Namespace
The namespace for stored scripts using both `lang` and `id` as a unique
identifier has been deprecated. The new namespace for stored scripts will
only use `id`. Stored scripts with the same `id`, but different `lang`'s
will no longer be allowed in 6.0. To comply with the new namespace for
stored scripts, existing stored scripts should be deleted and put again.
Any scripts that share an `id` but have different `lang`s will need to
be re-named. For example, take the following:
"id": "example", "lang": "painless"
"id": "example", "lang": "expressions"
The above scripts will conflict under the new namespace since the id's are
the same. At least one will have to be re-named to comply with the new
namespace of only `id`.
As a final caveat, stored search templates and stored scripts share
the same namespace, so if a search template has the same `id` as a
stored script, one of the two will have to be re-named as well using
delete and put requests.
==== Request Examples
The following are examples of stored script requests:
[source,js]
-----------------------------------
Change Namespace for Stored Script to Only Use Id (#22206) Currently, stored scripts use a namespace of (lang, id) to be put, get, deleted, and executed. This is not necessary since the lang is stored with the stored script. A user should only have to specify an id to use a stored script. This change makes that possible while keeping backwards compatibility with the previous namespace of (lang, id). Anywhere the previous namespace is used will log deprecation warnings. The new behavior is the following: When a user specifies a stored script, that script will be stored under both the new namespace and old namespace. Take for example script 'A' with lang 'L0' and data 'D0'. If we add script 'A' to the empty set, the scripts map will be ["A" -- D0, "A#L0" -- D0]. If a script 'A' with lang 'L1' and data 'D1' is then added, the scripts map will be ["A" -- D1, "A#L1" -- D1, "A#L0" -- D0]. When a user deletes a stored script, that script will be deleted from both the new namespace (if it exists) and the old namespace. Take for example a scripts map with {"A" -- D1, "A#L1" -- D1, "A#L0" -- D0}. If a script is removed specified by an id 'A' and lang null then the scripts map will be {"A#L0" -- D0}. To remove the final script, the deprecated namespace must be used, so an id 'A' and lang 'L0' would need to be specified. When a user gets/executes a stored script, if the new namespace is used then the script will be retrieved/executed using only 'id', and if the old namespace is used then the script will be retrieved/executed using 'id' and 'lang'
2017-01-31 16:27:02 -05:00
/_scripts/{id} <1>
-----------------------------------
Change Namespace for Stored Script to Only Use Id (#22206) Currently, stored scripts use a namespace of (lang, id) to be put, get, deleted, and executed. This is not necessary since the lang is stored with the stored script. A user should only have to specify an id to use a stored script. This change makes that possible while keeping backwards compatibility with the previous namespace of (lang, id). Anywhere the previous namespace is used will log deprecation warnings. The new behavior is the following: When a user specifies a stored script, that script will be stored under both the new namespace and old namespace. Take for example script 'A' with lang 'L0' and data 'D0'. If we add script 'A' to the empty set, the scripts map will be ["A" -- D0, "A#L0" -- D0]. If a script 'A' with lang 'L1' and data 'D1' is then added, the scripts map will be ["A" -- D1, "A#L1" -- D1, "A#L0" -- D0]. When a user deletes a stored script, that script will be deleted from both the new namespace (if it exists) and the old namespace. Take for example a scripts map with {"A" -- D1, "A#L1" -- D1, "A#L0" -- D0}. If a script is removed specified by an id 'A' and lang null then the scripts map will be {"A#L0" -- D0}. To remove the final script, the deprecated namespace must be used, so an id 'A' and lang 'L0' would need to be specified. When a user gets/executes a stored script, if the new namespace is used then the script will be retrieved/executed using only 'id', and if the old namespace is used then the script will be retrieved/executed using 'id' and 'lang'
2017-01-31 16:27:02 -05:00
<1> The `id` is a unique identifier for the stored script.
This example stores a Painless script called `calculate-score` in the cluster
state:
[source,js]
-----------------------------------
Change Namespace for Stored Script to Only Use Id (#22206) Currently, stored scripts use a namespace of (lang, id) to be put, get, deleted, and executed. This is not necessary since the lang is stored with the stored script. A user should only have to specify an id to use a stored script. This change makes that possible while keeping backwards compatibility with the previous namespace of (lang, id). Anywhere the previous namespace is used will log deprecation warnings. The new behavior is the following: When a user specifies a stored script, that script will be stored under both the new namespace and old namespace. Take for example script 'A' with lang 'L0' and data 'D0'. If we add script 'A' to the empty set, the scripts map will be ["A" -- D0, "A#L0" -- D0]. If a script 'A' with lang 'L1' and data 'D1' is then added, the scripts map will be ["A" -- D1, "A#L1" -- D1, "A#L0" -- D0]. When a user deletes a stored script, that script will be deleted from both the new namespace (if it exists) and the old namespace. Take for example a scripts map with {"A" -- D1, "A#L1" -- D1, "A#L0" -- D0}. If a script is removed specified by an id 'A' and lang null then the scripts map will be {"A#L0" -- D0}. To remove the final script, the deprecated namespace must be used, so an id 'A' and lang 'L0' would need to be specified. When a user gets/executes a stored script, if the new namespace is used then the script will be retrieved/executed using only 'id', and if the old namespace is used then the script will be retrieved/executed using 'id' and 'lang'
2017-01-31 16:27:02 -05:00
POST _scripts/calculate-score
{
Change Namespace for Stored Script to Only Use Id (#22206) Currently, stored scripts use a namespace of (lang, id) to be put, get, deleted, and executed. This is not necessary since the lang is stored with the stored script. A user should only have to specify an id to use a stored script. This change makes that possible while keeping backwards compatibility with the previous namespace of (lang, id). Anywhere the previous namespace is used will log deprecation warnings. The new behavior is the following: When a user specifies a stored script, that script will be stored under both the new namespace and old namespace. Take for example script 'A' with lang 'L0' and data 'D0'. If we add script 'A' to the empty set, the scripts map will be ["A" -- D0, "A#L0" -- D0]. If a script 'A' with lang 'L1' and data 'D1' is then added, the scripts map will be ["A" -- D1, "A#L1" -- D1, "A#L0" -- D0]. When a user deletes a stored script, that script will be deleted from both the new namespace (if it exists) and the old namespace. Take for example a scripts map with {"A" -- D1, "A#L1" -- D1, "A#L0" -- D0}. If a script is removed specified by an id 'A' and lang null then the scripts map will be {"A#L0" -- D0}. To remove the final script, the deprecated namespace must be used, so an id 'A' and lang 'L0' would need to be specified. When a user gets/executes a stored script, if the new namespace is used then the script will be retrieved/executed using only 'id', and if the old namespace is used then the script will be retrieved/executed using 'id' and 'lang'
2017-01-31 16:27:02 -05:00
"script": {
"lang": "painless",
"code": "Math.log(_score * 2) + params.my_modifier"
}
}
-----------------------------------
// CONSOLE
This same script can be retrieved with:
[source,js]
-----------------------------------
Change Namespace for Stored Script to Only Use Id (#22206) Currently, stored scripts use a namespace of (lang, id) to be put, get, deleted, and executed. This is not necessary since the lang is stored with the stored script. A user should only have to specify an id to use a stored script. This change makes that possible while keeping backwards compatibility with the previous namespace of (lang, id). Anywhere the previous namespace is used will log deprecation warnings. The new behavior is the following: When a user specifies a stored script, that script will be stored under both the new namespace and old namespace. Take for example script 'A' with lang 'L0' and data 'D0'. If we add script 'A' to the empty set, the scripts map will be ["A" -- D0, "A#L0" -- D0]. If a script 'A' with lang 'L1' and data 'D1' is then added, the scripts map will be ["A" -- D1, "A#L1" -- D1, "A#L0" -- D0]. When a user deletes a stored script, that script will be deleted from both the new namespace (if it exists) and the old namespace. Take for example a scripts map with {"A" -- D1, "A#L1" -- D1, "A#L0" -- D0}. If a script is removed specified by an id 'A' and lang null then the scripts map will be {"A#L0" -- D0}. To remove the final script, the deprecated namespace must be used, so an id 'A' and lang 'L0' would need to be specified. When a user gets/executes a stored script, if the new namespace is used then the script will be retrieved/executed using only 'id', and if the old namespace is used then the script will be retrieved/executed using 'id' and 'lang'
2017-01-31 16:27:02 -05:00
GET _scripts/calculate-score
-----------------------------------
// CONSOLE
// TEST[continued]
Change Namespace for Stored Script to Only Use Id (#22206) Currently, stored scripts use a namespace of (lang, id) to be put, get, deleted, and executed. This is not necessary since the lang is stored with the stored script. A user should only have to specify an id to use a stored script. This change makes that possible while keeping backwards compatibility with the previous namespace of (lang, id). Anywhere the previous namespace is used will log deprecation warnings. The new behavior is the following: When a user specifies a stored script, that script will be stored under both the new namespace and old namespace. Take for example script 'A' with lang 'L0' and data 'D0'. If we add script 'A' to the empty set, the scripts map will be ["A" -- D0, "A#L0" -- D0]. If a script 'A' with lang 'L1' and data 'D1' is then added, the scripts map will be ["A" -- D1, "A#L1" -- D1, "A#L0" -- D0]. When a user deletes a stored script, that script will be deleted from both the new namespace (if it exists) and the old namespace. Take for example a scripts map with {"A" -- D1, "A#L1" -- D1, "A#L0" -- D0}. If a script is removed specified by an id 'A' and lang null then the scripts map will be {"A#L0" -- D0}. To remove the final script, the deprecated namespace must be used, so an id 'A' and lang 'L0' would need to be specified. When a user gets/executes a stored script, if the new namespace is used then the script will be retrieved/executed using only 'id', and if the old namespace is used then the script will be retrieved/executed using 'id' and 'lang'
2017-01-31 16:27:02 -05:00
Stored scripts can be used by specifying the `stored` parameters as follows:
[source,js]
--------------------------------------------------
GET _search
{
"query": {
"script": {
"script": {
"stored": "calculate-score",
"params": {
"my_modifier": 2
}
}
}
}
}
--------------------------------------------------
// CONSOLE
// TEST[continued]
And deleted with:
[source,js]
-----------------------------------
Change Namespace for Stored Script to Only Use Id (#22206) Currently, stored scripts use a namespace of (lang, id) to be put, get, deleted, and executed. This is not necessary since the lang is stored with the stored script. A user should only have to specify an id to use a stored script. This change makes that possible while keeping backwards compatibility with the previous namespace of (lang, id). Anywhere the previous namespace is used will log deprecation warnings. The new behavior is the following: When a user specifies a stored script, that script will be stored under both the new namespace and old namespace. Take for example script 'A' with lang 'L0' and data 'D0'. If we add script 'A' to the empty set, the scripts map will be ["A" -- D0, "A#L0" -- D0]. If a script 'A' with lang 'L1' and data 'D1' is then added, the scripts map will be ["A" -- D1, "A#L1" -- D1, "A#L0" -- D0]. When a user deletes a stored script, that script will be deleted from both the new namespace (if it exists) and the old namespace. Take for example a scripts map with {"A" -- D1, "A#L1" -- D1, "A#L0" -- D0}. If a script is removed specified by an id 'A' and lang null then the scripts map will be {"A#L0" -- D0}. To remove the final script, the deprecated namespace must be used, so an id 'A' and lang 'L0' would need to be specified. When a user gets/executes a stored script, if the new namespace is used then the script will be retrieved/executed using only 'id', and if the old namespace is used then the script will be retrieved/executed using 'id' and 'lang'
2017-01-31 16:27:02 -05:00
DELETE _scripts/calculate-score
-----------------------------------
// CONSOLE
// TEST[continued]
[float]
[[modules-scripting-using-caching]]
=== Script Caching
All scripts are cached by default so that they only need to be recompiled
when updates occur. File scripts keep a static cache and will always reside
in memory. Both inline and stored scripts are stored in a cache that can evict
residing scripts. By default, scripts do not have a time-based expiration, but
you can change this behavior by using the `script.cache.expire` setting.
You can configure the size of this cache by using the `script.cache.max_size` setting.
By default, the cache size is `100`.
NOTE: The size of stored scripts is limited to 65,535 bytes. This can be
changed by setting `script.max_size_in_bytes` setting to increase that soft
limit, but if scripts are really large then alternatives like
<<modules-scripting-native,native>> scripts should be considered instead.