OpenSearch/docs/reference/modules/scripting/using.asciidoc

261 lines
7.8 KiB
Plaintext
Raw Normal View History

[[modules-scripting-using]]
=== How to use scripts
Wherever scripting is supported in the Elasticsearch API, the syntax follows
the same pattern:
[source,js]
-------------------------------------
"script": {
"lang": "...", <1>
"inline" | "stored" | "file": "...", <2>
"params": { ... } <3>
}
-------------------------------------
2016-08-30 19:51:37 -04:00
<1> The language the script is written in, which defaults to `painless`.
<2> The script itself which may be specified as `inline`, `stored`, or `file`.
<3> Any named parameters that should be passed into the script.
For example, the following script is used in a search request to return a
<<search-request-script-fields, scripted field>>:
[source,js]
-------------------------------------
PUT my_index/my_type/1
{
"my_field": 5
}
GET my_index/_search
{
"script_fields": {
"my_doubled_field": {
"script": {
"lang": "expression",
"inline": "doc['my_field'] * multiplier",
"params": {
"multiplier": 2
}
}
}
}
}
-------------------------------------
// CONSOLE
[float]
=== Script Parameters
`lang`::
Specifies the language the script is written in. Defaults to `groovy` but
may be set to any of languages listed in <<modules-scripting>>. The
default language may be changed in the `elasticsearch.yml` config file by
setting `script.default_lang` to the appropriate language.
`inline`, `id`, `file`::
Specifies the source of the script. An `inline` script is specified
`inline` as in the example above, a stored script with the specified `id`
is retrieved from the cluster state (see <<modules-scripting-stored-scripts,Stored Scripts>>),
and a `file` script is retrieved from a file in the `config/scripts`
directory (see <<modules-scripting-file-scripts, File Scripts>>).
+
While languages like `expression` and `painless` can be used out of the box as
inline or stored scripts, other languages like `groovy` can only be
specified as `file` unless you first adjust the default
<<modules-scripting-security,scripting security settings>>.
`params`::
Specifies any named parameters that are passed into the script as
variables.
[IMPORTANT]
[[prefer-params]]
.Prefer parameters
========================================
The first time Elasticsearch sees a new script, it compiles it and stores the
compiled version in a cache. Compilation can be a heavy process.
If you need to pass variables into the script, you should pass them in as
named `params` instead of hard-coding values into the script itself. For
example, if you want to be able to multiply a field value by different
multipliers, don't hard-code the multiplier into the script:
[source,js]
----------------------
"inline": "doc['my_field'] * 2"
----------------------
Instead, pass it in as a named parameter:
[source,js]
----------------------
"inline": "doc['my_field'] * multiplier",
"params": {
"multiplier": 2
}
----------------------
The first version has to be recompiled every time the multiplier changes. The
second version is only compiled once.
Circuit break the number of inline scripts compiled per minute When compiling many dynamically changing scripts, parameterized scripts (<https://www.elastic.co/guide/en/elasticsearch/reference/master/modules-scripting-using.html#prefer-params>) should be preferred. This enforces a limit to the number of scripts that can be compiled within a minute. A new dynamic setting is added - `script.max_compilations_per_minute`, which defaults to 15. If more dynamic scripts are sent, a user will get the following exception: ```json { "error" : { "root_cause" : [ { "type" : "circuit_breaking_exception", "reason" : "[script] Too many dynamic script compilations within one minute, max: [15/min]; please use on-disk, indexed, or scripts with parameters instead", "bytes_wanted" : 0, "bytes_limit" : 0 } ], "type" : "search_phase_execution_exception", "reason" : "all shards failed", "phase" : "query", "grouped" : true, "failed_shards" : [ { "shard" : 0, "index" : "i", "node" : "a5V1eXcZRYiIk8lecjZ4Jw", "reason" : { "type" : "general_script_exception", "reason" : "Failed to compile inline script [\"aaaaaaaaaaaaaaaa\"] using lang [painless]", "caused_by" : { "type" : "circuit_breaking_exception", "reason" : "[script] Too many dynamic script compilations within one minute, max: [15/min]; please use on-disk, indexed, or scripts with parameters instead", "bytes_wanted" : 0, "bytes_limit" : 0 } } } ], "caused_by" : { "type" : "general_script_exception", "reason" : "Failed to compile inline script [\"aaaaaaaaaaaaaaaa\"] using lang [painless]", "caused_by" : { "type" : "circuit_breaking_exception", "reason" : "[script] Too many dynamic script compilations within one minute, max: [15/min]; please use on-disk, indexed, or scripts with parameters instead", "bytes_wanted" : 0, "bytes_limit" : 0 } } }, "status" : 500 } ``` This also fixes a bug in `ScriptService` where requests being executed concurrently on a single node could cause a script to be compiled multiple times (many in the case of a powerful node with many shards) due to no synchronization between checking the cache and compiling the script. There is now synchronization so that a script being compiled will only be compiled once regardless of the number of concurrent searches on a node. Relates to #19396
2016-07-26 15:36:29 -04:00
If you compile too many unique scripts within a small amount of time,
Elasticsearch will reject the new dynamic scripts with a
`circuit_breaking_exception` error. By default, up to 15 inline scripts per
minute will be compiled. You can change this setting dynamically by setting
`script.max_compilations_per_minute`.
========================================
[float]
[[modules-scripting-file-scripts]]
=== File-based Scripts
To increase security, non-sandboxed languages can only be specified in script
files stored on every node in the cluster. File scripts must be saved in the
`scripts` directory whose default location depends on whether you use the
<<zip-targz-layout,`zip`/`tar.gz`>> (`$ES_HOME/config/scripts/`),
<<rpm-layout,RPM>>, or <<deb-layout,Debian>> package. The default may be
changed with the `path.scripts` setting.
The languages which are assumed to be safe by default are: `painless`,
`expression`, and `mustache` (used for search and query templates).
Any files placed in the `scripts` directory will be compiled automatically
when the node starts up and then <<reload-scripts,every 60 seconds thereafter>>.
The file should be named as follows: `{script-name}.{lang}`. For instance,
the following example creates a Groovy script called `calculate-score`:
[source,sh]
--------------------------------------------------
cat "log(_score * 2) + my_modifier" > config/scripts/calculate-score.groovy
--------------------------------------------------
This script can be used as follows:
[source,js]
--------------------------------------------------
GET my_index/_search
{
"query": {
"script": {
"script": {
"lang": "groovy", <1>
"file": "calculate-score", <2>
"params": {
"my_modifier": 2
}
}
}
}
}
--------------------------------------------------
<1> The language of the script, which should correspond with the script file suffix.
<2> The name of the script, which should be the name of the file.
The `script` directory may contain sub-directories, in which case the
hierarchy of directories is flattened and concatenated with underscores. A
script in `group1/group2/my_script.groovy` should use `group1_group2_myscript`
as the `file` name.
[[reload-scripts]]
[float]
==== Automatic script reloading
The `scripts` directory will be rescanned every `60s` (configurable with the
`resource.reload.interval` setting) and new, changed, or removed scripts will
be compiled, updated, or deleted from the script cache.
Script reloading can be completely disabled by setting
`script.auto_reload_enabled` to `false`.
[float]
[[modules-scripting-stored-scripts]]
=== Stored Scripts
Scripts may be stored in and retrieved from the cluster state using the
`_scripts` end-point:
[source,js]
-----------------------------------
/_scripts/{lang}/{id} <1> <2>
-----------------------------------
<1> The `lang` represents the script language.
<2> The `id` is a unique identifier or script name.
This example stores a Groovy script called `calculate-score` in the cluster
state:
[source,js]
-----------------------------------
POST _scripts/groovy/calculate-score
{
"script": "log(_score * 2) + my_modifier"
}
-----------------------------------
// CONSOLE
This same script can be retrieved with:
[source,js]
-----------------------------------
GET _scripts/groovy/calculate-score
-----------------------------------
// CONSOLE
// TEST[continued]
Stored scripts can be used by specifying the `lang` and `stored` parameters as follows:
[source,js]
--------------------------------------------------
GET _search
{
"query": {
"script": {
"script": {
"lang": "groovy",
"stored": "calculate-score",
"params": {
"my_modifier": 2
}
}
}
}
}
--------------------------------------------------
// CONSOLE
// TEST[continued]
And deleted with:
[source,js]
-----------------------------------
DELETE _scripts/groovy/calculate-score
-----------------------------------
// CONSOLE
// TEST[continued]
[float]
[[modules-scripting-using-caching]]
=== Script Caching
All scripts are cached by default so that they only need to be recompiled
when updates occur. File scripts keep a static cache and will always reside
in memory. Both inline and stored scripts are stored in a cache that can evict
residing scripts. By default, scripts do not have a time-based expiration, but
you can change this behavior by using the `script.cache.expire` setting.
You can configure the size of this cache by using the `script.cache.max_size` setting.
By default, the cache size is `100`.
NOTE: The size of stored scripts is limited to 65,535 bytes. This can be
changed by setting `script.max_size_in_bytes` setting to increase that soft
limit, but if scripts are really large then alternatives like
<<modules-scripting-native,native>> scripts should be considered instead.