2016-05-04 12:17:10 -04:00
|
|
|
[[modules-scripting-using]]
|
|
|
|
=== How to use scripts
|
|
|
|
|
|
|
|
Wherever scripting is supported in the Elasticsearch API, the syntax follows
|
|
|
|
the same pattern:
|
|
|
|
|
|
|
|
[source,js]
|
|
|
|
-------------------------------------
|
|
|
|
"script": {
|
|
|
|
"lang": "...", <1>
|
|
|
|
"inline" | "id" | "file": "...", <2>
|
|
|
|
"params": { ... } <3>
|
|
|
|
}
|
|
|
|
-------------------------------------
|
2016-08-30 19:51:37 -04:00
|
|
|
<1> The language the script is written in, which defaults to `painless`.
|
2016-05-04 12:17:10 -04:00
|
|
|
<2> The script itself which may be specfied as `inline`, `id`, or `file`.
|
|
|
|
<3> Any named parameters that should be passed into the script.
|
|
|
|
|
|
|
|
For example, the following script is used in a search request to return a
|
|
|
|
<<search-request-script-fields, scripted field>>:
|
|
|
|
|
|
|
|
[source,js]
|
|
|
|
-------------------------------------
|
|
|
|
PUT my_index/my_type/1
|
|
|
|
{
|
|
|
|
"my_field": 5
|
|
|
|
}
|
|
|
|
|
|
|
|
GET my_index/_search
|
|
|
|
{
|
|
|
|
"script_fields": {
|
|
|
|
"my_doubled_field": {
|
|
|
|
"script": {
|
|
|
|
"lang": "expression",
|
|
|
|
"inline": "doc['my_field'] * multiplier",
|
|
|
|
"params": {
|
|
|
|
"multiplier": 2
|
|
|
|
}
|
|
|
|
}
|
|
|
|
}
|
|
|
|
}
|
|
|
|
}
|
|
|
|
-------------------------------------
|
2016-05-09 09:42:23 -04:00
|
|
|
// CONSOLE
|
2016-05-04 12:17:10 -04:00
|
|
|
|
|
|
|
[float]
|
|
|
|
=== Script Parameters
|
|
|
|
|
|
|
|
`lang`::
|
|
|
|
|
|
|
|
Specifies the language the script is written in. Defaults to `groovy` but
|
|
|
|
may be set to any of languages listed in <<modules-scripting>>. The
|
|
|
|
default language may be changed in the `elasticsearch.yml` config file by
|
|
|
|
setting `script.default_lang` to the appropriate language.
|
|
|
|
|
|
|
|
|
|
|
|
`inline`, `id`, `file`::
|
|
|
|
|
|
|
|
Specifies the source of the script. An `inline` script is specified
|
|
|
|
`inline` as in the example above, a stored script with the specified `id`
|
|
|
|
is retrieved from the cluster state (see <<modules-scripting-stored-scripts,Stored Scripts>>),
|
|
|
|
and a `file` script is retrieved from a file in the `config/scripts`
|
|
|
|
directory (see <<modules-scripting-file-scripts, File Scripts>>).
|
|
|
|
+
|
|
|
|
While languages like `expression` and `painless` can be used out of the box as
|
|
|
|
inline or stored scripts, other languages like `groovy` can only be
|
|
|
|
specified as `file` unless you first adjust the default
|
|
|
|
<<modules-scripting-security,scripting security settings>>.
|
|
|
|
|
|
|
|
`params`::
|
|
|
|
|
|
|
|
Specifies any named parameters that are passed into the script as
|
|
|
|
variables.
|
|
|
|
|
|
|
|
[IMPORTANT]
|
2016-07-13 10:02:21 -04:00
|
|
|
[[prefer-params]]
|
2016-05-04 12:17:10 -04:00
|
|
|
.Prefer parameters
|
|
|
|
========================================
|
|
|
|
|
|
|
|
The first time Elasticsearch sees a new script, it compiles it and stores the
|
|
|
|
compiled version in a cache. Compilation can be a heavy process.
|
|
|
|
|
|
|
|
If you need to pass variables into the script, you should pass them in as
|
|
|
|
named `params` instead of hard-coding values into the script itself. For
|
|
|
|
example, if you want to be able to multiply a field value by different
|
|
|
|
multipliers, don't hard-code the multiplier into the script:
|
|
|
|
|
|
|
|
[source,js]
|
|
|
|
----------------------
|
|
|
|
"inline": "doc['my_field'] * 2"
|
|
|
|
----------------------
|
|
|
|
|
|
|
|
Instead, pass it in as a named parameter:
|
|
|
|
|
|
|
|
[source,js]
|
|
|
|
----------------------
|
|
|
|
"inline": "doc['my_field'] * multiplier",
|
|
|
|
"params": {
|
|
|
|
"multiplier": 2
|
|
|
|
}
|
|
|
|
----------------------
|
|
|
|
|
|
|
|
The first version has to be recompiled every time the multiplier changes. The
|
|
|
|
second version is only compiled once.
|
|
|
|
|
Circuit break the number of inline scripts compiled per minute
When compiling many dynamically changing scripts, parameterized
scripts (<https://www.elastic.co/guide/en/elasticsearch/reference/master/modules-scripting-using.html#prefer-params>)
should be preferred. This enforces a limit to the number of scripts that
can be compiled within a minute. A new dynamic setting is added -
`script.max_compilations_per_minute`, which defaults to 15.
If more dynamic scripts are sent, a user will get the following
exception:
```json
{
"error" : {
"root_cause" : [
{
"type" : "circuit_breaking_exception",
"reason" : "[script] Too many dynamic script compilations within one minute, max: [15/min]; please use on-disk, indexed, or scripts with parameters instead",
"bytes_wanted" : 0,
"bytes_limit" : 0
}
],
"type" : "search_phase_execution_exception",
"reason" : "all shards failed",
"phase" : "query",
"grouped" : true,
"failed_shards" : [
{
"shard" : 0,
"index" : "i",
"node" : "a5V1eXcZRYiIk8lecjZ4Jw",
"reason" : {
"type" : "general_script_exception",
"reason" : "Failed to compile inline script [\"aaaaaaaaaaaaaaaa\"] using lang [painless]",
"caused_by" : {
"type" : "circuit_breaking_exception",
"reason" : "[script] Too many dynamic script compilations within one minute, max: [15/min]; please use on-disk, indexed, or scripts with parameters instead",
"bytes_wanted" : 0,
"bytes_limit" : 0
}
}
}
],
"caused_by" : {
"type" : "general_script_exception",
"reason" : "Failed to compile inline script [\"aaaaaaaaaaaaaaaa\"] using lang [painless]",
"caused_by" : {
"type" : "circuit_breaking_exception",
"reason" : "[script] Too many dynamic script compilations within one minute, max: [15/min]; please use on-disk, indexed, or scripts with parameters instead",
"bytes_wanted" : 0,
"bytes_limit" : 0
}
}
},
"status" : 500
}
```
This also fixes a bug in `ScriptService` where requests being executed
concurrently on a single node could cause a script to be compiled
multiple times (many in the case of a powerful node with many shards)
due to no synchronization between checking the cache and compiling the
script. There is now synchronization so that a script being compiled
will only be compiled once regardless of the number of concurrent
searches on a node.
Relates to #19396
2016-07-26 15:36:29 -04:00
|
|
|
If you compile too many unique scripts within a small amount of time,
|
|
|
|
Elasticsearch will reject the new dynamic scripts with a
|
|
|
|
`circuit_breaking_exception` error. By default, up to 15 inline scripts per
|
|
|
|
minute will be compiled. You can change this setting dynamically by setting
|
|
|
|
`script.max_compilations_per_minute`.
|
|
|
|
|
2016-05-04 12:17:10 -04:00
|
|
|
========================================
|
|
|
|
|
|
|
|
|
|
|
|
[float]
|
|
|
|
[[modules-scripting-file-scripts]]
|
|
|
|
=== File-based Scripts
|
|
|
|
|
2016-05-11 10:06:07 -04:00
|
|
|
To increase security, non-sandboxed languages can only be specified in script
|
|
|
|
files stored on every node in the cluster. File scripts must be saved in the
|
|
|
|
`scripts` directory whose default location depends on whether you use the
|
|
|
|
<<zip-targz-layout,`zip`/`tar.gz`>> (`$ES_HOME/config/scripts/`),
|
|
|
|
<<rpm-layout,RPM>>, or <<deb-layout,Debian>> package. The default may be
|
2016-06-22 06:39:36 -04:00
|
|
|
changed with the `path.scripts` setting.
|
2016-05-11 10:06:07 -04:00
|
|
|
|
|
|
|
The languages which are assumed to be safe by default are: `painless`,
|
|
|
|
`expression`, and `mustache` (used for search and query templates).
|
2016-05-04 12:17:10 -04:00
|
|
|
|
|
|
|
Any files placed in the `scripts` directory will be compiled automatically
|
|
|
|
when the node starts up and then <<reload-scripts,every 60 seconds thereafter>>.
|
|
|
|
|
|
|
|
The file should be named as follows: `{script-name}.{lang}`. For instance,
|
|
|
|
the following example creates a Groovy script called `calculate-score`:
|
|
|
|
|
|
|
|
[source,sh]
|
|
|
|
--------------------------------------------------
|
|
|
|
cat "log(_score * 2) + my_modifier" > config/scripts/calculate-score.groovy
|
|
|
|
--------------------------------------------------
|
|
|
|
|
|
|
|
This script can be used as follows:
|
|
|
|
|
|
|
|
[source,js]
|
|
|
|
--------------------------------------------------
|
|
|
|
GET my_index/_search
|
|
|
|
{
|
|
|
|
"query": {
|
|
|
|
"script": {
|
|
|
|
"script": {
|
|
|
|
"lang": "groovy", <1>
|
|
|
|
"file": "calculate-score", <2>
|
|
|
|
"params": {
|
|
|
|
"my_modifier": 2
|
|
|
|
}
|
|
|
|
}
|
|
|
|
}
|
|
|
|
}
|
|
|
|
}
|
|
|
|
--------------------------------------------------
|
|
|
|
<1> The language of the script, which should correspond with the script file suffix.
|
|
|
|
<2> The name of the script, which should be the name of the file.
|
|
|
|
|
|
|
|
The `script` directory may contain sub-directories, in which case the
|
|
|
|
hierarchy of directories is flattened and concatenated with underscores. A
|
|
|
|
script in `group1/group2/my_script.groovy` should use `group1_group2_myscript`
|
|
|
|
as the `file` name.
|
|
|
|
|
|
|
|
|
|
|
|
[[reload-scripts]]
|
|
|
|
[float]
|
|
|
|
==== Automatic script reloading
|
|
|
|
|
|
|
|
The `scripts` directory will be rescanned every `60s` (configurable with the
|
|
|
|
`resource.reload.interval` setting) and new, changed, or removed scripts will
|
|
|
|
be compiled, updated, or deleted from the script cache.
|
|
|
|
|
|
|
|
Script reloading can be completely disabled by setting
|
|
|
|
`script.auto_reload_enabled` to `false`.
|
|
|
|
|
|
|
|
[float]
|
|
|
|
[[modules-scripting-stored-scripts]]
|
|
|
|
=== Stored Scripts
|
|
|
|
|
|
|
|
Scripts may be stored in and retrieved from the cluster state using the
|
|
|
|
`_scripts` end-point:
|
|
|
|
|
|
|
|
[source,js]
|
|
|
|
-----------------------------------
|
|
|
|
/_scripts/{lang}/{id} <1> <2>
|
|
|
|
-----------------------------------
|
|
|
|
<1> The `lang` represents the script language.
|
|
|
|
<2> The `id` is a unique identifier or script name.
|
|
|
|
|
|
|
|
This example stores a Groovy script called `calculate-score` in the cluster
|
|
|
|
state:
|
|
|
|
|
|
|
|
[source,js]
|
|
|
|
-----------------------------------
|
2016-04-29 10:42:03 -04:00
|
|
|
POST _scripts/groovy/calculate-score
|
2016-05-04 12:17:10 -04:00
|
|
|
{
|
|
|
|
"script": "log(_score * 2) + my_modifier"
|
|
|
|
}
|
|
|
|
-----------------------------------
|
2016-05-09 09:42:23 -04:00
|
|
|
// CONSOLE
|
2016-05-04 12:17:10 -04:00
|
|
|
|
|
|
|
This same script can be retrieved with:
|
|
|
|
|
|
|
|
[source,js]
|
|
|
|
-----------------------------------
|
2016-04-29 10:42:03 -04:00
|
|
|
GET _scripts/groovy/calculate-score
|
2016-05-04 12:17:10 -04:00
|
|
|
-----------------------------------
|
2016-05-09 09:42:23 -04:00
|
|
|
// CONSOLE
|
2016-04-29 10:42:03 -04:00
|
|
|
// TEST[continued]
|
2016-05-04 12:17:10 -04:00
|
|
|
|
|
|
|
Stored scripts can be used by specifying the `lang` and `id` parameters as follows:
|
|
|
|
|
|
|
|
[source,js]
|
|
|
|
--------------------------------------------------
|
2016-04-29 10:42:03 -04:00
|
|
|
GET _search
|
2016-05-04 12:17:10 -04:00
|
|
|
{
|
|
|
|
"query": {
|
|
|
|
"script": {
|
|
|
|
"script": {
|
|
|
|
"lang": "groovy",
|
|
|
|
"id": "calculate-score",
|
|
|
|
"params": {
|
|
|
|
"my_modifier": 2
|
|
|
|
}
|
|
|
|
}
|
|
|
|
}
|
|
|
|
}
|
|
|
|
}
|
|
|
|
--------------------------------------------------
|
2016-05-09 09:42:23 -04:00
|
|
|
// CONSOLE
|
2016-04-29 10:42:03 -04:00
|
|
|
// TEST[continued]
|
2016-05-04 12:17:10 -04:00
|
|
|
|
2016-04-29 10:42:03 -04:00
|
|
|
And deleted with:
|
|
|
|
|
|
|
|
[source,js]
|
|
|
|
-----------------------------------
|
|
|
|
DELETE _scripts/groovy/calculate-score
|
|
|
|
-----------------------------------
|
2016-05-09 09:42:23 -04:00
|
|
|
// CONSOLE
|
2016-04-29 10:42:03 -04:00
|
|
|
// TEST[continued]
|
2016-05-04 12:17:10 -04:00
|
|
|
|
2016-06-15 17:57:18 -04:00
|
|
|
[float]
|
|
|
|
[[modules-scripting-using-caching]]
|
|
|
|
=== Script Caching
|
|
|
|
|
|
|
|
All scripts are cached by default so that they only need to be recompiled
|
|
|
|
when updates occur. File scripts keep a static cache and will always reside
|
|
|
|
in memory. Both inline and stored scripts are stored in a cache that can evict
|
|
|
|
residing scripts. By default, scripts do not have a time-based expiration, but
|
|
|
|
you can change this behavior by using the `script.cache.expire` setting.
|
|
|
|
You can configure the size of this cache by using the `script.cache.max_size` setting.
|
|
|
|
By default, the cache size is `100`.
|
|
|
|
|
2016-05-04 12:17:10 -04:00
|
|
|
NOTE: The size of stored scripts is limited to 65,535 bytes. This can be
|
|
|
|
changed by setting `script.max_size_in_bytes` setting to increase that soft
|
|
|
|
limit, but if scripts are really large then alternatives like
|
|
|
|
<<modules-scripting-native,native>> scripts should be considered instead.
|