OpenSearch/docs/reference/search/search-template.asciidoc

710 lines
17 KiB
Plaintext
Raw Normal View History

[[search-template]]
=== Search Template
Allows you to use the mustache language to pre render search requests.
[source,console]
------------------------------------------
GET _search/template
{
"source" : {
"query": { "match" : { "{{my_field}}" : "{{my_value}}" } },
"size" : "{{my_size}}"
},
"params" : {
"my_field" : "message",
"my_value" : "some message",
"my_size" : 5
}
}
------------------------------------------
// TEST[setup:twitter]
[[search-template-api-request]]
==== {api-request-title}
`GET _search/template`
[[search-template-api-desc]]
==== {api-description-title}
The `/_search/template` endpoint allows you to use the mustache language to pre-
render search requests, before they are executed and fill existing templates
with template parameters.
For more information on how Mustache templating and what kind of templating you
can do with it check out the http://mustache.github.io/mustache.5.html[online
documentation of the mustache project].
NOTE: The mustache language is implemented in {es} as a sandboxed scripting
language, hence it obeys settings that may be used to enable or disable scripts
per type and context as described in the
<<allowed-script-types-setting, scripting docs>>.
[[search-template-api-path-params]]
==== {api-path-parms-title}
include::{docdir}/rest-api/common-parms.asciidoc[tag=index]
[[search-template-api-query-params]]
==== {api-query-parms-title}
include::{docdir}/rest-api/common-parms.asciidoc[tag=allow-no-indices]
`ccs_minimize_roundtrips`::
(Optional, boolean) If `true`, network round-trips are minimized for
cross-cluster search requests. Defaults to `true`.
include::{docdir}/rest-api/common-parms.asciidoc[tag=expand-wildcards]
`explain`::
(Optional, boolean) If `true`, the response includes additional details about
score computation as part of a hit. Defaults to `false`.
`ignore_throttled`::
(Optional, boolean) If `true`, specified concrete, expanded or aliased indices
are not included in the response when throttled. Defaults to `false`.
include::{docdir}/rest-api/common-parms.asciidoc[tag=index-ignore-unavailable]
include::{docdir}/rest-api/common-parms.asciidoc[tag=preference]
`profile`::
(Optional, boolean) If `true`, the query execution is profiled. Defaults
to `false`.
`rest_total_hits_as_int`::
(Optional, boolean) If `true`, `hits.total` are rendered as an integer in
the response. Defaults to `false`.
include::{docdir}/rest-api/common-parms.asciidoc[tag=routing]
include::{docdir}/rest-api/common-parms.asciidoc[tag=scroll]
include::{docdir}/rest-api/common-parms.asciidoc[tag=search_type]
`typed_keys`::
(Optional, boolean) If `true`, aggregation and suggester names are
prefixed by their respective types in the response. Defaults to `false`.
[[search-template-api-request-body]]
==== {api-request-body-title}
The API request body must contain the search definition template and its parameters.
[[search-template-api-example]]
==== {api-response-codes-title}
Scripting: add support for fine-grained settings Allow to on/off scripting based on their source (where they get loaded from), the operation that executes them and their language. The settings cover the following combinations: - mode: on, off, sandbox - source: indexed, dynamic, file - engine: groovy, expressions, mustache, etc - operation: update, search, aggs, mapping The following settings are supported for every engine: script.engine.groovy.indexed.update: sandbox/on/off script.engine.groovy.indexed.search: sandbox/on/off script.engine.groovy.indexed.aggs: sandbox/on/off script.engine.groovy.indexed.mapping: sandbox/on/off script.engine.groovy.dynamic.update: sandbox/on/off script.engine.groovy.dynamic.search: sandbox/on/off script.engine.groovy.dynamic.aggs: sandbox/on/off script.engine.groovy.dynamic.mapping: sandbox/on/off script.engine.groovy.file.update: sandbox/on/off script.engine.groovy.file.search: sandbox/on/off script.engine.groovy.file.aggs: sandbox/on/off script.engine.groovy.file.mapping: sandbox/on/off For ease of use, the following more generic settings are supported too: script.indexed: sandbox/on/off script.dynamic: sandbox/on/off script.file: sandbox/on/off script.update: sandbox/on/off script.search: sandbox/on/off script.aggs: sandbox/on/off script.mapping: sandbox/on/off These will be used to calculate the more specific settings, using the stricter setting of each combination. Operation based settings have precedence over conflicting source based ones. Note that the `mustache` engine is affected by generic settings applied to any language, while native scripts aren't as they are static by definition. Also, the previous `script.disable_dynamic` setting can now be deprecated. Closes #6418 Closes #10116 Closes #10274
2015-03-06 12:38:11 -05:00
[[pre-registered-templates]]
===== Store a search template
You can store a search template using the stored scripts API.
[source,console]
------------------------------------------
POST _scripts/<templateid>
{
"script": {
"lang": "mustache",
"source": {
"query": {
"match": {
"title": "{{query_string}}"
}
}
}
}
}
------------------------------------------
// TEST[continued]
//////////////////////////
The API returns the following result if the template has been successfully
created:
[source,console-result]
--------------------------------------------------
{
"acknowledged" : true
}
--------------------------------------------------
//////////////////////////
The template can be retrieved by calling
[source,console]
------------------------------------------
GET _scripts/<templateid>
------------------------------------------
// TEST[continued]
The API returns the following result:
[source,console-result]
------------------------------------------
{
"script" : {
"lang" : "mustache",
"source" : "{\"query\":{\"match\":{\"title\":\"{{query_string}}\"}}}",
"options": {
"content_type" : "application/json; charset=UTF-8"
}
},
"_id": "<templateid>",
"found": true
}
------------------------------------------
This template can be deleted by calling
[source,console]
------------------------------------------
DELETE _scripts/<templateid>
------------------------------------------
// TEST[continued]
[[use-registered-templates]]
===== Using a stored search template
To use a stored template at search time send the following request:
[source,console]
------------------------------------------
GET _search/template
{
"id": "<templateid>", <1>
"params": {
"query_string": "search for these words"
}
}
------------------------------------------
// TEST[catch:missing]
<1> Name of the stored template script.
[[_validating_templates]]
==== Validating a search template
A template can be rendered in a response with given parameters by using the
following request:
[source,console]
------------------------------------------
GET _render/template
{
"source": "{ \"query\": { \"terms\": {{#toJson}}statuses{{/toJson}} }}",
"params": {
"statuses" : {
"status": [ "pending", "published" ]
}
}
}
------------------------------------------
The API returns the rendered template:
[source,console-result]
------------------------------------------
{
"template_output": {
"query": {
"terms": {
"status": [ <1>
"pending",
"published"
]
}
}
}
}
------------------------------------------
<1> `status` array has been populated with values from the `params` object.
Stored templates can also be rendered by calling the following request:
[source,js]
------------------------------------------
GET _render/template/<template_name>
{
"params": {
"..."
}
}
------------------------------------------
// NOTCONSOLE
[[search-template-explain-parameter]]
===== Using the explain parameter
You can use the `explain` parameter when running a template:
[source,console]
------------------------------------------
GET _search/template
{
"id": "my_template",
"params": {
"status": [ "pending", "published" ]
},
"explain": true
}
------------------------------------------
// TEST[catch:missing]
[[search-template-profile-parameter]]
===== Profiling
You can use the `profile` parameter when running a template:
[source,console]
------------------------------------------
GET _search/template
{
"id": "my_template",
"params": {
"status": [ "pending", "published" ]
},
"profile": true
}
------------------------------------------
// TEST[catch:missing]
[[search-template-query-string-single]]
===== Filling in a query string with a single value
[source,console]
------------------------------------------
GET _search/template
{
"source": {
"query": {
"term": {
"message": "{{query_string}}"
}
}
},
"params": {
"query_string": "search for these words"
}
}
------------------------------------------
// TEST[setup:twitter]
[[search-template-converting-to-json]]
===== Converting parameters to JSON
The `{{#toJson}}parameter{{/toJson}}` function can be used to convert parameters
like maps and array to their JSON representation:
[source,console]
------------------------------------------
GET _search/template
{
"source": "{ \"query\": { \"terms\": {{#toJson}}statuses{{/toJson}} }}",
"params": {
"statuses" : {
"status": [ "pending", "published" ]
}
}
}
------------------------------------------
which is rendered as:
[source,js]
------------------------------------------
{
"query": {
"terms": {
"status": [
"pending",
"published"
]
}
}
}
------------------------------------------
// NOTCONSOLE
A more complex example substitutes an array of JSON objects:
[source,console]
------------------------------------------
GET _search/template
{
"source": "{\"query\":{\"bool\":{\"must\": {{#toJson}}clauses{{/toJson}} }}}",
"params": {
"clauses": [
{ "term": { "user" : "foo" } },
{ "term": { "user" : "bar" } }
]
}
}
------------------------------------------
which is rendered as:
[source,js]
------------------------------------------
{
"query" : {
"bool" : {
"must" : [
{
"term" : {
"user" : "foo"
}
},
{
"term" : {
"user" : "bar"
}
}
]
}
}
}
------------------------------------------
// NOTCONSOLE
[[search-template-concatenate-array]]
===== Concatenating array of values
The `{{#join}}array{{/join}}` function can be used to concatenate the
values of an array as a comma delimited string:
[source,console]
------------------------------------------
GET _search/template
{
"source": {
"query": {
"match": {
"emails": "{{#join}}emails{{/join}}"
}
}
},
"params": {
"emails": [ "username@email.com", "lastname@email.com" ]
}
}
------------------------------------------
which is rendered as:
[source,js]
------------------------------------------
{
"query" : {
"match" : {
"emails" : "username@email.com,lastname@email.com"
}
}
}
------------------------------------------
// NOTCONSOLE
The function also accepts a custom delimiter:
[source,console]
------------------------------------------
GET _search/template
{
"source": {
"query": {
"range": {
"born": {
"gte" : "{{date.min}}",
"lte" : "{{date.max}}",
"format": "{{#join delimiter='||'}}date.formats{{/join delimiter='||'}}"
}
}
}
},
"params": {
"date": {
"min": "2016",
"max": "31/12/2017",
"formats": ["dd/MM/yyyy", "yyyy"]
}
}
}
------------------------------------------
which is rendered as:
[source,js]
------------------------------------------
{
"query" : {
"range" : {
"born" : {
"gte" : "2016",
"lte" : "31/12/2017",
"format" : "dd/MM/yyyy||yyyy"
}
}
}
}
------------------------------------------
// NOTCONSOLE
[[search-template-default-values]]
===== Default values
A default value is written as `{{var}}{{^var}}default{{/var}}` for instance:
[source,js]
------------------------------------------
{
"source": {
"query": {
"range": {
"line_no": {
"gte": "{{start}}",
"lte": "{{end}}{{^end}}20{{/end}}"
}
}
}
},
"params": { ... }
}
------------------------------------------
// NOTCONSOLE
When `params` is `{ "start": 10, "end": 15 }` this query would be rendered as:
[source,js]
------------------------------------------
{
"range": {
"line_no": {
"gte": "10",
"lte": "15"
}
}
}
------------------------------------------
// NOTCONSOLE
But when `params` is `{ "start": 10 }` this query would use the default value
for `end`:
[source,js]
------------------------------------------
{
"range": {
"line_no": {
"gte": "10",
"lte": "20"
}
}
}
------------------------------------------
// NOTCONSOLE
[[search-template-conditional-clauses]]
===== Conditional clauses
Conditional clauses cannot be expressed using the JSON form of the template.
Instead, the template *must* be passed as a string. For instance, let's say
we wanted to run a `match` query on the `line` field, and optionally wanted
to filter by line numbers, where `start` and `end` are optional.
The `params` would look like:
[source,js]
------------------------------------------
{
"params": {
"text": "words to search for",
"line_no": { <1>
"start": 10, <1>
"end": 20 <1>
}
}
}
------------------------------------------
// NOTCONSOLE
<1> All three of these elements are optional.
We could write the query as:
[source,js]
------------------------------------------
{
"query": {
"bool": {
"must": {
"match": {
"line": "{{text}}" <1>
}
},
"filter": {
{{#line_no}} <2>
"range": {
"line_no": {
{{#start}} <3>
"gte": "{{start}}" <4>
{{#end}},{{/end}} <5>
{{/start}} <3>
{{#end}} <6>
"lte": "{{end}}" <7>
{{/end}} <6>
}
}
{{/line_no}} <2>
}
}
}
}
------------------------------------------
// NOTCONSOLE
<1> Fill in the value of param `text`
<2> Include the `range` filter only if `line_no` is specified
<3> Include the `gte` clause only if `line_no.start` is specified
<4> Fill in the value of param `line_no.start`
<5> Add a comma after the `gte` clause only if `line_no.start`
AND `line_no.end` are specified
<6> Include the `lte` clause only if `line_no.end` is specified
<7> Fill in the value of param `line_no.end`
[NOTE]
==================================
As written above, this template is not valid JSON because it includes the
_section_ markers like `{{#line_no}}`. For this reason, the template should
either be stored in a file (see <<pre-registered-templates>>) or, when used
via the REST API, should be written as a string:
[source,js]
--------------------
"source": "{\"query\":{\"bool\":{\"must\":{\"match\":{\"line\":\"{{text}}\"}},\"filter\":{{{#line_no}}\"range\":{\"line_no\":{{{#start}}\"gte\":\"{{start}}\"{{#end}},{{/end}}{{/start}}{{#end}}\"lte\":\"{{end}}\"{{/end}}}}{{/line_no}}}}}}"
--------------------
// NOTCONSOLE
==================================
[[search-template-encode-urls]]
===== Encoding URLs
The `{{#url}}value{{/url}}` function can be used to encode a string value
in a HTML encoding form as defined in by the
http://www.w3.org/TR/html4/[HTML specification].
As an example, it is useful to encode a URL:
[source,console]
------------------------------------------
GET _render/template
{
"source" : {
"query" : {
"term": {
"http_access_log": "{{#url}}{{host}}/{{page}}{{/url}}"
}
}
},
"params": {
"host": "https://www.elastic.co/",
"page": "learn"
}
}
------------------------------------------
The previous query will be rendered as:
[source,console-result]
------------------------------------------
{
"template_output" : {
"query" : {
"term" : {
"http_access_log" : "https%3A%2F%2Fwww.elastic.co%2F%2Flearn"
}
}
}
}
------------------------------------------
[[multi-search-template]]
=== Multi Search Template
Allows to execute several search template requests.
[[multi-search-template-api-request]]
==== {api-request-title}
`GET _msearch/template`
[[multi-search-template-api-desc]]
==== {api-description-title}
Allows to execute several search template requests within the same API using the
`_msearch/template` endpoint.
The format of the request is similar to the <<search-multi-search, Multi
Search API>> format:
[source,js]
--------------------------------------------------
header\n
body\n
header\n
body\n
--------------------------------------------------
// NOTCONSOLE
The header part supports the same `index`, `search_type`, `preference`, and
`routing` options as the Multi Search API.
The body includes a search template body request and supports inline, stored and
file templates.
[[multi-search-template-api-example]]
==== {api-examples-title}
[source,js]
--------------------------------------------------
$ cat requests
{"index": "test"}
{"source": {"query": {"match": {"user" : "{{username}}" }}}, "params": {"username": "john"}} <1>
{"source": {"query": {"{{query_type}}": {"name": "{{name}}" }}}, "params": {"query_type": "match_phrase_prefix", "name": "Smith"}}
{"index": "_all"}
{"id": "template_1", "params": {"query_string": "search for these words" }} <2>
$ curl -H "Content-Type: application/x-ndjson" -XGET localhost:9200/_msearch/template --data-binary "@requests"; echo
--------------------------------------------------
// NOTCONSOLE
// Not converting to console because this shows how curl works
<1> Inline search template request
<2> Search template request based on a stored template
The response returns a `responses` array, which includes the search template
response for each search template request matching its order in the original
multi search template request. If there was a complete failure for that specific
search template request, an object with `error` message will be returned in
place of the actual search response.