644 lines
14 KiB
Plaintext
644 lines
14 KiB
Plaintext
[[search-template]]
|
|
== Search Template
|
|
|
|
The `/_search/template` endpoint allows to use the mustache language to pre render search requests,
|
|
before they are executed and fill existing templates with template parameters.
|
|
|
|
[source,js]
|
|
------------------------------------------
|
|
GET _search/template
|
|
{
|
|
"source" : {
|
|
"query": { "match" : { "{{my_field}}" : "{{my_value}}" } },
|
|
"size" : "{{my_size}}"
|
|
},
|
|
"params" : {
|
|
"my_field" : "message",
|
|
"my_value" : "some message",
|
|
"my_size" : 5
|
|
}
|
|
}
|
|
------------------------------------------
|
|
// CONSOLE
|
|
// TEST[setup:twitter]
|
|
|
|
For more information on how Mustache templating and what kind of templating you
|
|
can do with it check out the http://mustache.github.io/mustache.5.html[online
|
|
documentation of the mustache project].
|
|
|
|
NOTE: The mustache language is implemented in Elasticsearch as a sandboxed
|
|
scripting language, hence it obeys settings that may be used to enable or
|
|
disable scripts per type and context as described in the
|
|
<<allowed-script-types-setting, scripting docs>>
|
|
|
|
[float]
|
|
==== More template examples
|
|
|
|
[float]
|
|
===== Filling in a query string with a single value
|
|
|
|
[source,js]
|
|
------------------------------------------
|
|
GET _search/template
|
|
{
|
|
"source": {
|
|
"query": {
|
|
"term": {
|
|
"message": "{{query_string}}"
|
|
}
|
|
}
|
|
},
|
|
"params": {
|
|
"query_string": "search for these words"
|
|
}
|
|
}
|
|
------------------------------------------
|
|
// CONSOLE
|
|
// TEST[setup:twitter]
|
|
|
|
[float]
|
|
===== Converting parameters to JSON
|
|
|
|
The `{{#toJson}}parameter{{/toJson}}` function can be used to convert parameters
|
|
like maps and array to their JSON representation:
|
|
|
|
[source,js]
|
|
------------------------------------------
|
|
GET _search/template
|
|
{
|
|
"source": "{ \"query\": { \"terms\": {{#toJson}}statuses{{/toJson}} }}",
|
|
"params": {
|
|
"statuses" : {
|
|
"status": [ "pending", "published" ]
|
|
}
|
|
}
|
|
}
|
|
------------------------------------------
|
|
// CONSOLE
|
|
|
|
which is rendered as:
|
|
|
|
[source,js]
|
|
------------------------------------------
|
|
{
|
|
"query": {
|
|
"terms": {
|
|
"status": [
|
|
"pending",
|
|
"published"
|
|
]
|
|
}
|
|
}
|
|
}
|
|
------------------------------------------
|
|
// NOTCONSOLE
|
|
|
|
A more complex example substitutes an array of JSON objects:
|
|
|
|
[source,js]
|
|
------------------------------------------
|
|
GET _search/template
|
|
{
|
|
"source": "{\"query\":{\"bool\":{\"must\": {{#toJson}}clauses{{/toJson}} }}}",
|
|
"params": {
|
|
"clauses": [
|
|
{ "term": { "user" : "foo" } },
|
|
{ "term": { "user" : "bar" } }
|
|
]
|
|
}
|
|
}
|
|
------------------------------------------
|
|
// CONSOLE
|
|
|
|
which is rendered as:
|
|
|
|
[source,js]
|
|
------------------------------------------
|
|
{
|
|
"query" : {
|
|
"bool" : {
|
|
"must" : [
|
|
{
|
|
"term" : {
|
|
"user" : "foo"
|
|
}
|
|
},
|
|
{
|
|
"term" : {
|
|
"user" : "bar"
|
|
}
|
|
}
|
|
]
|
|
}
|
|
}
|
|
}
|
|
------------------------------------------
|
|
// NOTCONSOLE
|
|
|
|
[float]
|
|
===== Concatenating array of values
|
|
|
|
The `{{#join}}array{{/join}}` function can be used to concatenate the
|
|
values of an array as a comma delimited string:
|
|
|
|
[source,js]
|
|
------------------------------------------
|
|
GET _search/template
|
|
{
|
|
"source": {
|
|
"query": {
|
|
"match": {
|
|
"emails": "{{#join}}emails{{/join}}"
|
|
}
|
|
}
|
|
},
|
|
"params": {
|
|
"emails": [ "username@email.com", "lastname@email.com" ]
|
|
}
|
|
}
|
|
------------------------------------------
|
|
// CONSOLE
|
|
|
|
which is rendered as:
|
|
|
|
[source,js]
|
|
------------------------------------------
|
|
{
|
|
"query" : {
|
|
"match" : {
|
|
"emails" : "username@email.com,lastname@email.com"
|
|
}
|
|
}
|
|
}
|
|
------------------------------------------
|
|
// NOTCONSOLE
|
|
|
|
The function also accepts a custom delimiter:
|
|
|
|
[source,js]
|
|
------------------------------------------
|
|
GET _search/template
|
|
{
|
|
"source": {
|
|
"query": {
|
|
"range": {
|
|
"born": {
|
|
"gte" : "{{date.min}}",
|
|
"lte" : "{{date.max}}",
|
|
"format": "{{#join delimiter='||'}}date.formats{{/join delimiter='||'}}"
|
|
}
|
|
}
|
|
}
|
|
},
|
|
"params": {
|
|
"date": {
|
|
"min": "2016",
|
|
"max": "31/12/2017",
|
|
"formats": ["dd/MM/yyyy", "yyyy"]
|
|
}
|
|
}
|
|
}
|
|
------------------------------------------
|
|
// CONSOLE
|
|
|
|
which is rendered as:
|
|
|
|
[source,js]
|
|
------------------------------------------
|
|
{
|
|
"query" : {
|
|
"range" : {
|
|
"born" : {
|
|
"gte" : "2016",
|
|
"lte" : "31/12/2017",
|
|
"format" : "dd/MM/yyyy||yyyy"
|
|
}
|
|
}
|
|
}
|
|
}
|
|
|
|
------------------------------------------
|
|
// NOTCONSOLE
|
|
|
|
[float]
|
|
===== Default values
|
|
|
|
A default value is written as `{{var}}{{^var}}default{{/var}}` for instance:
|
|
|
|
[source,js]
|
|
------------------------------------------
|
|
{
|
|
"source": {
|
|
"query": {
|
|
"range": {
|
|
"line_no": {
|
|
"gte": "{{start}}",
|
|
"lte": "{{end}}{{^end}}20{{/end}}"
|
|
}
|
|
}
|
|
}
|
|
},
|
|
"params": { ... }
|
|
}
|
|
------------------------------------------
|
|
// NOTCONSOLE
|
|
|
|
When `params` is `{ "start": 10, "end": 15 }` this query would be rendered as:
|
|
|
|
[source,js]
|
|
------------------------------------------
|
|
{
|
|
"range": {
|
|
"line_no": {
|
|
"gte": "10",
|
|
"lte": "15"
|
|
}
|
|
}
|
|
}
|
|
------------------------------------------
|
|
// NOTCONSOLE
|
|
|
|
But when `params` is `{ "start": 10 }` this query would use the default value
|
|
for `end`:
|
|
|
|
[source,js]
|
|
------------------------------------------
|
|
{
|
|
"range": {
|
|
"line_no": {
|
|
"gte": "10",
|
|
"lte": "20"
|
|
}
|
|
}
|
|
}
|
|
------------------------------------------
|
|
// NOTCONSOLE
|
|
|
|
[float]
|
|
===== Conditional clauses
|
|
|
|
Conditional clauses cannot be expressed using the JSON form of the template.
|
|
Instead, the template *must* be passed as a string. For instance, let's say
|
|
we wanted to run a `match` query on the `line` field, and optionally wanted
|
|
to filter by line numbers, where `start` and `end` are optional.
|
|
|
|
The `params` would look like:
|
|
[source,js]
|
|
------------------------------------------
|
|
{
|
|
"params": {
|
|
"text": "words to search for",
|
|
"line_no": { <1>
|
|
"start": 10, <1>
|
|
"end": 20 <1>
|
|
}
|
|
}
|
|
}
|
|
------------------------------------------
|
|
// NOTCONSOLE
|
|
<1> All three of these elements are optional.
|
|
|
|
We could write the query as:
|
|
|
|
[source,js]
|
|
------------------------------------------
|
|
{
|
|
"query": {
|
|
"bool": {
|
|
"must": {
|
|
"match": {
|
|
"line": "{{text}}" <1>
|
|
}
|
|
},
|
|
"filter": {
|
|
{{#line_no}} <2>
|
|
"range": {
|
|
"line_no": {
|
|
{{#start}} <3>
|
|
"gte": "{{start}}" <4>
|
|
{{#end}},{{/end}} <5>
|
|
{{/start}} <3>
|
|
{{#end}} <6>
|
|
"lte": "{{end}}" <7>
|
|
{{/end}} <6>
|
|
}
|
|
}
|
|
{{/line_no}} <2>
|
|
}
|
|
}
|
|
}
|
|
}
|
|
------------------------------------------
|
|
// NOTCONSOLE
|
|
<1> Fill in the value of param `text`
|
|
<2> Include the `range` filter only if `line_no` is specified
|
|
<3> Include the `gte` clause only if `line_no.start` is specified
|
|
<4> Fill in the value of param `line_no.start`
|
|
<5> Add a comma after the `gte` clause only if `line_no.start`
|
|
AND `line_no.end` are specified
|
|
<6> Include the `lte` clause only if `line_no.end` is specified
|
|
<7> Fill in the value of param `line_no.end`
|
|
|
|
[NOTE]
|
|
==================================
|
|
As written above, this template is not valid JSON because it includes the
|
|
_section_ markers like `{{#line_no}}`. For this reason, the template should
|
|
either be stored in a file (see <<pre-registered-templates>>) or, when used
|
|
via the REST API, should be written as a string:
|
|
|
|
[source,js]
|
|
--------------------
|
|
"source": "{\"query\":{\"bool\":{\"must\":{\"match\":{\"line\":\"{{text}}\"}},\"filter\":{{{#line_no}}\"range\":{\"line_no\":{{{#start}}\"gte\":\"{{start}}\"{{#end}},{{/end}}{{/start}}{{#end}}\"lte\":\"{{end}}\"{{/end}}}}{{/line_no}}}}}}"
|
|
--------------------
|
|
// NOTCONSOLE
|
|
==================================
|
|
|
|
|
|
[float]
|
|
===== Encoding URLs
|
|
|
|
The `{{#url}}value{{/url}}` function can be used to encode a string value
|
|
in a HTML encoding form as defined in by the http://www.w3.org/TR/html4/[HTML specification].
|
|
|
|
As an example, it is useful to encode a URL:
|
|
|
|
[source,js]
|
|
------------------------------------------
|
|
GET _render/template
|
|
{
|
|
"source" : {
|
|
"query" : {
|
|
"term": {
|
|
"http_access_log": "{{#url}}{{host}}/{{page}}{{/url}}"
|
|
}
|
|
}
|
|
},
|
|
"params": {
|
|
"host": "https://www.elastic.co/",
|
|
"page": "learn"
|
|
}
|
|
}
|
|
------------------------------------------
|
|
// CONSOLE
|
|
|
|
The previous query will be rendered as:
|
|
|
|
[source,js]
|
|
------------------------------------------
|
|
{
|
|
"template_output" : {
|
|
"query" : {
|
|
"term" : {
|
|
"http_access_log" : "https%3A%2F%2Fwww.elastic.co%2F%2Flearn"
|
|
}
|
|
}
|
|
}
|
|
}
|
|
------------------------------------------
|
|
// TESTRESPONSE
|
|
|
|
|
|
[float]
|
|
[[pre-registered-templates]]
|
|
===== Pre-registered template
|
|
|
|
You can register search templates by using the stored scripts api.
|
|
|
|
[source,js]
|
|
------------------------------------------
|
|
POST _scripts/<templatename>
|
|
{
|
|
"script": {
|
|
"lang": "mustache",
|
|
"source": {
|
|
"query": {
|
|
"match": {
|
|
"title": "{{query_string}}"
|
|
}
|
|
}
|
|
}
|
|
}
|
|
}
|
|
------------------------------------------
|
|
// CONSOLE
|
|
// TEST[continued]
|
|
|
|
//////////////////////////
|
|
|
|
We want to be sure that the template has been created,
|
|
because we'll use it later.
|
|
|
|
[source,js]
|
|
--------------------------------------------------
|
|
{
|
|
"acknowledged" : true
|
|
}
|
|
--------------------------------------------------
|
|
// TESTRESPONSE
|
|
|
|
//////////////////////////
|
|
|
|
This template can be retrieved by
|
|
|
|
[source,js]
|
|
------------------------------------------
|
|
GET _scripts/<templatename>
|
|
------------------------------------------
|
|
// CONSOLE
|
|
// TEST[continued]
|
|
|
|
which is rendered as:
|
|
|
|
[source,js]
|
|
------------------------------------------
|
|
{
|
|
"script" : {
|
|
"lang" : "mustache",
|
|
"source" : "{\"query\":{\"match\":{\"title\":\"{{query_string}}\"}}}",
|
|
"options": {
|
|
"content_type" : "application/json; charset=UTF-8"
|
|
}
|
|
},
|
|
"_id": "<templatename>",
|
|
"found": true
|
|
}
|
|
------------------------------------------
|
|
// TESTRESPONSE
|
|
|
|
This template can be deleted by
|
|
|
|
[source,js]
|
|
------------------------------------------
|
|
DELETE _scripts/<templatename>
|
|
------------------------------------------
|
|
// CONSOLE
|
|
// TEST[continued]
|
|
|
|
//////////////////////////
|
|
|
|
We want to be sure that the template has been created,
|
|
because we'll use it later.
|
|
|
|
[source,js]
|
|
--------------------------------------------------
|
|
{
|
|
"acknowledged" : true
|
|
}
|
|
--------------------------------------------------
|
|
// TESTRESPONSE
|
|
|
|
//////////////////////////
|
|
|
|
To use a stored template at search time use:
|
|
|
|
[source,js]
|
|
------------------------------------------
|
|
GET _search/template
|
|
{
|
|
"id": "<templateName>", <1>
|
|
"params": {
|
|
"query_string": "search for these words"
|
|
}
|
|
}
|
|
------------------------------------------
|
|
// CONSOLE
|
|
// TEST[catch:missing]
|
|
<1> Name of the stored template script.
|
|
|
|
[float]
|
|
==== Validating templates
|
|
|
|
A template can be rendered in a response with given parameters using
|
|
|
|
[source,js]
|
|
------------------------------------------
|
|
GET _render/template
|
|
{
|
|
"source": "{ \"query\": { \"terms\": {{#toJson}}statuses{{/toJson}} }}",
|
|
"params": {
|
|
"statuses" : {
|
|
"status": [ "pending", "published" ]
|
|
}
|
|
}
|
|
}
|
|
------------------------------------------
|
|
// CONSOLE
|
|
|
|
This call will return the rendered template:
|
|
|
|
[source,js]
|
|
------------------------------------------
|
|
{
|
|
"template_output": {
|
|
"query": {
|
|
"terms": {
|
|
"status": [ <1>
|
|
"pending",
|
|
"published"
|
|
]
|
|
}
|
|
}
|
|
}
|
|
}
|
|
------------------------------------------
|
|
// TESTRESPONSE
|
|
<1> `status` array has been populated with values from the `params` object.
|
|
|
|
Pre-registered templates can also be rendered using
|
|
|
|
[source,js]
|
|
------------------------------------------
|
|
GET _render/template/<template_name>
|
|
{
|
|
"params": {
|
|
"..."
|
|
}
|
|
}
|
|
------------------------------------------
|
|
// NOTCONSOLE
|
|
|
|
[float]
|
|
===== Explain
|
|
|
|
You can use `explain` parameter when running a template:
|
|
|
|
[source,js]
|
|
------------------------------------------
|
|
GET _search/template
|
|
{
|
|
"id": "my_template",
|
|
"params": {
|
|
"status": [ "pending", "published" ]
|
|
},
|
|
"explain": true
|
|
}
|
|
------------------------------------------
|
|
// CONSOLE
|
|
// TEST[catch:missing]
|
|
|
|
[float]
|
|
===== Profiling
|
|
|
|
You can use `profile` parameter when running a template:
|
|
|
|
[source,js]
|
|
------------------------------------------
|
|
GET _search/template
|
|
{
|
|
"id": "my_template",
|
|
"params": {
|
|
"status": [ "pending", "published" ]
|
|
},
|
|
"profile": true
|
|
}
|
|
------------------------------------------
|
|
// CONSOLE
|
|
// TEST[catch:missing]
|
|
|
|
[[multi-search-template]]
|
|
== Multi Search Template
|
|
|
|
The multi search template API allows to execute several search template
|
|
requests within the same API using the `_msearch/template` endpoint.
|
|
|
|
The format of the request is similar to the <<search-multi-search, Multi
|
|
Search API>> format:
|
|
|
|
[source,js]
|
|
--------------------------------------------------
|
|
header\n
|
|
body\n
|
|
header\n
|
|
body\n
|
|
--------------------------------------------------
|
|
// NOTCONSOLE
|
|
|
|
The header part supports the same `index`, `search_type`,
|
|
`preference`, and `routing` options as the usual Multi Search API.
|
|
|
|
The body includes a search template body request and supports inline,
|
|
stored and file templates. Here is an example:
|
|
|
|
[source,js]
|
|
--------------------------------------------------
|
|
$ cat requests
|
|
{"index": "test"}
|
|
{"source": {"query": {"match": {"user" : "{{username}}" }}}, "params": {"username": "john"}} <1>
|
|
{"source": {"query": {"{{query_type}}": {"name": "{{name}}" }}}, "params": {"query_type": "match_phrase_prefix", "name": "Smith"}}
|
|
{"index": "_all"}
|
|
{"id": "template_1", "params": {"query_string": "search for these words" }} <2>
|
|
|
|
$ curl -H "Content-Type: application/x-ndjson" -XGET localhost:9200/_msearch/template --data-binary "@requests"; echo
|
|
--------------------------------------------------
|
|
// NOTCONSOLE
|
|
// Not converting to console because this shows how curl works
|
|
<1> Inline search template request
|
|
|
|
<2> Search template request based on a stored template
|
|
|
|
The response returns a `responses` array, which includes the search template
|
|
response for each search template request matching its order in the original
|
|
multi search template request. If there was a complete failure for that specific
|
|
search template request, an object with `error` message will be returned in place
|
|
of the actual search response.
|