[[search-template]]
== Search Template
The `/_search/template` endpoint allows to use the mustache language to pre render search requests,
before they are executed and fill existing templates with template parameters.
[source,js]
------------------------------------------
GET _search/template
{
"source" : {
"query": { "match" : { "{{my_field}}" : "{{my_value}}" } },
"size" : "{{my_size}}"
},
"params" : {
"my_field" : "message",
"my_value" : "some message",
"my_size" : 5
}
}
------------------------------------------
// CONSOLE
// TEST[setup:twitter]
For more information on how Mustache templating and what kind of templating you
can do with it check out the http://mustache.github.io/mustache.5.html[online
documentation of the mustache project].
NOTE: The mustache language is implemented in Elasticsearch as a sandboxed
scripting language, hence it obeys settings that may be used to enable or
disable scripts per type and context as described in the
<<allowed-script-types-setting, scripting docs>>
[float]
==== More template examples
[float]
===== Filling in a query string with a single value
[source,js]
------------------------------------------
GET _search/template
{
"source": {
"query": {
"term": {
"message": "{{query_string}}"
}
}
},
"params": {
"query_string": "search for these words"
}
}
------------------------------------------
// CONSOLE
// TEST[setup:twitter]
[float]
===== Converting parameters to JSON
The `{{#toJson}}parameter{{/toJson}}` function can be used to convert parameters
like maps and array to their JSON representation:
[source,js]
------------------------------------------
GET _search/template
{
"source": "{ \"query\": { \"terms\": {{#toJson}}statuses{{/toJson}} }}",
"params": {
"statuses" : {
"status": [ "pending", "published" ]
}
}
}
------------------------------------------
// CONSOLE
which is rendered as:
[source,js]
------------------------------------------
{
"query": {
"terms": {
"status": [
"pending",
"published"
]
}
}
}
------------------------------------------
// NOTCONSOLE
A more complex example substitutes an array of JSON objects:
[source,js]
------------------------------------------
GET _search/template
{
"source": "{\"query\":{\"bool\":{\"must\": {{#toJson}}clauses{{/toJson}} }}}",
"params": {
"clauses": [
{ "term": { "user" : "foo" } },
{ "term": { "user" : "bar" } }
]
}
}
------------------------------------------
// CONSOLE
which is rendered as:
[source,js]
------------------------------------------
{
"query" : {
"bool" : {
"must" : [
{
"term" : {
"user" : "foo"
}
},
{
"term" : {
"user" : "bar"
}
}
]
}
}
}
------------------------------------------
// NOTCONSOLE
[float]
===== Concatenating array of values
The `{{#join}}array{{/join}}` function can be used to concatenate the
values of an array as a comma delimited string:
[source,js]
------------------------------------------
GET _search/template
{
"source": {
"query": {
"match": {
"emails": "{{#join}}emails{{/join}}"
}
}
},
"params": {
"emails": [ "username@email.com", "lastname@email.com" ]
}
}
------------------------------------------
// CONSOLE
which is rendered as:
[source,js]
------------------------------------------
{
"query" : {
"match" : {
"emails" : "username@email.com,lastname@email.com"
}
}
}
------------------------------------------
// NOTCONSOLE
The function also accepts a custom delimiter:
[source,js]
------------------------------------------
GET _search/template
{
"source": {
"query": {
"range": {
"born": {
"gte" : "{{date.min}}",
"lte" : "{{date.max}}",
"format": "{{#join delimiter='||'}}date.formats{{/join delimiter='||'}}"
}
}
}
},
"params": {
"date": {
"min": "2016",
"max": "31/12/2017",
"formats": ["dd/MM/yyyy", "yyyy"]
}
}
}
------------------------------------------
// CONSOLE
which is rendered as:
[source,js]
------------------------------------------
{
"query" : {
"range" : {
"born" : {
"gte" : "2016",
"lte" : "31/12/2017",
"format" : "dd/MM/yyyy||yyyy"
}
}
}
}
------------------------------------------
// NOTCONSOLE
[float]
===== Default values
A default value is written as `{{var}}{{^var}}default{{/var}}` for instance:
[source,js]
------------------------------------------
{
"source": {
"query": {
"range": {
"line_no": {
"gte": "{{start}}",
"lte": "{{end}}{{^end}}20{{/end}}"
}
}
}
},
"params": { ... }
}
------------------------------------------
// NOTCONSOLE
When `params` is `{ "start": 10, "end": 15 }` this query would be rendered as:
[source,js]
------------------------------------------
{
"range": {
"line_no": {
"gte": "10",
"lte": "15"
}
}
}
------------------------------------------
// NOTCONSOLE
But when `params` is `{ "start": 10 }` this query would use the default value
for `end`:
[source,js]
------------------------------------------
{
"range": {
"line_no": {
"gte": "10",
"lte": "20"
}
}
}
------------------------------------------
// NOTCONSOLE
[float]
===== Conditional clauses
Conditional clauses cannot be expressed using the JSON form of the template.
Instead, the template *must* be passed as a string. For instance, let's say
we wanted to run a `match` query on the `line` field, and optionally wanted
to filter by line numbers, where `start` and `end` are optional.
The `params` would look like:
[source,js]
------------------------------------------
{
"params": {
"text": "words to search for",
"line_no": { <1>
"start": 10, <1>
"end": 20 <1>
}
}
}
------------------------------------------
// NOTCONSOLE
<1> All three of these elements are optional.
We could write the query as:
[source,js]
------------------------------------------
{
"query": {
"bool": {
"must": {
"match": {
"line": "{{text}}" <1>
}
},
"filter": {
{{#line_no}} <2>
"range": {
"line_no": {
{{#start}} <3>
"gte": "{{start}}" <4>
{{#end}},{{/end}} <5>
{{/start}} <3>
{{#end}} <6>
"lte": "{{end}}" <7>
{{/end}} <6>
}
}
{{/line_no}} <2>
}
}
}
}
------------------------------------------
// NOTCONSOLE
<1> Fill in the value of param `text`
<2> Include the `range` filter only if `line_no` is specified
<3> Include the `gte` clause only if `line_no.start` is specified
<4> Fill in the value of param `line_no.start`
<5> Add a comma after the `gte` clause only if `line_no.start`
AND `line_no.end` are specified
<6> Include the `lte` clause only if `line_no.end` is specified
<7> Fill in the value of param `line_no.end`
[NOTE]
==================================
As written above, this template is not valid JSON because it includes the
_section_ markers like `{{#line_no}}`. For this reason, the template should
either be stored in a file (see <<pre-registered-templates>>) or, when used
via the REST API, should be written as a string:
[source,js]
--------------------
"source": "{\"query\":{\"bool\":{\"must\":{\"match\":{\"line\":\"{{text}}\"}},\"filter\":{{{#line_no}}\"range\":{\"line_no\":{{{#start}}\"gte\":\"{{start}}\"{{#end}},{{/end}}{{/start}}{{#end}}\"lte\":\"{{end}}\"{{/end}}}}{{/line_no}}}}}}"
--------------------
// NOTCONSOLE
==================================
[float]
===== Encoding URLs
The `{{#url}}value{{/url}}` function can be used to encode a string value
in a HTML encoding form as defined in by the http://www.w3.org/TR/html4/[HTML specification].
As an example, it is useful to encode a URL:
[source,js]
------------------------------------------
GET _render/template
{
"source" : {
"query" : {
"term": {
"http_access_log": "{{#url}}{{host}}/{{page}}{{/url}}"
}
}
},
"params": {
"host": "https://www.elastic.co/",
"page": "learn"
}
}
------------------------------------------
// CONSOLE
The previous query will be rendered as:
[source,js]
------------------------------------------
{
"template_output" : {
"query" : {
"term" : {
"http_access_log" : "https%3A%2F%2Fwww.elastic.co%2F%2Flearn"
}
}
}
}
------------------------------------------
// TESTRESPONSE
[float]
[[pre-registered-templates]]
===== Pre-registered template
You can register search templates by using the stored scripts api.
[source,js]
------------------------------------------
POST _scripts/<templatename>
{
"script": {
"lang": "mustache",
"source": {
"query": {
"match": {
"title": "{{query_string}}"
}
}
}
}
}
------------------------------------------
// CONSOLE
// TEST[continued]
//////////////////////////
We want to be sure that the template has been created,
because we'll use it later.
[source,js]
--------------------------------------------------
{
"acknowledged" : true
}
--------------------------------------------------
// TESTRESPONSE
//////////////////////////
This template can be retrieved by
[source,js]
------------------------------------------
GET _scripts/<templatename>
------------------------------------------
// CONSOLE
// TEST[continued]
which is rendered as:
[source,js]
------------------------------------------
{
"script" : {
"lang" : "mustache",
"source" : "{\"query\":{\"match\":{\"title\":\"{{query_string}}\"}}}",
"options": {
"content_type" : "application/json; charset=UTF-8"
}
},
"_id": "<templatename>",
"found": true
}
------------------------------------------
// TESTRESPONSE
This template can be deleted by
[source,js]
------------------------------------------
DELETE _scripts/<templatename>
------------------------------------------
// CONSOLE
// TEST[continued]
//////////////////////////
We want to be sure that the template has been created,
because we'll use it later.
[source,js]
--------------------------------------------------
{
"acknowledged" : true
}
--------------------------------------------------
// TESTRESPONSE
//////////////////////////
To use a stored template at search time use:
[source,js]
------------------------------------------
GET _search/template
{
"id": "<templateName>", <1>
"params": {
"query_string": "search for these words"
}
}
------------------------------------------
// CONSOLE
// TEST[catch:missing]
<1> Name of the stored template script.
[float]
==== Validating templates
A template can be rendered in a response with given parameters using
[source,js]
------------------------------------------
GET _render/template
{
"source": "{ \"query\": { \"terms\": {{#toJson}}statuses{{/toJson}} }}",
"params": {
"statuses" : {
"status": [ "pending", "published" ]
}
}
}
------------------------------------------
// CONSOLE
This call will return the rendered template:
[source,js]
------------------------------------------
{
"template_output": {
"query": {
"terms": {
"status": [ <1>
"pending",
"published"
]
}
}
}
}
------------------------------------------
// TESTRESPONSE
<1> `status` array has been populated with values from the `params` object.
Pre-registered templates can also be rendered using
[source,js]
------------------------------------------
GET _render/template/<template_name>
{
"params": {
"..."
}
}
------------------------------------------
// NOTCONSOLE
[float]
===== Explain
You can use `explain` parameter when running a template:
[source,js]
------------------------------------------
GET _search/template
{
"id": "my_template",
"params": {
"status": [ "pending", "published" ]
},
"explain": true
}
------------------------------------------
// CONSOLE
// TEST[catch:missing]
[float]
===== Profiling
You can use `profile` parameter when running a template:
[source,js]
------------------------------------------
GET _search/template
{
"id": "my_template",
"params": {
"status": [ "pending", "published" ]
},
"profile": true
}
------------------------------------------
// CONSOLE
// TEST[catch:missing]
[[multi-search-template]]
== Multi Search Template
The multi search template API allows to execute several search template
requests within the same API using the `_msearch/template` endpoint.
The format of the request is similar to the <<search-multi-search, Multi
Search API>> format:
[source,js]
--------------------------------------------------
header\n
body\n
header\n
body\n
--------------------------------------------------
// NOTCONSOLE
The header part supports the same `index`, `types`, `search_type`,
`preference`, and `routing` options as the usual Multi Search API.
The body includes a search template body request and supports inline,
stored and file templates. Here is an example:
[source,js]
--------------------------------------------------
$ cat requests
{"index": "test"}
{"source": {"query": {"match": {"user" : "{{username}}" }}}, "params": {"username": "john"}} <1>
{"index": "_all", "types": "accounts"}
{"source": {"query": {"{{query_type}}": {"name": "{{name}}" }}}, "params": {"query_type": "match_phrase_prefix", "name": "Smith"}}
{"index": "_all"}
{"id": "template_1", "params": {"query_string": "search for these words" }} <2>
$ curl -H "Content-Type: application/x-ndjson" -XGET localhost:9200/_msearch/template --data-binary "@requests"; echo
--------------------------------------------------
// NOTCONSOLE
// Not converting to console because this shows how curl works
<1> Inline search template request
<2> Search template request based on a stored template
The response returns a `responses` array, which includes the search template
response for each search template request matching its order in the original
multi search template request. If there was a complete failure for that specific
search template request, an object with `error` message will be returned in place
of the actual search response.