honeymoose/OpenSearch
1
0
Fork 0
mirror of https://github.com/honeymoose/OpenSearch.git synced 2025-03-24 17:09:48 +00:00
Code Issues Packages Projects Releases Wiki Activity

Query Templates: Adding dedicated /_search/template endpoint

Browse Source 
In order to simplify query template execution an own endpoint has been added

Closes #5353
This commit is contained in:
Alexander Reelsen 2014-03-17 11:30:47 +01:00 committed by Simon Willnauer
parent 7d6ad8d91c
commit 8f6e1d4720
12 changed files with 603 additions and 102 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

225
docs/reference/query-dsl/queries/template-query.asciidoc
View File

@ -8,7 +8,7 @@ template parameters.
[source,js]
------------------------------------------
GET _search
GET /_search
{
"query": {
"template": {
@ -23,15 +23,15 @@ GET _search
------------------------------------------
Alternatively escaping the template works as well:
Alternatively passing the template as an escaped string works as well:
[source,js]
------------------------------------------
GET _search
GET /_search
{
"query": {
"template": {
"query": "{\"match_{{template}}\": {}}\"",
"query": "{\"match_{{template}}\": {}}\"", <1>
"params" : {
"template" : "all"
}
@ -39,18 +39,21 @@ GET _search
}
}
------------------------------------------
<1> New line characters (`\n`) should be escaped as `\\n` or removed,
and quotes (`"`) should be escaped as `\\"`.
You register a template by storing it in the conf/scripts directory of
elasticsearch. In order to execute the stored template reference it in the query parameters:
You can register a template by storing it in the `config/scripts` directory.
In order to execute the stored template, reference it by name in the `query`
parameter:
[source,js]
------------------------------------------
GET _search
GET /_search
{
"query": {
"template": {
"query": "storedTemplate",
"query": "storedTemplate", <1>
"params" : {
"template" : "all"
}
@ -59,7 +62,7 @@ GET _search
}
------------------------------------------
<1> Name of the the query template in `config/scripts/`.
Templating is based on Mustache. For simple token substitution all you provide
is a query containing some variable that you want to substitute and the actual
@ -68,7 +71,7 @@ values:
[source,js]
------------------------------------------
GET _search
GET /_search
{
"query": {
"template": {
@ -79,23 +82,215 @@ GET _search
}
}
}
------------------------------------------
which is then turned into:
[source,js]
------------------------------------------
GET _search
{
"query": {
"match_all": {}
"match_all": {}
}
}
------------------------------------------
There is also a dedicated `template` endpoint, which allows you to specify the template query directly.
You can use the `/_search/template` endpoint for that.
[source,js]
------------------------------------------
GET /_search/template
{
"template" : {
"query": { "match" : { "{{my_field}}" : "{{my_value}}" } },
"size" : {{my_size}}
},
"params" : {
"my_field" : "foo",
"my_value" : "bar",
"my_size" : 5
}
}
------------------------------------------
For more information on how Mustache templating and what kind of templating you
can do with it check out the [online
documentation](http://mustache.github.io/mustache.5.html) of the mustache project.
can do with it check out the http://mustache.github.io/mustache.5.html[online
documentation of the mustache project].
[float]
==== More template examples
[float]
===== Filling in a query string with a single value
[source,js]
------------------------------------------
GET /_search/template
{
"template": {
"query": {
"match": {
"title": "{{query_string}}"
}
}
},
"params": {
"query_string": "search for these words"
}
}
------------------------------------------
[float]
===== Passing an array of strings
[source,js]
------------------------------------------
GET /_search/template
{
"template": {
"query": {
"terms": {
"status": [
"{{#status}}",
"{{.}}",
"{{/status}}"
]
}
}
},
"params": {
"status": [ "pending", "published" ]
}
}
------------------------------------------
which is rendered as:
[source,js]
------------------------------------------
{
"query": {
"terms": {
"status": [ "pending", "published" ]
}
}
------------------------------------------
[float]
===== Default values
A default value is written as `{{var}}{{^var}}default{{/var}}` for instance:
[source,js]
------------------------------------------
{
"template": {
"query": {
"range": {
"line_no": {
"gte": "{{start}}",
"lte": "{{end}}{{^end}}20{{/end}}"
}
}
}
},
"params": { ... }
}
------------------------------------------
When `params` is `{ "start": 10, "end": 15 }` this query would be rendered as:
[source,js]
------------------------------------------
{
"range": {
"line_no": {
"gte": "10",
"lte": "15"
}
}
}
------------------------------------------
But when `params` is `{ "start": 10 }` this query would use the default value
for `end`:
[source,js]
------------------------------------------
{
"range": {
"line_no": {
"gte": "10",
"lte": "20"
}
}
}
------------------------------------------
[float]
===== Conditional clauses
Conditional clauses cannot be expressed using the JSON form of the template.
Instead, the template *must* be passed as a string. For instance, let's say
we wanted to run a `match` query on the `line` field, and optionally wanted
to filter by line numbers, where `start` and `end` are optional.
The `params` would look like:
[source,js]
------------------------------------------
{
"params": {
"text": "words to search for",
"line_no": { <1>
"start": 10, <1>
"end": 20 <1>
}
}
}
------------------------------------------
<1> All three of these elements are optional.
We could write the query as:
[source,js]
------------------------------------------
{
"filtered": {
"query": {
"match": {
"line": "{{text}}" <1>
}
},
"filter": {
{{#line_no}} <2>
"range": {
"line_no": {
{{#start}} <3>
"gte": "{{start}}" <4>
{{#end}},{{/end}} <5>
{{/start}} <3>
{{#end}} <6>
"lte": "{{end}}" <7>
{{/end}} </6>
}
}
{{/line_no}} <2>
}
}
}
------------------------------------------
<1> Fill in the value of param `text`
<2> Include the `range` filter only if `line_no` is specified
<3> Include the `gte` clause only if `line_no.start` is specified
<4> Fill in the value of param `line_no.start`
<5> Add a comma after the `gte` clause only if `line_no.start`
AND `line_no.end` are specified
<6> Include the `lte` clause only if `line_no.end` is specified
<7> Fill in the value of param `line_no.end`
As written above, this template is not valid JSON because it includes the
_section_ markers like `{{#line_no}}`. For this reason, the template
can only be written as a string.

23
rest-api-spec/api/search.template.json Normal file
View File

@ -0,0 +1,23 @@
{
"search-template": {
"documentation": "http://www.elasticsearch.org/guide/en/elasticsearch/reference/master/search-search.html",
"methods": ["GET", "POST"],
"url": {
"path": "/_search/template",
"paths": ["/_search/template", "/{index}/_search/template", "/{index}/{type}/_search/template"],
"parts": {
"index": {
"type" : "list",
"description" : "A comma-separated list of index names to search; use `_all` or empty string to perform the operation on all indices"
},
"type": {
"type" : "list",
"description" : "A comma-separated list of document types to search; leave empty to perform the operation on all types"
}
}
},
"body": {
"description": "The search definition template and its params"
}
}
}

29
rest-api-spec/test/search/40_search_request_template.yaml Normal file
View File

@ -0,0 +1,29 @@
---
"Template search request":
- do:
index:
index: test
type: testtype
id: 1
body: { "text": "value1" }
- do:
index:
index: test
type: testtype
id: 2
body: { "text": "value2" }
- do:
indices.refresh: {}
- do:
search-template:
body: { "template" : { "query": { "term": { "text": { "value": "{{template}}" } } } }, "params": { "template": "value1" } }
- match: { hits.total: 1 }
- do:
search-template:
body: { "template" : { "query": { "match_{{template}}": {} } }, "params" : { "template" : "all" } }
- match: { hits.total: 2 }

84
src/main/java/org/elasticsearch/action/search/SearchRequest.java
View File

@ -21,6 +21,7 @@ package org.elasticsearch.action.search;
import org.elasticsearch.ElasticsearchGenerationException;
import org.elasticsearch.ElasticsearchIllegalArgumentException;
import org.elasticsearch.Version;
import org.elasticsearch.action.ActionRequest;
import org.elasticsearch.action.ActionRequestValidationException;
import org.elasticsearch.action.support.IndicesOptions;
@ -39,6 +40,7 @@ import org.elasticsearch.search.Scroll;
import org.elasticsearch.search.builder.SearchSourceBuilder;
import java.io.IOException;
import java.util.Collections;
import java.util.Map;
import static org.elasticsearch.search.Scroll.readScroll;
@ -69,6 +71,11 @@ public class SearchRequest extends ActionRequest<SearchRequest> {
@Nullable
private String preference;
private BytesReference templateSource;
private boolean templateSourceUnsafe;
private String templateName;
private Map<String, String> templateParams = Collections.emptyMap();
private BytesReference source;
private boolean sourceUnsafe;
@ -123,6 +130,10 @@ public class SearchRequest extends ActionRequest<SearchRequest> {
extraSource = extraSource.copyBytesArray();
extraSourceUnsafe = false;
}
if (templateSource != null && templateSourceUnsafe) {
templateSource = templateSource.copyBytesArray();
templateSourceUnsafe = false;
}
}
/**
@ -327,6 +338,13 @@ public class SearchRequest extends ActionRequest<SearchRequest> {
return source;
}
/**
* The search source template to execute.
*/
public BytesReference templateSource() {
return templateSource;
}
/**
* Allows to provide additional source that will be used as well.
*/
@ -395,6 +413,52 @@ public class SearchRequest extends ActionRequest<SearchRequest> {
return this;
}
/**
* Allows to provide template as source.
*/
public SearchRequest templateSource(BytesReference template, boolean unsafe) {
this.templateSource = template;
this.templateSourceUnsafe = unsafe;
return this;
}
/**
* The template of the search request.
*/
public SearchRequest templateSource(String source) {
this.source = new BytesArray(source);
this.sourceUnsafe = false;
return this;
}
/**
* The name of the stored template
*/
public void templateName(String name) {
this.templateName = name;
}
/**
* Template parameters used for rendering
*/
public void templateParams(Map<String, String> params) {
this.templateParams = params;
}
/**
* The name of the stored template
*/
public String templateName() {
return templateName;
}
/**
* Template parameters used for rendering
*/
public Map<String, String> templateParams() {
return templateParams;
}
/**
* Additional search source to execute.
*/
@ -471,6 +535,15 @@ public class SearchRequest extends ActionRequest<SearchRequest> {
types = in.readStringArray();
indicesOptions = IndicesOptions.readIndicesOptions(in);
if (in.getVersion().onOrAfter(Version.V_1_1_0)) {
templateSourceUnsafe = false;
templateSource = in.readBytesReference();
templateName = in.readOptionalString();
if (in.readBoolean()) {
templateParams = (Map<String, String>) in.readGenericValue();
}
}
}
@Override
@ -497,5 +570,16 @@ public class SearchRequest extends ActionRequest<SearchRequest> {
out.writeBytesReference(extraSource);
out.writeStringArray(types);
indicesOptions.writeIndicesOptions(out);
if (out.getVersion().onOrAfter(Version.V_1_1_0)) {
out.writeBytesReference(templateSource);
out.writeOptionalString(templateName);
boolean existTemplateParams = templateParams != null;
out.writeBoolean(existTemplateParams);
if (existTemplateParams) {
out.writeGenericValue(templateParams);
}
}
}
}

24
src/main/java/org/elasticsearch/action/search/SearchRequestBuilder.java
View File

@ -981,6 +981,30 @@ public class SearchRequestBuilder extends ActionRequestBuilder<SearchRequest, Se
return this;
}
/**
* template stuff
*/
public SearchRequestBuilder setTemplateName(String templateName) {
request.templateName(templateName);
return this;
}
public SearchRequestBuilder setTemplateParams(Map<String,String> templateParams) {
request.templateParams(templateParams);
return this;
}
public SearchRequestBuilder setTemplateSource(String source) {
request.templateSource(source);
return this;
}
public SearchRequestBuilder setTemplateSource(BytesReference source) {
request.templateSource(source, true);
return this;
}
/**
* Sets the source builder to be used with this request. Note, any operations done
* on this require builder before are discarded as this internal builder replaces

95
src/main/java/org/elasticsearch/index/query/TemplateQueryParser.java
View File

@ -25,12 +25,10 @@ import org.elasticsearch.common.inject.Inject;
import org.elasticsearch.common.xcontent.XContentBuilder;
import org.elasticsearch.common.xcontent.XContentFactory;
import org.elasticsearch.common.xcontent.XContentParser;
import org.elasticsearch.common.xcontent.json.JsonXContent;
import org.elasticsearch.script.ExecutableScript;
import org.elasticsearch.script.ScriptService;
import java.io.IOException;
import java.util.HashMap;
import java.util.Map;
/**
@ -45,20 +43,14 @@ public class TemplateQueryParser implements QueryParser {
public static final String QUERY = "query";
/** Name of query parameter containing the template parameters. */
public static final String PARAMS = "params";
/** This is what we are registered with for query executions. */
private final ScriptService scriptService;
/**
* @param scriptService will automatically be wired by Guice
* */
@Inject
public TemplateQueryParser(ScriptService scriptService) {
this.scriptService = scriptService;
}
/**
* @return a list of names this query is registered under.
* */
@Override
public String[] names() {
return new String[] {NAME};
@ -68,46 +60,8 @@ public class TemplateQueryParser implements QueryParser {
@Nullable
public Query parse(QueryParseContext parseContext) throws IOException {
XContentParser parser = parseContext.parser();
String template = "";
Map<String, Object> vars = new HashMap<String, Object>();
String currentFieldName = null;
XContentParser.Token token;
while ((token = parser.nextToken()) != XContentParser.Token.END_OBJECT) {
if (token == XContentParser.Token.FIELD_NAME) {
currentFieldName = parser.currentName();
} else if (QUERY.equals(currentFieldName)) {
if (token == XContentParser.Token.START_OBJECT && ! parser.hasTextCharacters()) {
// when called with un-escaped json string
XContentBuilder builder = XContentBuilder.builder(JsonXContent.jsonXContent);
builder.copyCurrentStructure(parser);
template = builder.string();
} else {
// when called with excaped json string or when called with filename
template = parser.text();
}
} else if (PARAMS.equals(currentFieldName)) {
XContentParser.Token innerToken;
String key = null;
while ((innerToken = parser.nextToken()) != XContentParser.Token.END_OBJECT) {
// parsing template parameter map
if (innerToken == XContentParser.Token.FIELD_NAME) {
key = parser.currentName();
} else {
if (key != null) {
vars.put(key, parser.text());
} else {
throw new IllegalStateException("Template parameter key must not be null.");
}
key = null;
}
}
}
}
ExecutableScript executable = this.scriptService.executable("mustache", template, vars);
TemplateContext templateContext = parse(parser, QUERY, PARAMS);
ExecutableScript executable = this.scriptService.executable("mustache", templateContext.template(), templateContext.params());
BytesReference querySource = (BytesReference) executable.run();
XContentParser qSourceParser = XContentFactory.xContent(querySource).createParser(querySource);
@ -121,4 +75,47 @@ public class TemplateQueryParser implements QueryParser {
qSourceParser.close();
}
}
public static TemplateContext parse(XContentParser parser, String templateFieldname, String paramsFieldname) throws IOException {
Map<String, Object> params = null;
String templateNameOrTemplateContent = null;
String currentFieldName = null;
XContentParser.Token token;
while ((token = parser.nextToken()) != XContentParser.Token.END_OBJECT) {
if (token == XContentParser.Token.FIELD_NAME) {
currentFieldName = parser.currentName();
} else if (templateFieldname.equals(currentFieldName)) {
if (token == XContentParser.Token.START_OBJECT && !parser.hasTextCharacters()) {
XContentBuilder builder = XContentBuilder.builder(parser.contentType().xContent());
builder.copyCurrentStructure(parser);
templateNameOrTemplateContent = builder.string();
} else {
templateNameOrTemplateContent = parser.text();
}
} else if (paramsFieldname.equals(currentFieldName)) {
params = parser.map();
}
}
return new TemplateContext(templateNameOrTemplateContent, params);
}
public static class TemplateContext {
private Map<String, Object> params;
private String template;
public TemplateContext(String templateName, Map<String, Object> params) {
this.params = params;
this.template = templateName;
}
public Map<String, Object> params() {
return params;
}
public String template() {
return template;
}
}
}

31
src/main/java/org/elasticsearch/rest/action/search/RestSearchAction.java
View File

@ -55,12 +55,18 @@ public class RestSearchAction extends BaseRestHandler {
@Inject
public RestSearchAction(Settings settings, Client client, RestController controller) {
super(settings, client);
controller.registerHandler(GET, "/_search", this);
controller.registerHandler(GET, "/_search", this);
controller.registerHandler(POST, "/_search", this);
controller.registerHandler(GET, "/{index}/_search", this);
controller.registerHandler(GET, "/{index}/_search", this);
controller.registerHandler(POST, "/{index}/_search", this);
controller.registerHandler(GET, "/{index}/{type}/_search", this);
controller.registerHandler(GET, "/{index}/{type}/_search", this);
controller.registerHandler(POST, "/{index}/{type}/_search", this);
controller.registerHandler(GET, "/_search/template", this);
controller.registerHandler(POST, "/_search/template", this);
controller.registerHandler(GET, "/{index}/_search/template", this);
controller.registerHandler(POST, "/{index}/_search/template", this);
controller.registerHandler(GET, "/{index}/{type}/_search/template", this);
controller.registerHandler(POST, "/{index}/{type}/_search/template", this);
}
@Override
@ -121,16 +127,29 @@ public class RestSearchAction extends BaseRestHandler {
String[] indices = Strings.splitStringByCommaToArray(request.param("index"));
SearchRequest searchRequest = new SearchRequest(indices);
// get the content, and put it in the body
// add content/source as template if template flag is set
boolean isTemplateRequest = request.path().endsWith("/template");
if (request.hasContent()) {
searchRequest.source(request.content(), request.contentUnsafe());
if (isTemplateRequest) {
searchRequest.templateSource(request.content(), request.contentUnsafe());
} else {
searchRequest.source(request.content(), request.contentUnsafe());
}
} else {
String source = request.param("source");
if (source != null) {
searchRequest.source(source);
if (isTemplateRequest) {
searchRequest.templateSource(source);
} else {
searchRequest.source(source);
}
}
}
// add extra source based on the request parameters
searchRequest.extraSource(parseSearchSource(request));
if (!isTemplateRequest) {
searchRequest.extraSource(parseSearchSource(request));
}
searchRequest.searchType(request.param("search_type"));

41
src/main/java/org/elasticsearch/search/SearchService.java
View File

@ -26,7 +26,9 @@ import com.google.common.collect.ImmutableMap;
import org.apache.lucene.index.AtomicReaderContext;
import org.apache.lucene.index.NumericDocValues;
import org.apache.lucene.search.TopDocs;
import org.apache.lucene.util.IOUtils;
import org.elasticsearch.ElasticsearchException;
import org.elasticsearch.ElasticsearchParseException;
import org.elasticsearch.ExceptionsHelper;
import org.elasticsearch.action.search.SearchType;
import org.elasticsearch.cache.recycler.CacheRecycler;
@ -42,9 +44,7 @@ import org.elasticsearch.common.unit.TimeValue;
import org.elasticsearch.common.util.BigArrays;
import org.elasticsearch.common.util.concurrent.ConcurrentCollections;
import org.elasticsearch.common.util.concurrent.ConcurrentMapLong;
import org.elasticsearch.common.xcontent.XContentFactory;
import org.elasticsearch.common.xcontent.XContentHelper;
import org.elasticsearch.common.xcontent.XContentParser;
import org.elasticsearch.common.xcontent.*;
import org.elasticsearch.index.engine.Engine;
import org.elasticsearch.index.fielddata.FieldDataType;
import org.elasticsearch.index.fielddata.IndexFieldDataService;
@ -53,6 +53,7 @@ import org.elasticsearch.index.mapper.FieldMapper;
import org.elasticsearch.index.mapper.FieldMapper.Loading;
import org.elasticsearch.index.mapper.MapperService;
import org.elasticsearch.index.mapper.internal.ParentFieldMapper;
import org.elasticsearch.index.query.TemplateQueryParser;
import org.elasticsearch.index.search.stats.StatsGroupsParseElement;
import org.elasticsearch.index.service.IndexService;
import org.elasticsearch.index.shard.service.IndexShard;
@ -60,6 +61,7 @@ import org.elasticsearch.indices.IndicesLifecycle;
import org.elasticsearch.indices.IndicesService;
import org.elasticsearch.indices.warmer.IndicesWarmer;
import org.elasticsearch.indices.warmer.IndicesWarmer.WarmerContext;
import org.elasticsearch.script.ExecutableScript;
import org.elasticsearch.script.ScriptService;
import org.elasticsearch.search.dfs.CachedDfSource;
import org.elasticsearch.search.dfs.DfsPhase;
@ -73,6 +75,7 @@ import org.elasticsearch.search.query.*;
import org.elasticsearch.search.warmer.IndexWarmersMetaData;
import org.elasticsearch.threadpool.ThreadPool;
import java.io.IOException;
import java.util.HashMap;
import java.util.Iterator;
import java.util.Map;
@ -81,6 +84,7 @@ import java.util.concurrent.Executor;
import java.util.concurrent.ScheduledFuture;
import java.util.concurrent.atomic.AtomicLong;
import static org.elasticsearch.common.Strings.hasLength;
import static org.elasticsearch.common.unit.TimeValue.timeValueMinutes;
/**
@ -499,6 +503,7 @@ public class SearchService extends AbstractLifecycleComponent<SearchService> {
try {
context.scroll(request.scroll());
parseTemplate(request);
parseSource(context, request.source());
parseSource(context, request.extraSource());
@ -567,6 +572,36 @@ public class SearchService extends AbstractLifecycleComponent<SearchService> {
SearchContext.removeCurrent();
}
private void parseTemplate(ShardSearchRequest request) {
if (hasLength(request.templateName())) {
ExecutableScript executable = this.scriptService.executable("mustache", request.templateName(), request.templateParams());
BytesReference processedQuery = (BytesReference) executable.run();
request.source(processedQuery);
} else {
if (request.templateSource() == null || request.templateSource().length() == 0) {
return;
}
XContentParser parser = null;
try {
parser = XContentFactory.xContent(request.templateSource()).createParser(request.templateSource());
TemplateQueryParser.TemplateContext templateContext = TemplateQueryParser.parse(parser, "template", "params");
if (!hasLength(templateContext.template())) {
throw new ElasticsearchParseException("Template must have [template] field configured");
}
ExecutableScript executable = this.scriptService.executable("mustache", templateContext.template(), templateContext.params());
BytesReference processedQuery = (BytesReference) executable.run();
request.source(processedQuery);
} catch (IOException e) {
logger.error("Error trying to parse template: ", e);
} finally {
IOUtils.closeWhileHandlingException(parser);
}
}
}
private void parseSource(SearchContext context, BytesReference source) throws SearchParseException {
// nothing to parse...
if (source == null || source.length() == 0) {

38
src/main/java/org/elasticsearch/search/internal/ShardSearchRequest.java
View File

@ -19,6 +19,7 @@
package org.elasticsearch.search.internal;
import org.elasticsearch.Version;
import org.elasticsearch.action.search.SearchRequest;
import org.elasticsearch.action.search.SearchType;
import org.elasticsearch.cluster.routing.ShardRouting;
@ -30,6 +31,7 @@ import org.elasticsearch.search.Scroll;
import org.elasticsearch.transport.TransportRequest;
import java.io.IOException;
import java.util.Map;
import static org.elasticsearch.search.Scroll.readScroll;
@ -68,6 +70,9 @@ public class ShardSearchRequest extends TransportRequest {
private BytesReference source;
private BytesReference extraSource;
private BytesReference templateSource;
private String templateName;
private Map<String, String> templateParams;
private long nowInMillis;
@ -82,6 +87,9 @@ public class ShardSearchRequest extends TransportRequest {
this.searchType = searchRequest.searchType();
this.source = searchRequest.source();
this.extraSource = searchRequest.extraSource();
this.templateSource = searchRequest.templateSource();
this.templateName = searchRequest.templateName();
this.templateParams = searchRequest.templateParams();
this.scroll = searchRequest.scroll();
this.types = searchRequest.types();
@ -132,6 +140,18 @@ public class ShardSearchRequest extends TransportRequest {
return this;
}
public BytesReference templateSource() {
return this.templateSource;
}
public String templateName() {
return templateName;
}
public Map<String, String> templateParams() {
return templateParams;
}
public ShardSearchRequest nowInMillis(long nowInMillis) {
this.nowInMillis = nowInMillis;
return this;
@ -185,6 +205,14 @@ public class ShardSearchRequest extends TransportRequest {
types = in.readStringArray();
filteringAliases = in.readStringArray();
nowInMillis = in.readVLong();
if (in.getVersion().onOrAfter(Version.V_1_1_0)) {
templateSource = in.readBytesReference();
templateName = in.readOptionalString();
if (in.readBoolean()) {
templateParams = (Map<String, String>) in.readGenericValue();
}
}
}
@Override
@ -205,5 +233,15 @@ public class ShardSearchRequest extends TransportRequest {
out.writeStringArray(types);
out.writeStringArrayNullable(filteringAliases);
out.writeVLong(nowInMillis);
if (out.getVersion().onOrAfter(Version.V_1_1_0)) {
out.writeBytesReference(templateSource);
out.writeOptionalString(templateName);
boolean existTemplateParams = templateParams != null;
out.writeBoolean(existTemplateParams);
if (existTemplateParams) {
out.writeGenericValue(templateParams);
}
}
}
}

22
src/test/java/org/elasticsearch/index/query/TemplateQueryParserTest.java
View File

@ -31,6 +31,8 @@ import org.elasticsearch.common.settings.Settings;
import org.elasticsearch.common.settings.SettingsModule;
import org.elasticsearch.common.xcontent.XContentFactory;
import org.elasticsearch.common.xcontent.XContentParser;
import org.elasticsearch.env.Environment;
import org.elasticsearch.env.EnvironmentModule;
import org.elasticsearch.index.Index;
import org.elasticsearch.index.IndexNameModule;
import org.elasticsearch.index.analysis.AnalysisModule;
@ -58,13 +60,15 @@ public class TemplateQueryParserTest extends ElasticsearchTestCase {
private Injector injector;
private QueryParseContext context;
@Before
public void setup() {
Settings settings = ImmutableSettings.Builder.EMPTY_SETTINGS;
public void setup() throws IOException {
String scriptPath = this.getClass().getResource("config").getPath();
Settings settings = ImmutableSettings.settingsBuilder().put("path.conf", scriptPath).build();
Index index = new Index("test");
injector = new ModulesBuilder().add(
new EnvironmentModule(new Environment(settings)),
new SettingsModule(settings),
new CacheRecyclerModule(settings),
new CodecModule(settings),
@ -105,4 +109,16 @@ public class TemplateQueryParserTest extends ElasticsearchTestCase {
Query query = parser.parse(context);
assertTrue("Parsing template query failed.", query instanceof ConstantScoreQuery);
}
@Test
public void testParserCanExtractTemplateNames() throws Exception {
String templateString = "{ \"template\": { \"query\": \"storedTemplate\" ,\"params\":{\"template\":\"all\" } } } ";
XContentParser templateSourceParser = XContentFactory.xContent(templateString).createParser(templateString);
context.reset(templateSourceParser);
TemplateQueryParser parser = injector.getInstance(TemplateQueryParser.class);
Query query = parser.parse(context);
assertTrue("Parsing template query failed.", query instanceof ConstantScoreQuery);
}
}

87
src/test/java/org/elasticsearch/index/query/TemplateQueryTest.java
View File

@ -18,7 +18,11 @@
*/
package org.elasticsearch.index.query;
import com.google.common.collect.Maps;
import org.elasticsearch.action.search.SearchRequest;
import org.elasticsearch.action.search.SearchResponse;
import org.elasticsearch.common.bytes.BytesArray;
import org.elasticsearch.common.bytes.BytesReference;
import org.elasticsearch.common.settings.ImmutableSettings;
import org.elasticsearch.common.settings.Settings;
import org.elasticsearch.test.ElasticsearchIntegrationTest;
@ -30,6 +34,11 @@ import java.io.IOException;
import java.util.HashMap;
import java.util.Map;
import static org.elasticsearch.common.settings.ImmutableSettings.settingsBuilder;
import static org.elasticsearch.common.xcontent.XContentFactory.jsonBuilder;
import static org.elasticsearch.test.hamcrest.ElasticsearchAssertions.assertHitCount;
import static org.hamcrest.Matchers.is;
/**
* Full integration test of the template query plugin.
* */
@ -37,17 +46,21 @@ import java.util.Map;
public class TemplateQueryTest extends ElasticsearchIntegrationTest {
@Before
public void setup() {
public void setup() throws IOException {
createIndex("test");
ensureGreen();
ensureGreen("test");
client().prepareIndex("test", "testtype").setId("1")
.setSource("text", "value1").get();
client().prepareIndex("test", "testtype").setId("2")
.setSource("text", "value2").get();
index("test", "testtype", "1", jsonBuilder().startObject().field("text", "value1").endObject());
index("test", "testtype", "2", jsonBuilder().startObject().field("text", "value2").endObject());
refresh();
}
@Override
public Settings nodeSettings(int nodeOrdinal) {
String scriptPath = this.getClass().getResource("config").getPath();
return settingsBuilder().put("path.conf", scriptPath).build();
}
@Test
public void testTemplateInBody() throws IOException {
Map<String, Object> vars = new HashMap<String, Object>();
@ -57,7 +70,7 @@ public class TemplateQueryTest extends ElasticsearchIntegrationTest {
"{\"match_{{template}}\": {}}\"", vars);
SearchResponse sr = client().prepareSearch().setQuery(builder)
.execute().actionGet();
ElasticsearchAssertions.assertHitCount(sr, 2);
assertHitCount(sr, 2);
}
@Test
@ -68,7 +81,7 @@ public class TemplateQueryTest extends ElasticsearchIntegrationTest {
"{\"match_all\": {}}\"", vars);
SearchResponse sr = client().prepareSearch().setQuery(builder)
.execute().actionGet();
ElasticsearchAssertions.assertHitCount(sr, 2);
assertHitCount(sr, 2);
}
@Test
@ -80,7 +93,7 @@ public class TemplateQueryTest extends ElasticsearchIntegrationTest {
"storedTemplate", vars);
SearchResponse sr = client().prepareSearch().setQuery(builder)
.execute().actionGet();
ElasticsearchAssertions.assertHitCount(sr, 2);
assertHitCount(sr, 2);
}
@ -88,37 +101,59 @@ public class TemplateQueryTest extends ElasticsearchIntegrationTest {
public void testRawEscapedTemplate() throws IOException {
String query = "{\"template\": {\"query\": \"{\\\"match_{{template}}\\\": {}}\\\"\",\"params\" : {\"template\" : \"all\"}}}";
SearchResponse sr = client().prepareSearch().setQuery(query)
.execute().actionGet();
ElasticsearchAssertions.assertHitCount(sr, 2);
SearchResponse sr = client().prepareSearch().setQuery(query).get();
assertHitCount(sr, 2);
}
@Test
public void testRawTemplate() throws IOException {
String query = "{\"template\": {\"query\": {\"match_{{template}}\": {}},\"params\" : {\"template\" : \"all\"}}}";
SearchResponse sr = client().prepareSearch().setQuery(query)
.execute().actionGet();
ElasticsearchAssertions.assertHitCount(sr, 2);
SearchResponse sr = client().prepareSearch().setQuery(query).get();
assertHitCount(sr, 2);
}
@Test
public void testRawFSTemplate() throws IOException {
String query = "{\"template\": {\"query\": \"storedTemplate\",\"params\" : {\"template\" : \"all\"}}}";
SearchResponse sr = client().prepareSearch().setQuery(query)
.execute().actionGet();
ElasticsearchAssertions.assertHitCount(sr, 2);
SearchResponse sr = client().prepareSearch().setQuery(query).get();
assertHitCount(sr, 2);
}
@Override
public Settings nodeSettings(int nodeOrdinal) {
String scriptPath = this.getClass()
.getResource("config").getPath();
@Test
public void testSearchRequestTemplateSource() throws Exception {
SearchRequest searchRequest = new SearchRequest();
searchRequest.indices("_all");
Settings settings = ImmutableSettings
.settingsBuilder()
.put("path.conf", scriptPath).build();
String query = "{ \"template\" : { \"query\": {\"match_{{template}}\": {} } }, \"params\" : { \"template\":\"all\" } }";
BytesReference bytesRef = new BytesArray(query);
searchRequest.templateSource(bytesRef, false);
return settings;
SearchResponse searchResponse = client().search(searchRequest).get();
assertHitCount(searchResponse, 2);
}
@Test
public void testThatParametersCanBeSet() throws Exception {
index("test", "type", "1", jsonBuilder().startObject().field("theField", "foo").endObject());
index("test", "type", "2", jsonBuilder().startObject().field("theField", "foo 2").endObject());
index("test", "type", "3", jsonBuilder().startObject().field("theField", "foo 3").endObject());
index("test", "type", "4", jsonBuilder().startObject().field("theField", "foo 4").endObject());
index("test", "type", "5", jsonBuilder().startObject().field("otherField", "foo").endObject());
refresh();
Map<String, String> templateParams = Maps.newHashMap();
templateParams.put("mySize", "2");
templateParams.put("myField", "theField");
templateParams.put("myValue", "foo");
SearchResponse searchResponse = client().prepareSearch("test").setTypes("type").setTemplateName("full-query-template").setTemplateParams(templateParams).get();
assertHitCount(searchResponse, 4);
// size kicks in here...
assertThat(searchResponse.getHits().getHits().length, is(2));
templateParams.put("myField", "otherField");
searchResponse = client().prepareSearch("test").setTypes("type").setTemplateName("full-query-template").setTemplateParams(templateParams).get();
assertHitCount(searchResponse, 1);
}
}

6
src/test/resources/org/elasticsearch/index/query/config/scripts/full-query-template.mustache Normal file
View File

@ -0,0 +1,6 @@
{
"query": {
"match": { "{{myField}}" : "{{myValue}}" }
},
"size" : {{mySize}}
}