[[query-string-syntax]] ===== Query string syntax The query string ``mini-language'' is used by the <> and by the `q` query string parameter in the <>. The query string is parsed into a series of _terms_ and _operators_. A term can be a single word -- `quick` or `brown` -- or a phrase, surrounded by double quotes -- `"quick brown"` -- which searches for all the words in the phrase, in the same order. Operators allow you to customize the search -- the available options are explained below. ====== Field names You can specify fields to search in the query syntax: * where the `status` field contains `active` status:active * where the `title` field contains `quick` or `brown` title:(quick OR brown) * where the `author` field contains the exact phrase `"john smith"` author:"John Smith" * where the `first name` field contains `Alice` (note how we need to escape the space with a backslash) first\ name:Alice * where any of the fields `book.title`, `book.content` or `book.date` contains `quick` or `brown` (note how we need to escape the `*` with a backslash): book.\*:(quick OR brown) * where the field `title` has any non-null value: _exists_:title [[query-string-wildcard]] ====== Wildcards Wildcard searches can be run on individual terms, using `?` to replace a single character, and `*` to replace zero or more characters: qu?ck bro* Be aware that wildcard queries can use an enormous amount of memory and perform very badly -- just think how many terms need to be queried to match the query string `"a* b* c*"`. [WARNING] ======= Pure wildcards `\*` are rewritten to <> queries for efficiency. As a consequence, the wildcard `"field:*"` would match documents with an empty value like the following: ``` { "field": "" } ``` \... and would **not** match if the field is missing or set with an explicit null value like the following: ``` { "field": null } ``` ======= [WARNING] ======= Allowing a wildcard at the beginning of a word (eg `"*ing"`) is particularly heavy, because all terms in the index need to be examined, just in case they match. Leading wildcards can be disabled by setting `allow_leading_wildcard` to `false`. ======= Only parts of the analysis chain that operate at the character level are applied. So for instance, if the analyzer performs both lowercasing and stemming, only the lowercasing will be applied: it would be wrong to perform stemming on a word that is missing some of its letters. By setting `analyze_wildcard` to true, queries that end with a `*` will be analyzed and a boolean query will be built out of the different tokens, by ensuring exact matches on the first N-1 tokens, and prefix match on the last token. ====== Regular expressions Regular expression patterns can be embedded in the query string by wrapping them in forward-slashes (`"/"`): name:/joh?n(ath[oa]n)/ The supported regular expression syntax is explained in <>. [WARNING] ======= The `allow_leading_wildcard` parameter does not have any control over regular expressions. A query string such as the following would force Elasticsearch to visit every term in the index: /.*n/ Use with caution! ======= [[query-string-fuzziness]] ====== Fuzziness We can search for terms that are similar to, but not exactly like our search terms, using the ``fuzzy'' operator: quikc~ brwn~ foks~ This uses the {wikipedia}/Damerau-Levenshtein_distance[Damerau-Levenshtein distance] to find all terms with a maximum of two changes, where a change is the insertion, deletion or substitution of a single character, or transposition of two adjacent characters. The default _edit distance_ is `2`, but an edit distance of `1` should be sufficient to catch 80% of all human misspellings. It can be specified as: quikc~1 [[avoid-widlcards-fuzzy-searches]] [WARNING] .Avoid mixing fuzziness with wildcards ==== Mixing <> and <> operators is _not_ supported. When mixed, one of the operators is not applied. For example, you can search for `app~1` (fuzzy) or `app*` (wildcard), but searches for `app*~1` do not apply the fuzzy operator (`~1`). ==== ====== Proximity searches While a phrase query (eg `"john smith"`) expects all of the terms in exactly the same order, a proximity query allows the specified words to be further apart or in a different order. In the same way that fuzzy queries can specify a maximum edit distance for characters in a word, a proximity search allows us to specify a maximum edit distance of words in a phrase: "fox quick"~5 The closer the text in a field is to the original order specified in the query string, the more relevant that document is considered to be. When compared to the above example query, the phrase `"quick fox"` would be considered more relevant than `"quick brown fox"`. ====== Ranges Ranges can be specified for date, numeric or string fields. Inclusive ranges are specified with square brackets `[min TO max]` and exclusive ranges with curly brackets `{min TO max}`. * All days in 2012: date:[2012-01-01 TO 2012-12-31] * Numbers 1..5 count:[1 TO 5] * Tags between `alpha` and `omega`, excluding `alpha` and `omega`: tag:{alpha TO omega} * Numbers from 10 upwards count:[10 TO *] * Dates before 2012 date:{* TO 2012-01-01} Curly and square brackets can be combined: * Numbers from 1 up to but not including 5 count:[1 TO 5} Ranges with one side unbounded can use the following syntax: age:>10 age:>=10 age:<10 age:<=10 [NOTE] ==================================================================== To combine an upper and lower bound with the simplified syntax, you would need to join two clauses with an `AND` operator: age:(>=10 AND <20) age:(+>=10 +<20) ==================================================================== The parsing of ranges in query strings can be complex and error prone. It is much more reliable to use an explicit <>. ====== Boosting Use the _boost_ operator `^` to make one term more relevant than another. For instance, if we want to find all documents about foxes, but we are especially interested in quick foxes: quick^2 fox The default `boost` value is 1, but can be any positive floating point number. Boosts between 0 and 1 reduce relevance. Boosts can also be applied to phrases or to groups: "john smith"^2 (foo bar)^4 ====== Boolean operators By default, all terms are optional, as long as one term matches. A search for `foo bar baz` will find any document that contains one or more of `foo` or `bar` or `baz`. We have already discussed the `default_operator` above which allows you to force all terms to be required, but there are also _boolean operators_ which can be used in the query string itself to provide more control. The preferred operators are `+` (this term *must* be present) and `-` (this term *must not* be present). All other terms are optional. For example, this query: quick brown +fox -news states that: * `fox` must be present * `news` must not be present * `quick` and `brown` are optional -- their presence increases the relevance The familiar boolean operators `AND`, `OR` and `NOT` (also written `&&`, `||` and `!`) are also supported but beware that they do not honor the usual precedence rules, so parentheses should be used whenever multiple operators are used together. For instance the previous query could be rewritten as: `((quick AND fox) OR (brown AND fox) OR fox) AND NOT news`:: This form now replicates the logic from the original query correctly, but the relevance scoring bears little resemblance to the original. In contrast, the same query rewritten using the <> would look like this: { "bool": { "must": { "match": "fox" }, "should": { "match": "quick brown" }, "must_not": { "match": "news" } } } ====== Grouping Multiple terms or clauses can be grouped together with parentheses, to form sub-queries: (quick OR brown) AND fox Groups can be used to target a particular field, or to boost the result of a sub-query: status:(active OR pending) title:(full text search)^2 ====== Reserved characters If you need to use any of the characters which function as operators in your query itself (and not as operators), then you should escape them with a leading backslash. For instance, to search for `(1+1)=2`, you would need to write your query as `\(1\+1\)\=2`. When using JSON for the request body, two preceding backslashes (`\\`) are required; the backslash is a reserved escaping character in JSON strings. [source,console] ---- GET /my-index-000001/_search { "query" : { "query_string" : { "query" : "kimchy\\!", "fields" : ["user.id"] } } } ---- // TEST[setup:my_index] The reserved characters are: `+ - = && || > < ! ( ) { } [ ] ^ " ~ * ? : \ /` Failing to escape these special characters correctly could lead to a syntax error which prevents your query from running. NOTE: `<` and `>` can't be escaped at all. The only way to prevent them from attempting to create a range query is to remove them from the query string entirely. ====== Whitespaces and empty queries Whitespace is not considered an operator. If the query string is empty or only contains whitespaces the query will yield an empty result set.