478 lines
22 KiB
Plaintext
478 lines
22 KiB
Plaintext
[[java-rest-high-search]]
|
|
=== Search API
|
|
|
|
[[java-rest-high-document-search-request]]
|
|
==== Search Request
|
|
|
|
The `SearchRequest` is used for any operation that has to do with searching
|
|
documents, aggregations, suggestions and also offers ways of requesting
|
|
highlighting on the resulting documents.
|
|
|
|
In its most basic form, we can add a query to the request:
|
|
|
|
["source","java",subs="attributes,callouts,macros"]
|
|
--------------------------------------------------
|
|
include-tagged::{doc-tests}/SearchDocumentationIT.java[search-request-basic]
|
|
--------------------------------------------------
|
|
|
|
<1> Creates the `SeachRequest`. Without arguments this runs against all indices.
|
|
<2> Most search parameters are added to the `SearchSourceBuilder`. It offers setters for everything that goes into the search request body.
|
|
<3> Add a `match_all` query to the `SearchSourceBuilder`.
|
|
|
|
===== Optional arguments
|
|
|
|
Let's first look at some of the optional arguments of a `SearchRequest`:
|
|
|
|
["source","java",subs="attributes,callouts,macros"]
|
|
--------------------------------------------------
|
|
include-tagged::{doc-tests}/SearchDocumentationIT.java[search-request-indices-types]
|
|
--------------------------------------------------
|
|
<1> Restricts the request to an index
|
|
<2> Limits the request to a type
|
|
|
|
There are a couple of other interesting optional parameters:
|
|
|
|
["source","java",subs="attributes,callouts,macros"]
|
|
--------------------------------------------------
|
|
include-tagged::{doc-tests}/SearchDocumentationIT.java[search-request-routing]
|
|
--------------------------------------------------
|
|
<1> Set a routing parameter
|
|
|
|
["source","java",subs="attributes,callouts,macros"]
|
|
--------------------------------------------------
|
|
include-tagged::{doc-tests}/SearchDocumentationIT.java[search-request-indicesOptions]
|
|
--------------------------------------------------
|
|
<1> Setting `IndicesOptions` controls how unavailable indices are resolved and
|
|
how wildcard expressions are expanded
|
|
|
|
["source","java",subs="attributes,callouts,macros"]
|
|
--------------------------------------------------
|
|
include-tagged::{doc-tests}/SearchDocumentationIT.java[search-request-preference]
|
|
--------------------------------------------------
|
|
<1> Use the preference parameter e.g. to execute the search to prefer local
|
|
shards. The default is to randomize across shards.
|
|
|
|
===== Using the SearchSourceBuilder
|
|
|
|
Most options controlling the search behavior can be set on the
|
|
`SearchSourceBuilder`,
|
|
which contains more or less the equivalent of the options in the search request
|
|
body of the Rest API.
|
|
|
|
Here are a few examples of some common options:
|
|
|
|
["source","java",subs="attributes,callouts,macros"]
|
|
--------------------------------------------------
|
|
include-tagged::{doc-tests}/SearchDocumentationIT.java[search-source-basics]
|
|
--------------------------------------------------
|
|
<1> Create a `SearchSourceBuilder` with default options.
|
|
<2> Set the query. Can be any type of `QueryBuilder`
|
|
<3> Set the `from` option that determines the result index to start searching
|
|
from. Defaults to 0.
|
|
<4> Set the `size` option that determines the number of search hits to return.
|
|
Defaults to 10.
|
|
<5> Set an optional timeout that controls how long the search is allowed to
|
|
take.
|
|
|
|
After this, the `SearchSourceBuilder` only needs to be added to the
|
|
`SearchRequest`:
|
|
|
|
["source","java",subs="attributes,callouts,macros"]
|
|
--------------------------------------------------
|
|
include-tagged::{doc-tests}/SearchDocumentationIT.java[search-source-setter]
|
|
--------------------------------------------------
|
|
|
|
===== Building queries
|
|
|
|
Search queries are created using `QueryBuilder` objects. A `QueryBuilder` exists
|
|
for every search query type supported by Elasticsearch's {ref}/query-dsl.html[Query DSL].
|
|
|
|
A `QueryBuilder` can be created using its constructor:
|
|
|
|
["source","java",subs="attributes,callouts,macros"]
|
|
--------------------------------------------------
|
|
include-tagged::{doc-tests}/SearchDocumentationIT.java[search-query-builder-ctor]
|
|
--------------------------------------------------
|
|
<1> Create a full text {ref}/query-dsl-match-query.html[Match Query] that matches
|
|
the text "kimchy" over the field "user".
|
|
|
|
Once created, the `QueryBuilder` object provides methods to configure the options
|
|
of the search query it creates:
|
|
|
|
["source","java",subs="attributes,callouts,macros"]
|
|
--------------------------------------------------
|
|
include-tagged::{doc-tests}/SearchDocumentationIT.java[search-query-builder-options]
|
|
--------------------------------------------------
|
|
<1> Enable fuzzy matching on the match query
|
|
<2> Set the prefix length option on the match query
|
|
<3> Set the max expansion options to control the fuzzy
|
|
process of the query
|
|
|
|
`QueryBuilder` objects can also be created using the `QueryBuilders` utility class.
|
|
This class provides helper methods that can be used to create `QueryBuilder` objects
|
|
using a fluent programming style:
|
|
|
|
["source","java",subs="attributes,callouts,macros"]
|
|
--------------------------------------------------
|
|
include-tagged::{doc-tests}/SearchDocumentationIT.java[search-query-builders]
|
|
--------------------------------------------------
|
|
|
|
Whatever the method used to create it, the `QueryBuilder` object must be added
|
|
to the `SearchSourceBuilder` as follows:
|
|
|
|
["source","java",subs="attributes,callouts,macros"]
|
|
--------------------------------------------------
|
|
include-tagged::{doc-tests}/SearchDocumentationIT.java[search-query-setter]
|
|
--------------------------------------------------
|
|
|
|
The <<java-rest-high-search-queries, Search Queries>> page gives a list of all available search queries with
|
|
their corresponding `QueryBuilder` objects and `QueryBuilders` helper methods.
|
|
|
|
|
|
===== Specifying Sorting
|
|
|
|
The `SearchSourceBuilder` allows to add one or more `SortBuilder` instances. There are four special implementations (Field-, Score-, GeoDistance- and ScriptSortBuilder).
|
|
|
|
["source","java",subs="attributes,callouts,macros"]
|
|
--------------------------------------------------
|
|
include-tagged::{doc-tests}/SearchDocumentationIT.java[search-source-sorting]
|
|
--------------------------------------------------
|
|
<1> Sort descending by `_score` (the default)
|
|
<2> Also sort ascending by `_id` field
|
|
|
|
===== Source filtering
|
|
|
|
By default, search requests return the contents of the document `_source` but like in the Rest API you can overwrite this behavior. For example, you can turn off `_source` retrieval completely:
|
|
|
|
["source","java",subs="attributes,callouts,macros"]
|
|
--------------------------------------------------
|
|
include-tagged::{doc-tests}/SearchDocumentationIT.java[search-source-filtering-off]
|
|
--------------------------------------------------
|
|
|
|
The method also accepts an array of one or more wildcard patterns to control which fields get included or excluded in a more fine grained way:
|
|
|
|
["source","java",subs="attributes,callouts,macros"]
|
|
--------------------------------------------------
|
|
include-tagged::{doc-tests}/SearchDocumentationIT.java[search-source-filtering-includes]
|
|
--------------------------------------------------
|
|
|
|
[[java-rest-high-request-highlighting]]
|
|
===== Requesting Highlighting
|
|
|
|
Highlighting search results can be achieved by setting a `HighlightBuilder` on the
|
|
`SearchSourceBuilder`. Different highlighting behaviour can be defined for each
|
|
fields by adding one or more `HighlightBuilder.Field` instances to a `HighlightBuilder`.
|
|
|
|
["source","java",subs="attributes,callouts,macros"]
|
|
--------------------------------------------------
|
|
include-tagged::{doc-tests}/SearchDocumentationIT.java[search-request-highlighting]
|
|
--------------------------------------------------
|
|
<1> Creates a new `HighlightBuilder`
|
|
<2> Create a field highlighter for the `title` field
|
|
<3> Set the field highlighter type
|
|
<4> Add the field highlighter to the highlight builder
|
|
|
|
There are many options which are explained in detail in the Rest API documentation. The Rest
|
|
API parameters (e.g. `pre_tags`) are usually changed by
|
|
setters with a similar name (e.g. `#preTags(String ...)`).
|
|
|
|
Highlighted text fragments can <<java-rest-high-retrieve-highlighting,later be retrieved>> from the `SearchResponse`.
|
|
|
|
===== Requesting Aggregations
|
|
|
|
Aggregations can be added to the search by first creating the appropriate
|
|
`AggregationBuilder` and then setting it on the `SearchSourceBuilder`. In the
|
|
following example we create a `terms` aggregation on company names with a
|
|
sub-aggregation on the average age of employees in the company:
|
|
|
|
["source","java",subs="attributes,callouts,macros"]
|
|
--------------------------------------------------
|
|
include-tagged::{doc-tests}/SearchDocumentationIT.java[search-request-aggregations]
|
|
--------------------------------------------------
|
|
|
|
The <<java-rest-high-aggregations, Aggregations>> page gives a list of all available aggregations with
|
|
their corresponding `AggregationBuilder` objects and `AggregationBuilders` helper methods.
|
|
|
|
We will later see how to <<java-rest-high-retrieve-aggs,access aggregations>> in the `SearchResponse`.
|
|
|
|
===== Requesting Suggestions
|
|
|
|
To add Suggestions to the search request, use one of the `SuggestionBuilder` implementations
|
|
that are easily accessible from the `SuggestBuilders` factory class. Suggestion builders
|
|
need to be added to the top level `SuggestBuilder`, which itself can be set on the `SearchSourceBuilder`.
|
|
|
|
["source","java",subs="attributes,callouts,macros"]
|
|
--------------------------------------------------
|
|
include-tagged::{doc-tests}/SearchDocumentationIT.java[search-request-suggestion]
|
|
--------------------------------------------------
|
|
<1> Creates a new `TermSuggestionBuilder` for the `user` field and
|
|
the text `kmichy`
|
|
<2> Adds the suggestion builder and names it `suggest_user`
|
|
|
|
We will later see how to <<java-rest-high-retrieve-suggestions,retrieve suggestions>> from the
|
|
`SearchResponse`.
|
|
|
|
===== Profiling Queries and Aggregations
|
|
|
|
The {ref}/search-profile.html[Profile API] can be used to profile the execution of queries and aggregations for
|
|
a specific search request. in order to use it, the profile flag must be set to true on the `SearchSourceBuilder`:
|
|
|
|
["source","java",subs="attributes,callouts,macros"]
|
|
--------------------------------------------------
|
|
include-tagged::{doc-tests}/SearchDocumentationIT.java[search-request-profiling]
|
|
--------------------------------------------------
|
|
|
|
Once the `SearchRequest` is executed the corresponding `SearchResponse` will
|
|
<<java-rest-high-retrieve-profile-results,contain the profiling results>>.
|
|
|
|
[[java-rest-high-document-search-sync]]
|
|
==== Synchronous Execution
|
|
|
|
When executing a `SearchRequest` in the following manner, the client waits
|
|
for the `SearchResponse` to be returned before continuing with code execution:
|
|
|
|
["source","java",subs="attributes,callouts,macros"]
|
|
--------------------------------------------------
|
|
include-tagged::{doc-tests}/SearchDocumentationIT.java[search-execute]
|
|
--------------------------------------------------
|
|
|
|
[[java-rest-high-document-search-async]]
|
|
==== Asynchronous Execution
|
|
|
|
|
|
Executing a `SearchRequest` can also be done in an asynchronous fashion so that
|
|
the client can return directly. Users need to specify how the response or
|
|
potential failures will be handled by passing in appropriate listeners:
|
|
|
|
["source","java",subs="attributes,callouts,macros"]
|
|
--------------------------------------------------
|
|
include-tagged::{doc-tests}/SearchDocumentationIT.java[search-execute-async]
|
|
--------------------------------------------------
|
|
<1> Called when the execution is successfully completed.
|
|
<2> Called when the whole `SearchRequest` fails.
|
|
|
|
==== SearchResponse
|
|
|
|
The `SearchResponse` that is returned by executing the search provides details
|
|
about the search execution itself as well as access to the documents returned.
|
|
First, there is useful information about the request execution itself, like the
|
|
HTTP status code, execution time or wether the request terminated early or timed
|
|
out:
|
|
|
|
["source","java",subs="attributes,callouts,macros"]
|
|
--------------------------------------------------
|
|
include-tagged::{doc-tests}/SearchDocumentationIT.java[search-response-1]
|
|
--------------------------------------------------
|
|
|
|
Second, the response also provides information about the execution on the
|
|
shard level by offering statistics about the total number of shards that were
|
|
affected by the search, and the successful vs. unsuccessful shards. Possible
|
|
failures can also be handled by iterating over an array off
|
|
`ShardSearchFailures` like in the following example:
|
|
|
|
["source","java",subs="attributes,callouts,macros"]
|
|
--------------------------------------------------
|
|
include-tagged::{doc-tests}/SearchDocumentationIT.java[search-response-2]
|
|
--------------------------------------------------
|
|
|
|
[[java-rest-high-retrieve-searchHits]]
|
|
===== Retrieving SearchHits
|
|
|
|
To get access to the returned documents, we need to first get the `SearchHits`
|
|
contained in the response:
|
|
|
|
["source","java",subs="attributes,callouts,macros"]
|
|
--------------------------------------------------
|
|
include-tagged::{doc-tests}/SearchDocumentationIT.java[search-hits-get]
|
|
--------------------------------------------------
|
|
|
|
The `SearchHits` provides global information about all hits, like total number
|
|
of hits or the maximum score:
|
|
|
|
["source","java",subs="attributes,callouts,macros"]
|
|
--------------------------------------------------
|
|
include-tagged::{doc-tests}/SearchDocumentationIT.java[search-hits-info]
|
|
--------------------------------------------------
|
|
|
|
Nested inside the `SearchHits` are the individual search results that can
|
|
be iterated over:
|
|
|
|
|
|
["source","java",subs="attributes,callouts,macros"]
|
|
--------------------------------------------------
|
|
include-tagged::{doc-tests}/SearchDocumentationIT.java[search-hits-singleHit]
|
|
--------------------------------------------------
|
|
|
|
The `SearchHit` provides access to basic information like index, type, docId and
|
|
score of each search hit:
|
|
|
|
["source","java",subs="attributes,callouts,macros"]
|
|
--------------------------------------------------
|
|
include-tagged::{doc-tests}/SearchDocumentationIT.java[search-hits-singleHit-properties]
|
|
--------------------------------------------------
|
|
|
|
Furthermore, it lets you get back the document source, either as a simple
|
|
JSON-String or as a map of key/value pairs. In this map, regular fields
|
|
are keyed by the field name and contain the field value. Multi-valued fields are
|
|
returned as lists of objects, nested objects as another key/value map. These
|
|
cases need to be cast accordingly:
|
|
|
|
["source","java",subs="attributes,callouts,macros"]
|
|
--------------------------------------------------
|
|
include-tagged::{doc-tests}/SearchDocumentationIT.java[search-hits-singleHit-source]
|
|
--------------------------------------------------
|
|
|
|
[[java-rest-high-retrieve-highlighting]]
|
|
===== Retrieving Highlighting
|
|
|
|
If <<java-rest-high-request-highlighting,requested>>, highlighted text fragments can be retrieved from each `SearchHit` in the result. The hit object offers
|
|
access to a map of field names to `HighlightField` instances, each of which contains one
|
|
or many highlighted text fragments:
|
|
|
|
|
|
["source","java",subs="attributes,callouts,macros"]
|
|
--------------------------------------------------
|
|
include-tagged::{doc-tests}/SearchDocumentationIT.java[search-request-highlighting-get]
|
|
--------------------------------------------------
|
|
<1> Get the highlighting for the `title` field
|
|
<2> Get one or many fragments containing the highlighted field content
|
|
|
|
[[java-rest-high-retrieve-aggs]]
|
|
===== Retrieving Aggregations
|
|
|
|
Aggregations can be retrieved from the `SearchResponse` by first getting the
|
|
root of the aggregation tree, the `Aggregations` object, and then getting the
|
|
aggregation by name.
|
|
|
|
["source","java",subs="attributes,callouts,macros"]
|
|
--------------------------------------------------
|
|
include-tagged::{doc-tests}/SearchDocumentationIT.java[search-request-aggregations-get]
|
|
--------------------------------------------------
|
|
<1> Get the `by_company` terms aggregation
|
|
<2> Get the buckets that is keyed with `Elastic`
|
|
<3> Get the `average_age` sub-aggregation from that bucket
|
|
|
|
Note that if you access aggregations by name, you need to specify the
|
|
aggregation interface according to the type of aggregation you requested,
|
|
otherwise a `ClassCastException` will be thrown:
|
|
|
|
["source","java",subs="attributes,callouts,macros"]
|
|
--------------------------------------------------
|
|
include-tagged::{doc-tests}/SearchDocumentationIT.java[search-request-aggregations-get-wrongCast]
|
|
--------------------------------------------------
|
|
<1> This will throw an exception because "by_company" is a `terms` aggregation
|
|
but we try to retrieve it as a `range` aggregation
|
|
|
|
It is also possible to access all aggregations as a map that is keyed by the
|
|
aggregation name. In this case, the cast to the proper aggregation interface
|
|
needs to happen explicitly:
|
|
|
|
["source","java",subs="attributes,callouts,macros"]
|
|
--------------------------------------------------
|
|
include-tagged::{doc-tests}/SearchDocumentationIT.java[search-request-aggregations-asMap]
|
|
--------------------------------------------------
|
|
|
|
There are also getters that return all top level aggregations as a list:
|
|
|
|
["source","java",subs="attributes,callouts,macros"]
|
|
--------------------------------------------------
|
|
include-tagged::{doc-tests}/SearchDocumentationIT.java[search-request-aggregations-asList]
|
|
--------------------------------------------------
|
|
|
|
And last but not least you can iterate over all aggregations and then e.g.
|
|
decide how to further process them based on their type:
|
|
|
|
["source","java",subs="attributes,callouts,macros"]
|
|
--------------------------------------------------
|
|
include-tagged::{doc-tests}/SearchDocumentationIT.java[search-request-aggregations-iterator]
|
|
--------------------------------------------------
|
|
|
|
[[java-rest-high-retrieve-suggestions]]
|
|
===== Retrieving Suggestions
|
|
|
|
To get back the suggestions from a `SearchResponse`, use the `Suggest` object as an entry point and then retrieve the nested suggestion objects:
|
|
|
|
["source","java",subs="attributes,callouts,macros"]
|
|
--------------------------------------------------
|
|
include-tagged::{doc-tests}/SearchDocumentationIT.java[search-request-suggestion-get]
|
|
--------------------------------------------------
|
|
<1> Use the `Suggest` class to access suggestions
|
|
<2> Suggestions can be retrieved by name. You need to assign them to the correct
|
|
type of Suggestion class (here `TermSuggestion`), otherwise a `ClassCastException` is thrown
|
|
<3> Iterate over the suggestion entries
|
|
<4> Iterate over the options in one entry
|
|
|
|
[[java-rest-high-retrieve-profile-results]]
|
|
===== Retrieving Profiling Results
|
|
|
|
Profiling results are retrieved from a `SearchResponse` using the `getProfileResults()` method. This
|
|
method returns a `Map` containing a `ProfileShardResult` object for every shard involved in the
|
|
`SearchRequest` execution. `ProfileShardResult` are stored in the `Map` using a key that uniquely
|
|
identifies the shard the profile result corresponds to.
|
|
|
|
Here is a sample code that shows how to iterate over all the profiling results of every shard:
|
|
|
|
["source","java",subs="attributes,callouts,macros"]
|
|
--------------------------------------------------
|
|
include-tagged::{doc-tests}/SearchDocumentationIT.java[search-request-profiling-get]
|
|
--------------------------------------------------
|
|
<1> Retrieve the `Map` of `ProfileShardResult` from the `SearchResponse`
|
|
<2> Profiling results can be retrieved by shard's key if the key is known, otherwise it might be simpler
|
|
to iterate over all the profiling results
|
|
<3> Retrieve the key that identifies which shard the `ProfileShardResult` belongs to
|
|
<4> Retrieve the `ProfileShardResult` for the given shard
|
|
|
|
The `ProfileShardResult` object itself contains one or more query profile results, one for each query
|
|
executed against the underlying Lucene index:
|
|
|
|
["source","java",subs="attributes,callouts,macros"]
|
|
--------------------------------------------------
|
|
include-tagged::{doc-tests}/SearchDocumentationIT.java[search-request-profiling-queries]
|
|
--------------------------------------------------
|
|
<1> Retrieve the list of `QueryProfileShardResult`
|
|
<2> Iterate over each `QueryProfileShardResult`
|
|
|
|
Each `QueryProfileShardResult` gives access to the detailed query tree execution, returned as a list of
|
|
`ProfileResult` objects:
|
|
|
|
["source","java",subs="attributes,callouts,macros"]
|
|
--------------------------------------------------
|
|
include-tagged::{doc-tests}/SearchDocumentationIT.java[search-request-profiling-queries-results]
|
|
--------------------------------------------------
|
|
<1> Iterate over the profile results
|
|
<2> Retrieve the name of the Lucene query
|
|
<3> Retrieve the time in millis spent executing the Lucene query
|
|
<4> Retrieve the profile results for the sub-queries (if any)
|
|
|
|
The Rest API documentation contains more information about {ref}/_profiling_queries.html[Profiling Queries] with
|
|
a description of the {ref}/_profiling_queries.html#_literal_query_literal_section[query profiling information]
|
|
|
|
The `QueryProfileShardResult` also gives access to the profiling information for the Lucene collectors:
|
|
|
|
["source","java",subs="attributes,callouts,macros"]
|
|
--------------------------------------------------
|
|
include-tagged::{doc-tests}/SearchDocumentationIT.java[search-request-profiling-queries-collectors]
|
|
--------------------------------------------------
|
|
<1> Retrieve the profiling result of the Lucene collector
|
|
<2> Retrieve the name of the Lucene collector
|
|
<3> Retrieve the time in millis spent executing the Lucene collector
|
|
<4> Retrieve the profile results for the sub-collectors (if any)
|
|
|
|
The Rest API documentation contains more information about profiling information
|
|
{ref}/_profiling_queries.html#_literal_collectors_literal_section[for Lucene collectors].
|
|
|
|
In a very similar manner to the query tree execution, the `QueryProfileShardResult` objects gives access
|
|
to the detailed aggregations tree execution:
|
|
|
|
["source","java",subs="attributes,callouts,macros"]
|
|
--------------------------------------------------
|
|
include-tagged::{doc-tests}/SearchDocumentationIT.java[search-request-profiling-aggs]
|
|
--------------------------------------------------
|
|
<1> Retrieve the `AggregationProfileShardResult`
|
|
<2> Iterate over the aggregation profile results
|
|
<3> Retrieve the type of the aggregation (corresponds to Java class used to execute the aggregation)
|
|
<4> Retrieve the time in millis spent executing the Lucene collector
|
|
<5> Retrieve the profile results for the sub-aggregations (if any)
|
|
|
|
The Rest API documentation contains more information about {ref}/_profiling_aggregations.html[Profiling Aggregations]
|