OpenSearch/docs/reference
Jim Ferenczi 5288235ca3
Optimize the composite aggregation for match_all and range queries (#28745)
This change refactors the composite aggregation to add an execution mode that visits documents in the order of the values
present in the leading source of the composite definition. This mode does not need to visit all documents since it can early terminate
the collection when the leading source value is greater than the lowest value in the queue.
Instead of collecting the documents in the order of their doc_id, this mode uses the inverted lists (or the bkd tree for numerics) to collect documents
in the order of the values present in the leading source.
For instance the following aggregation:

```
"composite" : {
  "sources" : [
    { "value1": { "terms" : { "field": "timestamp", "order": "asc" } } }
  ],
  "size": 10
}
```
... can use the field `timestamp` to collect the documents with the 10 lowest values for the field instead of visiting all documents.
For composite aggregation with more than one source the execution can early terminate as soon as one of the 10 lowest values produces enough
composite buckets. For instance if visiting the first two lowest timestamp created 10 composite buckets we can early terminate the collection since it
is guaranteed that the third lowest timestamp cannot create a composite key that compares lower than the one already visited.

This mode can execute iff:
 * The leading source in the composite definition uses an indexed field of type `date` (works also with `date_histogram` source), `integer`, `long` or `keyword`.
 * The query is a match_all query or a range query over the field that is used as the leading source in the composite definition.
 * The sort order of the leading source is the natural order (ascending since postings and numerics are sorted in ascending order only).

If these conditions are not met this aggregation visits each document like any other agg.
2018-03-26 09:51:37 +02:00
..
aggregations Optimize the composite aggregation for match_all and range queries (#28745) 2018-03-26 09:51:37 +02:00
analysis [DOCS] Add supported token filters 2018-02-13 14:10:25 -08:00
cat Replace jvm-example by two plugin examples (#28339) 2018-01-26 17:34:24 +01:00
cluster Add a note about using the `retry_failed` flag before accepting data loss (#29160) 2018-03-20 18:11:45 +01:00
docs Docs: Update docs/index_.asciidoc (#29172) 2018-03-23 11:35:10 -04:00
how-to [Docs]Corrected spelling errors. (#28976) 2018-03-19 10:22:40 -07:00
images
index-modules Improve similarity docs. (#29089) 2018-03-21 10:41:10 +01:00
indices Reject updates to the `_default_` mapping. (#29165) 2018-03-21 10:44:11 +01:00
ingest Docs: Support triple quotes (#28915) 2018-03-16 12:46:39 -04:00
mapping [DOCS] Remove ignore_z_value parameter link 2018-03-23 11:07:24 -05:00
migration Remove deprecated options for query_string (#29203) 2018-03-22 18:37:08 +01:00
modules [DOCS] Unregister repository instead of deleting it (#29206) 2018-03-23 15:53:36 +01:00
query-dsl Remove deprecated options for query_string (#29203) 2018-03-22 18:37:08 +01:00
release-notes Remove tribe node support (#28443) 2018-01-30 20:40:19 +01:00
search [Docs] Add rank_eval size parameter k (#29218) 2018-03-23 18:04:32 +01:00
setup Add error file docs to important settings 2018-03-22 23:06:53 -04:00
testing
upgrade [DOCS] Add X-Pack upgrade details (#29038) 2018-03-15 11:40:20 -07:00
aggregations.asciidoc
analysis.asciidoc Allow `_doc` as a type. (#27816) 2017-12-14 17:47:53 +01:00
api-conventions.asciidoc Fix incorrect datemath example (#28904) 2018-03-06 15:46:21 +01:00
cat.asciidoc Add a note to the docs that _cat api `help` option cannot be used if an optional url param is used (#28686) 2018-02-15 15:56:01 +01:00
cluster.asciidoc
docs.asciidoc
getting-started.asciidoc Update "_doc" to "account" type for bulk example (#28786) 2018-03-13 15:51:59 -07:00
glossary.asciidoc
gs-index.asciidoc
how-to.asciidoc
index-modules.asciidoc Limit analyzed text for highlighting (improvements) (#28808) 2018-03-02 08:09:05 -08:00
index-shared1.asciidoc [DOCS] Added link to upgrade guide and bumped the upgrade topic up to the top level (#27621) 2017-12-05 10:58:52 -08:00
index-shared2.asciidoc
index-shared3.asciidoc
index-shared4.asciidoc
index-shared5.asciidoc
index.asciidoc
index.x.asciidoc [DOCS] Build Elasticsearch Reference from elasticsearch repo (#28469) 2018-02-01 13:36:06 -08:00
indices.asciidoc
ingest.asciidoc [Docs] Changes to ingest.asciidoc (#28212) 2018-01-16 09:36:19 +01:00
mapping.asciidoc
modules.asciidoc Remove left-over tribe reference 2018-01-30 21:44:21 +01:00
query-dsl.asciidoc Update query-dsl.asciidoc (#27669) 2017-12-11 18:06:08 +01:00
redirects.asciidoc
release-notes.asciidoc
search.asciidoc Move search concurrency and parallelism paragraphs 2018-02-26 07:47:57 -08:00
setup.asciidoc Clarify using supported LTS versions of the Java 2017-12-14 07:47:50 -05:00
testing.asciidoc
upgrade.asciidoc [DOCS] Removed prerelease footnote from upgrade table. 2018-03-16 10:33:10 -07:00