OpenSearch/docs/reference/mapping/params/eager-global-ordinals.asciidoc

91 lines
3.6 KiB
Plaintext
Raw Normal View History

[[eager-global-ordinals]]
=== `eager_global_ordinals`
Global ordinals is a data-structure on top of doc values, that maintains an
incremental numbering for each unique term in a lexicographic order. Each
term has a unique number and the number of term 'A' is lower than the
number of term 'B'. Global ordinals are only supported with
<<keyword,`keyword`>> and <<text,`text`>> fields. In `keyword` fields, they
are available by default but `text` fields can only use them when `fielddata`,
with all of its associated baggage, is enabled.
Doc values (and fielddata) also have ordinals, which is a unique numbering for
all terms in a particular segment and field. Global ordinals just build on top
of this, by providing a mapping between the segment ordinals and the global
ordinals, the latter being unique across the entire shard. Given that global
ordinals for a specific field are tied to _all the segments of a shard_, they
need to be entirely rebuilt whenever a once new segment becomes visible.
Global ordinals are used for features that use segment ordinals, such as
the <<search-aggregations-bucket-terms-aggregation,`terms` aggregation>>,
to improve the execution time. A terms aggregation relies purely on global
ordinals to perform the aggregation at the shard level, then converts global
ordinals to the real term only for the final reduce phase, which combines
results from different shards.
The loading time of global ordinals depends on the number of terms in a field,
but in general it is low, since it source field data has already been loaded.
The memory overhead of global ordinals is a small because it is very
efficiently compressed.
By default, global ordinals are loaded at search-time, which is the right
trade-off if you are optimizing for indexing speed. However, if you are more
interested in search speed, it could be beneficial to set
`eager_global_ordinals: true` on fields that you plan to use in terms
aggregations:
[source,js]
------------
Update the default for include_type_name to false. (#37285) * Default include_type_name to false for get and put mappings. * Default include_type_name to false for get field mappings. * Add a constant for the default include_type_name value. * Default include_type_name to false for get and put index templates. * Default include_type_name to false for create index. * Update create index calls in REST documentation to use include_type_name=true. * Some minor clean-ups around the get index API. * In REST tests, use include_type_name=true by default for index creation. * Make sure to use 'expression == false'. * Clarify the different IndexTemplateMetaData toXContent methods. * Fix FullClusterRestartIT#testSnapshotRestore. * Fix the ml_anomalies_default_mappings test. * Fix GetFieldMappingsResponseTests and GetIndexTemplateResponseTests. We make sure to specify include_type_name=true during xContent parsing, so we continue to test the legacy typed responses. XContent generation for the typeless responses is currently only covered by REST tests, but we will be adding unit test coverage for these as we implement each typeless API in the Java HLRC. This commit also refactors GetMappingsResponse to follow the same appraoch as the other mappings-related responses, where we read include_type_name out of the xContent params, instead of creating a second toXContent method. This gives better consistency in the response parsing code. * Fix more REST tests. * Improve some wording in the create index documentation. * Add a note about types removal in the create index docs. * Fix SmokeTestMonitoringWithSecurityIT#testHTTPExporterWithSSL. * Make sure to mention include_type_name in the REST docs for affected APIs. * Make sure to use 'expression == false' in FullClusterRestartIT. * Mention include_type_name in the REST templates docs.
2019-01-14 16:08:01 -05:00
PUT my_index/_mapping
{
"properties": {
"tags": {
"type": "keyword",
"eager_global_ordinals": true
}
}
}
------------
// CONSOLE
// TEST[s/^/PUT my_index\n/]
This will shift the cost of building the global ordinals from search-time to
refresh-time. Elasticsearch will make sure that global ordinals are built
before exposing to searches any changes to the content of the index.
Elasticsearch will also eagerly build global ordinals when starting a new copy
of a shard, such as when increasing the number of replicas or when relocating a
shard onto a new node.
If a shard has been <<indices-forcemerge,force-merged>> down to a single
segment then its global ordinals are identical to the ordinals for its unique
segment, which means there is no extra cost for using global ordinals on such a
shard. Note that for performance reasons you should only force-merge an index
to which you will never write again.
On a <<frozen-indices,frozen index>>, global ordinals are discarded after each
search and rebuilt again on the next search if needed or if
`eager_global_ordinals` is set. This means `eager_global_ordinals` should not
be used on frozen indices. Instead, force-merge an index to a single segment
before freezing it so that global ordinals need not be built separately on each
search.
If you ever decide that you do not need to run `terms` aggregations on this
field anymore, then you can disable eager loading of global ordinals at any
time:
[source,js]
------------
Update the default for include_type_name to false. (#37285) * Default include_type_name to false for get and put mappings. * Default include_type_name to false for get field mappings. * Add a constant for the default include_type_name value. * Default include_type_name to false for get and put index templates. * Default include_type_name to false for create index. * Update create index calls in REST documentation to use include_type_name=true. * Some minor clean-ups around the get index API. * In REST tests, use include_type_name=true by default for index creation. * Make sure to use 'expression == false'. * Clarify the different IndexTemplateMetaData toXContent methods. * Fix FullClusterRestartIT#testSnapshotRestore. * Fix the ml_anomalies_default_mappings test. * Fix GetFieldMappingsResponseTests and GetIndexTemplateResponseTests. We make sure to specify include_type_name=true during xContent parsing, so we continue to test the legacy typed responses. XContent generation for the typeless responses is currently only covered by REST tests, but we will be adding unit test coverage for these as we implement each typeless API in the Java HLRC. This commit also refactors GetMappingsResponse to follow the same appraoch as the other mappings-related responses, where we read include_type_name out of the xContent params, instead of creating a second toXContent method. This gives better consistency in the response parsing code. * Fix more REST tests. * Improve some wording in the create index documentation. * Add a note about types removal in the create index docs. * Fix SmokeTestMonitoringWithSecurityIT#testHTTPExporterWithSSL. * Make sure to mention include_type_name in the REST docs for affected APIs. * Make sure to use 'expression == false' in FullClusterRestartIT. * Mention include_type_name in the REST templates docs.
2019-01-14 16:08:01 -05:00
PUT my_index/_mapping
{
"properties": {
"tags": {
"type": "keyword",
"eager_global_ordinals": false
}
}
}
------------
// CONSOLE
// TEST[continued]