OpenSearch/docs/reference
Simon Willnauer a0becd26b1 Optimize indexing for the autogenerated ID append-only case (#20211)
If elasticsearch controls the ID values as well as the documents
version we can optimize the code that adds / appends the documents
to the index. Essentially we an skip the version lookup for all
documents unless the same document is delivered more than once.

On the lucene level we can simply call IndexWriter#addDocument instead
of #updateDocument but on the Engine level we need to ensure that we deoptimize
the case once we see the same document more than once.

This is done as follows:

1. Mark every request with a timestamp. This is done once on the first node that
receives a request and is fixed for this request. This can be even the
machine local time (see why later). The important part is that retry
requests will have the same value as the original one.

2. In the engine we make sure we keep the highest seen time stamp of "retry" requests.
This is updated while the retry request has its doc id lock. Call this `maxUnsafeAutoIdTimestamp`

3. When the engine runs an "optimized" request comes, it compares it's timestamp with the
current `maxUnsafeAutoIdTimestamp` (but doesn't update it). If the the request
timestamp is higher it is safe to execute it as optimized (no retry request with the same
timestamp has been run before). If not we fall back to "non-optimzed" mode and run the request as a retry one
and update the `maxUnsafeAutoIdTimestamp` unless it's been updated already to a higher value

Relates to #19813
2016-09-01 10:39:40 +02:00
..
aggregations Add the ability to disable the retrieval of the stored fields entirely 2016-08-24 16:40:08 +02:00
analysis Add the ability to disable the retrieval of the stored fields entirely 2016-08-24 16:40:08 +02:00
cat Improve cat thread pool API 2016-08-03 23:02:13 -04:00
cluster Parameter improvements to Cluster Health API wait for shards (#20223) 2016-08-31 11:58:19 -04:00
docs Merge pull request #20213 from jimferenczi/painless_list_add 2016-08-30 09:33:59 +02:00
how-to Add `scaled_float`. #19264 2016-07-18 12:36:23 +02:00
images Docs: Add more points to the chart that gives accuracy for the cardinality aggregation. 2016-07-20 10:37:12 +02:00
index-modules Update docs for Log4j 2 2016-08-31 15:51:52 -04:00
indices [docs] Remove coming in 2.0.0 2016-08-31 14:43:04 -04:00
ingest Add support for parameters to the script ingest processor 2016-08-24 16:49:48 -04:00
mapping Support downgrading keyword/text into string 2016-08-29 11:27:37 -04:00
migration Optimize indexing for the autogenerated ID append-only case (#20211) 2016-09-01 10:39:40 +02:00
modules Use `painless` as language for painless snippets (#20185) 2016-08-26 15:39:44 -04:00
query-dsl Docs: clarify scale is applied at origin+offest (#20242) 2016-08-31 17:02:59 +02:00
release-notes Documented netty4 Expect bug in release notes and breaking changes 2016-08-10 10:16:25 +02:00
search [doc] Remove leftover from CONSOLE conversion 2016-08-31 13:14:47 -04:00
setup Update docs for Log4j 2 2016-08-31 15:51:52 -04:00
testing Introduce dedicated master nodes in testing infrastructure (#18514) 2016-05-27 08:44:20 +02:00
aggregations.asciidoc refactor matrix agg documentation from modules to main agg section 2016-06-06 07:39:00 -05:00
analysis.asciidoc First pass at improving analyzer docs (#18269) 2016-05-11 14:17:56 +02:00
api-conventions.asciidoc Add exclusion filters support to filter_path 2016-08-30 09:08:30 +02:00
cat.asciidoc Add support for documented byte/size units and for micros as a time unit in _cat API 2016-04-15 20:55:41 +02:00
cluster.asciidoc Update task management docs to reflect the latest changes in the interface 2016-03-29 12:26:37 -04:00
docs.asciidoc Add support for waiting until a refresh occurs 2016-06-06 11:37:53 -04:00
getting-started.asciidoc Documentation changes for wait_for_active_shards (#19581) 2016-08-02 09:15:01 -04:00
glossary.asciidoc Improve glossary to not refer to types as "like a table" (#17704) 2016-04-13 14:29:47 +02:00
how-to.asciidoc Add notes about sparsity. 2016-07-07 17:47:19 +02:00
index-modules.asciidoc Remove `_timestamp` and `_ttl` on 5.x indices. #18980 2016-06-22 08:35:54 +02:00
index.asciidoc Bumped version in docs 2016-08-09 15:57:35 +02:00
indices.asciidoc [DOCS] fix missing rollover-index link 2016-06-17 12:14:45 -04:00
ingest.asciidoc Renamed all AUTOSENSE snippets to CONSOLE (#18210) 2016-05-09 15:42:23 +02:00
mapping.asciidoc Renamed all AUTOSENSE snippets to CONSOLE (#18210) 2016-05-09 15:42:23 +02:00
modules.asciidoc refactor matrix agg documentation from modules to main agg section 2016-06-06 07:39:00 -05:00
query-dsl.asciidoc Fixed broken xrefs to query-dsl-not-query, which has been removed. 2015-10-20 13:01:37 -07:00
redirects.asciidoc Docs: Complete rewrite of setup, installation, and configuration docs 2016-04-03 16:09:48 +02:00
release-notes.asciidoc Added release notes for 5.0.0-alpha5 2016-07-29 14:43:31 +02:00
search.asciidoc Add documentation for global search timeout 2016-01-07 10:42:18 -05:00
setup.asciidoc Die with dignity 2016-07-07 14:44:03 -04:00
testing.asciidoc [DOCS] Test framework documentation 2013-12-02 18:01:45 +01:00