mirror of
https://github.com/honeymoose/OpenSearch.git
synced 2025-02-07 21:48:39 +00:00
1cf62e7782
Currently we use 5k operations as a flush threshold. Indexing 5k documents per second is rather common which would cause the index to be committed on the lucene level each time the flush logic runs which is 5 seconds by default. We should rather use a size based threshold similar to the lucene index writer that doesn't cause such agressive commits which can slow down indexing significantly especially since they cause the underlying devices to fsync their data.
29 lines
1.2 KiB
Plaintext
29 lines
1.2 KiB
Plaintext
[[index-modules-translog]]
|
|
== Translog
|
|
|
|
Each shard has a transaction log or write ahead log associated with it.
|
|
It allows to guarantee that when an index/delete operation occurs, it is
|
|
applied atomically, while not "committing" the internal Lucene index for
|
|
each request. A flush ("commit") still happens based on several
|
|
parameters:
|
|
|
|
[cols="<,<",options="header",]
|
|
|=======================================================================
|
|
|Setting |Description
|
|
|index.translog.flush_threshold_ops |After how many operations to flush.
|
|
Defaults to `unlimited`.
|
|
|
|
|index.translog.flush_threshold_size |Once the translog hits this size,
|
|
a flush will happen. Defaults to `200mb`.
|
|
|
|
|index.translog.flush_threshold_period |The period with no flush
|
|
happening to force a flush. Defaults to `30m`.
|
|
|
|
|index.translog.interval |How often to check if a flush is needed, randomized
|
|
between the interval value and 2x the interval value. Defaults to `5s`.
|
|
|=======================================================================
|
|
|
|
Note: these parameters can be updated at runtime using the Index
|
|
Settings Update API (for example, these number can be increased when
|
|
executing bulk updates to support higher TPS)
|