honeymoose
/
OpenSearch
mirror of https://github.com/honeymoose/OpenSearch.git
1
0
Fork 0
Code Issues Packages Projects Releases Wiki Activity

Use unlimited flush_threshold_ops for translog 

Currently we use 5k operations as a flush threshold. Indexing 5k documents
per second is rather common which would cause the index to be committed on
the lucene level each time the flush logic runs which is 5 seconds by default.
We should rather use a size based threshold similar to the lucene index writer
that doesn't cause such agressive commits which can slow down indexing significantly
especially since they cause the underlying devices to fsync their data.
This commit is contained in:
Simon Willnauer 2014-04-22 14:29:26 +02:00
parent 3ba8fbbef8
commit 1cf62e7782
2 changed files with 2 additions and 2 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

2
docs/reference/index-modules/translog.asciidoc
View File

@ -11,7 +11,7 @@ parameters:
|======================================================================= |=======================================================================
|Setting |Description |Setting |Description
|index.translog.flush_threshold_ops |After how many operations to flush. |index.translog.flush_threshold_ops |After how many operations to flush.
Defaults to `5000`. Defaults to `unlimited`.
|index.translog.flush_threshold_size |Once the translog hits this size, |index.translog.flush_threshold_size |Once the translog hits this size,
a flush will happen. Defaults to `200mb`. a flush will happen. Defaults to `200mb`.

2
src/main/java/org/elasticsearch/index/translog/TranslogService.java
View File

@ -67,7 +67,7 @@ public class TranslogService extends AbstractIndexShardComponent {
this.indexShard = indexShard; this.indexShard = indexShard;
this.translog = translog; this.translog = translog;
this.flushThresholdOperations = componentSettings.getAsInt("flush_threshold_ops", componentSettings.getAsInt("flush_threshold", 5000)); this.flushThresholdOperations = componentSettings.getAsInt("flush_threshold_ops", componentSettings.getAsInt("flush_threshold", Integer.MAX_VALUE));
this.flushThresholdSize = componentSettings.getAsBytesSize("flush_threshold_size", new ByteSizeValue(200, ByteSizeUnit.MB)); this.flushThresholdSize = componentSettings.getAsBytesSize("flush_threshold_size", new ByteSizeValue(200, ByteSizeUnit.MB));
this.flushThresholdPeriod = componentSettings.getAsTime("flush_threshold_period", TimeValue.timeValueMinutes(30)); this.flushThresholdPeriod = componentSettings.getAsTime("flush_threshold_period", TimeValue.timeValueMinutes(30));
this.interval = componentSettings.getAsTime("interval", timeValueMillis(5000)); this.interval = componentSettings.getAsTime("interval", timeValueMillis(5000));