Today the engine writes the transaction log itself as well as manages
all the commit / translog mapping internally. Yet, if an engine is closed
and reopend it doesn't replay it's translog or does anything to be consistent
with it's latest state again.
This change moves the transaction log replay code into the Engine / InternalEngine
and adds unittests for replaying and consistency.
Closes#10452
At the moment, we are very strict when handling data folders containing corrupted shards and will fail any recovery attempt into it. Typically this wouldn't be a problem as the shard will be assigned to another node (which we try first anyway when a shard fails). However, it has been proven to be too strict for smaller clusters which may not have an extra node available (either because of allocation filtering, disk space issues etc.). This commit changes the behavior to force a full recovery. Once all the new files are verified we remove the old corrupted data and start the shard.
This also fixes a small issue where the shard state file wasn't deleted on an engine failure (we had a protection against deleting the state file on an active shard, but in this case the shard is still active but will be removed). The state deletion is also moved to before the failure handlers are called, to avoid race conditions when calling the master (it will potentially try to read it when allocating the shard)
Closes#10558
ShapeBuilder's coordinate parser expected 2 double values for every coordinate array. If > 2 doubles were provided the parser terminated parsing of the coordinate array. This resulted in an invalid Shape state leaving LineStrings, LinearRings, and Polygons with a single coordinate. An incorrect parse exception was thrown. This corrects the parser to ignore those values in the 3rd+ dimension, correctly parsing the rest of the coordinate array.
Unit tests have been updated to verify the fix.
closes#10510
Prevents the user from changing strategies, tree, tree_level or precision. distance_error_pct changes are allowed as they do not compromise the integrity of the index. A separate issue is open for allowing users to change tree_level or precision.
OGC SFA 2.1.10 assertion 3 allows interior boundaries to touch exterior boundaries provided they intersect at a single point. Issue #9511 provides an example where a valid shape is incorrectly interpreted as invalid (a false violation of assertion 3). When the intersecting point appears as the first and last coordinate of the interior boundary in a polygon, the ShapeBuilder incorrectly counted this as multiple intersecting vertices. The fix required a little more than just a logic check. Passing the duplicate vertices resulted in a connected component in the edge graph causing an invalid self crossing polygon. This required additional logic to the edge assignment in order to correctly segment the connected components. Finally, an additional hole validation has been added along with proper unit tests for testing valid and invalid conditions (including dateline crossing polys).
closes#9511
Update Eclipse core prefs
Eclipse Luna overwrites the prefs file, putting all the settings
in alphabetical order and removing comments. This causes the prefs
files to be modified in the git workspace.
Update the file with the version generated by Eclipse to prevent it
from being modified every time.
No settings values are modified by this change.
This also adds another plugin to the lifecycle mapping in the pom.xml which was missed in https://github.com/elastic/elasticsearch/pull/10524.
This is really a Collector instead of a filter. This commit deprecates the
`limit` filter, makes it a no-op and recommends to use the `terminate_after`
parameter instead that we introduced in the meantime.
Most tests don't "really" need to fsync, and this is costly (makes
tests slower, wears out our SSDs).
This change makes it uncommon to actually fsync when Lucene asks for
it. It's just a workaround (in MockDirectoryHelper) until we can
cutover Elasticseach to use MockFileSystem like Lucene.
Closes#10516
This change logs total space, free space, usable free space, an
estimate of whether the IO system spins (e.g., SSD or not), the mount
point and filesystem type, on node startup.
It produces log output like this:
[2015-04-09 12:09:30,244][INFO ][env ] [node_t0] node data locations details:
-> /l/es.logspins/target/J0/data/TEST-haswell-CHILD_VM=[0]-CLUSTER_SEED=[2926863498862121027]-HASH=[AFC194B1B384B]/nodes/0, free_space [260.6gb], usable_space [256.3gb], total_space [465gb], spins? [no], mount [/ (/dev/mapper/haswell--vg-root)], type [btrfs]
Closes#10502
Today we force a flush before check index to ensure we have an index
to check on. Yet if the index is large and the FS is slow this can have
significant impact on the index deletion performance. This commit introduces
a check if there are any uncommitted changes in order to skip the additional commit.
Closes#10505
ReduceContext contains the list of aggregations to reduce but these aggregations are set as null half of the time. This change makes the reduce(ReduceContext) method changed to reduce(List<InternalAggregation>, ReduceContext) and ReduceContext now only holds the BigArrays and Script services.
* In code, we mark `River`, `AbstractRiverComponent`, `RiverComponent` and `RiverName` classes as deprecated
* We log that information when a cluster is still using it
* We add this information in the plugins list as well
Today we check every regular expression eagerly against every possible term.
This can be very slow if you have lots of unique terms, and even the bottleneck
if your query is selective.
This commit switches to Lucene regular expressions instead of Java (not exactly
the same syntax yet most existing regular expressions should keep working) and
uses the same logic as RegExpQuery to intersect the regular expression with the
terms dictionary. I wrote a quick benchmark (in the PR) to make sure it made
things faster and the same request that took 750ms on master now takes 74ms with
this change.
Close#7526
The refactoring in #9544 introduced a regression that broke multi-level
aggregations using breadth-first. This was due to sub-aggregators creating
deferred collectors before their parent aggregator and then the parent
aggregator trying to collect sub aggregators directly instead of going through
the deferred wrapper.
This commit fixes the issue but we should try to simplify all the pre/post
collection logic that we have.
Also `breadth_first` is now automatically ignored if the sub aggregators need
scores (just like we ignore `execution_mode` when the value does not make sense
like using ordinals on a script).
Close#9823
To ensure subclasses like MockInternalEngine which is in a different
package (test.engine) are logging under the same logger name this commit
moves to a static logger class to determin the logger name. This way
all subclasses of engine will log under `index.engine` which also plays
nicely with `@TestLogging` where log messages sometimes disappeared since
they were enabled for the `index.engine` package but not for `test.engine`
Commit 168238dab6 declared multiple maven-resources-plugin usages instead of declaring multiple executions for the same plugin... resulting to Maven warnings.
Closes#10433
Simon Willnauer
Clinton Gormley
Boaz Leskes
Benoit Delbosc
David Pursehouse
Uwe Schindler
Britta Weber
Michael McCandless
Nicholas Knize
Adrien Grand
Robert Muir
Colin Goodheart-Smithe
Ryan Ernst
javanna
David Pilato
Tanguy Leroux
Spencer Alger