This is a start to exposing memory stats improvements from Lucene 5.0.
This adds the following categories of Lucene index pieces to index stats:
* Terms
* Stored fields
* Term Vectors
* Norms
* Doc values
This commit add the engines reference to the store out of the actual
implementation into the hodler since the holder manages the actual lifcycle.
Engine internal references like per searcher or per recovery are kept inside
the actual implemenation since the have a different lifecycle.
Once the current engine is started you can only close it once. Once closed the engine cannot be started again. This commit adds a stop method which signals the engine to free it's resources but in a way that allows restarting.
This is done by introducing InternalEngineHolder which is a wrapper around InternalEngine. This allows to add the stop() method without adding complexity the engine implementation. InternalEngineHolder also serves an entry point for listeners (incoming and outgoing) to other ES components, which removes the needs add/remove them if the engine is stopped.
Closes#8784
Today we try to fetch a shard Id for a given IndexReader / LeafReader
by walking it's tree until the lucene internal SegmentReader and then
casting the directory into a StoreDirecotory. This class is fully internal
to Elasticsearch and should not be exposed outside of the Store.
This commit makes StoreDirectory a private inner class and adds dedicated
ElasticsearchDirectoryReader / ElasticserachLeafReader exposing a ShardId
getter to obtain information about the shard the index / segment belogs to.
These classes can be used to expose other segment specific information in
the future more easily.
When a scoring script returns not a number, the current message is confusing (IllegalArgumentException[docID must be >= 0 and < maxDoc=3 (got docID=2147483647)]). This commit adds the error message ScriptException[script score function returns a wrong score: NaN].
Closes#2426
The packaged init scripts could return an error, if the file
/proc/sys/vm/max_map_count was not existing and we still called
sysctl.
This is primarly to prevent confusing error messages when elasticsearch
is started under virtualized environments without a proc file system.
Closes#4978
Instead of iterating all shards of all indices to get all relocating
shards for a given node we can just use the RoutingNode#shardsWithState
method and fetch all INITIALIZING / RELOCATING shards and check if they
are relocating. This operation is much faster and uses pre-build
data-structures.
Relates to #6372
This commit changes internal callback to be clear
about when they are called and also provide the
exception that was potentially thrown as a callback argument.
Closes#5945
Previously it was possible for the field data clearing in this test to
take too long, causing the test to time out.
This also switches to using `scaledRandomIntBetween` for the number of
fields.
This change will chown /usr/share/elasticsearch/plugins to the elasticsearch
user (the directory was formerly owned by root). This enables the ES user to
manage plugins.
Also, /usr/share/elasticsearch/plugins is now removed when the elasticsearch
package is un-installed. Previously it was left lying there.
Closes#8732
Signed-off-by: Thilo Fromm <github@thilo-fromm.de>
This change adds a systemd service configuration file, and adds systemd logic
to installation and de-installation scripts. The upcoming Debian 8 "Jessie"
release will use systemd.
fixes#8943
Signed-off-by: Thilo Fromm <github@thilo-fromm.de>
the recovery diff can return file in the `different` category
since it's conservative if it can't tell if the files are the same.
Yet, the cleanup code only needs to ensure both ends of the recovery
are consistent. If we have a very old segments_N file no checksum is present
but for the delete files they might be such that the segments file passes
the consistency check but the .del file doesn't sicne it's in-fact the same
but this check was missing in the last commit.
Original indices are optional in ShardDeleteByQueryRequest only for backwards compatibility, see #7406. We can remove this in master since 2.0 will require a full cluster restart.
Closes#8777
This commit factors out the PID file creation from bootstrap and adds
tests for error conditions etc. We also can't rely on DELETE_ON_CLOSE
since it might not even write the file depending on the OS and JVM implementation.
This impl uses a shutdown hook to best-effort remove the pid file if it was written.
Closes#8771
This commit adds a very lightweight action to the transport
serivce that allows to fetch clustername and the discovery node
from a node. This is used by transport clients to test liveness of
a node without using the nodesinfo API which can be blocking if management
threadpools are busy.
Closes#8763
The bwc layer added with #7105 is not needed in master as a full cluster restart will be required, thus from 2.0 on the only supported action names are compliant to the defined conventions and don't need to be converted to the old format
Closes#8758
Related to #8667:
Some QueryBuilders have been deprecated in 1.x branches. We removed them in 2.0.
Removed
-------
* `textPhrase(...)`
* `textPhrasePrefix(...)`
* `textPhrasePrefixQuery(...)`
* `filtered(...)`
* `inQuery(...)`
* `commonTerms(...)`
* `queryString(...)`
* `simpleQueryString(...)`
Closes#8721.
Simon Willnauer
Ryan Ernst
Robert Muir
Michael McCandless
tlrx
Kevin Kluge
Boaz Leskes
Lee Hinman
Alexander Reelsen
Jun Ohtani
Patrick Koenig
Igor Motov
Thilo Fromm
Adrien Grand
tristanbob
Louis Sautier
John Michael Luy
Martijn van Groningen
javanna
Sebastian Ziebell
David Pilato