On ubuntu 14.04, which uses upstart, where as our debian package uses
sysvinit, there is no stdout/stderr message printed when starting up,
because the start-stop-daemon swallows it.
As Elasticsearch is started to daemonize, we can remove the background
flag from the start-stop-daemon and thus see, if the system does not have
enough memory for starting up - something that happens often on VMs, since
Elasticsearch 5.0 uses 2gb by default instead of one.
Relates #21300
Relates #12716
We plan to deprecate `_suggest` during 5.0 so it isn't worth fixing
it to support the `_source` parameter for `_source` filtering. But we
should fix the docs so they are accurate.
Since this removes the last non-`// CONSOLE` line in
`completion-suggest.asciidoc` this also removes it from the list of
files that have non-`// CONSOLE` docs.
Closes#20482
Adds support for `?slices=N` to reindex which automatically
parallelizes the process using parallel scrolls on `_uid`. Performance
testing sees a 3x performance improvement for simple docs
on decent hardware, maybe 30% performance improvement
for more complex docs. Still compelling, especially because
clusters should be able to get closer to the 3x than the 30%
number.
Closes#20624
Today when writing an unbounded queue_size in the cat thread pool API,
we write null. This commit modifies this so that the output is -1 so
that the output is always present, and always a numeric value.
Relates #21342
Exist requests are supposed to never throw an exception, but rather return true or false depending on whether some resource exists or not. Indices exists does that for indices and accepts wildcard expressions too. The way the api works internally is by resolving indices and catching IndexNotFoundException: if an exception is thrown the index does not exist hence it returns false, otherwise it returns true. That works ok only if ignore_unavailable and allow_no_indices indices options are both set to false, meaning that they are strict and any missing index or wildcard expressions that resolves to no indices will lead to an exception that can be thrown and cause false to be returned.
Unfortunately the indices options have been configurable up until now for this request, meaning that one can set ignore_unavailable or allow_no_indices to true and have the indices exist request return true for indices that really don't exist, which makes very little sense in the context of this api.
This commit removes the indicesOptions setter from the IndicesExistsRequest and makes settable only expandWildcardsOpen and expandWildcardsClosed, hence a subset of the available indices options. This way we can guarantee more consistent behaviour of the indices exists api. We can then remove the ignore_unavailable and allow_no_indices option from indices exists api spec
Today if you start Elasticsearch with the status logger configured to
the warn level, or use a transport client with the default status logger
level, you will see warn messages about deprecation loggers being
created with different message factories and that formatting might be
broken. This happens because the deprecation logger is constructed using
the message factory from its parent, an artifact leftover from the first
Log4j 2 implementation that used a custom message factory. When that
custom message factory was removed, this constructor invocation should
have been changed to not explicitly use the message factory from the
parent. This commit fixes this invocation. However, we also had some
status checking to all tests to ensure that there are no warn status log
messages that might indicate a configuration problem with Log4j 2. These
assertions blow up badly without the fix for the deprecation logger
construction, and also caught a misconfiguration in one of the logging
tests.
Relates #21339
This commit fixes an issue with the cat thread pool API spec. Namely,
the name of the variable for the thread pool patterns parameter was
misnamed in the spec.
Relates #21332
Moves the `_flush` in the `_cat/indices` snippets testing framework
to the very first test. We need to flush super early because index
size is cached for a few seconds so we really need to read a
consistent size on the first read so we can sort by it properly.
Closes#21062
Today we validate the target index name late and therefore don't fail for instance
if the target index already exists and `dry_run=true` was specified. This change
validates the index name before we early terminate if dry_run is set.
Closes#21149
Today we still have a leftover from older percolators where lucene
query instances where created ahead of time and rewritten later.
This `LateParsingQuery` was resolving `now()` when it's really used which we
don't need anymore. As a side-effect this failed to execute some highlighting
queries when they get rewritten since at that point `now` access it not permitted
anymore to prevent bugs when queries get cached.
Closes#21295
This commit introduces a new execution mode for the query_string query, which
is intended down the road to be a replacement for the current _all field.
It now does auto-field-expansion and auto-leniency when the following criteria
are ALL met:
The _all field is disabled
No default_field has been set in the index settings
No default_field has been set in the request
No fields are specified in the request
Additionally, a user can force the "all-like" execution by setting the
all_fields parameter to true.
When executing in all field mode, the query_string query will look at all the
fields in the mapping that are not metafields and can be searched, and
automatically expand the list of fields that are going to be queried.
Relates to #19784
According to ISO 8601, a time zone offset of zero, can be stated numerically as
"+00:00", "+0000", or "00". The Joda library also seems to allow for "-00:00",
"-00" and "-0000". This adds some test to the DateMathParserTests that check
that we also conform to this.
Closes#21320
The usage information for `elasticsearch-plugin` is quiet verbose and makes the
actual error message that is shown when trying to remove an non-existing plugin
hard to spot. This changes the error code to not trigger printing the usage
information.
Closes#21250
Dependencies are currently marked as non-transitive in generated POM files by adding a wildcard (*) exclusion. This breaks compatibility with the dependency manager Apache Ivy as it incorrectly translates POMs with * excludes to Ivy XML with * excludes which results in the main artifact being excluded as well (see https://issues.apache.org/jira/browse/IVY-1531). To stay compatible with the current release of Ivy this commit uses explicit excludes for each transitive artifact instead to ensure that the main artifact is not excluded. This should be revisited when we upgrade Gradle to a higher version as the current one (2.13) as Gradle automatically translates non-transitive dependencies to * excludes in 2.14+.
This is a bit funky to do with junit because we need per test state
but we only want to log it per suite. So we use a static flag that
we test per test and reset before every suite.
ShardInfo#toString resorts to calling ShardInfo#toXContent via
Strings#toString. However, the resulting XContent object will not start
with an object, and this is a violation of the generator state
machine. This commit fixes this issue by replacing the override of
toString to provide simple non-toXContent output of the ShardInfo
instance. Without this fix, ShardInfo#toString will instead produce
"Error building toString out of XContent:
com.fasterxml.jackson.core.JsonGenerationException: Can not write a
field name, expecting a value"
Relates #21319
Our test infrastructure checks after running each test that there are no more in-flight requests on the shard level. Whenever the check fails, we only know that there were in-flight requests but don't know what requests were causing this issue. This commit adds the replication tasks that are still active at that moment to the assertion error.
With #21099 we removed support for the ignored allow_no_indices parameter in indices upgrade API. Truth is that ignore_unavailable and expand_wildcards were also ignored, in indices upgrade as well as upgrade status API. Those parameters are though supported internally and settable through java API, hence they should be all supported on the REST layer too.
`FilterAggregationBuilder` today misses to rewrite queries which causes failures
if a query that uses a client for instance to lookup terms since it must be rewritten first.
This change also ensures that if a client is used from the rewrite context we mark the query as
non-cacheable.
Closes#21301
Plugins: Remove pluggability of ZenPing
ZenPing is the part of zen discovery which knows how to ping nodes.
There is only one alternative implementation, which is just for testing.
This change removes the ability to add custom zen pings, and instead
hooks in the MockZenPing for tests through an overridden method in
MockNode. This also folds in the ZenPingService (which was really just a
single method) into ZenDiscovery, and removes the idea of having
multiple ZenPing instances. Finally, this was the last usage of the
ExtensionPoint classes, so that is also removed here.
Currently the default S3 buffer size is 100MB, which can be a lot for small
heaps. This pull request updates the default to be 100MB for heaps that are
greater than 2GB and 5% of the heap size otherwise.
The important settings docs previously referred to a section regarding
the node.max_local_storage_nodes setting. This section was removed, but
the link was not. This commit removes that link.
Previously node.max_local_storage_nodes defaulted to fifty, and this
permitted users to start multiple instances of Elasticsearch sharing the
same data folder. This can be dangerous, and usually it does not make
sense to run more than one instance of Elasticsearch on a single
server. Because of this, we had a note in the important settings docs
advising users to set this setting to one. However, we have since
changed the default value of this setting to one so this advise is no
longer needed.
Relates #21305
Since we now validate all consumed request parameter, users can't specify
`_cat/nodes?full_id=true|false` anymore since this parameter is consumed late.
This commit adds a test for this parameter and consumes it before request is processed.
Closes#21266
This would allow MapperService to still be usable in contexts when a
QueryShardContext cannot be obtained, for instance in the case that a
MapperService needs to be created only to merge mappings.
Currently the request cache adds a `CacheEntity` object. It looks quite innocent
but in practice it has a reference to a lambda that knows how to create a value.
The issue is that this lambda has implicit references to the SearchContext
object, which can be quite heavy since it holds a structured representation of
aggregations for instance.
This pull request splits the `CacheEntity` object from the object that generates
cache values.
Stored scripts and ingest node configuration are important parts of the overall cluster state and should be included into a snapshot together with index templates and persistent settings if the includeGlobalState is set to true.
Closes#21184
These tests had a single method due to the fact that es didn't support resetting settings when they were first written. They can now be rewritten to have separate methods and an after method that resets the setting that is left behind.
Adrien Grand
Clinton Gormley
Alexander Reelsen
Alexander Kazakov
markharwood
Nik Everett
Loek van Gool
Jason Tedor
Luca Cavanna
Andreas Creten
Jayson Minard
Simon Willnauer
Simon Willnauer
Lee Hinman
Christoph Büscher
Yannick Welsch
Chris Earle
Ryan Ernst
Boaz Leskes
Igor Motov
Areek Zillur
Shane Connelly