Prior to this change, if the persistent tasks framework noticed that a
job was running on a node that was isolated but has rejoined the cluster
then it would close that job. This was not ideal, because then the job
would persist state from the autodetect process that was isolated. This
commit changes the behaviour to kill the autodetect process associated
with such a job, so that it does not interfere with the autodetect process
that is running on the node where the persistent tasks framework thinks it
should be running.
In order to achieve this a change has also been made to the behaviour of
force-close. Previously this would result in the autodetect process being
gracefully shut down asynchronously to the force-close request. However,
the mechanism by which this happened was the same as the mechanism for
cancelling tasks that end up running on more than one node due to nodes
becoming isolated from the cluster. Therefore, force-close now also kills
the autodetect process rather than gracefully stopping it. The documentation
has been changed to reflect this. It should not be a problem as force-close
is supposed to be a last resort for when normal close fails.
relates elastic/x-pack-elasticsearch#1186
Original commit: elastic/x-pack-elasticsearch@578c944371
This came up in a forum post. An NPE was raised, when a search input
contained a search that did not contain a body, but just specified
indices or types.
This commit allows for empty bodies, and also makes sure there are
no null pointer exceptions by using empty bytes references otherwise.
In addition a suite scoped integration test was converted to a unit
test.
Original commit: elastic/x-pack-elasticsearch@29be2976fc
This introduces a new index setting called xpack.internal.format to
x-pack, which is configured for all of our index templates and set to
"v6". This indicates the version of compatibility of this index.
In addition a setting named index.xpack.version has been removed,
as it was unused.
Watcher does not start, if the watches and the triggered watches
index is not compatible with this setting.
Original commit: elastic/x-pack-elasticsearch@e430691c51
* [DOCS] Add Anomaly Explorer limitation re scripted fields
* [DOCS] Update Anomaly Explorer limitation re scripted fields
Original commit: elastic/x-pack-elasticsearch@7c9971ada0
Removes the `assemble` task from the `build` task when we have
removed `assemble` from the project. We removed `assemble` from
projects that aren't published so our releases will be faster. But
That broke CI because CI builds with `gradle precommit build` and,
it turns out, that `build` includes `check` and `assemble`. With
this change CI will only run `check` for projects without an
`assemble`.
Original commit: elastic/x-pack-elasticsearch@d01b0df1d9
Before the event stats were mislabeled, so were not being indexed, and
the ephemeral_id was only in the _state document.
Original commit: elastic/x-pack-elasticsearch@ca0ec81aa5
Adds tests similar to `:qa:full-cluster-restart` for x-pack. You
run them with `gradle :x-pack:qa:full-cluster-restart:check`.
The actual tests are as basic as it gets: create a doc and load it,
shut down, upgrade to master, startup, and load it. Create a user
and load it, shut down, upgrade to master, startup, and load it.
Relates to elastic/x-pack-elasticsearch#1629
Original commit: elastic/x-pack-elasticsearch@8994bec8e7
This commit removes unnecessary initialization of the system key in tests that no longer make use
of the system key. It also removes the feature usage for the system key in the SecurityFeatureSet.
Original commit: elastic/x-pack-elasticsearch@b9fffe0bd3
Logstash now has ephemeral id at the instance level and also at the
pipeline level, we need to add them to the logstash monitoring template.
Original commit: elastic/x-pack-elasticsearch@dfac702d59
* Use bulk request to persist model plots and model size stats
* Revert persisting model size stats in the bulk request
* Refactor results persister
Original commit: elastic/x-pack-elasticsearch@f51297bfc2
We recently added logging for critical authentication failures as they had previously been silent (with respect to logs) but would cause authentication processing to stop.
However, the reserved realm intentionally uses exceptions to stop any other realm authenticating a reserved user if the password is entered incorrectly.
Since this is the most common use of exceptions in the authc chain, we reduce the logging verbosity in normal cases (drop the stack trace, remove "unexpected") and only log the full details in debug.
Original commit: elastic/x-pack-elasticsearch@686a98010b
This commit fixes the default password migration tests that had been failing reproducibly. The
first fix skips tests using the set enabled api when running against a version prior to 5.1.2 as
this api would otherwise trip an assertion that the xcontent builder was not closed. The second
fix is to ensure we set the password field in the user object.
relates elastic/x-pack-elasticsearch#1529
relates elastic/x-pack-elasticsearch#1516
Original commit: elastic/x-pack-elasticsearch@2f9c804309
This is related to elastic/x-pack-elasticsearch#1217. This change introduces a tool
bin/x-pack/setup-passwords that will streamline the setting of
internal user passwords. There are two modes of operation. One mode
called auto, automatically generates passwords and prints them to
the console. The second mode called interactive allows the user to
enter passwords.
All passwords are changed using the elastic superuser. The elastic
password is the first password to be set.
Original commit: elastic/x-pack-elasticsearch@00974234a2
After improving the authorization of scroll requests and backporting to 5.x, we no longer need to
have any signing code in master. This commit removes it.
Original commit: elastic/x-pack-elasticsearch@8b65fd9338
If an exception occurs while sending the initial setup messages to the autodetect
such that it fails rather than reaching the open state then the autodetect process
needs to be killed to prevent it hogging resources.
Relates elastic/x-pack-elasticsearch#1684
Original commit: elastic/x-pack-elasticsearch@1ee80ed9b0
In does not make sense for the time_field in the data_description to
be used as a by/over/partition field name, nor the summary_count_field,
categorization_field or as an influencer. Therefore, configurations
where the time_field in the data_description is used in the
analysis_config are now rejected.
Additionally, it causes a problem communicating with the C++ code if
the control field name (which is '.') is used in the analysis_config,
so this is also rejected at the validation stage.
Relates elastic/x-pack-elasticsearch#1684
Original commit: elastic/x-pack-elasticsearch@e6750a2cda
- Don't attempt to upgrade from 2.x
- Attempt up to 10 retries if the migration fails (with increasing back-off between attempts)
- If a cached user is disabled, recheck with the underlying store
The last change is required if the migration takes a long time.
While users are being migrated, they might be marked as disabled, but when the migration is complete they need to be usable immediately.
Original commit: elastic/x-pack-elasticsearch@2621867014
Removes the `assemble` task from projects that aren't published
to speed up `gradle assemble` so the unified release can call it.
Original commit: elastic/x-pack-elasticsearch@43dfcc15f3
This commit increases the amount of time to wait for green cluster health during a rolling upgrade
to account for the time that may be needed in the case of delayed shards. Additionally some old
timeout values were removed as they were used due to the default timeout of 30s.
Relates elastic/x-pack-elasticsearch#1683
Original commit: elastic/x-pack-elasticsearch@9996673db0
These tests are starting their own nodes and do not use the testing
trigger schedule class.
There are occasional test failure due to a race condition where watcher
is in the process of being started, but cannot be shut down properly,
because starting up was not finished when the shut down was called for.
These filter tests do not rely on watcher, so we can disable them for
now, but we still need to fix a race condition in starting/stopping
watcher.
relates elastic/x-pack-elasticsearch#1422
Original commit: elastic/x-pack-elasticsearch@f13bb7a6fb
There is no need to handle any _status field in
the 6.0 release from now on, as everything has been
taken care in the upgrade API.
Original commit: elastic/x-pack-elasticsearch@606581f4d7
This changes part of the logic that was added in elastic/x-pack-elasticsearch#644 and extended
in elastic/x-pack-elasticsearch#1495 so that when ML is disabled we never try to communicate with
the native controller during node shutdown.
The original reason for needing to communicate with the native controller
when ML is disabled was the problem of elastic/prelert-legacy#803.
However, this was fixed in a better way in elastic/elasticsearch#24579.
Now there is considerable benefit in never talking to the native
controller from the plugin code when ML is disabled, because it means
anyone suffering some obscure problem with ML can disable it without
running the risk of uncovering some other obscure problem with shutdown.
Original commit: elastic/x-pack-elasticsearch@9d329483a7
This adds a check in the REST tests to ensure that
watcher is started, and if not, tries to start watcher.
This eliminates test failures where watcher was not in
the correct state due to other tests stopping watcher.
Original commit: elastic/x-pack-elasticsearch@fc547d49b4
This commit adds a new Logstash component to x-pack to support the config management work. Currently, the functionality in this component is really simple; all it does is upload a new index template for `.logstash` index. This index stores the actual LS configuration.
On this template is bootstrapped in ES, Kibana can write user-created LS configs which adhere to the mapping defined here. In the future, we're looking into adding more functionality on the ES side to handle config documents, but for now, this is simple.
relates elastic/x-pack-elasticsearch#1499, relates elastic/x-pack-elasticsearch#1471
Original commit: elastic/x-pack-elasticsearch@d7cc8675f7
In the case where a field is a text multi-field, it has
no doc values and it is not in source. Thus, the datafeed
will not be able to extract it.
However, it is possible to extract it by getting its parent
field instead. This commit implements the logic to look
in parent fields when the field in question is a text field.
Original commit: elastic/x-pack-elasticsearch@f116e89921
Alexander Reelsen
Dimitris Athanasiou
David Roberts
Simon Willnauer
Lisa Cawley
Nik Everett
Andrew Cholakian
Luca Cavanna
Jay Modi
Pier-Hugues Pellerin
David Kyle
Tim Vernum
Martijn van Groningen
jaymode
Tim Brooks
Jim Ferenczi
Jason Tedor
Boaz Leskes
Igor Motov
Chris Earle
Ryan Ernst
Suyog Rao