The watcher thread pool is scaled by the number of CPUs and has by
default up to 5x the number of cores. This is needed because we assumme
I/O based waiting workloads for watcher. However if the node is not a
data node, there will not be any execution of watches with the exception
of a user calling the execute watch API on that node.
This means, we can get away with just one thread, so that there is no
need for the JVM to manage more threads on master/client or ingest only
nodes.
Original commit: elastic/x-pack-elasticsearch@b5899401d3
The current toXContent serialization of a failed hipchat message writes
the same field called status twice and thus cannot be stored in the
watch history.
This commit ensures the field gets only written once.
relates elastic/x-pack-elasticsearch#3919
Original commit: elastic/x-pack-elasticsearch@fb499e8055
... yet support updates. This commit introduces a few changes of how
watches are put.
The GET Watch API will never return credentials like basic auth
passwords, but a placeholder instead now. If the watcher is enabled to
encrypt sensitive settings, then the original encrypted value is
returned otherwise a "::es_redacted::" place holder.
There have been several Put Watch API changes.
The API now internally uses the Update API and versioning. This has
several implications. First if no version is supplied, we assume an
initial creation. This will work as before, however if a credential is
marked as redacted we will reject storing the watch, so users do not
accidentally store the wrong watch.
The watch xcontent parser now has an additional methods to tell the
caller if redacted passwords have been found. Based on this information
an error can be thrown.
If the user now wants to store a watch that contains a password marked
as redacted, this password will not be part of the toXContent
representation of the watch and in combinatination with update request
the existing password will be merged in. If the encrypted password is
supplied this one will be stored.
The serialization for GetWatchResponse/PutWatchRequest has changed.
The version checks for this will be put into the 6.x branch.
The Watcher UI now needs specify the version, when it wants to store a
watch. This also prevents last-write-wins scenarios and is the reason
why the put/get watch response now contains the internal version.
relates elastic/x-pack-elasticsearch#3089
Original commit: elastic/x-pack-elasticsearch@bb63be9f79
* [Tests] Ensure that Watcher templates are created before creating alias
The createWatcherIndicesOrAliases() method randomly created watcher
indices with or without aliases. On slow machines it is possible that
the indices get created before WatcherIndexTemplateRegistry put the
templates. When it happens, the indices will be created without the
right index.format value, preventing the WatcherService to start (as
it checks the index.format before starting, in case an upgrade is
required)
relates elastic/x-pack-elasticsearch#3965
Original commit: elastic/x-pack-elasticsearch@82946a2be0
In order to prevent exceptions to bubble up to the thread pool exception
handler, this properly wraps all the calls for the watcher service
within an executor into an AbstractRunnable to catch and log a possible
exception.
relates elastic/x-pack-elasticsearch#3854
Original commit: elastic/x-pack-elasticsearch@c0b39e6b5b
This commit introduces a new watcher setting to defer starting watcher
until it has been called with the API for the first time. This is
primarily useful in testing environments, as this ensures that watcher
does not try to reload itself because of starting first and then
creating watcher indices.
In addition the undocumented and unused option
xpack.watcher.start_immediately has been removed.
Relates elastic/x-pack-elasticsearch#3854
Original commit: elastic/x-pack-elasticsearch@2b55aec4ad
This fixes a regression introduced in Elasticsearch 6.0, when switching
from HttpURLConnection to the Apache HTTP Client.
In the old implementation there was no way to specify if you wanted to use HTTP
or HTTPS for your proxy, only HTTP. If people needed to use HTTPs, they
could just use the CONNECT feature of the proxy.
The new implementation used the scheme of the request that was about to
be sent out as the proxy scheme to be used. So if the request was HTTPS
but the proxy server was HTTP this created a problem.
This commit changes the default scheme to be just HTTP, so that then the
standard CONNECT procecure is taken care off.
Without a real proxy server this is super hard to test. I have verified
this with the following test against a tinyproxy running on port 8888,
but I do not have a great idea how to test this in a unit testable way using a real proxy.
Original commit: elastic/x-pack-elasticsearch@f68e72d8f1
The watcher index uses auto expanding of replicas nowadays, so there is
no need to set the number of replicas.
Also ensuring that all watcher indices are green on startup has been moved
to one call, in order to prevent indices being moved around after adding
the triggered watches or watch history index.
Original commit: elastic/x-pack-elasticsearch@dbe04adf64
java.time features it's own halted clock, called a fixed clock, we can
use that one.
On top of that the watcher xcontent parser does not need a clock at all,
just a timestamp when parsing happened.
Original commit: elastic/x-pack-elasticsearch@2061aeffe1
This adds support to allow different schemes in a proxy being used
compared to what the actual request requires. So if your proxy runs via
HTTP, but the endpoint you want to connect to uses HTTPS, this is now
possible to configure the proxy explicitely.
Also a small unit test for parsing this has been added.
relates elastic/x-pack-elasticsearch#3596
Original commit: elastic/x-pack-elasticsearch@176f7cdf0e
Tests have been failing because of out of order cluster state processing
or because of stopping/starting was still in progress. Current tests do
not do further stop/start tries after the first leading to potential
being stuck in an unwanted state. This commit removes the methods
checking for the state being started or stopped in favor of using the
stopWatcher/startWatcher methods which now check if the desired state is
reached and otherwise issue another start/stop command.
Original commit: elastic/x-pack-elasticsearch@97b3232a6a
The WatcherLifeCycleService is responsible for deciding if watches need
to be reloaded while running. In order to do this, the service stores
the currently local shard allocation ids in a List.
This data structure however was not properly updated all the time, when
it should have been - for example when a master node is not available.
This lead to unintended reloads, even though there was no change in the
allocated shards. This in turn lead to unwanted executions and unwanted
loading of triggered watches.
This commit should also fix one of the more nasty ongoing test failures,
where the test returns with an exception that only parts of watcher have
been started. The AbstractWatcherIntegrationTestCase now properly waits
until watcher is started before starting the test case itself.
Original commit: elastic/x-pack-elasticsearch@097f12a900
In order to more easily integrate xpack once it moves into the
elasticsearch repo, references to the existing x-pack-elasticsearch need
to be reduced. This commit introduces a few helper "methods" available
to any project within xpack (through gradle project extension
properties, as closures). All refeerences to project paths now use these
helper methods, except for those pertaining to bwc, which will be
handled in a followup.
Original commit: elastic/x-pack-elasticsearch@850668744c
The EmailSecretsIntegrationTests did not properly wait for watcher to be
stopped before starting it again.
This test failed a few times per week across master and 6.x branches.
Original commit: elastic/x-pack-elasticsearch@cf20d58b0b
This change moves watcher's Cron class in the schedule package of xpack-core so that it can be used by other projects (rollup).
Original commit: elastic/x-pack-elasticsearch@f0aa32ccc2
This commit reenables running ITs in xpack by adding an internalClusterTest to xpack modules that contain ESIntegTestCase tests. The new task allows us to run these independently of rest integ tests, which are disabled for xpack modules because installing the bundled plugins directly is not quite the same as installing via the meta plugin. Some tests (ML) are moved to their own qa module to accommodate the need for a real cluster. A couple tests (monitoring and upgrade) have been marked as AwaitsFix.
Commits that have been folded into this commit:
* Move ML IT tests to qa/ml-native-tests
* Add internalClusterTest task and disable rest integ tests for xpack
modules. Also tweak ML tests and get upgrade tests working
* Adding the keystore and security back to the ml native tests
* Fixing native integ test
* Fix last ML test, add awaits fix to monitoring and upgrade tests
* cleanup PR
* fix checkstyle
Original commit: elastic/x-pack-elasticsearch@3c0ed6fd3b
The security manager permissions were copied wholesale from pre-split
X-Pack. However, this grants unnecessary permissions to the child
plugins. This commit is a simple attempt at removing permissions that
are not needed in the child plugins.
Relates elastic/x-pack-elasticsearch#3651
Original commit: elastic/x-pack-elasticsearch@8325ed83d7
Some tests seem to be pretty flaky due to concurrent watcher restarts.
This change makes sure we never restart watcher once the node is shutting down.
Original commit: elastic/x-pack-elasticsearch@f0bed7269b
x-pack makes extensive use of group settings in its exporters as well as
its notification services. This does not cope well with settings being
marked as filtered.
This replaces a fair share of group settings (unfortunately not all of
them) with affix key settings, allowing for better filtering and
registering of settings.
Original commit: elastic/x-pack-elasticsearch@968bc8c2ee
This commit moves the source file in x-pack-core to a org.elasticsearch.xpack.core package. This is to prevent issues where we have compile-time success reaching through packages that will cross module boundaries at runtime (due to being in different classloaders). By moving these to a separate package, we have compile-time safety. Follow-ups can consider build time checking that only this package is defined in x-pack-core, or sealing x-pack-core until modules arrive for us.
Original commit: elastic/x-pack-elasticsearch@232e156e0e
This reverts commit elastic/x-pack-elasticsearch@f91c401a60 due to
failing tests, like
./gradlew :x-pack-elasticsearch:plugin:watcher:test -Dtests.seed=AE30350FCE96D26D -Dtests.class=org.elasticsearch.xpack.watcher.watch.WatchTests -Dtests.method="testParserSelfGenerated" -Dtests.security.manager=true -Dtests.locale=ja-JP -Dtests.timezone=EET
Original commit: elastic/x-pack-elasticsearch@e45d79d643
The HaltedClock was a leftover from moving over from our own Clock
implementation to a java.time one. java.time already has a fixed clock,
this one is not needed.
Original commit: elastic/x-pack-elasticsearch@f91c401a60
We are relying on the fact that :x-pack-elasticsearch:plugin:core is
configured before these. This happens when these projects are
alphabetically after core, but is not the case when they are not
(ccr). To address this, we simplfy be explicit about evaluation order in
all sub-plugins.
Relates elastic/x-pack-elasticsearch#3663
Original commit: elastic/x-pack-elasticsearch@755f84258b
These were copied wholesale from the pre-split X-Pack
descriptor. However, only ML has a native controller. This removes the
plugin installation asking multiple times to approve the existence of a
native controlled for every bundled plugin.
Relates elastic/x-pack-elasticsearch#3650
Original commit: elastic/x-pack-elasticsearch@4fca606243
With plugins of a meta-plugin now loaded in separate classloaders, we
should not be loading all classes in all plugins of a meta-plugin when
executing scripts. This is particularly problematic in the case of
security extensions where the install plugin extension command would be
running with the classpath of all plugins. However, if there is JAR hell
in this classpath, installation would fail. This is not realistic though
since the plugins are run in separate classloaders. To fix this, for the
scripts of a plugin, we only set the classpath to include the JARs for
that plugin and the JARs of core. This leads us to the introduction of
plugin-specific env scripts.
Relates elastic/x-pack-elasticsearch#3649
Original commit: elastic/x-pack-elasticsearch@543df37eed
Some imports were changed in 6.x to address line-length issues
there. This commit pulls the same changes to master to keep the branches
consistent to simplify backports.
Original commit: elastic/x-pack-elasticsearch@190f9d41f5
This is no longer needed for general X-Pack since the tests will not be
depending on ML. We move this class to an ML specific directory and
remove the dependency from other tests.
Original commit: elastic/x-pack-elasticsearch@9b287f7460
* Rename folder x-pack-core -> core
The jar remains 'x-pack-core-*.jar'
* Put group in top-level build.gradle instead of plugin/core/build.gradle
Original commit: elastic/x-pack-elasticsearch@b23452fa55
This creates a basic skeleton for the plugin split by adding folders and example
`build.gradle` files. It also includes a non-implemented `migrate-plugins.sh`
script that we can fill in at a later time.
Relates to elastic/x-pack-elasticsearch#2925
Original commit: elastic/x-pack-elasticsearch@2ab035d6b6
Alexander Reelsen
Luca Cavanna
Lee Hinman
Tanguy Leroux
Adrien Grand
Tim Brooks
Martijn van Groningen
Ryan Ernst
Michael Basnight
Jim Ferenczi
Jason Tedor
Simon Willnauer