A number of REST requests require a body but did not explicitly validate for it.
This would typically cause a NPE if they were called with no body.
Original commit: elastic/x-pack-elasticsearch@863ac89429
The test also used the timewarp trigger for watches to be executed, but it is sufficient to just call the execute watch API to make this test faster.
Original commit: elastic/x-pack-elasticsearch@3a4165f72c
When getting a single bucket, the get_buckets API can take a timestamp
either in the body or in the URL. Prior to this change, if a timestamp
was specified in the URL but a body not containing a timestamp was specified
(either empty or containing other parameters like exclude_interim or sort)
then it would cause a bad_request exception. This in turn causes problems
for clients that cannot send a body when GETting and always send a body when
POSTing.
This change fixes get_buckets to always read any timestamp in the URL, even
when a body is sent.
relates elastic/x-pack-elasticsearch#2515
Original commit: elastic/x-pack-elasticsearch@5c23dd972e
If a job close is requested after a job was opened but before
its process was launched, the job close returns successfully
without doing anything. The result is that the process hangs
around. This has been causing test failures as documented
int elastic/x-pack-elasticsearch#2360 and elastic/x-pack-elasticsearch#1270.
This commit fixes this problem by refactoring the
AutodetectProcessManager. It introduces a state pattern
to make clear the states of the process and it uses locking
to ensure a close waits for the job process to be created.
relates elastic/x-pack-elasticsearch#1270
Original commit: elastic/x-pack-elasticsearch@ff858bd136
The AuthenticationService#testInvalidToken would cause a suite timeout in the case of an exception
due to a incorrect stream size as the latch was never counted down. This fixes the missing latch
countdown.
relates elastic/x-pack-elasticsearch#2615
Original commit: elastic/x-pack-elasticsearch@e838e6e912
Checking the size of the map doesn't make sense when each filter is
checked independently right after.
Original commit: elastic/x-pack-elasticsearch@58e5d3401d
This is related to elastic/x-pack-elasticsearch#1941.
Currently we support self-generating either a basic or trial license at
cluster startup. With the addition of the basic option, it is possible
that a user would choose to self-generate and eventually register a
basic license.
This commit allows a user to upgrade to a 30-day trial license if they
have not already utilized this 30-day trial license before. Additionally
it adds a get route to check if the user is eligible to upgrade. This
route will allow kibana to implement a cleaner UI.
Original commit: elastic/x-pack-elasticsearch@7f19b33a08
This removes the creation and handling of the Beats monitoring template and its data until we actually expect to support it (most likely 6.2 - 6.3).
Original commit: elastic/x-pack-elasticsearch@2dc8abbb37
* [Monitoring/Cluster Alerts] Fix the email message for cluster license expiration resolved
* fix making payload.message show only when new
Original commit: elastic/x-pack-elasticsearch@6d54b02913
When the datafeed uses aggregations and in order to accommodate
derivatives, an extra bucket is queried at the beginning of each
search. In order to avoid visiting the same bucket twice, we need
to search buckets aligned to the histogram interval. This allows
us to steer away from partial buckets, and thus avoid the problem
of dropping or duplicating data.
relates elastic/x-pack-elasticsearch#2519
Original commit: elastic/x-pack-elasticsearch@e03dde5fea
There are two rest tests that rely on hardcoded license
signatures that use the dev public key. These tests fail
when tests are run with build.snapshot=false. This Commit
blacklists these two tests in that scenario
relates elastic/x-pack-elasticsearch#2527.
Original commit: elastic/x-pack-elasticsearch@7581e8d699
Some model plot documents should not have an "actual" value, for example
when no input events were seen for a meean/min/max detector in a particular
bucket. Prior to this change we would set the "actual" value to 0 for such
model plot documents. Following this change no "actual" value will be
present in these documents.
Only newly created model plot documents are affected. Model plot documents
that were incorrectly written in the past will remain wrong forever.
relates elastic/x-pack-elasticsearch#2528
Original commit: elastic/x-pack-elasticsearch@47a7365f59
The problem here is that the code was ignoring buckets
whose start time was before the start time of the extractor.
However, this is not a good enough condition. For example,
when there are no data in the bucket extra bucket that is
being queried, the first bucket will be the one containing
the start time.
This commit fixes the issue by changing the condition to
ignore buckets before the first bucket that includes the
start time of the extraction.
relates elastic/x-pack-elasticsearch#2519
Original commit: elastic/x-pack-elasticsearch@15c7d2655f
If the keystore type is not explicitly specified, infer it from the filename.
Treats .p12, .pfx and .pkcs12 as being PKCS12, all others as jks.
This will allow certgen to produce PKCSelastic/x-pack-elasticsearch#12 files by default and make it easy to use them as x-pack keystores
Original commit: elastic/x-pack-elasticsearch@fc361f0d87
Beginning with 7.0, the cleaner service will no longer automatically cleanup .marvel indices regardless of their age.
Original commit: elastic/x-pack-elasticsearch@5b90e6f62a
This changes Monitoring's Cleaner Service to remove any legacy Monitoring index that is appropriately old.
This includes any `.marvel-*` index and also the "data" indices used by both Marvel and 5.0 - 5.4 versions of X-Pack monitoring, as well as the legacy alerts index.
Original commit: elastic/x-pack-elasticsearch@8d99f5518b
Since the template upgrade service was added, upgrades should
be performed by a node with the highest version in the cluster,
which may not be the master node.
Original commit: elastic/x-pack-elasticsearch@d66145de54
This commit adds back the ability to disable TLS on the transport layer and also disables TLS by
default to restore the 5.x behavior. The auto generation of key/cert and bundled CA certificate
have also been removed.
Relates elastic/x-pack-elasticsearch#2463
Original commit: elastic/x-pack-elasticsearch@abc66ec67d
if a user tries to upgrade a license to a production license and has security
enabled we prevent the upgrade unless TLS is setup. This is a requirement now
if a cluster with security is running in prodcution.
Relates to elastic/x-pack-elasticsearch#2463
Original commit: elastic/x-pack-elasticsearch@d61ef3bcb1
This change removes security index access from the xpack user by creating its own specific role
and adds a xpack security user that maintains the superuser role so that it can perform all
operations necessary for security.
Original commit: elastic/x-pack-elasticsearch@ad906bc913
This change will enforce transport SSL to be enforced if security is enabled and the
license in the clusterstate is a production license. The cluster state is loaded from
local storage such that we don't need to join a cluster to make these checks. Yet, the cluster
might have already got a different license if the node got disconnected while the license got
downgraded and then TLS got disabled. This corner case requires manual intervention which
we consider ok given the simplicity of this change.
Relates to elastic/x-pack-elasticsearch#2463
Original commit: elastic/x-pack-elasticsearch@5765b7cd21
integrate forecasting feature branch into master
- add endpoint xpack/ml/job/forecast to request forecasting on data of ml-jobs
- current parameters: end time
- persists forecast results into shared or own index
- different runs are separated by a 'forecast id'
relates elastic/x-pack-elasticsearch#1838
Original commit: elastic/x-pack-elasticsearch@f9d701a6bc
After the addition of the secure settings in 5.6, the truststore.password setting for the PKI realm
was no longer registered. This would cause new nodes to fail for customers that were upgrading and
had configured a PKI realm with a truststore. This change registers the setting and adds a test to
ensure a realm configuration with the old setting passes validation.
Relates elastic/support-dev-help#2505
Original commit: elastic/x-pack-elasticsearch@54da044a27
This change prevents a node from joining a cluster with a production license (gold, platinum, standard) iff the cluster doesn't have TLS setup. This is mainly a BWC oriented change that prevents joining old 5.x clusters without a TLS setup.
Relates to elastic/x-pack-elasticsearch#2463
Original commit: elastic/x-pack-elasticsearch@21f5a58472
Currently the maintenance task is executed at 30 minutes past
midnight of each day. In the scenario where multiple clusters
are running on the same hardware infrastructure they all will
be running at the same time, competing for resources.
This commit changes this by adding a random offset to the
execution time which ranges from 0 to 119 minutes. The
minute granularity means that different offsets give at
least 1 minute for the maintenance task to end. Moreover,
the 2 hour window gives enough slots for different offsets
to occur and remains within what most people would think
as "middle of the night".
relates elastic/x-pack-elasticsearch#2273
Original commit: elastic/x-pack-elasticsearch@b538923aca
Changes the default query delay from 1m to a random
value between 1m and 2m. The motivation is to avoid
having multiple jobs firing their searches at the same
time which may potentially lead to increased load
on the machine.
relates elastic/x-pack-elasticsearch#2472
Original commit: elastic/x-pack-elasticsearch@3224e836fa
This change removes `xpack.security.authc.token.passphrase` entirely since from
6.0 onwards we use randomly generated keys by the master there is no need for
this setting anymore. This setting will be deprecated from 6.0 onwards.
Original commit: elastic/x-pack-elasticsearch@37ba90359e
As there are no master node operations anymore.
* TransportActions are regular Actions now
* Watcher requests are now ActionRequests, no MasterNodeRequests anymore
* REST spec does not contain master node timeout parameters anymore
* WatcherLifeCycleService does not have a check anymore if watcher is able to run distributed, this will be a given in 7.0
* Some serialization BWC checks against version 5 have been removed
Original commit: elastic/x-pack-elasticsearch@4607dd538c
The datafeed runs on frequency-aligned intervals behind
query_delay. Currently, when a real-time run is triggered,
we subtract query_delay from now and then we take the aligned
interval. This results into running frequency + query_delay
behind now. The fix involves simply adding the query_delay
into the time real-time runs occur.
Relates elastic/x-pack-elasticsearch#2426
Original commit: elastic/x-pack-elasticsearch@61ceaaca8f
"Established" memory use will be one of the building blocks for smarter node
allocation.
In order for a job to be considered to have established memory usage it must:
- Have generated at least 20 buckets of results
- Have generated at least one model size stats document
- Have low variability of model bytes in model size stats documents in the
time period covered by the last 20 buckets, which is defined as having a
coefficient of variation of no more than 0.1
Relates elastic/x-pack-elasticsearch#546
Original commit: elastic/x-pack-elasticsearch@5032eb01d8
The changes made for elastic/x-pack-elasticsearch#2369 showed that the ML security tests were seriously
weakened by the decision to grant many "minimal" privileges to all users
involved in the tests. A better solution is to override the auth header
such that a superuser runs setup actions and assertions that work by
querying raw documents in ways that an end user wouldn't. Then the ML
endpoints can be called with the privileges provided by the ML roles and
nothing else.
Original commit: elastic/x-pack-elasticsearch@4de42d9e54
When watcher is loading it must only load the watches
which are active instead of all possible watches.
This loading happens on start up as well as when shards
relocate.
Original commit: elastic/x-pack-elasticsearch@29df56b99d
Implementation details of ML endpoints should be performed using the
internal client, so that the end user only requires permissions for
the public ML endpoints and does not need to know how they are
implemented. This change fixes some instances where this rule was
not adhered to.
Original commit: elastic/x-pack-elasticsearch@01c8f5172c
* Add support for authz checks at on shard requests
* Add Rest Tests for authorization
* Bulk security - Only reject individual items, rather than a whole shard
* Sync with core change
* Grant "delete" priv in ML smoketest
This role had index and+bulk privileges but it also needs delete (in order to delete ML model-snapshots)
Original commit: elastic/x-pack-elasticsearch@830e89e652
The method to check if watcher was enabled was returning
`randomBoolean()` and thus could change during test runs.
This fixes the test to ensure that always the same value
is returned and documents this requirement.
relates elastic/x-pack-elasticsearch#1783
Original commit: elastic/x-pack-elasticsearch@97bf3cfc29
This parameter ceased to work when Elasticsearch 5 introduced strict
parameter handling, because of a missing test.
This commit adds the parameter to the rest handler responseParams()
and adds a test along with the needed YAML definition.
relates elastic/x-pack-elasticsearch#2396
Original commit: elastic/x-pack-elasticsearch@8638df336c
If the duration time was 0 (and this might happen due to
using System.currentTimeMillis), the is_true check
still returns false.
The correct fix will be done later to replace the offending
measurement calls and replace them. Then we can add back this
line.
Original commit: elastic/x-pack-elasticsearch@076a9a37cc
A bunch of integration tests should have been built as unit tests
or already have unit test equivalents.
This commit removes integration tests as well as adding REST equivalents
or creating unit tests instead of extending from AbstractWatcherIntegrationTestCase
Original commit: elastic/x-pack-elasticsearch@a97b99467d
Tim Vernum
Simon Willnauer
Alexander Reelsen
David Roberts
Dimitris Athanasiou
Tanguy Leroux
jaymode
Tim Brooks
Martijn van Groningen
Jason Tedor
Chris Earle
David Kyle
Albert Zaharovits
Tim Sullivan
Igor Motov
Hendrik Muhs
Yannick Welsch
Tal Levy
Nik Everett
Suyog Rao
Michael Basnight
Andy Bristol
Jay Modi
Ryan Ernst
Jim Ferenczi