This change prevents a node from joining a cluster with a production license (gold, platinum, standard) iff the cluster doesn't have TLS setup. This is mainly a BWC oriented change that prevents joining old 5.x clusters without a TLS setup.
Relates to elastic/x-pack-elasticsearch#2463
Original commit: elastic/x-pack-elasticsearch@21f5a58472
Currently the maintenance task is executed at 30 minutes past
midnight of each day. In the scenario where multiple clusters
are running on the same hardware infrastructure they all will
be running at the same time, competing for resources.
This commit changes this by adding a random offset to the
execution time which ranges from 0 to 119 minutes. The
minute granularity means that different offsets give at
least 1 minute for the maintenance task to end. Moreover,
the 2 hour window gives enough slots for different offsets
to occur and remains within what most people would think
as "middle of the night".
relates elastic/x-pack-elasticsearch#2273
Original commit: elastic/x-pack-elasticsearch@b538923aca
Changes the default query delay from 1m to a random
value between 1m and 2m. The motivation is to avoid
having multiple jobs firing their searches at the same
time which may potentially lead to increased load
on the machine.
relates elastic/x-pack-elasticsearch#2472
Original commit: elastic/x-pack-elasticsearch@3224e836fa
This change removes `xpack.security.authc.token.passphrase` entirely since from
6.0 onwards we use randomly generated keys by the master there is no need for
this setting anymore. This setting will be deprecated from 6.0 onwards.
Original commit: elastic/x-pack-elasticsearch@37ba90359e
As there are no master node operations anymore.
* TransportActions are regular Actions now
* Watcher requests are now ActionRequests, no MasterNodeRequests anymore
* REST spec does not contain master node timeout parameters anymore
* WatcherLifeCycleService does not have a check anymore if watcher is able to run distributed, this will be a given in 7.0
* Some serialization BWC checks against version 5 have been removed
Original commit: elastic/x-pack-elasticsearch@4607dd538c
The datafeed runs on frequency-aligned intervals behind
query_delay. Currently, when a real-time run is triggered,
we subtract query_delay from now and then we take the aligned
interval. This results into running frequency + query_delay
behind now. The fix involves simply adding the query_delay
into the time real-time runs occur.
Relates elastic/x-pack-elasticsearch#2426
Original commit: elastic/x-pack-elasticsearch@61ceaaca8f
"Established" memory use will be one of the building blocks for smarter node
allocation.
In order for a job to be considered to have established memory usage it must:
- Have generated at least 20 buckets of results
- Have generated at least one model size stats document
- Have low variability of model bytes in model size stats documents in the
time period covered by the last 20 buckets, which is defined as having a
coefficient of variation of no more than 0.1
Relates elastic/x-pack-elasticsearch#546
Original commit: elastic/x-pack-elasticsearch@5032eb01d8
Instead of throwing and catching an exception for invalid
indices this returns *why* they are invalid in a convenient
object form that can be thrown as an exception when the index
is required or the entire index can be ignored when listing
indices.
Original commit: elastic/x-pack-elasticsearch@f45cbce647
This integrates SQL's metadata calls with security by creating
`SqlIndicesAction` and routing all of SQL's metadata calls through
it. Since it *does* know up from which indices it is working against
it can be an `IndicesRequest.Replaceable` and integrate with the
existing security infrastructure for filtering indices.
This request is implemented fairly similarly to the `GetIndexAction`
with the option to read from the master or from a local copy of
cluster state. Currently SQL forces it to run on the local copy
because the request doesn't properly support serialization. I'd
like to implement that in a followup.
Original commit: elastic/x-pack-elasticsearch@15f9512820
The changes made for elastic/x-pack-elasticsearch#2369 showed that the ML security tests were seriously
weakened by the decision to grant many "minimal" privileges to all users
involved in the tests. A better solution is to override the auth header
such that a superuser runs setup actions and assertions that work by
querying raw documents in ways that an end user wouldn't. Then the ML
endpoints can be called with the privileges provided by the ML roles and
nothing else.
Original commit: elastic/x-pack-elasticsearch@4de42d9e54
When watcher is loading it must only load the watches
which are active instead of all possible watches.
This loading happens on start up as well as when shards
relocate.
Original commit: elastic/x-pack-elasticsearch@29df56b99d
Implementation details of ML endpoints should be performed using the
internal client, so that the end user only requires permissions for
the public ML endpoints and does not need to know how they are
implemented. This change fixes some instances where this rule was
not adhered to.
Original commit: elastic/x-pack-elasticsearch@01c8f5172c
* Add support for authz checks at on shard requests
* Add Rest Tests for authorization
* Bulk security - Only reject individual items, rather than a whole shard
* Sync with core change
* Grant "delete" priv in ML smoketest
This role had index and+bulk privileges but it also needs delete (in order to delete ML model-snapshots)
Original commit: elastic/x-pack-elasticsearch@830e89e652
The method to check if watcher was enabled was returning
`randomBoolean()` and thus could change during test runs.
This fixes the test to ensure that always the same value
is returned and documents this requirement.
relates elastic/x-pack-elasticsearch#1783
Original commit: elastic/x-pack-elasticsearch@97bf3cfc29
This parameter ceased to work when Elasticsearch 5 introduced strict
parameter handling, because of a missing test.
This commit adds the parameter to the rest handler responseParams()
and adds a test along with the needed YAML definition.
relates elastic/x-pack-elasticsearch#2396
Original commit: elastic/x-pack-elasticsearch@8638df336c
If the duration time was 0 (and this might happen due to
using System.currentTimeMillis), the is_true check
still returns false.
The correct fix will be done later to replace the offending
measurement calls and replace them. Then we can add back this
line.
Original commit: elastic/x-pack-elasticsearch@076a9a37cc
A bunch of integration tests should have been built as unit tests
or already have unit test equivalents.
This commit removes integration tests as well as adding REST equivalents
or creating unit tests instead of extending from AbstractWatcherIntegrationTestCase
Original commit: elastic/x-pack-elasticsearch@a97b99467d
Scrolling was only implemented for the `SqlAction` (not jdbc or cli)
and it was implemented by keeping request state on the server. On
principle we try to avoid adding extra state to elasticsearch where
possible because it creates extra points of failure and tends to
have lots of hidden complexity.
This replaces the state on the server with serializing state to the
client. This looks to the user like a "next_page" key with fairly
opaque content. It actually consists of an identifier for the *kind*
of scroll, the scroll id, and a base64 string containing the field
extractors.
Right now this only implements scrolling for `SqlAction`. The plan
is to reuse the same implementation for jdbc and cli in a followup.
This also doesn't implement all of the required serialization.
Specifically it doesn't implement serialization of
`ProcessingHitExtractor` because I haven't implemented serialization
for *any* `ColumnProcessors`.
Original commit: elastic/x-pack-elasticsearch@a8567bc5ec
* Don’t count incomplete buckets in data stream diagnostics
* Fix tests now bucket_count doesn’t include partial buckets
Original commit: elastic/x-pack-elasticsearch@bc1a7bd9e7
This commit adapts to the renaming of the TransportResyncReplicationAction in core and also adds
an assertion to the check for a user being present when sending a request. The assertion is added
so that we can hopefully catch these scenarios in our testing as the assertion error will cause the
node to die but the ISE will just be seen in the logs. Since we do not run with assertions enabled
in production, the ISE is left to handle those cases.
relates elastic/x-pack-elasticsearch#2335
Original commit: elastic/x-pack-elasticsearch@c5ce0c93af
In `mutateInstance()` the from or size could become negative if the other one was pushed over the limit for `from + size`. This change fixes this case to make sure after the mutate method is called the from and size obey the limit but are also both `>= 0`
relates elastic/x-pack-elasticsearch#2344
Original commit: elastic/x-pack-elasticsearch@a8a7072fcc
* Add reserved dashboards_only_user role
* Fix line too long
* add tests for new reserved role
* rename role, hopefully fix tests
* Fix test
Original commit: elastic/x-pack-elasticsearch@99f6718c7c
