* Fix license messaging for Logstash functionality
With a Basic license, users are still able to perform CRUD operations on the `.logstash` index, therefore manage their Logstash pipelines. However, Logstash itself will not pick up any changes from this index and act on them. With an expired license Logstash functionality continues to operate as normal.
* Fixing messages after feedback
* Removing extraneous tabs at end of line
* Fixing typo
Original commit: elastic/x-pack-elasticsearch@bc069cf00f
This adds:
* The CLI jar itself into the `bin`. It is an executable jar.
* A shell and bat script to start the CLI. This isn't strictly required but folks will appreciate the consistency.
* Basic packaging tests for the CLI.
Relates to elastic/x-pack-elasticsearch#2979
Original commit: elastic/x-pack-elasticsearch@158f70a530
Verify that the configuration directory `$ES_PATH_CONF/x-pack`
exists before attempting to run any of the `users` commands, and
return a helpful error message to the user if it doesn't.
Original commit: elastic/x-pack-elasticsearch@6d663b6654
This changes the default behavior of .monitoring indices to be green on one-node clusters, instead of constantly yellow.
Note: This only impacts .monitoring* indices. Watcher indices currently still require a replica.
Original commit: elastic/x-pack-elasticsearch@6eb8a48a9f
* Rename folder x-pack-core -> core
The jar remains 'x-pack-core-*.jar'
* Put group in top-level build.gradle instead of plugin/core/build.gradle
Original commit: elastic/x-pack-elasticsearch@b23452fa55
This commit adds additional checks around resize operations and alias creation operations to
add an extra layer of security around these APIs.
Original commit: elastic/x-pack-elasticsearch@b79f16673c
Upon selecting a node to run a datafeed we normally check that the
data indices exists and their primaries are active. However, these
checks cannot be applied for CCS to a remote cluster. This commit
skips these checks for remote indices.
This removes the last obstacle for running CCS datafeeds.
Relates elastic/x-pack-elasticsearch#1071
Original commit: elastic/x-pack-elasticsearch@092f44feee
SQL used to have some changes to security. We've since reverted them but
we have some leftover stuff like import reordering and spacing changes.
We may as well remove them so merging SQL to master is smaller.
Original commit: elastic/x-pack-elasticsearch@c632256ddd
This commits adds a new end point for closing in-flight cursors, it also ensures that all cursors are properly closed by adding after test checks that ensures that we don't leave any search context open.
relates elastic/x-pack-elasticsearch#2878
Original commit: elastic/x-pack-elasticsearch@1052ea28dc
This commit updates x-pack to be compatible with
elastic/elasticsearch#27711. That commit removed the need for channels
to be internally tracked inside transport implementations. This commit
removes a test mocking class that is not necessary after that change.
Original commit: elastic/x-pack-elasticsearch@75d99ba1d1
This creates a basic skeleton for the plugin split by adding folders and example
`build.gradle` files. It also includes a non-implemented `migrate-plugins.sh`
script that we can fill in at a later time.
Relates to elastic/x-pack-elasticsearch#2925
Original commit: elastic/x-pack-elasticsearch@2ab035d6b6
Generate passwords from [A-Za-z0-9] so that they are safe to be
used in shell scripts.
Entropy deterioration is not significant (124.9 -> 119), generated
passwords still meet guidelines and best practices regarding length
and complexity.
Resolveselastic/x-pack-elasticsearch#3087
Original commit: elastic/x-pack-elasticsearch@078639e7c2
Hopefully fixes the Windows CI failures that break on cloning the repository into a target directory with a lengthy path name.
Original commit: elastic/x-pack-elasticsearch@fe18e95d3f
When using the security networking implementations, the Netty jars that
are in play are those that are loaded in the X-Pack classloader. This
means that permissions granted to the Netty jars loaded in the
transport-netty4 module classloader do nothing. Instead, we have to
grant the same permissions to the Netty jars in the X-Pack
classloader. This commit does this.
Relates elastic/x-pack-elasticsearch#3247
Original commit: elastic/x-pack-elasticsearch@91780597b9
* Add Special Event
* Add special events to update process
* Add time condition and skip rule actions.
* Update special events
* Address review comments
Original commit: elastic/x-pack-elasticsearch@80500ded76
Given that we get now filtered mappings directly from the get index API (in case security is configured with FLS), we don't need the security filter nor the filtered catalog. That means we can remove the delayed action support also from AuthorizationService and rather make SQLAction a composite action like others. It will be authorized as an action, but its indices won't be checked while that will happen with its inner actions (get index and search) which need to be properly authorized.
Also, SQLGetIndicesAction is not needed anymore, as its purpose was to retrieve the indices access resolver put in the context by the security plugin for delayed actions, which are not supported anymore.
This commit kind of reverts elastic/x-pack-elasticsearch#2162, as it is now possible to integrate with security out-of-the-box
relates elastic/x-pack-elasticsearch#2934
Original commit: elastic/x-pack-elasticsearch@64d5044426
This PR uses a new extension point that's being added to Elasticsearch (see https://github.com/elastic/elasticsearch/pull/27603) so that the security plugin can filter the mappings fields returned by get index, get mappings, get field mappings and field capabilities API.
This effort aims at filtering information returned by API in the `indices/admin` category and field capabilities. It doesn't filter what the cluster state api returns as that is a cluster level operation.
One question is about backwards compatibility given that we would like to have this in 6.2. Shall we treat this as a bug as mappings should have been filtered before? Not sure if it's going to break existing integrations.
relates elastic/x-pack-elasticsearch#340
Original commit: elastic/x-pack-elasticsearch@d7e3fd3fa1
Before this was done it was easy to get into the situation where a
job created in 5.x with a default model memory limit of 4GB could not
be opened on any node in the cluster. Following this change this
problem will no longer occur for jobs that ran for a decent amount of
time on the old cluster.
relates elastic/x-pack-elasticsearch#3181
Original commit: elastic/x-pack-elasticsearch@cb029debba
The watcher threadpool size was always five times the CPU core
count, resulting in a huge threadpool when with even 24 cores.
This changes the behaviour to be five times the number of cores
by default - as watcher is usually waiting on I/O you should have more
threads than cores, but it maxes out with 50 threads, unless the number
of available cores is higher than that.
relates elastic/x-pack-elasticsearch#3052
Original commit: elastic/x-pack-elasticsearch@eab5deb113
In order to support the source directory repo split, this commit
disables security for the regular integration tests.
The MonitoringSettingsFilterTests already existed as REST test, so
this test has been removed.
Relates elastic/x-pack-elasticsearch#2925
Original commit: elastic/x-pack-elasticsearch@519154dd5f
The pagerduty action allows to send contexts, which contains an array
of texts or images, each with a link.
The field of this data was named 'context' instead of 'contexts' and
thus those contects were never correctly parsed on the pagerduty side.
Unfortunately pagerduty accepts any JSON, thus this was not caught so
far.
This commit allows parsing of the old field name to retain BWC, but when
written out via toXContent, it will always use the 'contexts' field name.
relates elastic/x-pack-elasticsearch#3184
Original commit: elastic/x-pack-elasticsearch@50f0b65d56
This commit fixes the minimum value being smaller than the maximum value in a call to the
scaledRandomIntBetween, which causes the test to fail.
Original commit: elastic/x-pack-elasticsearch@da7d0ce0ce
When you click "delete" in the UI it force-deletes the datafeed then
force-deletes the job. For a datafeed doing lookback, this results
in a close followed very quickly by a kill on the autodetect process.
Depending on thread scheduling this could cause a lot of spurious
errors and exception traces to be logged.
This change prevents the log spam in this scenario.
relates elastic/x-pack-elasticsearch#3149
Original commit: elastic/x-pack-elasticsearch@091240f32a
The IndexLifecycleManagerIntegTests has a method that tests concurrency and executes a large number
of requests in parallel. On some machines, this can actually overwhelm a thread pool and cause the
test to fail for the wrong reasons. This commit bounds the total number of requests to 100.
Original commit: elastic/x-pack-elasticsearch@07613ada51
Created a smoke-test-monitoring-with-watcher project that runs REST
tests with watcher enabled to ensure that the proper watcher are
installed either when the local or the HTTP exporter are set up.
Also removed two more watcher imports in the tests.
Relates elastic/x-pack-elasticsearch#2925
Original commit: elastic/x-pack-elasticsearch@0a9abc3185
This was due to a hardcoded job ID that wouldn't exist when two random
numbers generated by the test framework were both small.
Original commit: elastic/x-pack-elasticsearch@9529e6d280
* [Monitoring] Update Beats Template with Metrics
- remove source_node.timestamp
- includes mapping for beat type = Metricbeat
* remove metricbeat and xpack objects which are not used in the UI
* use long instead of integer type
Original commit: elastic/x-pack-elasticsearch@083b9cc575
SQL: Introduce PreAnalyze phase to resolve catalogs async
The new preanalyze phase collects all unresolved relations and tries
to resolve them as indices through typical async calls _before_ starting the analysis process.
The result is loaded into a catalog which is then passed to the analyzer.
While at it, the analyzer was made singleton and state across the engine
is done through SqlSession#currentContext().
Commit missing fix
Fix typo
Fix license
Fix line length
remove redundant static modifier
Remove redundant generics type
Rename catalogResolver instance member to indexResolver
Fix translate action to return a response through the listener, it hangs otherwise
IndexResolver improvements
Make sure that get index requests calls are locally executed by providing local flag.
Don't replace index/alias name with concrete index name in asCatalog response conversion. We need to preserve the original alias name for security, so it is reused in the subsequent search.
Update roles and actions names for security tests
Get index is now executed instead of sql get indices, and sql get indices has been removed.
Also made cluster privileges more restrictive to make sure that cluster state calls are no longer executed.
Fix most of the security IT tests
indices options are now unified, always lenient. The only situation where we get authorization exception back is when the user is not authorized for the sql action (besides for which indices).
Improve SessionContext handling
Fix context being invalid in non-executable phases
Make Explain & Debug command fully async
Resolve checkstyle error about redundant modifiers
Temporarily restore SqlGetIndicesAction
SqlGetIndicesAction action is still needed in RestSqlJdbcAction (metaTable and metaColumn methods), where we can't at the moment call IndexResolver directly, as security (FLS) needs index resolver to be called as part of the execution of an indices action. Once mappings are returned filtered, delayed action and the security filter will go away, as well as SqlGetIndicesAction.
SqlGetIndicesAction doesn't need to be a delayed action, my bad
[TEST] remove unused expectSqlWithAsyncLookup and rename expectSqlWithSyncLookup to expectSqlCompositeAction
Polish and feedback
Add unit test for PreAnalyzer
Original commit: elastic/x-pack-elasticsearch@57846ed613
This change adds some debug and trace logging when we look up role names, to explain how each role was resolved.
At the moment we have very little insight into how roles are being resolved which can make it difficult to diagnose some issues.
Original commit: elastic/x-pack-elasticsearch@1b3c246186
Register LoggingAuditTrail as a ClusterStateListener.
Avoid querying for the localNode while on the ClusterStateApplier thread,
which tripps assertion. This can happen when logging audit events that
originate from the system.
relates elastic/x-pack-elasticsearch#3057
Original commit: elastic/x-pack-elasticsearch@66bc59682d
If the KeyStoreWrapper is closed, then we cannot validate secure settings (because we no longer have access to them)
The Realm group setting uses the "validate" method to ensure that child settings are correct, but it must ignore secure settings as it might get called
after startup (e.g. during a settings diff)
Original commit: elastic/x-pack-elasticsearch@b30db6bc62
This commit uses SPI to get the list of system wide reserved roles in
security. It does not yet split out the RoleDescriptor to a common
location so the implementing services still depend on security. Each
role, however, only depends on its own feature as well as security.
ref elastic/x-pack-elasticsearch#2925
Original commit: elastic/x-pack-elasticsearch@efebc3e5c8
* Use XPackRestIT as base class for XDocsClientYamlTestSuiteIT
* Remove the XPackRestTestCase class
* Address review comments
* Fix checkstyle checks
Original commit: elastic/x-pack-elasticsearch@c2a5e60c12
This change fixes a problem that would cause job deletion to fail if ANY
index had a block on it, e.g. read-only.
The problem was that we were requesting the job aliases be deleted from
ALL indices in the system due to a misunderstanding with the format of the
get_aliases response. This didn't usually cause any noticable effects, as
only the ML indices would have the aliases. But in the case of a read-only
index it would cause an error, leading to unnecessary failure of the job
deletion.
Fixeselastic/machine-learning-cpp#428
Original commit: elastic/x-pack-elasticsearch@a573f85a00
Following the changes of elastic/x-pack-elasticsearch#2975 the hard limit on the number of ML jobs
per node is no longer the only limiting factor. Additionally there is
now a limit based on the estimated memory usage of the jobs, and this is
expected to provide a more sensible limit that accounts for differing
resource requirements per job.
As a result, it makes sense to raise the default for the hard limit on
the number of jobs, on the assumption that the memory limit will prevent
the node becoming overloaded if an attempt is made to run many large jobs.
Increasing the hard limit will allow more small jobs to be run than was
previously the case by default.
Of course, this change to the default will have no effect for customers
who have already overridden the default in their config files.
Original commit: elastic/x-pack-elasticsearch@9fed1d1237
The issue here is that if the first search request fails
(initScroll), then we do not have a scroll_id. However,
in order to retry the search, we reset the scroll. That
involves clearing the current search, but since we do
not have a scroll_id, the clear scroll request fails.
We end up reporting the failure for the scroll clearing,
rather than the actual problem.
This commit fixes that by avoiding clearing the scroll
when the scroll_id is null.
relates elastic/x-pack-elasticsearch#3146
Original commit: elastic/x-pack-elasticsearch@b5086028f6
This commit removes the KnownActionsTests as it no longer serves the intended purpose of catching
actions/handlers added to elasticsearch that security needs to be aware of. Today, it is common
for this test to break the build and as a mechanical response developers just add to the actions
or handlers file to get the build green again.
Relates elastic/x-pack-elasticsearch#1489
Original commit: elastic/x-pack-elasticsearch@0bdb5341f5
In order to support the repository split, this changes the
`AbstractWatcherIntegrationTestCase` to not run with security enabled.
We have a dedicated QA project called `smoke-test-watcher-with-security`,
where tests that explicitely need security should be running.
This commit removes the possibility to enable security as part of the
test case. In addition some tests have been moved over to the dedicated
project.
In addition the `timewarp` functionality cannot be configured with a
system property anymore. This would not have worked anyway, because
tests were already dependent on that functionality and did not have any
other means of running. A bit of redundant code was removed due to this.
Relates elastic/x-pack-elasticsearch#2925
Original commit: elastic/x-pack-elasticsearch@b24b365ad1
This change applies the same pattern that was applied in elastic/x-pack-elasticsearch#3054 to the
ML batched results iterators, which are used to scroll through ML results
during some internal ML implementation details, such as renormalization
and nightly maintenance.
Additionally the thread context is reset before submitting the results
processor to a thread pool, to avoid masking the problem in situations
where the user opening the job coincidentally had workable permissions.
Fixeselastic/machine-learning-cpp#438
Original commit: elastic/x-pack-elasticsearch@bd1e2dc7d4
The /_sql endpoint now returns the results in the text format by default. Structured formats are also supported using the format parameter or accept header similar to _cat endpoints.
Original commit: elastic/x-pack-elasticsearch@4353793b83
The chained input in watcher is a useful feature to
call several endpoints before execution a condition.
However it was pretty hard to modify data from a previous
input in order to be able to execute it in another input.
This commit adds a another input, called a `transform` input,
which allows you to do a transform as another input in a chained
input.
See this example
```
"input" : {
"chain" : {
"inputs" : [ <1>
{
"first" : {
"simple" : { "path" : "/_search" }
}
},
{
"second" : {
"transform" : {
"script" : "return [ 'path' : 'ctx.payload.first.path' + '/' ]"
}
}
},
{
"third" : {
"http" : {
"request" : {
"host" : "localhost",
"port" : 9200,
"path" : "{{ctx.payload.second.path}}" <2>
}
}
}
}
]
}
}
```
This allows for far more flexibility before executing the next input in a chained
one.
Original commit: elastic/x-pack-elasticsearch@3af9ba6e9b
We have to ensure logging is configured for any CLI command that depends
on core Elasticsearch (since it might directly or indirectly touch
logging). This commit does this for all commands in X-Pack.
Relates elastic/x-pack-elasticsearch#3112
Original commit: elastic/x-pack-elasticsearch@f77f9b5052
In order to be able to execute a watch as the user, who stored the
watch, this commit stores certain headers of the thread context, that
was used when the watch was stored.
Upon loading the watch the headers are loaded and applied for the
following watcher execution features
* search transform
* search input
* index action
A special case is the execute watch API, which overrides the headers loaded
from the watch with the one of the current request, so that a user
cannot execute this watch with other privileges of the user who stored it.
Only the headers "es-security-runas-user", "_xpack_security_authentication" are
copied for now, as those are needed for our security features.
The headers are stored in watch status in the watch and are not returned by default,
when the GET Watch API is used. A search reveals those of course.
relates elastic/x-pack-elasticsearch#2201
Original commit: elastic/x-pack-elasticsearch@9803bd51c2
Fixes to the build system, particularly around BWC testing, and to make future
version bumps less painful.
Original commit: elastic/x-pack-elasticsearch@a1d456f30a
Instead of returning "error response" objects and then translating them
into SQL exceptions this just throws the SQL exceptions directly. This
means the CLI catches exceptions and prints out the messages which isn't
ideal if this were hot code but it isn't and this is a much simpler way
of doing things.
Original commit: elastic/x-pack-elasticsearch@08431d3941
This change removes the InternalClient and the InternalSecurityClient. These are replaced with
usage of the ThreadContext and a transient value, `action.origin`, to indicate which component the
request came from. The security code has been updated to look for this value and ensure the
request is executed as the proper user. This work comes from elastic/x-pack-elasticsearch#2808 where @s1monw suggested
that we do this.
While working on this, I came across index template registries and rather than updating them to use
the new method, I replaced the ML one with the template upgrade framework so that we could
remove this template registry. The watcher template registry is still needed as the template must be
updated for rolling upgrades to work (see elastic/x-pack-elasticsearch#2950).
Original commit: elastic/x-pack-elasticsearch@7dbf2f263e
Room names in hipchat were not properly URL encoded, thus room names
with spaces would not work as expected. This fixes all the hipchat
accounts by properly using spaces.
Also the hipchat tests are reenabled, as the IT team gave me new access to hipchat,
allowing to create a fresh set of oauth tokens for the integration account type.
The HipchatServiceTests have also been converted to XPackSingleNodeTestCase
relates elastic/x-pack-elasticsearch#2371
relates elastic/x-pack-elasticsearch#2429
Original commit: elastic/x-pack-elasticsearch@9f8872f686
This commit changes the IndexLifecycleManager's handling of variables about an index to only update
all of the values at a single time. Previously, all of the index state variables were volatile
members of the IndexLifecycleManager, which meant we could get an inconsistent view of the index
state. Although rare, this is still incorrect so this change adds a single volatile variable that
holds the state as of the last processed cluster state update.
Additionally, the IndexLifecycleManagerIntegTests were updated to have more concurrency and further
stress this portion of the code and its checks.
relates elastic/x-pack-elasticsearch#2973
Original commit: elastic/x-pack-elasticsearch@5f1552b298
This commit adds the manage_index_templates permission to the kibana_system role that is used by
the kibana system user. This is needed due to an upcoming feature in kibana where a index template
will be used to create the saved objects index.
relates elastic/x-pack-elasticsearch#2937
Original commit: elastic/x-pack-elasticsearch@85a67c73aa
Some of our REST tests open many jobs, and assuming each will use 1GB of
RAM on a single node cluster could fail the test. The solution is to
explicitly say the test jobs will use very little RAM.
Original commit: elastic/x-pack-elasticsearch@a3fcfc4589
Currently, any errors that occur during the DeleteExpiredDataAction are logged and the deletion proceeds to the next job. The user will get no indication in the response that something went wrong although nothing should really go wrong unless the cluster is messed up.
This commit changes this so that errors are reported back to the action.
Original commit: elastic/x-pack-elasticsearch@489cf03c3e
This change modifies the way ML jobs are assigned to nodes to primarily
base the decision on the estimated memory footprint of the jobs. The
memory footprint comes from the model size stats if the job has been
running long enough, otherwise from the model memory limit. In addition,
an allowance for the program code and stack is added.
If insufficient information is available to base the allocation decision on
memory requirements then the decision falls back to using simple job
counts per node.
relates elastic/x-pack-elasticsearch#546
Original commit: elastic/x-pack-elasticsearch@b276aedf2f
tool. However, this is a forbidden API so this commit replaces it with URI#getPath. Additionally,
the tests fail with a security manager permission error due to the use of Mockito for exception
throwing. This commit still uses Mockito for throwing exceptions but does it differently in a way
that is acceptable by our test security policy.
Original commit: elastic/x-pack-elasticsearch@5e1d45acf8
Fixes bug when the url option had trailing slashes. The URL built
was invalid (consecutive fwd slashes) but the failure errors of
the subsequent requests were ignored.
URL is built correctly from the option spec.
True HTTP errors and Exceptions are logged and the cmd fails.
relates elastic/x-pack-elasticsearch#2778
Original commit: elastic/x-pack-elasticsearch@62b2d94ca0
Changes to further prepare for feature split with regards to watcher:
- CryptoService has been moved into watcher
- CryptoService.generateKey() has been moved into SystemKeyTools, only
used there
- The creation of the http client/notification classes have been moved
into watcher, no further dependencies on watcher in XPackPlugin
- Each subproject now registers it's own named writeables
Relates elastic/x-pack-elasticsearch#2925
Original commit: elastic/x-pack-elasticsearch@a60c98ba7e
This problem was introduced due to distributed watch execution.
When a node newer than the master node joins the cluster and gets a
.watches shard assigned it is supposed to start watcher. However
when a new version of the watch history template is part of that new
node (and we might increase that version anytime), this template does
not get installed, because only the master node is updating watcher
templates.
This commit checks if the local node version is higher than the master
node version and then also puts missing templates.
Currently this is done for all watcher templates, not only the watcher history.
relates elastic/x-pack-elasticsearch#2944
Original commit: elastic/x-pack-elasticsearch@4960231ea7
* Add "client-api-objects" dependency for xpack plugin and transport-client
This adds another gradle project, "client-api-objects" which is intended to be a
common dependency so that the xpack plugin and transport-client can share the
same Request and Response objects.
Relates to elastic/x-pack-elasticsearch#2925
Original commit: elastic/x-pack-elasticsearch@a6d83299d0
This is related to elastic/elasticsearch#elastic/x-pack-elasticsearch#27388. It modifies x-pack to
be compatible with the removal of the delegating transport channel.
Original commit: elastic/x-pack-elasticsearch@3bd7bf6773
When there were no accounts configured, watcher returned a cryptic
error message containing 'null' in the description. This fix returns
a more clear error message.
On top a dedicated NotificationServiceTests class was added to remove
redundant test cases in the hipchat/jira/slack unit tests, that all
basically tested NotificationService capabilties.
relates elastic/x-pack-elasticsearch#2666
Original commit: elastic/x-pack-elasticsearch@45d0d1df31
* Rename REST spec xpack.deprecation.info to xpack.migration.deprecations
* Fixed parameter-type naming in xpack.ml.get_model_snapshots
* Fixed QS multi-cluster search test to use cluster.remote_info
Original commit: elastic/x-pack-elasticsearch@ccd35b4a6c
Today persistent tasks is only usable from machine learning, but others like ccr will need to use it too.
With this change ccr and other will be able to make use of it too.
Original commit: elastic/x-pack-elasticsearch@c90f01d5f6
This commit adds checks to the TribeWithSecurityIT tests to ensure that the security index is
writeable before making modification operations. Otherwise, we hit errors in tests that are not
always reproducible.
relates elastic/x-pack-elasticsearch#2977
Original commit: elastic/x-pack-elasticsearch@c29bdff7ae
In elastic/x-pack-elasticsearch#2901, the dependency on the tribe module was removed but a few leftover references were missed
in the build.gradle file of the x-pack-elasticsearch plugin. This commit removes these leftover
references.
Original commit: elastic/x-pack-elasticsearch@03f1cae1f5
In order to prepare for separate source directories, this commit moves
a few packages back into the watcher namespaces. A few of them have been
moved out previously as we thought that it might make sense to have a
dedicated notification API. This wont be the case for watcher on ES
anymore, so we can safely move those back into the watcher space.
Packages affected by this move:
* org.elasticsearch.xpack.common.http
* org.elasticsearch.xpack.common.text
* org.elasticsearch.xpack.common.secret
* org.elasticsearch.xpack.common.stats
* org.elasticsearch.xpack.support
* org.elasticsearch.xpack.notification
Tests have been moved accordingly.
The class `XContentUtils` has been split into one implementation for
watcher and one for security as different methods were used.
Relates elastic/x-pack-elasticsearch#2925
Original commit: elastic/x-pack-elasticsearch@0aec64a7e2