Eagerly authorizing CompositeIndicesRequests allowed the security plugin to fail fast up until now, but it makes it very hard to reason about each specific item in a multi items request. Either all items fail, or none do. We would rather want to adopt a similar behaviour to es core, where individual items fail without affecting other items that are part of the same request. We can rely on the fact that es core always authorizes both main action and every subaction too, and skip authorization for the main action. By subaction we mean either all sub search requests in msearch, as well as each shard level get in mget or shard level bulk request for bulk.
BulkRequestInterceptor was converted to intercept BulkShardRequests rather than BulkRequest as that is where bulk is authorized after this change.
Split IndicesAndAliasesResolverIntegrationTests into ReadActionsTests and WriteActionsTests as they require different set of permissions, lots of tests added.
Explicitly listing the composite actions makes sure that the actions that can bypass security are known, somebody adding a similar action must to add it to the list, so we know it doesn't happen by mistake. At this point the CompositeIndicesRequest can be used as a marker interface only (it is not really needed but can be used to verify that composite actions use a request that implements such interface).
Given that we don't authorize composite actions based on their indices anymore, but only their sub-requests which implement IndicesRequest, printing out the indices names in the audit log for requests like bulk and msearch is confusing. Removed support for that.
Authorize composite indices actions based on their name only, their indices will be authorized at the sub-request/shard level
Rather than simply granting bulk, mget, msearch etc. and relying on authorization at the sub-request/shard level, we check that the current user can at least execute the action. This justifies the grant line that gets written in the audit log, the action is potentially possible without looking at the indices. Each specific item will fail or succeed later and will yield its own specific audit log entry.
Original commit: elastic/x-pack-elasticsearch@4570caf019
Like es core does in TransportIndicesExistsAction, we should only consider expandWildcardsOpen and expandWildcardsClosed out of the indices options passed in with IndicesExistsRequest. ignore_unavailable and allow_no_indices should always be considered both true, to prevent the request from throwing exception as it is supposed to return true or false, no exceptions.
Original commit: elastic/x-pack-elasticsearch@daa274b3fd
Supporting allowNoIndices means that the security plugin has a behaviour much more similar to vanilla es when dealing with wildcard expressions that match no indices, or empty clusters. The default for most request is to allow no indices, but security plugin could only disallow no indices all the time up until now.
The technical problem was that when anything gets resolved to an empty set of indices, we couldn't let that go through to es core, as that would become resolved to all indices by es core, which would be a security hole. We have now found a way though to replace an empty set of indices with something that es core will for sure resolve to no indices, so we can let the request through. We simply replace empty indices with '-*'.
Multi apis requests (e.g. _msearch) have yet to be fixed, as all their indices end up in the same bucket while they should each be authorized separately, so that every specific item can fail or be let through.
Original commit: elastic/x-pack-elasticsearch@0f67a0bfea
For all the requests that support multiple indices and wildcards, hence implementing IndicesRequest.Replaceable, we replace the wildcard expressions with the explicit names of the authorized indices they match. _all or empty indices is treated as a wildcard expression. We can also honour the ignore_unavailable option by going over all the explicit names and filter out the non authorized ones when ignore_unavailable is set to true. If ignore_unavailable is set to false, we leave everything as-is, which will cause an authorization exception to be thrown if only one of those explicit indices is not authorized for the current user.
This is the first step towards resolving elastic/elasticsearch#1250. The remaining issue is that in case we are left with no indices after stripping out the ones that the user is not authorized for, we throw an authorization exception rather than returning an empty response. That will require honouring the allow_no_indices option, which will also change the behaviour when a cluster is empty.
Relates to elastic/elasticsearch#1250
Original commit: elastic/x-pack-elasticsearch@e4ca940d05
The superuser role is the only user assignable role that grants access to the .security index, but when
resolving wildcards the index was not getting resolved. The resolution of indices and aliases explicitly
excludes the .security index for users that are not the internal user without checking if the user has the
superuser role. This commit adds a check in for the superuser role.
Original commit: elastic/x-pack-elasticsearch@02ee0a8740
The role cache was previously using the wrong time unit for its expire after write time; the
value passed to the cache was milliseconds instead of nanoseconds.
Original commit: elastic/x-pack-elasticsearch@65f7b08763
The anonymous role was being applied to other users for index access control but was not being applied
in terms of action level access control. This change makes the minimum required change to apply the
anonymous role for all users when anonymous is enabled. Additionally, some minor changes were made to the native roles store to not lookup roles before the service is started.
Closeselastic/elasticsearch#3711
Original commit: elastic/x-pack-elasticsearch@a9398e178d
When adding a watch which has a painless component, the scriptexception
was wrapped into a deprecated exception which means, that the awesome
painless descriptions were lost. This wrapping has been removed.
Closeselastic/elasticsearch#3161
Original commit: elastic/x-pack-elasticsearch@1703fe4eb6
[Monitoring UI] Default to hide system indices on shards view, with a checkbox to enable showing system indices
Original commit: elastic/x-pack-elasticsearch@27cb79ea6b
This test has been blacklisted and deactivated months ago. This commit reenables this test and moves it at the right place. It also change the test to use the Execute Watch API instead of being sleep based.
Original commit: elastic/x-pack-elasticsearch@e7a9689375
in core we wrap request handlers with an asserting one to ensure we can serialize messages
with different versions. Yet, xpack uses the same functionality to add security aspects to
the network layer. These tests assert that the right handlers are in-place.
Original commit: elastic/x-pack-elasticsearch@e39c8995ae
Fixes to x-plugins code now that DateMathParser accepts a LongSupplier rather than a Callable to get the value of now
Relates to elastic/elasticsearchelastic/elasticsearch#20796
Original commit: elastic/x-pack-elasticsearch@99fc47a8a7
This change moves to using SSLParameters as the configuration source for SSLEngine and SSLSocket
objects that are configured by the SSLService. Previously we used a mix of specific methods and
SSLParameters, which resulted in issues where ordering of calls is important. For example, if configuring
client authentication directly on the engine prior to setting the SSLParameters resulted in the client
authentication configuration being reset to the default.
Additionally, this change also sets use cipher suite order to true to ensure preferred ciphers will be used.
Original commit: elastic/x-pack-elasticsearch@8ddecdc20c
This change ensures we wait for a response before the async http client is closed. Otherwise we can
close the client during the connection to the remote endpoint or never even connect to the remote
endpoint.
Closeselastic/elasticsearch#3640
Original commit: elastic/x-pack-elasticsearch@54900b1b4a
This changes does two things in the tribe tests. The first is that when we split data up between
multiple clusters, we always force create the security index so that randomization does not cause
edge cases like the index not existing in the preferred cluster. The second is we look at the cluster
state of the nodes and ensure the tribe node sees the indices and has all primaries active.
Separate tests were also added to cover the scenario where the security index only exists in the non
preferred node.
Original commit: elastic/x-pack-elasticsearch@17b78ec837
This is the last action that needs additional support for proxies.
You can set a proxy in the JSON like this:
```
"actions" : {
"notify-pagerduty" : {
"pagerduty" : {
"description" : "Main system down, please check!",
"proxy" : { "host" : "localhost", "port" : 8080 }
}
}
}
```
Closeselastic/elasticsearch#3372
Original commit: elastic/x-pack-elasticsearch@b99969fd6b
You can set it like this in the JSON
"actions" : {
"notify-slack" : {
"slack" : {
"account" : "integration-account",
"proxy" : {
"host" : "localhost",
"port" : 8080
},
"message" : {
...
}
}
}
}
Relates elastic/elasticsearch#3372
Original commit: elastic/x-pack-elasticsearch@de86233d4f
Watcher uses a custom thread pool. This is because executing watches can
be long-running tasks that often block on I/O and it is best to not
consume the core thread pools with these tasks. Today this thread pool
is fixed, and sized at five times the bounded number of cores (so 160 on
a 32-core box). It makes sense for there to possibly be so many threads,
again because these tasks can block on I/O and having excess capacity
lets unblocked watches execute. It's the fixed size that can cause
problem, all these threads are always consuming resources even when
there are no or not that many watches running. This commit changes this
thread pool to be a scaling thread pool.
Relates elastic/elasticsearch#3660
Original commit: elastic/x-pack-elasticsearch@3cafab6e83
We need to special case IndicesAliasesRequest as it doesn't implement CompositeIndicesRequest anymore. Note that the similar loop for CompositeIndicesRequests's subrequests will soon go away
Relates to elastic/elasticsearch#3638
Original commit: elastic/x-pack-elasticsearch@50d119ff61
javanna
Jay Modi
Shaunak Kashyap
Areek Zillur
Tim Sullivan
CJ Cenizal
Tanguy Leroux
Yannick Welsch
Alexander Reelsen
Nik Everett
Joe Fleming
jaymode
Simon Willnauer
Simon Willnauer
Colin Goodheart-Smithe
Igor Motov
jaymode
Jason Tedor
Luca Cavanna
Ryan Ernst