This PR changes how we use roles and how we look at the roles of a user. Previously we looked up each role individually, parsed each into their own `Role` object, and had a wrapper that essentially served as an iterator over the roles. The same pattern was also followed for the permissions that composed a role (ClusterPermission, IndicesPermission, and RunAsPermission). This resulted in a lot of code that was hard to follow and could be inefficient.
Now, we look up the roles for a user in bulk and only get the RoleDescriptor for each role. Once all role descriptors have been retrieved, we build a single Role that represents the user's permissions and we also cache this combination for better performance as authorization can happen many times for a single top level request as we authorize the top level request and any sub requests, which could be a large number in the case of shard requests.
This change also enabled a large cleanup of our permission and privilege classes, which should reduce the footprint of what needs to be followed. Some of the notable changes are:
* Consolidation of GeneralPrivilege and AbstractAutomatonPrivilege into the Privilege class
* The DefaultRole class has been removed and the permissions it provided were moved into the AuthorizationService
* The GlobalPermission class was removed as there is a single role that represents a user's permissions
* The Global inner classes for the various permissions were removed
* The Core inner class was removed and ClusterPermission, IndexPermission, RunAsPermission became final classes instead of interfaces
* The Permission interface has been removed. The isEmpty() method defined by this interface is not needed as we can simply evaluate the permission to get the same effect
* The ClusterPermission#check method only takes the action name again
* The AutomatonPredicate class was removed and replaced by Automatons#predicate
* IndicesAccessControl objects no longer need to be merged when evaluating permissions
* MergedFieldPermissions has been removed
* The Name class that was used to hold an array of strings has been removed and replaced with the use of a Set
* Privilege resolution is more efficient by only combining automata once
Other items:
* NativeRolesStore no longer does caching, so the RoleAndVersion class could be removed
* FileRolesStore doesn't need to be an AbstractLifecycleComponent
Relates elastic/elasticsearch#4327
Original commit: elastic/x-pack-elasticsearch@c1901bc82e
When attempting to a get a watch that does not exist, the listener is called once inside
an if statement and the execution of the method continues as if the watch existed. This
causes failures to happen including a NPE. This commit wraps the execution in a if-else
to prevent this.
Original commit: elastic/x-pack-elasticsearch@27f09852e7
We are trying to get rid of guava, so it doesn't make much sense to use it as a dependency unless we really need to.
Original commit: elastic/x-pack-elasticsearch@e2e2d58fba
Switches custom cluster state components from PROTO-based de-serialization to named objects based de-serialization
Original commit: elastic/x-pack-elasticsearch@e1be8597ee
This commit adapts to the change of the method name
ESTestCase#randomPostivieLong to ESTestCase#randomNonNegativeLong.
Original commit: elastic/x-pack-elasticsearch@689429cb54
This "super registry" will eventually replace things like
`IndiciesQueriesRegistry` but for now it is just another thing
to plumb across requests.
Original commit: elastic/x-pack-elasticsearch@da26a42b36
When the index action is used to do some bulk indexing, the single
items of the response were not checked to have been indexed successful.
This could lead to NPEs due to an index response being null when the index
operation had failed. The action was still logged as a success though.
This commit only returns SUCCESS for the action, if all items were indexed
successfully. If all items failed, the result will be FAILED as well. Lastly
the result status PARTIAL_FAILURE is used if there were successful and unsuccessful
index operations.
Additionally some minor cleanups happened, like changing equals/hashcode.
Closeselastic/elasticsearch#4416
Original commit: elastic/x-pack-elasticsearch@692687e1af
Removing the WatchLockService could result in duplication of wids, because of a wrong
call to replace underscores with dashes. As UUIDs.createBase64UUID() can contain underscores
but they are kind of reserved in the Wid class due to handling of watch ids, this just uses
the toString() representation of a random UUID.
Closeselastic/elasticsearch#4422
Original commit: elastic/x-pack-elasticsearch@dceb01ae5e
This test fails due to a changing cluster state(?). The
test checks that a local exporter is ready and then continues.
However, during the test, we see output similar to:
skipping exporter [_local] as it isn't ready yet
Which indicates that the cluster state has changed and the
exporter does not return a bulk anymore. Hence, the test is
failing although at one point in time it returned a bulk.
By enabling trace logging we should be able to find out
what's going on.
Original commit: elastic/x-pack-elasticsearch@d7e2200dd9
The watch lock service is not really needed, as there is already
a data structure that has information about the currently executing
watches, that can be consulted before executed.
This change will now check, if there is already a watch running with
the current id. If there is not, execution will happen as usual. If
there is however, than a watch record will be created, stating that
the watch is currently being executed - which means that it is either
being executed or in the list of planned executions.
This way users can check in the watch history, if a watch has been executed
more often than it should.
In order to easily search for this, a new execution state called
`NOT_EXECUTED_ALREADY_QUEUED` has been added.
Original commit: elastic/x-pack-elasticsearch@867acec3c3
In LoggingAuditTrail and the audit-event filtering, we distinguish between single-realm authentication failures ("realm_authentication_failure") and global failures ("authentication_failure").
Update the message output of IndexAuditTrail to reflect this distinction.
Original commit: elastic/x-pack-elasticsearch@56802ae2df
With this commit we enable the Jackson feature 'STRICT_DUPLICATE_DETECTION'
by default for all XContent types (not only JSON).
We have also changed the name of the system property to disable this feature
from `es.json.strict_duplicate_detection` to the now more appropriate name
`es.xcontent.strict_duplicate_detection`.
Relates elastic/elasticsearchelastic/elasticsearch#19614
Relates elastic/elasticsearchelastic/elasticsearch#22073
Original commit: elastic/x-pack-elasticsearch@c92e2a6c55
* Build: Convert xplugins to use new extra projects setup
This change makes the gradle initialization for xplugins look in the
correct location for elasticsearch, which is now as a sibling of an
elasticsearch-extra directory, with x-plugins as a child of the extra
directory.
The elasticsearch side of this change is
elastic/elasticsearchelastic/elasticsearch#21773. This change will enable renaming x-plugins
to x-pack, see elastic/elasticsearch#3643.
Original commit: elastic/x-pack-elasticsearch@09398aea5a
The new MockWebServer assumed that responses are inserted at the beginning
and removed later on. This was not thread safe.
Also this fixes a bug in the HttpExporterIT where there was no wait time
for a bulk request, even though the request execution is asynchronous.
Closeselastic/elasticsearch#4335
Original commit: elastic/x-pack-elasticsearch@11f31f68bd
In order to prepare to the distributed watch execution, this commit
removes the in memory watch store.
Whenever a watch is needed now, a get request is executed and the parsing
is done. This happens when
* Put
* Get
* Ack
* Activate/Deactivate
* Execute
Note: This also means there are no usage stats currently regarding
the watch count, because we would need to execute a query. This would
require the usage stats to be async, see elastic/elasticsearch#3569
Another advantage is, that there is no dirty flag in the watch itself
needed anymore, because the watch is always the latest. Also write
operations store immediately and dont leave anything in memory.
Also ActionListener.wrap() was used a lot instead of more verbose anonmyous
inner classes.
Original commit: elastic/x-pack-elasticsearch@c47465b47c
Modified the definition and loading of settings in Security to provide early detection and failure of invalid (unrecognised or syntactically invalid) settings.
Also consolidates the number of places where settings were defined.
Each realm now defines its supported settings.
This is facilitated for custom realms via a new "getRealmSettings" method on XPackExtension.
The realm group setting performs validation of the child settings with reference to the "type".
For backwards compatibility reasons, realm types that have no configuration defined, will be accepted during validation, but may fail at realm creation time.
All SSL settings have been centralised into SSLConfigurationSettings, which supports a variable "prefix" to accommodate the multiple places we define SSL config.
HTTP Proxy settings are explicitly defined rather than being a generic group.
Where possible all security settings now reference a Setting object, and there are less magic strings scattered in the code.
Closes: elastic/elasticsearch#3965
Original commit: elastic/x-pack-elasticsearch@2c76a137a9
In the triggered watch store a list of integers was returned to keep track
which watches had been successfully stored and thus should be executed.
This means, that an arraylist, plus autoboxing/unboxing needs to be done for
all the triggered watches. This data structure can easily be replaced with a BitSet,
resulting in much less objects being created or parsed - also it's a bit faster.
Original commit: elastic/x-pack-elasticsearch@e9fba67e34
This change removes the deprecated methods from the realm class. These methods include
blocking authentication and lookup and the lookup supported method.
Original commit: elastic/x-pack-elasticsearch@cff21e21ee
With this commit we enable the Jackson feature 'STRICT_DUPLICATE_DETECTION'
by default. This ensures that JSON keys are always unique. While this has
a performance impact, benchmarking has indicated that the typical drop in
indexing throughput is around 1 - 2%.
As a last resort, we allow users to still disable strict duplicate checks
by setting `-Des.json.strict_duplicate_detection=false` which is
intentionally undocumented.
Relates elastic/elasticsearchelastic/elasticsearch#19614
Original commit: elastic/x-pack-elasticsearch@cced57b884
The latest release of the mock web server requires more security permissions, and
we dont need all the functionality anyway.
This introduces a small MockWebServer using the JDK internal HttpServer, yet fullfilling
all our needs and supporting SSL as well for testing.
The MockWebServer allows to enqueue responses and also requires you to enqueue as many responses
as you requests will be executed - there is no fallback at the moment.
SSL is also supported by passing an SSL context - for which the TestsSSLService is needed, which
makes the required methods public.
Original commit: elastic/x-pack-elasticsearch@55f4a172a2
Jay Modi
jaymode
Adrien Grand
Igor Motov
Tim Vernum
Yannick Welsch
Jason Tedor
Nik Everett
Alexander Reelsen
Daniel Mitterdorfer
Ryan Ernst
javanna
Daniel Mitterdorfer
Simon Willnauer
Areek Zillur
Boaz Leskes
Luca Cavanna