Note currently this just installs the plugin. we need to add some
more logic to really get some kind of integ test working with rest tests.
e.g. we need to install the license plugin. I will look into this as a followup.
Original commit: elastic/x-pack-elasticsearch@ebc507a1fb
- Removed `AuthorizationException` and `AuthenticationException`, both were replaced by the std `ElasticsearchSecurityException`.
- Added utility class `Exceptions` to help create authorization/authentication exceptions.
- Added a utility class `ShieldTestsUtils` that provides methods for asserting authentication/authorization exceptions
Original commit: elastic/x-pack-elasticsearch@b59360ebc7
This exception was removed from the license plugin. We use `ElasticsearchException` instead with `es.license.expired.feature` header to hold the expired feature. We use `LicenseUtils.newExpirationException` to create and throw the proper exception.
Original commit: elastic/x-pack-elasticsearch@5e722be1cc
Attaching the zip assembly as an artifact to the build will include it when deploying to maven repositories. This will in turn facilitate testing Elasticsearch snapshot versions in a Found environment. We need to to that in order to cut down the delay from a release is made until it is available on Found.
Original commit: elastic/x-pack-elasticsearch@27516c7243
Currently, any plugin that is on the classpath will be loaded by the TransportClient we create
for remote audit indexing. The only plugin that should be loaded for this client is the Shield
plugin. To accomplish this, we disable loading of plugins from the classpath and specify that
the Shield plugin should be loaded.
Closeselastic/elasticsearch#941
Original commit: elastic/x-pack-elasticsearch@2bc0a8ce17
This changes the mappings for the audit indices to use doc_values for all fields
other than the request_body, which will have a lot of variance. Additionally, the
request_body field is no longer indexed.
Closeselastic/elasticsearch#918
Original commit: elastic/x-pack-elasticsearch@4917529ffa
The index audit trail is currently using a BulkProcessor directly, which under
certain conditions can result in a deadlock. This occurs when the BulkProcessor
is executing a bulk request that triggers another request on the same node and
a flush of the BulkProcessor is also triggered at the same time. The flush
operation holds the lock on the bulk processor but block on acquiring a permit
from the semaphore. The request that was triggered by the bulk request blocks
the release of the semaphore permit since it needs to add a new audit message
to the BulkProcessor.
This commit works around this issue by making use of a bounded queue between the
index audit trail and the BulkProcessor with a consumer thread that handles the
add calls to the BulkProcessor.
Additionally, a new state, INITIALIZED, was added for the lifecycle of the index
audit trail. This is needed for tests since the audit trail can stop, a new
cluster state update is received, and the ShieldLifecycleService will restart the
index audit trail. At the end of the tests, the test infrastructure interrupts all
the threads and this was causing tests to fail with a InterruptedException.
Finally, the test infrastructure was also deleting the template for the index audit
logs, so this commit adds the necessary logic to prevent the deletion of this
template.
Closeselastic/elasticsearch#920
Original commit: elastic/x-pack-elasticsearch@f1b0b47b99
The change fixes two bugs in the index audit trail implementation. The first is that
we did not always store the origin type with rest requests. The second is that a
conditional statement controlled the storage of the rest requests content, but the
conditional was based on a log level that had nothing to do with the index based
audit implementation.
Closeselastic/elasticsearch#932
Original commit: elastic/x-pack-elasticsearch@b309e261c3
we're not using the _timestamp field and the path option is no longer supported
in elasticsearch 2.0 so this commit removes the field from the mapping.
Original commit: elastic/x-pack-elasticsearch@399d835d1f
- lowercase `beta` and `rc`
- replaced `.betaXXX` and `.rcXXXX` suffix with `-betaXXX` and `-rcXXX`
Original commit: elastic/x-pack-elasticsearch@843d01c647
This change renames the shield audit indices to keep naming consistent with other plugins.
The name of the index uses '_' to separate words, a '-' to separate the prefix from the time
portion, and '.'s to separate the different portions of the date.
Closeselastic/elasticsearch#925
Original commit: elastic/x-pack-elasticsearch@8ca6856e4a
Previously, we were just using the current time in milliseconds from the system
for dates and the indices were not being created for UTC dates. This change
uses UTC dates for timestamps and indices resolution for index auditing.
This also ensures that custom shield forbidden apis for tests are enforced.
Closeselastic/elasticsearch#916
Original commit: elastic/x-pack-elasticsearch@724d12cb7a
We currently store the names of indices as a comma separated string instead
of an array. An array is the proper format for this information so this commit
changes the index audit trail to store the indices as an array.
Closeselastic/elasticsearch#917
Original commit: elastic/x-pack-elasticsearch@025393d91c
The beforeIndexDeletion method expects that nothing is still indexing when it runs
but this is not the case as the index audit trail will continue indexing events that
occur as checks are being performed in the cluster.
Original commit: elastic/x-pack-elasticsearch@02001a5222
Previously we relied on the default dynamic mapping for the audit index output, which did
not create an ideal mapping. This change adds a mapping file with default settings and
dynamic mapping disabled for the audit indexes.
Additionally, the ability to override settings for the audit indexes has been provided so that
users can customize the number of shards and replicas to meet their needs.
In order to implement these changes, the index audit service had to be moved from a lifecycle
component to an abstract component that had its own custom lifecycle on top of the cluster
state. A ShieldLifecycleService class was added to accomplish this. In the future, this class
can be used for other services that need to perform index based operations.
Closeselastic/elasticsearch#913
Original commit: elastic/x-pack-elasticsearch@231740c1cc
If auditing is not enabled, we should not bind the index audit trail as a lifecycle component
since we do not bind all of its dependent classes when auditing is disabled.
Original commit: elastic/x-pack-elasticsearch@39f736ca6a
This changes how the user defines which events should be indexed. Previously, there were
several boolean settings being used. This condenses to an include and exclude setting.
Additionally, the IndexAuditTrail needed to become a lifecycle component since parsing the
enum could throw exceptions. Given this, the IndexBulkProcessor was condensed into the
IndexAuditTrail since it did not make sense to have two lifecycle components for an index
audit trail.
Closeselastic/elasticsearch#900
Original commit: elastic/x-pack-elasticsearch@4b4d824f5e
Today, a realm could throw an exception, which will not be caught and cause the audit
trail to be skipped. This commit catches exceptions thrown by realms and ensures that
the audit trail is called and then re-throws the exceptions.
Closeselastic/elasticsearch#901
Original commit: elastic/x-pack-elasticsearch@d260c00b33
Today, some system actions could be logged by default when the actions performed are
not internal. Additionally for internal actions, we never checked if the user was the system
user.
This adds a check to ensure the user is the System user and that the actions that are being
suppressed are known system actions.
Closeselastic/elasticsearch#902
Original commit: elastic/x-pack-elasticsearch@b107994692
