For the RemoteIndexAuditTrailStartingTests we start a remote cluster and index into the test cluster and after
the test we stop the remote cluster but indexing operations can still be ongoing in the test cluster and thus cause
locked shards causing lots of noise in CI. We simply skip this check for these tests.
Closeselastic/elasticsearch#598
Original commit: elastic/x-pack-elasticsearch@e678f82f29
This change makes the environment optional for the ClientSSLService, which is used for
Transport Clients and also used for LDAP client connections. Since we use the ClientSSLService
for LDAP connections, we still need the environment when running as a node under the
security manager to resolve paths from the configuration directory.
Closeselastic/elasticsearch#573
Original commit: elastic/x-pack-elasticsearch@862fafffe3
Instead of wrapping the IndexSearcher and applying the role query during the rewrite, the role query gets applied in a custom filtered reader that applies the query via the live docs.
The big advantage is that DLS is being applied in all document based APIs instead of just the _search and _percolate APIs.
In order to better deal with the cost of converting the role query to a bitset, the bitsets are cached in the bitset filter cache
and if the role query bitset is sparse the role query and main query will execute in a leapfrog manner to make executing queries faster.
If the role query bitset isn't sparse, we fallback to livedocs.
Closeselastic/elasticsearch#537
Original commit: elastic/x-pack-elasticsearch@330b96e1f2
This commit removes and now forbids all uses of
com.google.common.base.Strings across the codebase.
For uses of com.google.common.base.Strings.isNullOrEmpty, use
org.elasticsearch.common.Strings.isNullOrEmpty.
For uses of com.google.common.base.Strings.padStart use
org.elasticsearch.common.Strings.padStart.
For uses of com.google.common.base.Strings.nullToEmpty use
org.elasticsearch.common.Strings.coalesceToEmpty.
Relates elastic/elasticsearchelastic/elasticsearch#13224
Original commit: elastic/x-pack-elasticsearch@d6e349e654
This commit removes and now forbids all uses of
com.google.common.base.Predicate and com.google.common.base.Predicates
across the codebase. This is one of the many steps in the eventual
removal of Guava as a dependency. This was enabled by
elastic/elasticsearchelastic/elasticsearch#13314.
Relates elastic/elasticsearchelastic/elasticsearch#13224, elastic/elasticsearchelastic/elasticsearch#13349
Original commit: elastic/x-pack-elasticsearch@968b81849f
The InternalTestCluster checks after the tests run if there are any locked shards, which is still
possible because the IndexAuditTrail is running and auditing events. This change makes the
test stop the IndexAuditTrail if they are running in the local cluster.
Original commit: elastic/x-pack-elasticsearch@f7fb564525
The ShieldWrappingCluster was added to prevent deletion of the IndexAuditTrail template when
running tests since the wipe() method will delete all templates. However, the cluster wrapping
is problematic with indexRandom() and causes noise in CI. This change removes the
ShieldWrappingCluster and changes the IndexAuditTrail to recreate the index template if it is
removed.
Closeselastic/elasticsearch#562
Original commit: elastic/x-pack-elasticsearch@0403ea7cef
This commit adds a test to ensure that all request handlers are wrapped
by ProfileSecuredRequestHandler.
Original commit: elastic/x-pack-elasticsearch@26473d0ddc
Currently we copy the authorization header from every rest request to the action request. This is not
necessary because the user associated with each request is copied into the context and then if the
request leaves the node, the user will be serialized into a string and attached as a header.
This commit removes the copying of the authorization header as it is not necessary and by not copying
it, we limit the amount of copies we make of this sensitive information.
Original commit: elastic/x-pack-elasticsearch@4e5ba4b4aa
This commit adds document and field level security to Shield.
Field level security can be enabled by adding the `fields` option to a role in the `role.yml` file.
For example:
```yaml
customer_care:
indices:
'*':
privileges: read
fields:
- issue_id
- description
- customer_handle
- customer_email
- customer_address
- customer_phone
```
The `fields` list is an inclusive list of fields that controls what fields should be accessible for that role. By default all meta fields (_uid, _type, _source, _ttl etc) are also included, otherwise ES or specific features stop working. The `_all` field if configured, isn't included by default, since that actually contains data from all the other fields. If the `_all` field is required then this needs to be added to the `fields` list in a role. In the case of the content of the `_source` field and `_field_names` there is special filtering in place so that only the content relevant for the role are being returned.
If no `fields` is specified then field level security is disabled for that role and all fields in an index are accessible.
Field level security can be setup per index group.
Field level security is implemented at the Lucene level by wrapping a directory index reader and hides fields away that aren't in the `field` list defined with the role of the current user. It as if the other fields never existed.
* Any `realtime` read operation from the translog is disabled. Instead this operations fall back to the Lucene index, which makes these operations compatible with field level security, but there aren't realtime.
* If user with role A executes first and the result gets cached and then a user with role B executes the same query results from the query executed with role A would be returned. This is bad and therefore the query cache is disabled.
* For the same reason the request cache is also disabled.
* The update API is blocked. An update request needs to be executed via a role that doesn't have field level security enabled.
Document level security can be enabled by adding the `query` option to a role in the `role.yml` file:
```yaml
customer_care:
indices:
'*':
privileges: read
query:
term:
department_id: 12
```
Document level security is implemented as a filter that filters out documents there don't match with the query. This is like index aliases, but better, because the role query is embedded on the lowest level possible in ES (Engine level) and on all places the acquire an IndexSearcher the role query will always be included. While alias filters are applied at a higher level (after the searcher has been acquired)
Document level security can be setup per index group.
Right now like alias filters the document level security isn't applied on all APIs. Like for example the get api, term vector api, which ignore the alias filter. These apis do acquire an IndexSearcher, but don't use the IndexSearcher itself and directly use the index reader to access the inverted index and there for bypassing the role query. If it is required to these apis need document level security too the the implementation for document level security needs to change.
Closeselastic/elasticsearch#341
Original commit: elastic/x-pack-elasticsearch@fac085dca6
Martijn van Groningen
jaymode
Ryan Ernst
Adrien Grand
Jason Tedor
uboness
Areek Zillur
Britta Weber
debadair
David Pilato
Boaz Leskes