Instead of wrapping the IndexSearcher and applying the role query during the rewrite, the role query gets applied in a custom filtered reader that applies the query via the live docs.
The big advantage is that DLS is being applied in all document based APIs instead of just the _search and _percolate APIs.
In order to better deal with the cost of converting the role query to a bitset, the bitsets are cached in the bitset filter cache
and if the role query bitset is sparse the role query and main query will execute in a leapfrog manner to make executing queries faster.
If the role query bitset isn't sparse, we fallback to livedocs.
Closeselastic/elasticsearch#537
Original commit: elastic/x-pack-elasticsearch@330b96e1f2
This commit adds document and field level security to Shield.
Field level security can be enabled by adding the `fields` option to a role in the `role.yml` file.
For example:
```yaml
customer_care:
indices:
'*':
privileges: read
fields:
- issue_id
- description
- customer_handle
- customer_email
- customer_address
- customer_phone
```
The `fields` list is an inclusive list of fields that controls what fields should be accessible for that role. By default all meta fields (_uid, _type, _source, _ttl etc) are also included, otherwise ES or specific features stop working. The `_all` field if configured, isn't included by default, since that actually contains data from all the other fields. If the `_all` field is required then this needs to be added to the `fields` list in a role. In the case of the content of the `_source` field and `_field_names` there is special filtering in place so that only the content relevant for the role are being returned.
If no `fields` is specified then field level security is disabled for that role and all fields in an index are accessible.
Field level security can be setup per index group.
Field level security is implemented at the Lucene level by wrapping a directory index reader and hides fields away that aren't in the `field` list defined with the role of the current user. It as if the other fields never existed.
* Any `realtime` read operation from the translog is disabled. Instead this operations fall back to the Lucene index, which makes these operations compatible with field level security, but there aren't realtime.
* If user with role A executes first and the result gets cached and then a user with role B executes the same query results from the query executed with role A would be returned. This is bad and therefore the query cache is disabled.
* For the same reason the request cache is also disabled.
* The update API is blocked. An update request needs to be executed via a role that doesn't have field level security enabled.
Document level security can be enabled by adding the `query` option to a role in the `role.yml` file:
```yaml
customer_care:
indices:
'*':
privileges: read
query:
term:
department_id: 12
```
Document level security is implemented as a filter that filters out documents there don't match with the query. This is like index aliases, but better, because the role query is embedded on the lowest level possible in ES (Engine level) and on all places the acquire an IndexSearcher the role query will always be included. While alias filters are applied at a higher level (after the searcher has been acquired)
Document level security can be setup per index group.
Right now like alias filters the document level security isn't applied on all APIs. Like for example the get api, term vector api, which ignore the alias filter. These apis do acquire an IndexSearcher, but don't use the IndexSearcher itself and directly use the index reader to access the inverted index and there for bypassing the role query. If it is required to these apis need document level security too the the implementation for document level security needs to change.
Closeselastic/elasticsearch#341
Original commit: elastic/x-pack-elasticsearch@fac085dca6
Today the XContent building of the response for the ClearRealmsCacheResponse is broken and causes
an exception to be thrown. This fixes the building of the response and adds tests that call the HTTP
endpoint and do a basic check on the response.
Closeselastic/elasticsearch#390
Original commit: elastic/x-pack-elasticsearch@8ad9dae4ea
This commit changes the groupId to the above mentioned one
so that S3 uploads will end up in the right bucket. This will
allow the Elasticsearch plugin manager to install the commercial
plugins like
```
bin/plugin install {watcher,shield,license,marvel}
```
like the official ones.
Original commit: elastic/x-pack-elasticsearch@642f1f006a
In the documentation reorgantization for Shield, the type of the active directory realm changed
and is not valid. Users are configuring Shield incorrectly and running into issues because of
this. This changes the realm type to the correct value.
Closeselastic/elasticsearch#291
Original commit: elastic/x-pack-elasticsearch@7f8b65eda7
Martijn van Groningen
jaymode
debadair
Alexander Reelsen
Suyog Rao
Clinton Gormley
uboness