Added document and field level security
This commit adds document and field level security to Shield.
Field level security can be enabled by adding the `fields` option to a role in the `role.yml` file.
For example:
```yaml
customer_care:
indices:
'*':
privileges: read
fields:
- issue_id
- description
- customer_handle
- customer_email
- customer_address
- customer_phone
```
The `fields` list is an inclusive list of fields that controls what fields should be accessible for that role. By default all meta fields (_uid, _type, _source, _ttl etc) are also included, otherwise ES or specific features stop working. The `_all` field if configured, isn't included by default, since that actually contains data from all the other fields. If the `_all` field is required then this needs to be added to the `fields` list in a role. In the case of the content of the `_source` field and `_field_names` there is special filtering in place so that only the content relevant for the role are being returned.
If no `fields` is specified then field level security is disabled for that role and all fields in an index are accessible.
Field level security can be setup per index group.
Field level security is implemented at the Lucene level by wrapping a directory index reader and hides fields away that aren't in the `field` list defined with the role of the current user. It as if the other fields never existed.
* Any `realtime` read operation from the translog is disabled. Instead this operations fall back to the Lucene index, which makes these operations compatible with field level security, but there aren't realtime.
* If user with role A executes first and the result gets cached and then a user with role B executes the same query results from the query executed with role A would be returned. This is bad and therefore the query cache is disabled.
* For the same reason the request cache is also disabled.
* The update API is blocked. An update request needs to be executed via a role that doesn't have field level security enabled.
Document level security can be enabled by adding the `query` option to a role in the `role.yml` file:
```yaml
customer_care:
indices:
'*':
privileges: read
query:
term:
department_id: 12
```
Document level security is implemented as a filter that filters out documents there don't match with the query. This is like index aliases, but better, because the role query is embedded on the lowest level possible in ES (Engine level) and on all places the acquire an IndexSearcher the role query will always be included. While alias filters are applied at a higher level (after the searcher has been acquired)
Document level security can be setup per index group.
Right now like alias filters the document level security isn't applied on all APIs. Like for example the get api, term vector api, which ignore the alias filter. These apis do acquire an IndexSearcher, but don't use the IndexSearcher itself and directly use the index reader to access the inverted index and there for bypassing the role query. If it is required to these apis need document level security too the the implementation for document level security needs to change.
Closes elastic/elasticsearch#341
Original commit: elastic/x-pack-elasticsearch@fac085dca6c3e1957eae0cf13d86ba93d20107c1
2015-08-27 11:53:10 -04:00
[[setting-up-field-and-document-level-security]]
=== Setting Up Field and Document Level Security.
You can control access to data within an index by adding field and document level security permissions to a role.
Field level security permissions restrict access to particular fields within a document.
Document level security permissions restrict access to particular documents within an index.
Field and document level permissions are specified separately, but a role can define both field and document level permissions.
Field and document level security permissions can be configured on a per-index basis.
==== Field Level Security
To enable field level security, you specify the fields that each role can access in the `roles.yml` file.
You list the allowed fields with the `fields` option. Fields are associated with a particular index or index pattern and
operate in conjunction with the privileges specified for the indices.
[source,yaml]
--------------------------------------------------
<role_name>:
indices:
<index_permission_expression>:
privileges: <privileges>
fields:
- <allowed_field_1>
- <allowed_field_2>
- <allowed_field_N>
--------------------------------------------------
To allow access to the `_all` meta field, you must explicitly list it as an allowed field. Access to the following meta fields
is always allowed: _id, _type, _parent, _routing, _timestamp, _ttl, _size and _index. If you specify an empty list of fields,
only these meta fields are accessible.
NOTE: Omitting the fields entry entirely disables field-level security.
For example, the following `customer_care` role grants read access to six fields in any index:
[source,yaml]
--------------------------------------------------
customer_care:
indices:
'*':
privileges: read
fields:
- issue_id
- description
- customer_handle
- customer_email
- customer_address
- customer_phone
--------------------------------------------------
2015-08-28 13:18:51 -04:00
Also wildcard field expressions can be added to the `fields` options in the `roles.yml` file. For example the following
example has the same effect as the previous example:
[source,yaml]
--------------------------------------------------
customer_care:
indices:
'*':
privileges: read
fields:
- issue_id
- description
- 'customer_*'
--------------------------------------------------
If documents are more complex and contain json objects then the fields with dot notion should be used.
Assume the following document:
[source,json]
--------------------------------------------------
{
"customer": {
"handle": "Jim",
"email": "jim@mycompany.com",
"phone": "555-555-5555"
}
}
--------------------------------------------------
If only access to the `handle` field is allowed then the following role should be setup:
[source,yaml]
--------------------------------------------------
my_role:
indices:
'*':
privileges: read
fields:
- customer.handle
--------------------------------------------------
If access to the entire `customer` object is allowed then the wildcard dot notation can be used to make this easier:
[source,yaml]
--------------------------------------------------
my_role:
indices:
'*':
privileges: read
fields:
- customer.*
--------------------------------------------------
Added document and field level security
This commit adds document and field level security to Shield.
Field level security can be enabled by adding the `fields` option to a role in the `role.yml` file.
For example:
```yaml
customer_care:
indices:
'*':
privileges: read
fields:
- issue_id
- description
- customer_handle
- customer_email
- customer_address
- customer_phone
```
The `fields` list is an inclusive list of fields that controls what fields should be accessible for that role. By default all meta fields (_uid, _type, _source, _ttl etc) are also included, otherwise ES or specific features stop working. The `_all` field if configured, isn't included by default, since that actually contains data from all the other fields. If the `_all` field is required then this needs to be added to the `fields` list in a role. In the case of the content of the `_source` field and `_field_names` there is special filtering in place so that only the content relevant for the role are being returned.
If no `fields` is specified then field level security is disabled for that role and all fields in an index are accessible.
Field level security can be setup per index group.
Field level security is implemented at the Lucene level by wrapping a directory index reader and hides fields away that aren't in the `field` list defined with the role of the current user. It as if the other fields never existed.
* Any `realtime` read operation from the translog is disabled. Instead this operations fall back to the Lucene index, which makes these operations compatible with field level security, but there aren't realtime.
* If user with role A executes first and the result gets cached and then a user with role B executes the same query results from the query executed with role A would be returned. This is bad and therefore the query cache is disabled.
* For the same reason the request cache is also disabled.
* The update API is blocked. An update request needs to be executed via a role that doesn't have field level security enabled.
Document level security can be enabled by adding the `query` option to a role in the `role.yml` file:
```yaml
customer_care:
indices:
'*':
privileges: read
query:
term:
department_id: 12
```
Document level security is implemented as a filter that filters out documents there don't match with the query. This is like index aliases, but better, because the role query is embedded on the lowest level possible in ES (Engine level) and on all places the acquire an IndexSearcher the role query will always be included. While alias filters are applied at a higher level (after the searcher has been acquired)
Document level security can be setup per index group.
Right now like alias filters the document level security isn't applied on all APIs. Like for example the get api, term vector api, which ignore the alias filter. These apis do acquire an IndexSearcher, but don't use the IndexSearcher itself and directly use the index reader to access the inverted index and there for bypassing the role query. If it is required to these apis need document level security too the the implementation for document level security needs to change.
Closes elastic/elasticsearch#341
Original commit: elastic/x-pack-elasticsearch@fac085dca6c3e1957eae0cf13d86ba93d20107c1
2015-08-27 11:53:10 -04:00
===== Limitations
When field level security is enabled for an index:
* The get, multi get, termsvector and multi termsvector APIs aren't executed in real time. The realtime option for these APIs is forcefully set to false.
* The query cache and the request cache are disabled for search requests.
* The update API is blocked. An update request needs to be executed via a role that doesn't have field level security enabled.
==== Document Level Security
2015-09-08 05:04:10 -04:00
Enabling document level security restricts which documents can be accessed from any document based API.
Added document and field level security
This commit adds document and field level security to Shield.
Field level security can be enabled by adding the `fields` option to a role in the `role.yml` file.
For example:
```yaml
customer_care:
indices:
'*':
privileges: read
fields:
- issue_id
- description
- customer_handle
- customer_email
- customer_address
- customer_phone
```
The `fields` list is an inclusive list of fields that controls what fields should be accessible for that role. By default all meta fields (_uid, _type, _source, _ttl etc) are also included, otherwise ES or specific features stop working. The `_all` field if configured, isn't included by default, since that actually contains data from all the other fields. If the `_all` field is required then this needs to be added to the `fields` list in a role. In the case of the content of the `_source` field and `_field_names` there is special filtering in place so that only the content relevant for the role are being returned.
If no `fields` is specified then field level security is disabled for that role and all fields in an index are accessible.
Field level security can be setup per index group.
Field level security is implemented at the Lucene level by wrapping a directory index reader and hides fields away that aren't in the `field` list defined with the role of the current user. It as if the other fields never existed.
* Any `realtime` read operation from the translog is disabled. Instead this operations fall back to the Lucene index, which makes these operations compatible with field level security, but there aren't realtime.
* If user with role A executes first and the result gets cached and then a user with role B executes the same query results from the query executed with role A would be returned. This is bad and therefore the query cache is disabled.
* For the same reason the request cache is also disabled.
* The update API is blocked. An update request needs to be executed via a role that doesn't have field level security enabled.
Document level security can be enabled by adding the `query` option to a role in the `role.yml` file:
```yaml
customer_care:
indices:
'*':
privileges: read
query:
term:
department_id: 12
```
Document level security is implemented as a filter that filters out documents there don't match with the query. This is like index aliases, but better, because the role query is embedded on the lowest level possible in ES (Engine level) and on all places the acquire an IndexSearcher the role query will always be included. While alias filters are applied at a higher level (after the searcher has been acquired)
Document level security can be setup per index group.
Right now like alias filters the document level security isn't applied on all APIs. Like for example the get api, term vector api, which ignore the alias filter. These apis do acquire an IndexSearcher, but don't use the IndexSearcher itself and directly use the index reader to access the inverted index and there for bypassing the role query. If it is required to these apis need document level security too the the implementation for document level security needs to change.
Closes elastic/elasticsearch#341
Original commit: elastic/x-pack-elasticsearch@fac085dca6c3e1957eae0cf13d86ba93d20107c1
2015-08-27 11:53:10 -04:00
To enable document level security, you use a query to specify the documents that each role can access in the `roles.yml` file.
You specify the document query with the `query` option. The document query is associated with a particular index or index pattern and
operates in conjunction with the privileges specified for the indices.
[source,yaml]
--------------------------------------------------
<role_name>:
indices:
<index_permission_expression>:
privileges: <privileges>
query:
<query>
--------------------------------------------------
NOTE: Omitting the `query` entry entirely disables document-level security.
The `query` should follow the same format as if a query was defined in the request body of a search request,
but here the format is YAML. Any query from the query-dsl can be defined in the `query` entry.
For example, the following `customer_care` role grants read access to all indices, but restricts access to documents whose `department_id` equals `12`.
[source,yaml]
--------------------------------------------------
customer_care:
indices:
'*':
privileges: read
query:
term:
department_id: 12
--------------------------------------------------
Alternatively the query can also be defined in JSON as a string. This makes it easier to define queries that already have
been defined in the JSON body of search request body elsewhere.
[source,yaml]
--------------------------------------------------
customer_care:
indices:
'*':
privileges: read
2015-09-08 05:04:10 -04:00
query: '{"term" : {"department_id" : "12"}}''
--------------------------------------------------
===== Limitations
When document level security is enabled for an index:
* The get, multi get, termsvector and multi termsvector APIs aren't executed in real time. The realtime option for these APIs is forcefully set to false.
* Document level security isn't applied for APIs that aren't document based oriented. For example this is the case for the field stats API.
* Document level security doesn't affect global index statistics that relevancy scoring uses. So this means that scores are computed without taking the role query into account.
Note that, documents not matching with the role query are never returned.
* The `has_child` and `has_parent` queries aren't supported as role query in the `roles.yml` file.
The `has_child` and `has_parent` queries can be used in the search API with document level security enabled.
