This commit adds document and field level security to Shield.
Field level security can be enabled by adding the `fields` option to a role in the `role.yml` file.
For example:
```yaml
customer_care:
indices:
'*':
privileges: read
fields:
- issue_id
- description
- customer_handle
- customer_email
- customer_address
- customer_phone
```
The `fields` list is an inclusive list of fields that controls what fields should be accessible for that role. By default all meta fields (_uid, _type, _source, _ttl etc) are also included, otherwise ES or specific features stop working. The `_all` field if configured, isn't included by default, since that actually contains data from all the other fields. If the `_all` field is required then this needs to be added to the `fields` list in a role. In the case of the content of the `_source` field and `_field_names` there is special filtering in place so that only the content relevant for the role are being returned.
If no `fields` is specified then field level security is disabled for that role and all fields in an index are accessible.
Field level security can be setup per index group.
Field level security is implemented at the Lucene level by wrapping a directory index reader and hides fields away that aren't in the `field` list defined with the role of the current user. It as if the other fields never existed.
* Any `realtime` read operation from the translog is disabled. Instead this operations fall back to the Lucene index, which makes these operations compatible with field level security, but there aren't realtime.
* If user with role A executes first and the result gets cached and then a user with role B executes the same query results from the query executed with role A would be returned. This is bad and therefore the query cache is disabled.
* For the same reason the request cache is also disabled.
* The update API is blocked. An update request needs to be executed via a role that doesn't have field level security enabled.
Document level security can be enabled by adding the `query` option to a role in the `role.yml` file:
```yaml
customer_care:
indices:
'*':
privileges: read
query:
term:
department_id: 12
```
Document level security is implemented as a filter that filters out documents there don't match with the query. This is like index aliases, but better, because the role query is embedded on the lowest level possible in ES (Engine level) and on all places the acquire an IndexSearcher the role query will always be included. While alias filters are applied at a higher level (after the searcher has been acquired)
Document level security can be setup per index group.
Right now like alias filters the document level security isn't applied on all APIs. Like for example the get api, term vector api, which ignore the alias filter. These apis do acquire an IndexSearcher, but don't use the IndexSearcher itself and directly use the index reader to access the inverted index and there for bypassing the role query. If it is required to these apis need document level security too the the implementation for document level security needs to change.
Closeselastic/elasticsearch#341
Original commit: elastic/x-pack-elasticsearch@fac085dca6
We have different types of templates in watcher - http request template, email template, hipchat message template, and simple text template... to avoid confusion, and clean up the codebase, this commit renames the `Template` class to `TextTemplate` to better convey what this template is about.
Original commit: elastic/x-pack-elasticsearch@8e5202019c
An action capable of sending notifications to rooms and users on hipchat. This actions support three types of HipChat APIs:
- `v1` - The (now deprecated) legacy API where a token can be registered at the group level, and the `v1` version of the API can be used. This API only supports room notification (users cannot be notified). multi-room notification is supported.
- `integration` - The basic integration that one can create in HipChat (it is using the `v2` API version), where notifications can be sent to a single room. User notification is unsupported by this API
- `user` - this API uses an API token of a specific user. An admin user can create an API token and configure it to have access to room notification and user private messaging. This API supports multi-room and multi-user notifications.
The settings for `hipchat` are very similar to the `email` infrastructure in nature. It is possible to configure multiple/different hipchat account, each is associated with the api type (a.k.a profile) - can be `v1`, `integration` or `user`, and the respective `auth_token`. When configuring the action in the watch, one can specify what hipchat account they would like to use (when not specifying an account, the `default_account` will be used). Each account can also specify its own unique `host`/`port` for the hipchat server - for full flexibility.
Closeselastic/elasticsearch#462
Original commit: elastic/x-pack-elasticsearch@9d9ee13542
PreProcessModule was an alternate way to customize another module's
behavior inside plugins. The preferred (and only in the future) way to
do this is with onModule in the plugin itself. This change moves the
only two remaining users of PreProcessModule to do so in their
respective plugins. The use case was adding roles for shield
authorization, but these roles were really static, so there was no
reason they could not be configured up front.
Original commit: elastic/x-pack-elasticsearch@e67ac2dcb6
SpawnModules will be going away very soon as part of
elastic/elasticsearchelastic/elasticsearch#12783. This change removes its use from all
x-plugins.
Most spawnmodules uses here were to either collect a number of modules
into one (so the modules were just moved up into the plugin itself), or
to spawn a module which interacted with an extension point from ES. This
change moves those, as well as most uses of PreProcessModule, to use
onModule.
Original commit: elastic/x-pack-elasticsearch@6430e35379
only run watcher rest tests during verify phase
never run the rest tests with shield enabled, because that is now tested in the new qa module
removed the disabled license watcher rest tests, because the disabled license use case is already tested by the LicenseIntegrationTests
enabled the getting started rest test
Closeselastic/elasticsearch#403
Original commit: elastic/x-pack-elasticsearch@67f0f7f596
For now this just tries to install license, marvel and watcher, and then checks
that these plugins are listed in the node infos. I can do shield once I figure
out how to set it up for REST tests.
Original commit: elastic/x-pack-elasticsearch@8549f4bc5a
* The watcher+shield test failed, but the error was ignored due to a buggy if statement that existed for the hijack rest test.
* Blacklisted the hijack test.
* Also ade sure that we run the watcher+rest test with an user that doesn't have credentials.
Original commit: elastic/x-pack-elasticsearch@61b1bf0142
This commit changes the groupId to the above mentioned one
so that S3 uploads will end up in the right bucket. This will
allow the Elasticsearch plugin manager to install the commercial
plugins like
```
bin/plugin install {watcher,shield,license,marvel}
```
like the official ones.
Original commit: elastic/x-pack-elasticsearch@642f1f006a
With elastic/elasticsearchelastic/elasticsearch#12623 base test classes were renamed
to use "TestCase" suffix. This updates x-plugins to reflect those
name changes. It also renames some tests that were marked
with @Slow (which was forbidden with elastic/elasticsearchelastic/elasticsearch#12617 and
elastic/elasticsearch elastic/elasticsearch#12618) to use the IT suffix to run
under `mvn verify`.
Original commit: elastic/x-pack-elasticsearch@05ffe2f202
homeFile() is removed and should not be used, we need to cleanup,
but this is just a rote change to get builds green.
Original commit: elastic/x-pack-elasticsearch@05d0fb4a7c
Martijn van Groningen
uboness
Areek Zillur
Simon Willnauer
Ryan Ernst
jaymode
debadair
David Pilato
Adrien Grand
Tanguy Leroux
Alexander Reelsen
Robert Muir