A watch may contain sensitive data that typically you don't want to expose in plain text. Exposing means:
- storing it as plain text in the `.watches` and `.watch_history` indices
- storing it in memory in plain text (can be access via mem dump)
- returning it to the user via API in plain text
Examples of such sensitive data:
- The `password` for the email service (can be configured on the watch itself)
- The `password` for http input when using basic auth
- The `passowrd` for webhook action when using basic auth
A new `SecretService` (you heard it right... secret service) was added to handel the secrets across the board. When a watch is first added to watcher, this service converts all the sensitive data to secrets. From that moment on, all sensitive data associated with the watch (whether in stored in the index or in memory) is hidden behind the secret. This service is also used to "reveal" the original sensitive data on-demand when needed (for example, when the email is sent, it is sent with the original text).
There are two implementations for the `SecretService`. The default one is "plain text" where the created secrets don't really hide anything. The second implementation is based on Shield. If Shield is installed and enabled, the `ShieldSecretService` is used which uses shield's crypto service to potentially encrypt the sensitive data (only potentially because Shield's system key must be defined for encryption to take effect, without the system key, the crypto service will not encrypt and instead return the sensitive data in plain text)
Note, even when Shield is installed, the encryption of sensitive data will only be applied if the `watcher.shield.encrypt_sensitive_data` setting is set to `true`. By default it is set to `false`.
The `get watch` and `execute watch` APIs were updated to filter out sensitive data (using special "hide secrets" parameter).
When shield is integrated, we use shield's settings filter to filter out sensitive settings from the REST nodes info API (when shield is not installed or enabled, we don't do this filtering).
For this change several other refactoring needed to take place
- The http auth codebase was refactored to be more modular. Just like with other modular constructs in watcher, we separated `HttpAuth` from `ApplicableHttpAuth` where the former is the configuration construct and tha latter is the applicable ("executable") construct.
- Changed `WatchStore#put` to accept a watch (instead of the watch source). That's more natural way of looking at a store. Also, a `Watch` can now create and return itself as `ByteReference`. In addition, we now don't directly store the watch source as it was sent by the user, instead, we first parse it to a watch (important step to both validate the source and convert all sensitive data to secrets) and then serialize the watch back to `ByteReference`. This way we're sure that only the secrets are stored and not the original sensitive data.
- All `ToXContent` implementation were updated to properly propagate the `Params`
Docs were added to the Shield Integration chapter
Original commit: elastic/x-pack-elasticsearch@4490fb0ab8
Before this change DateTimes were being constructed without providing a time zone,
this was causing some non-utc time leakage. In particular watch record ids were being created with non utc dates and
watch records were going to the wrong .watch_history index.
Add Clock.now(DateTimeZone) to allow callers to get UTC now.
Also construct the DateTimes with UTC timezones when we construct from millis.
Add all constuctors of DateTime that do not specify a time zone to the forbidden APIs.
This change makes constructing a `DateTime` object without providing a `DateTimeZone` forbidden.
This is much safer and makes time zone errors much more unlikely to occur.
Statically import DateTimeZone.UTC everywhere it was being used
Now all calls that use DateTimeZone.UTC just refrence UTC.
Fixeselastic/elasticsearch#150
Original commit: elastic/x-pack-elasticsearch@7f23ce605e
- fixed the use of the found forbidden APIs
- changed `FiredAlert.State` values to lower case (for consistency sake)
Original commit: elastic/x-pack-elasticsearch@9b3f8383d9
uboness
Brian Murphy