This change will attempt to compile scripts at construction time and throw an exception if a script is bad or missing.
With this change the script is compiled at Executable ctor and then the compiled script is used to get the executable.
Fixeselastic/elasticsearch#262
Original commit: elastic/x-pack-elasticsearch@b49c9debf2
This change changes the watch specific actions put/get/delete/execute/ack to be watcher.<action>_watch.
Original commit: elastic/x-pack-elasticsearch@86a91cb141
ElasticsearchIntegrationTest now has a method called httpClient with a different return type than the watcher AbstractWatcherTests httpclient.
This change renamed the `AbstractWatcherTests.httpClient` to `AbstractWatcherTests.watcherHttpClient`.
Original commit: elastic/x-pack-elasticsearch@d35f076cf2
A watch may contain sensitive data that typically you don't want to expose in plain text. Exposing means:
- storing it as plain text in the `.watches` and `.watch_history` indices
- storing it in memory in plain text (can be access via mem dump)
- returning it to the user via API in plain text
Examples of such sensitive data:
- The `password` for the email service (can be configured on the watch itself)
- The `password` for http input when using basic auth
- The `passowrd` for webhook action when using basic auth
A new `SecretService` (you heard it right... secret service) was added to handel the secrets across the board. When a watch is first added to watcher, this service converts all the sensitive data to secrets. From that moment on, all sensitive data associated with the watch (whether in stored in the index or in memory) is hidden behind the secret. This service is also used to "reveal" the original sensitive data on-demand when needed (for example, when the email is sent, it is sent with the original text).
There are two implementations for the `SecretService`. The default one is "plain text" where the created secrets don't really hide anything. The second implementation is based on Shield. If Shield is installed and enabled, the `ShieldSecretService` is used which uses shield's crypto service to potentially encrypt the sensitive data (only potentially because Shield's system key must be defined for encryption to take effect, without the system key, the crypto service will not encrypt and instead return the sensitive data in plain text)
Note, even when Shield is installed, the encryption of sensitive data will only be applied if the `watcher.shield.encrypt_sensitive_data` setting is set to `true`. By default it is set to `false`.
The `get watch` and `execute watch` APIs were updated to filter out sensitive data (using special "hide secrets" parameter).
When shield is integrated, we use shield's settings filter to filter out sensitive settings from the REST nodes info API (when shield is not installed or enabled, we don't do this filtering).
For this change several other refactoring needed to take place
- The http auth codebase was refactored to be more modular. Just like with other modular constructs in watcher, we separated `HttpAuth` from `ApplicableHttpAuth` where the former is the configuration construct and tha latter is the applicable ("executable") construct.
- Changed `WatchStore#put` to accept a watch (instead of the watch source). That's more natural way of looking at a store. Also, a `Watch` can now create and return itself as `ByteReference`. In addition, we now don't directly store the watch source as it was sent by the user, instead, we first parse it to a watch (important step to both validate the source and convert all sensitive data to secrets) and then serialize the watch back to `ByteReference`. This way we're sure that only the secrets are stored and not the original sensitive data.
- All `ToXContent` implementation were updated to properly propagate the `Params`
Docs were added to the Shield Integration chapter
Original commit: elastic/x-pack-elasticsearch@4490fb0ab8
SearchInput using inline, indexed and on disk templates.
ScriptedCondition where the script accesses the ctx.
ScriptedCondition where the script throws an exception.
ScriptedCondition where the script doesn't return a boolean.
Webhook tests using templated body, path and parameters.
Some REST test fixes.
Original commit: elastic/x-pack-elasticsearch@d02b6d1d7b
The HttpInputIntegrationTest sometimes unsetted the 'shieldEnabled' field for subsequent test methods, this caused http 401 errors, because the nodes were started with Shield enabled.
Original commit: elastic/x-pack-elasticsearch@b39ca9647d
This change employs the same kind of logic that `HttpClientTest` uses to find an open port for it's server. This should prevent the rare build failures we have seen.
Original commit: elastic/x-pack-elasticsearch@f3b68adad5
If there were many watches concurrently executing that take some time to execute it can a long time to shutdown a node. This change introduces a Timeout when stopping the WatchLockService.
Make watcher lock service timeout configurable. The configuration setting is `watcher.stop.timeout` the default is 30s.
Fixeselastic/elasticsearch#216
Original commit: elastic/x-pack-elasticsearch@06bf029a54
* The validation happens separately from the actual starting and loading. Also the validation happens on the cluster update thread, so that we don't miss any cluster state update.
* The starting/loading part happens on a forked thread, but if it fails then it always retry after a small timeout.
* Simplified the starting code, so that we don't need the callback interface.
Closeselastic/elasticsearch#212
Original commit: elastic/x-pack-elasticsearch@b5cd48e5bb
Before this change DateTimes were being constructed without providing a time zone,
this was causing some non-utc time leakage. In particular watch record ids were being created with non utc dates and
watch records were going to the wrong .watch_history index.
Add Clock.now(DateTimeZone) to allow callers to get UTC now.
Also construct the DateTimes with UTC timezones when we construct from millis.
Add all constuctors of DateTime that do not specify a time zone to the forbidden APIs.
This change makes constructing a `DateTime` object without providing a `DateTimeZone` forbidden.
This is much safer and makes time zone errors much more unlikely to occur.
Statically import DateTimeZone.UTC everywhere it was being used
Now all calls that use DateTimeZone.UTC just refrence UTC.
Fixeselastic/elasticsearch#150
Original commit: elastic/x-pack-elasticsearch@7f23ce605e
Mainly how templates read/write themselves from/to xcontent. Instead of using `text`, use `template`
Original commit: elastic/x-pack-elasticsearch@0d6f317539
* Split the transform into two constructs: `Transform` and `ExecutableTransform`. The former holds all the transform configuration, the latter can execute the transform based on that configuration (an executable transform holds a transform)
- This makes the code clearer to understand and maintain.
- This also enabled to pull some common implementation code into the `ExecutableTransform` and by that reduce the implementation details of each executable to the minimum required.
* Also, extracted the `Transform.Parser` to its own top level class, and renamed it to - `TransformFactory`. The main thing that the factory does is: 1) delegate to the parsing to the `Transform` class, 2) construct & wire up the `ExecutableTransform`.
* With the introduction of `Transform`, we no longer need the `SourceBuilder` for transforms. Instead, we have `Transform.Builder` that help you build a transform. This is much more intuitive from the client perspective.
Original commit: elastic/x-pack-elasticsearch@f6ee0d0c75
* Tune the index templates for higher index throughput.
* Trigger events should be processed and indexed in an async manner.
* Moved the job name into the TriggerEvent
* Added support for fire multiple events at the same time. If multiple events are fired at the same time then use async bulk to persist the watch record.
* Cut simple ticker and timer ticker over to fire multiple events at the same time
* Don't fire more than 1000 events at the time. (this also may result in large bulk requests)
Original commit: elastic/x-pack-elasticsearch@c7f6bd3812
Brian Murphy
Martijn van Groningen
uboness