This is a step between the old and elastic/elasticsearch#184 this change unifies the logic that generates the search requests from prototype for SearchTransforms and SearchInputs.
This change also adds the `executed_request` field to the `SearchTransform.Result` that was missing.
Add tests for SearchTransform.Result parsing and all different templating options for SearchTransform along with testing handling of search_type in SearchTransform which was being missed previously.
Fixeselastic/elasticsearch#311
Original commit: elastic/x-pack-elasticsearch@26e68576f5
This input_result may contain different types for the same field names. This will cause mapping failures when we try to write the updated watch record.
This change disables this field and adds a test in the `ManualExecutionTests` to test this case.
Fixeselastic/elasticsearch#307
Original commit: elastic/x-pack-elasticsearch@5340761343
This change will attempt to compile scripts at construction time and throw an exception if a script is bad or missing.
With this change the script is compiled at Executable ctor and then the compiled script is used to get the executable.
Fixeselastic/elasticsearch#262
Original commit: elastic/x-pack-elasticsearch@b49c9debf2
This change changes the watch specific actions put/get/delete/execute/ack to be watcher.<action>_watch.
Original commit: elastic/x-pack-elasticsearch@86a91cb141
ElasticsearchIntegrationTest now has a method called httpClient with a different return type than the watcher AbstractWatcherTests httpclient.
This change renamed the `AbstractWatcherTests.httpClient` to `AbstractWatcherTests.watcherHttpClient`.
Original commit: elastic/x-pack-elasticsearch@d35f076cf2
A watch may contain sensitive data that typically you don't want to expose in plain text. Exposing means:
- storing it as plain text in the `.watches` and `.watch_history` indices
- storing it in memory in plain text (can be access via mem dump)
- returning it to the user via API in plain text
Examples of such sensitive data:
- The `password` for the email service (can be configured on the watch itself)
- The `password` for http input when using basic auth
- The `passowrd` for webhook action when using basic auth
A new `SecretService` (you heard it right... secret service) was added to handel the secrets across the board. When a watch is first added to watcher, this service converts all the sensitive data to secrets. From that moment on, all sensitive data associated with the watch (whether in stored in the index or in memory) is hidden behind the secret. This service is also used to "reveal" the original sensitive data on-demand when needed (for example, when the email is sent, it is sent with the original text).
There are two implementations for the `SecretService`. The default one is "plain text" where the created secrets don't really hide anything. The second implementation is based on Shield. If Shield is installed and enabled, the `ShieldSecretService` is used which uses shield's crypto service to potentially encrypt the sensitive data (only potentially because Shield's system key must be defined for encryption to take effect, without the system key, the crypto service will not encrypt and instead return the sensitive data in plain text)
Note, even when Shield is installed, the encryption of sensitive data will only be applied if the `watcher.shield.encrypt_sensitive_data` setting is set to `true`. By default it is set to `false`.
The `get watch` and `execute watch` APIs were updated to filter out sensitive data (using special "hide secrets" parameter).
When shield is integrated, we use shield's settings filter to filter out sensitive settings from the REST nodes info API (when shield is not installed or enabled, we don't do this filtering).
For this change several other refactoring needed to take place
- The http auth codebase was refactored to be more modular. Just like with other modular constructs in watcher, we separated `HttpAuth` from `ApplicableHttpAuth` where the former is the configuration construct and tha latter is the applicable ("executable") construct.
- Changed `WatchStore#put` to accept a watch (instead of the watch source). That's more natural way of looking at a store. Also, a `Watch` can now create and return itself as `ByteReference`. In addition, we now don't directly store the watch source as it was sent by the user, instead, we first parse it to a watch (important step to both validate the source and convert all sensitive data to secrets) and then serialize the watch back to `ByteReference`. This way we're sure that only the secrets are stored and not the original sensitive data.
- All `ToXContent` implementation were updated to properly propagate the `Params`
Docs were added to the Shield Integration chapter
Original commit: elastic/x-pack-elasticsearch@4490fb0ab8
SearchInput using inline, indexed and on disk templates.
ScriptedCondition where the script accesses the ctx.
ScriptedCondition where the script throws an exception.
ScriptedCondition where the script doesn't return a boolean.
Webhook tests using templated body, path and parameters.
Some REST test fixes.
Original commit: elastic/x-pack-elasticsearch@d02b6d1d7b
The HttpInputIntegrationTest sometimes unsetted the 'shieldEnabled' field for subsequent test methods, this caused http 401 errors, because the nodes were started with Shield enabled.
Original commit: elastic/x-pack-elasticsearch@b39ca9647d
This change employs the same kind of logic that `HttpClientTest` uses to find an open port for it's server. This should prevent the rare build failures we have seen.
Original commit: elastic/x-pack-elasticsearch@f3b68adad5
If there were many watches concurrently executing that take some time to execute it can a long time to shutdown a node. This change introduces a Timeout when stopping the WatchLockService.
Make watcher lock service timeout configurable. The configuration setting is `watcher.stop.timeout` the default is 30s.
Fixeselastic/elasticsearch#216
Original commit: elastic/x-pack-elasticsearch@06bf029a54
* The validation happens separately from the actual starting and loading. Also the validation happens on the cluster update thread, so that we don't miss any cluster state update.
* The starting/loading part happens on a forked thread, but if it fails then it always retry after a small timeout.
* Simplified the starting code, so that we don't need the callback interface.
Closeselastic/elasticsearch#212
Original commit: elastic/x-pack-elasticsearch@b5cd48e5bb
Brian Murphy
Martijn van Groningen
uboness