moved from `text_body` and `html_body` to a more structured `body` object as follows:
```
{
"body" : {
"text" : "the text body",
"html" : "the html body"
}
}
```
`body` can also accpet a string, in which case it will default to the text body of the email:
```
{
"body" : "the text body of the email"
}
```
the above is a syntactic sugar for the following:
```
{
"body" : {
"text" : "the text body of the email"
}
}
```
Original commit: elastic/x-pack-elasticsearch@92406ac2a1
Bye default, when `attach_data` is set to `true` we attache the watch execution data as a `yaml` file. Now it's possible to configure the format of the file by setting the `attach_data` to an object with a `format` field (can either be set to `yaml` or `json`). Example:
```
"attach_data" : {
"format" : "json"
}
```
Original commit: elastic/x-pack-elasticsearch@5cab59a676
Scripts and Templates are very much alike. There are only two main differences between them when it comes to watcher:
1. The template doesn't have a language type - it's always mustache.
2. Templates are often used to render xcontent (json/yaml). So there's build in support for content type in it
Traditionally, in es core, the configuration of the scripts and templates was always a bit of a mess. There was no consistency between the two and when it comes to the templates configuration, they're even not future proof (future additions to the search requests body can break the templates).
After a long discussion, we've decided that we need to change the way users configure templates & scripts in es core... and align the two. These changes will only come in es 2.0. That said, it's important that watcher will be aligned with es core in how templates and scripts are configured and therefore we need to change it for that purpose.
Watcher will come with support for the scripts & templates configuration format of es 2.0. We want to future proof the configuration from day one.
here are the configuration:
Inline:
```
{
"inline" : "script here",
"lang" : "lang here",
"params" : { parameters here }
}
```
File:
```
{
"file" : "file name here",
"lang" : "language here",
"params" : { parameters here }
}
```
Indexed:
```
{
"id" : "template id here",
"lang" : "language here",
"params" : { parameters here }
}
```
For templates it's the same configuration except there is no `lang` field. Also, for templates, there's native support for xcontent:
```
{
"inline" : { "key" : "{{param1}}" },
"params" : { "param1" : "value1" }
}
}
```
Original commit: elastic/x-pack-elasticsearch@4a31114f35
This change changes how the ScheduledTriggerEvent is parsed to parse using DateMath instead of just as a date. This will allow the execute API to use such constructs as
```
POST _watcher/watch/test_watch/_execute
{
"ignore_throttle" : true,
"trigger_event" : {
"schedule" : {
"triggered_time": "now-5h",
"scheduled_time": "now"
}
}
}
```
Fixes: elastic/elasticsearch#374
Original commit: elastic/x-pack-elasticsearch@fa286b217e
This change changes the manual trigger event to be a wrapper for another `TriggerEvent`.
TriggerEvent is required field for the execute API either via setTriggerEvent on the request or requestBuilder in the java API or `trigger_event` in the REST API
Original commit: elastic/x-pack-elasticsearch@790bdb96a5
This change removes these jars from shading and changes the assemblies file so that they are included as is.
Original commit: elastic/x-pack-elasticsearch@695f5e2b20
Also removed the version checking from the index template support, so that if the settings are changed these settings always get stored in the index template. This means we always put two index templates once watcher started.
And also shutdown watcher after the cluster service has been stopped.
This was done caused deadlock in the integration test framework when we are stopping the test cluster.
If multiple nodes are stopped, a node may briefly try to or run Watcher during the test cluster shutdown.
As part if starting we always put an index template, which will block any other start or stop calls for the WatcherService.
A node was running a put index template call as part of the start procedure while the start framework would stop the node.
The stop call would wait because it doesn't own the lock yet.
The put index template call didn't return, which caused the stop call the keep waiting.
The reason why put index template was hanging is that it failed to return anything while it the node was stopped.
Likely the put index template request got queued up waiting for a change in the cluster state, the cluster service
got stopped and the request got lost, but the watcher start procedure was still waiting.
By stopping Watcher before we stop the cluster service we avoid the scenario described above.
Original commit: elastic/x-pack-elasticsearch@db94b2279e
- wrote an intro where we tell how to get the watcher jar and we tall about the `WatcherClient`
- cleaned up the code snippets
- removed a reference to the `Watch.Parser`
- Also renamed `WatcherStatsResponse#getWatchServiceState` to `watcherStatsResponse.getWatcherState`
Original commit: elastic/x-pack-elasticsearch@814329cba7
An absraction aover XContent byte reference that:
- implements `ToXContent`
- can resolve values given a dot-notation path
- Used in `GetWatchResponse` and `ExecuteWatchResponse`
Also
- Moved `WatchExecutionResult` to the `execution` package
Original commit: elastic/x-pack-elasticsearch@0b41f53f38
- Moved it to its own top level enum `WatcherState`
- Also changed the key holding the current state in the `stats` API to `watcher_state`
Original commit: elastic/x-pack-elasticsearch@2e5ca90f57
Change them to consistently serialize themselves the same way they deserialize themselves.
If the script (or template) is read from a `string` value, it will serialize it self to xcontent as a `string` value as well. Otherwise it will serialize as an object, holding only those fields that where configured in the first place.
Original commit: elastic/x-pack-elasticsearch@52a82e0bbe
This change moves the sanitization of the html body of emails to render time instead of at message send time.
Move the sanitization code the the EmailTemplate class.
Add <hr> tag to allowed html tags.
Add global setting `watcher.actions.email.sanitize_html` that defaults to true. If this is set to false html sanitization will be disabled
Add documentation for sanitization disable setting.
Enhance email tests to verify that sanitization is happening at construction time.
Fixeselastic/elasticsearch#356
Original commit: elastic/x-pack-elasticsearch@282a2d85c2
The body of the http input may hold xcontent data (json/yaml). The problem is that today, the body only accepts text and writing
escapted json in text makes it very hard to maintain.
This commit introduces another field settings - `xbody`. Now, use `xbody` for inlined xcontent and `body` for any text content. When `xbody` is used, we keep around the xcontent type and the http request that is then executed will include a content-type header that will match the xcontent type of input.
Original commit: elastic/x-pack-elasticsearch@7210908064
- added dynamic_templates to the index template that make sure not to analyze email addresses and http `path` and `host` under the `watch_execution` object. This will enabled building aggregations over these fields. Also make sure that different time fields in thw `watch_record` are mapped as date types.
- While at it changed the `watch_execution.input_result` mapping such that it's enabled and only disabled `watch_execution.input_result.payload` as different payloads from different sources may conflict with one another in terms of mappings
- Fixed a bug in the `EmailTemplate` building of the `reply_to` field.
- Also, added missing `execution_time` to the `watch_record` (under `watch_execution` object).
Closeselastic/elasticsearch#335
Original commit: elastic/x-pack-elasticsearch@dd28c70bca
The meta data field was known and `metadata` in the docs but the parsing logic expected `meta`.
This change unifies everything to use `metadata`. Also clean up Watch parsing to fail in a more consistent way when unknown fields are encountered.
Fixeselastic/elasticsearch#329
Original commit: elastic/x-pack-elasticsearch@a9013127d8
This change hijacks the document APIs for the .watches index and prevents direct access to these APIs via REST.
This hijacking is controlled by the "watcher.index.rest.direct_access" setting, this setting defaults to false which restricts access to the index. To allow direct access to the .watches index from the REST API set this setting to `true`
Copied json files from core es to test this feature and added comments to the json files indicating why they have been copied from es core.
Fixeselastic/elasticsearch#336
Original commit: elastic/x-pack-elasticsearch@22335750bd
Also added assertions to verify if all watch record not null field are expected to be there at parse time.
Original commit: elastic/x-pack-elasticsearch@42052ae02c
The problem we have today is that since all write operations on a watch are blocked on the watch lock, if a watch is executing for a long time (for whatever reason) the action will simply block and by that hold a `management` thread for potentially a long time. This impacts elasticsearch as a whole - if for example there's an infinite loop in the condition script, then you try to delete/ack/update the watch 5 times (the default size of the management thread pool in elasticsearch is 5). You effectively block all management threads in elasticsearch and from that moment on, the only thing left to do is kill the node.
This commit introduces a timeout for the tree write watch actions - `put`, `ack` and `delete`. It is set to 10 seconds by default, but you can customize it both in java and in the rest API using the standard `master_timeout` query parameter.
Updated the relevant docs.
Fixeselastic/elasticsearch#309
Original commit: elastic/x-pack-elasticsearch@03292db37b
This change disallows the SCAN search type in our search requests used by search input and search transform.
Add tests for this and update the current tests to both detect SCAN and not use SCAN in valid tests.
Original commit: elastic/x-pack-elasticsearch@c9d61930c8
This is a step between the old and elastic/elasticsearch#184 this change unifies the logic that generates the search requests from prototype for SearchTransforms and SearchInputs.
This change also adds the `executed_request` field to the `SearchTransform.Result` that was missing.
Add tests for SearchTransform.Result parsing and all different templating options for SearchTransform along with testing handling of search_type in SearchTransform which was being missed previously.
Fixeselastic/elasticsearch#311
Original commit: elastic/x-pack-elasticsearch@26e68576f5
This input_result may contain different types for the same field names. This will cause mapping failures when we try to write the updated watch record.
This change disables this field and adds a test in the `ManualExecutionTests` to test this case.
Fixeselastic/elasticsearch#307
Original commit: elastic/x-pack-elasticsearch@5340761343
This change will attempt to compile scripts at construction time and throw an exception if a script is bad or missing.
With this change the script is compiled at Executable ctor and then the compiled script is used to get the executable.
Fixeselastic/elasticsearch#262
Original commit: elastic/x-pack-elasticsearch@b49c9debf2
ElasticsearchIntegrationTest now has a method called httpClient with a different return type than the watcher AbstractWatcherTests httpclient.
This change renamed the `AbstractWatcherTests.httpClient` to `AbstractWatcherTests.watcherHttpClient`.
Original commit: elastic/x-pack-elasticsearch@d35f076cf2
A watch may contain sensitive data that typically you don't want to expose in plain text. Exposing means:
- storing it as plain text in the `.watches` and `.watch_history` indices
- storing it in memory in plain text (can be access via mem dump)
- returning it to the user via API in plain text
Examples of such sensitive data:
- The `password` for the email service (can be configured on the watch itself)
- The `password` for http input when using basic auth
- The `passowrd` for webhook action when using basic auth
A new `SecretService` (you heard it right... secret service) was added to handel the secrets across the board. When a watch is first added to watcher, this service converts all the sensitive data to secrets. From that moment on, all sensitive data associated with the watch (whether in stored in the index or in memory) is hidden behind the secret. This service is also used to "reveal" the original sensitive data on-demand when needed (for example, when the email is sent, it is sent with the original text).
There are two implementations for the `SecretService`. The default one is "plain text" where the created secrets don't really hide anything. The second implementation is based on Shield. If Shield is installed and enabled, the `ShieldSecretService` is used which uses shield's crypto service to potentially encrypt the sensitive data (only potentially because Shield's system key must be defined for encryption to take effect, without the system key, the crypto service will not encrypt and instead return the sensitive data in plain text)
Note, even when Shield is installed, the encryption of sensitive data will only be applied if the `watcher.shield.encrypt_sensitive_data` setting is set to `true`. By default it is set to `false`.
The `get watch` and `execute watch` APIs were updated to filter out sensitive data (using special "hide secrets" parameter).
When shield is integrated, we use shield's settings filter to filter out sensitive settings from the REST nodes info API (when shield is not installed or enabled, we don't do this filtering).
For this change several other refactoring needed to take place
- The http auth codebase was refactored to be more modular. Just like with other modular constructs in watcher, we separated `HttpAuth` from `ApplicableHttpAuth` where the former is the configuration construct and tha latter is the applicable ("executable") construct.
- Changed `WatchStore#put` to accept a watch (instead of the watch source). That's more natural way of looking at a store. Also, a `Watch` can now create and return itself as `ByteReference`. In addition, we now don't directly store the watch source as it was sent by the user, instead, we first parse it to a watch (important step to both validate the source and convert all sensitive data to secrets) and then serialize the watch back to `ByteReference`. This way we're sure that only the secrets are stored and not the original sensitive data.
- All `ToXContent` implementation were updated to properly propagate the `Params`
Docs were added to the Shield Integration chapter
Original commit: elastic/x-pack-elasticsearch@4490fb0ab8
SearchInput using inline, indexed and on disk templates.
ScriptedCondition where the script accesses the ctx.
ScriptedCondition where the script throws an exception.
ScriptedCondition where the script doesn't return a boolean.
Webhook tests using templated body, path and parameters.
Some REST test fixes.
Original commit: elastic/x-pack-elasticsearch@d02b6d1d7b
The HttpInputIntegrationTest sometimes unsetted the 'shieldEnabled' field for subsequent test methods, this caused http 401 errors, because the nodes were started with Shield enabled.
Original commit: elastic/x-pack-elasticsearch@b39ca9647d
This change employs the same kind of logic that `HttpClientTest` uses to find an open port for it's server. This should prevent the rare build failures we have seen.
Original commit: elastic/x-pack-elasticsearch@f3b68adad5
If there were many watches concurrently executing that take some time to execute it can a long time to shutdown a node. This change introduces a Timeout when stopping the WatchLockService.
Make watcher lock service timeout configurable. The configuration setting is `watcher.stop.timeout` the default is 30s.
Fixeselastic/elasticsearch#216
Original commit: elastic/x-pack-elasticsearch@06bf029a54
* The validation happens separately from the actual starting and loading. Also the validation happens on the cluster update thread, so that we don't miss any cluster state update.
* The starting/loading part happens on a forked thread, but if it fails then it always retry after a small timeout.
* Simplified the starting code, so that we don't need the callback interface.
Closeselastic/elasticsearch#212
Original commit: elastic/x-pack-elasticsearch@b5cd48e5bb
Before this change DateTimes were being constructed without providing a time zone,
this was causing some non-utc time leakage. In particular watch record ids were being created with non utc dates and
watch records were going to the wrong .watch_history index.
Add Clock.now(DateTimeZone) to allow callers to get UTC now.
Also construct the DateTimes with UTC timezones when we construct from millis.
Add all constuctors of DateTime that do not specify a time zone to the forbidden APIs.
This change makes constructing a `DateTime` object without providing a `DateTimeZone` forbidden.
This is much safer and makes time zone errors much more unlikely to occur.
Statically import DateTimeZone.UTC everywhere it was being used
Now all calls that use DateTimeZone.UTC just refrence UTC.
Fixeselastic/elasticsearch#150
Original commit: elastic/x-pack-elasticsearch@7f23ce605e
Mainly how templates read/write themselves from/to xcontent. Instead of using `text`, use `template`
Original commit: elastic/x-pack-elasticsearch@0d6f317539
* Split the transform into two constructs: `Transform` and `ExecutableTransform`. The former holds all the transform configuration, the latter can execute the transform based on that configuration (an executable transform holds a transform)
- This makes the code clearer to understand and maintain.
- This also enabled to pull some common implementation code into the `ExecutableTransform` and by that reduce the implementation details of each executable to the minimum required.
* Also, extracted the `Transform.Parser` to its own top level class, and renamed it to - `TransformFactory`. The main thing that the factory does is: 1) delegate to the parsing to the `Transform` class, 2) construct & wire up the `ExecutableTransform`.
* With the introduction of `Transform`, we no longer need the `SourceBuilder` for transforms. Instead, we have `Transform.Builder` that help you build a transform. This is much more intuitive from the client perspective.
Original commit: elastic/x-pack-elasticsearch@f6ee0d0c75
* Tune the index templates for higher index throughput.
* Trigger events should be processed and indexed in an async manner.
* Moved the job name into the TriggerEvent
* Added support for fire multiple events at the same time. If multiple events are fired at the same time then use async bulk to persist the watch record.
* Cut simple ticker and timer ticker over to fire multiple events at the same time
* Don't fire more than 1000 events at the time. (this also may result in large bulk requests)
Original commit: elastic/x-pack-elasticsearch@c7f6bd3812
* Changed the isolated ScheduleEngineBenchmark to tests all engines and collect statistics during a test run.
* Updated the isolated schedule benchmark to just test the trigger part of the scheduler and capture some basic matrics.
Original commit: elastic/x-pack-elasticsearch@88bbf3688d
- `TimerScheduleTriggerEngine` - a single threaded Java `Timer`based scheduler. "Ticks" every second and checks all the registered schedules.
- `SimpleTickerScheduleTriggerEngine` - a single threaded scheduler. "Ticks" every second and checks all the registered schedules
- `SchedulerScheduleTriggerEngine` - a single threaded engine based on Java's schedule executor service. Here, every job is added as a scheduled task to the executor and each job is managing its own execution times.
- `HashWheelScheduleTriggerEngine` - a single threaded engine based on Netty's `HashWheelTimer`. Like with the `scheduler` above, every job is added as a scheduled task to the executor and each job is managing its own execution times.
Also:
- Added an undocumented feature to configure the schedule engine in the settings using `watcher.trigger.schedule.engine` (optional values right now are `quartz`, `simple`, `timer`, `hashwheel` and `scheduler`)
- `Cron` is a fork/copy of quartz `CronExpression`.. a bit cleaned up though.
- `Schedule` now exposes `nextScheduledTimeAfter` to return the next scheduled time after the given one.
- `CronnableSchedule` is now based on `Cron` (this exposed bugs in the schedule tests where we generated invalid cron expression. Now, since `Cronnable` creates the actual cron, validation is in place to make sure only valid expressions are created)
- While at it... refactored how the thread pool settings are set. Removed it from the plugin class, now each module is responsible for the settings of its own TPs. Also, if the thread pools are already configured in node settings we don't configure our default ones. This will enable users to configure the TPs in `elasticsearch.yml`
- Also updated `CronEvalTool` to work with `DateTime` construct (instead of java's `Date`)
Original commit: elastic/x-pack-elasticsearch@40d107c66e
* Split the action into two constructs: `Input` and `ExecutableInput`. The former holds all the input configuration, the latter can execute the input based on that configuration (an executable input holds an input)
- This the code clearer to understand and maintain.
- This also enabled to pull some common implementation code into the `ExecutableInput` and by that reduce the implementation details of each executable to the minimum required.
* Also, extracted the `Input.Parser` to its own top level class, and renamed it to - `InputFactory`. The main thing that the factory does is: 1) delegate to the parsing to the `Input` class, 2) construct & wire up the `ExecutableInput`.
* With the introduction of `Input`, we no longer need the `SourceBuilder` for inputs. Instead, we have an `Input.Builder` that help you build an input. This is much more intuitive from the client perspective.
* Changed the `request` xcontent field in the http input result to `sent_request` for clarity
* Changed the `request` xcontent field in the search input result to `executed_request` for clarity
Original commit: elastic/x-pack-elasticsearch@63b93f9c7b
The default mustache template that is supported by elasticsearch doesn't support array/list access. This poses a real limitation for watcher as with `search` input, the hits are returned as an array/list. To bypass this limitation, an extra (tedious) step is required in order to transform the hits to a data structure that is supported by mustache.
This commit adds a new mustache script engine - `xmustache` to elasticsearch that supports array/list access in the form of `array.X` where `X` is the index into the array/list. This enables accessing the search results without using a transform. The following example will fetch the `"key"` field of the 3rd hit in the search result: `ctx.payload.hits.hits.3._source.key`.
This array/list support will be added to elasticsearch, but it'll only be available in later versions. For now, the default template in watcher will therefore be `xmustache`.
Added docs for templates
Fixeselastic/elasticsearch#230
Original commit: elastic/x-pack-elasticsearch@b09cad7f8b
The watch metadata is now accessible to the templates and scripts (under `ctx.metadata`).
Also, changed the default email attachment to include the ctx model as a whole (not just the payload). This provides more context to the watch exectuion (e.g. watch id, execution time, metadata, etc...)
Original commit: elastic/x-pack-elasticsearch@c5dde855d2
The `WatchLockService` was not using fair locks. This could result in out of order execution of queue executions of the same watch.
Fork KeydLock tests from core and make global lock fair.
Fixeselastic/elasticsearch#225
Original commit: elastic/x-pack-elasticsearch@77382e09ca
* Split the action into two constructs: `Condition` and `ExecutableCondition`. The former holds all the condition configuration, the latter can execute the condition based on that configuration (an executable condition holds a condition)
- This the code clearer to understand and maintain.
- This also enabled to pull some common implementation code into the `ExecutableCondition` and by that reduce the implementation details of each executable to the minimum required.
* Also, extracted the `Condition.Parser` to its own top level class, and renamed it to - `ConditionFactory`. The main thing that the factory does is: 1) delegate to the parsing to the `Condition` class, 2) construct & wire up the `ExecutableCondition`.
* With the introduction of `Condition`, we no longer need the `SourceBuilder` for conditions. Instead, we have `Condiiton.Builder` that help you build condition. This is much more intuitive from the client perspective.
* Renamed `always_true` condition type to `always`
* Renamed `always_false` condition type to `never`
* Updated docs (also added docs for the `never` condition)
Original commit: elastic/x-pack-elasticsearch@0f8eb494ca
We weren't properly registering the chain or script transforms so that they were accessable from the Watch.parse and PUT Watch APIs.
Original commit: elastic/x-pack-elasticsearch@2a236d62f3
This change makes the default throttle period 5s.
This default can be controlled from config with the parameter `watcher.throttle.period.default_period`.
Added test and updated docs.
Original commit: elastic/x-pack-elasticsearch@cf8f5de724
This change persists the Watch.Status if needed by marking the Status as `dirty` if the status mutates during watch execution.
If the status is dirty it will be persisted at the end of execution while the execution lock is held.
Change record_in_history to record_execution which also controls weither the status will be updated.
Fixeselastic/elasticsearch#222
Original commit: elastic/x-pack-elasticsearch@25869cabf0
There were a few corner cases where a `HistoryStore.put` operation was underway while we were shutting down since we were not holding any lock during this time.
This change adds a ReadWriteLock and uses the ReadLock for updates and put operations and the WriteLock for shutdown.
Also add a lock around the manual execution of a watch since it could change the status.
Fixeselastic/elasticsearch#204
Original commit: elastic/x-pack-elasticsearch@60ef78eff5
If we failed to parse a `Watch` or `WatchRecord` at startup we currently get stuck as this behaviour is undefined.
This commit just adds some logging so we can tell that this is what happened.
Original commit: elastic/x-pack-elasticsearch@aa6d95c172
* Split the action into two constructs: `Action` and `ExecutableAction`. The former holds all the action configuration, the latter can execute the action based on that configuration (an executable action holds an action)
- This the code clearer to understand and maintain.
- This also enabled to pull some common implementation code into the `ExecutableAction` and by that reduce the implementation details of each executable to the minimum required.
* Also, extracted teh `Action.Parser` to its own top level class, and renamed it to - `ActionFactory`. The main thing that the parser does is: 1) delegate to the parsing to the `Action` class, 2) construct & wire up the `ExecutableAction`.
* Cleaned up and aligned the log messages across all actions. Introduced
- introduced parameterized exceptions (now the exception messages are constructed exactly like that log messages - using `{}` params)
- Introduced the notion of `Wid` - Watch (execution) ID`. Now the `WatchExecutionContext` and the `WatchRecord` hold this construct as their ID (instead of an opaque string). The Wid has a well defined structure where it's prefixed with the watch id. This is very helpful to provide context to parsing the different constructs (actions in our case) as when parsing error occurs we can point directly to the place were we found the error (ie. watch id, record id and action id).
* With the introduction of `Action`, we no longer need the `SourceBuilder` for actions. Instead, we have `Action.Builder` that help you build actions. This is much more intuitive from the client perspective.
Original commit: elastic/x-pack-elasticsearch@df22da4e1f
This change adds a new test class that runs the REST tests with the license disabled.
It checks the error message to make sure that all api access is blocked from all the apis we normally test.
Fixeselastic/elasticsearch#193
Original commit: elastic/x-pack-elasticsearch@a70de9b7a7
`
{
"script" : "<script>"
}
`
Should be a valid script condition. This was silently failing to parse creating invalid watches in the system that could not be executed or interacted with except via DELETE.
Added a test for this as well.
Fixeselastic/elasticsearch#191
Original commit: elastic/x-pack-elasticsearch@25ed27bf2e
- Introduced `TemplateEngine` and its only implementation `MustacheTemplateEngine`
- Change `Template` to only hold the template config.. the rendering is now done by the engine
- Introduced `EmailTemplate` that can render itself to an `Email` (given an engine and model)
- Renamed `TemplatedHttpRequest` to `HttpRequestTemplate` and it has a `render` method that given a model and an engine it creates an `HttpRequest` construct.
- also fixed a bug where when the http request is build, the configured scheme wasn't set.
- cleaned up `HttpRequest` - made it immutable and now we use a builder to build it.
- also fixed a bug in `HttpRequest` where the xcontent didn't render the scheme
- Got rid of all old template implementations... also, parsing a template is now simply `Template.parse(...)` (no need for a special `TemplateParser`.
- While at it, change the action builder to only deal with actions and not know anything about the action id. The id is now passed to the watch builder directly when adding an action
Original commit: elastic/x-pack-elasticsearch@1a7acc5868
- What used to be the `HistoryService` is not named `ExecutionService`. In fact, it always was an execution context and was simply named wrong and place in the wrog place in the code.
- A new `execution` package was created, to contain all execution related contsructs which includes the new `ExecutionService`, `WatchExecutionContext`, `WatchExecutor`, `InternalWatchExecutor`.
- The `WatchExecutionContext` role in the execution process was extended to also enable it to "control" the execution. "control" in the sence that it can now provide default results to some of the contructs (for example, it can provide a default input and by that skip the execution of the input). Also, it enables simulating actions (more bellow).
- There are two implementations now of the `WatchExecutionContext`. A `TriggeredExecutionContext` which is used when the trigger service triggers the execution of a watch. And `ManualExecutionContext` enabling to manually execute a watch. The latter, can be configured such that different phases in the normal executions of the watch can be skipped. This is great for simulating a watch execution, that is, executing a watch without actually fully executing the actions (e.g. create the emails that the email action would have normally send, but without actually sending it). Both of these new context implementations also sit in the new `execution` package.
- With the changes described above, the actions now consult the execution context before actually performing their tasks. If the execution context indicates that the action should run in a "simulated" mode, the action will semi execute and return a `Simulated` result. Each action type defines what is "simulated mode" for it. For example, the email action creates the email and renders all its templats but doesn't actually send it, the returned `Email.Simulated` result will contain the email that would have been sent if it was running under normal conditions (non-simulated mode). The `index` action, doesn't actually indexes anything (just creating the index request that would have been executed), and the `webhook` action only creates the HTTP request but doesn't actually send it.
- A new `execute` API was introduced that utilizes all of the above and enables to manually execute registered watches. The API provides all the tools to control what actions are simulated and enables skipping the execution of the watch `input` and `condition`. By default it also skip the recording of the execution in the watch history (though this can be enabled if desired). This API is exposed both by the Java `WatcherClient` and via a dedicated REST endpoint. (documentation around this new API was added as well)
Closeselastic/elasticsearch#149
Original commit: elastic/x-pack-elasticsearch@ee8fe0490a
something seems to be broken... either in the integration test infra or somewhere else.. .investigation required
Original commit: elastic/x-pack-elasticsearch@c27bc41ef4
- Added watch CRUD tests + cleanup
- Changed and aligned terminology across the board. No more "watch name".. from now on we'll refer to it as "watch id" (aligned with the public APIs). Also the templates now refer to `"{{ctx.watch_id}}"` (instead of `"{{ctx.watch_name}}"`)
- cleaned up the Response constructs (removed dependency on index/delete/get responses)
Fixeselastic/elasticsearch#202
Original commit: elastic/x-pack-elasticsearch@3296a69128
Super useful when dealing with the `cron` schedule. It enables to:
1. validate a cron expression. If the cron expression is incorrect it will try to output a descriptive/guiding error message
2. show future times in which the cron expression will be triggered (based on the current time)
When deployed, once can access this tool from ES_HOME by `bin/watcher/croneval`
Also updated the docs
Original commit: elastic/x-pack-elasticsearch@2666c32a69
This change adds support for testing all watcher REST endpoints.
It also updates the api docs to be current with the latest code.
Change GetWatchResponse to only have the information in needs
GetWatchResponse used to contain a GetResponse this is not needed. Now it just contains the needed fields.
Closeselastic/elasticsearch#35
Original commit: elastic/x-pack-elasticsearch@905c5da318
- When the license is disabled, all actions of all watches are throttled, and all watcher APIs are blocked.
- before license expiry we're logging warning messages to the standard log file.
- Added a check on startup verifying the installed license plugin confirms to the min compatibility requirement of watcher
- Added a check on startup verifying the installed shield plugin (if one is installed) confirms to the min compatibility requirement of watcher
- Fixed a bug in simple input deserialization where it expected a `payload` field that should be there.
- Fixed a bug in `WatcherServiceResponse` where the acknowledged state wasn't de/serialized
Closeselastic/elasticsearch#30
Original commit: elastic/x-pack-elasticsearch@35dcaf4feb
Mainly useful for testing & debugging, but might be even useful as an actual action in production.
(for now, we won't document it)
Original commit: elastic/x-pack-elasticsearch@726049cece
- actions are not identified by a unique id. The actions in a "watch" definition are now keyed (instead of an array)
- the `ActionWrapper` is now a construct that holds: and action, the id of the action and optionally a transform
- the `webhook` action is now structured based on the templated http request it holds (no need for extra `request` element there`
- the `webhook` reponse holds the http response (not the `status` and `body` directly)
- the action builders now accept an `id` when created
- introduced `WatchSourceBuilders` for quickly access various source builders
- introduced `Template.SourceBuilder` to help building a template
- changed templated http request builder to accept template source builders (instead of templates directly)
- changed `HttpResponse` to accept the body as a `ByteReference` in the ctor (this construct used to be in an inconsistent state holding an input stream that could have already been read)
Original commit: elastic/x-pack-elasticsearch@296350b6a8
- now enables defining `types` (for document types)
- enables defining `types` and `indices` as comma-delimited strings (not just string arrays)
- aligned the parsing in `WatcherUtils` with the way we're parsing xcontent across the board (e.g. using `ParseField`)
- Added additional unit test to test deserialization
Original commit: elastic/x-pack-elasticsearch@5491b85f75
This commit enables the script transform return any value. If the value is of type `Map` we'll just use it as the payload itself. Otherwise (any other value type) we'll take that value and put it in a map under the `"_value"` key, and that map will be the new payload.
This will simplify writing scripts (as it doesn't require the script to build a map)
Original commit: elastic/x-pack-elasticsearch@73fab7dc2b
uboness
Brian Murphy
Martijn van Groningen
Bill Hwang