SearchInput using inline, indexed and on disk templates.
ScriptedCondition where the script accesses the ctx.
ScriptedCondition where the script throws an exception.
ScriptedCondition where the script doesn't return a boolean.
Webhook tests using templated body, path and parameters.
Some REST test fixes.
Original commit: elastic/x-pack-elasticsearch@d02b6d1d7b
The HttpInputIntegrationTest sometimes unsetted the 'shieldEnabled' field for subsequent test methods, this caused http 401 errors, because the nodes were started with Shield enabled.
Original commit: elastic/x-pack-elasticsearch@b39ca9647d
This change employs the same kind of logic that `HttpClientTest` uses to find an open port for it's server. This should prevent the rare build failures we have seen.
Original commit: elastic/x-pack-elasticsearch@f3b68adad5
If there were many watches concurrently executing that take some time to execute it can a long time to shutdown a node. This change introduces a Timeout when stopping the WatchLockService.
Make watcher lock service timeout configurable. The configuration setting is `watcher.stop.timeout` the default is 30s.
Fixeselastic/elasticsearch#216
Original commit: elastic/x-pack-elasticsearch@06bf029a54
* The validation happens separately from the actual starting and loading. Also the validation happens on the cluster update thread, so that we don't miss any cluster state update.
* The starting/loading part happens on a forked thread, but if it fails then it always retry after a small timeout.
* Simplified the starting code, so that we don't need the callback interface.
Closeselastic/elasticsearch#212
Original commit: elastic/x-pack-elasticsearch@b5cd48e5bb
Before this change DateTimes were being constructed without providing a time zone,
this was causing some non-utc time leakage. In particular watch record ids were being created with non utc dates and
watch records were going to the wrong .watch_history index.
Add Clock.now(DateTimeZone) to allow callers to get UTC now.
Also construct the DateTimes with UTC timezones when we construct from millis.
Add all constuctors of DateTime that do not specify a time zone to the forbidden APIs.
This change makes constructing a `DateTime` object without providing a `DateTimeZone` forbidden.
This is much safer and makes time zone errors much more unlikely to occur.
Statically import DateTimeZone.UTC everywhere it was being used
Now all calls that use DateTimeZone.UTC just refrence UTC.
Fixeselastic/elasticsearch#150
Original commit: elastic/x-pack-elasticsearch@7f23ce605e
Mainly how templates read/write themselves from/to xcontent. Instead of using `text`, use `template`
Original commit: elastic/x-pack-elasticsearch@0d6f317539
* Split the transform into two constructs: `Transform` and `ExecutableTransform`. The former holds all the transform configuration, the latter can execute the transform based on that configuration (an executable transform holds a transform)
- This makes the code clearer to understand and maintain.
- This also enabled to pull some common implementation code into the `ExecutableTransform` and by that reduce the implementation details of each executable to the minimum required.
* Also, extracted the `Transform.Parser` to its own top level class, and renamed it to - `TransformFactory`. The main thing that the factory does is: 1) delegate to the parsing to the `Transform` class, 2) construct & wire up the `ExecutableTransform`.
* With the introduction of `Transform`, we no longer need the `SourceBuilder` for transforms. Instead, we have `Transform.Builder` that help you build a transform. This is much more intuitive from the client perspective.
Original commit: elastic/x-pack-elasticsearch@f6ee0d0c75
* Tune the index templates for higher index throughput.
* Trigger events should be processed and indexed in an async manner.
* Moved the job name into the TriggerEvent
* Added support for fire multiple events at the same time. If multiple events are fired at the same time then use async bulk to persist the watch record.
* Cut simple ticker and timer ticker over to fire multiple events at the same time
* Don't fire more than 1000 events at the time. (this also may result in large bulk requests)
Original commit: elastic/x-pack-elasticsearch@c7f6bd3812
* Changed the isolated ScheduleEngineBenchmark to tests all engines and collect statistics during a test run.
* Updated the isolated schedule benchmark to just test the trigger part of the scheduler and capture some basic matrics.
Original commit: elastic/x-pack-elasticsearch@88bbf3688d
- `TimerScheduleTriggerEngine` - a single threaded Java `Timer`based scheduler. "Ticks" every second and checks all the registered schedules.
- `SimpleTickerScheduleTriggerEngine` - a single threaded scheduler. "Ticks" every second and checks all the registered schedules
- `SchedulerScheduleTriggerEngine` - a single threaded engine based on Java's schedule executor service. Here, every job is added as a scheduled task to the executor and each job is managing its own execution times.
- `HashWheelScheduleTriggerEngine` - a single threaded engine based on Netty's `HashWheelTimer`. Like with the `scheduler` above, every job is added as a scheduled task to the executor and each job is managing its own execution times.
Also:
- Added an undocumented feature to configure the schedule engine in the settings using `watcher.trigger.schedule.engine` (optional values right now are `quartz`, `simple`, `timer`, `hashwheel` and `scheduler`)
- `Cron` is a fork/copy of quartz `CronExpression`.. a bit cleaned up though.
- `Schedule` now exposes `nextScheduledTimeAfter` to return the next scheduled time after the given one.
- `CronnableSchedule` is now based on `Cron` (this exposed bugs in the schedule tests where we generated invalid cron expression. Now, since `Cronnable` creates the actual cron, validation is in place to make sure only valid expressions are created)
- While at it... refactored how the thread pool settings are set. Removed it from the plugin class, now each module is responsible for the settings of its own TPs. Also, if the thread pools are already configured in node settings we don't configure our default ones. This will enable users to configure the TPs in `elasticsearch.yml`
- Also updated `CronEvalTool` to work with `DateTime` construct (instead of java's `Date`)
Original commit: elastic/x-pack-elasticsearch@40d107c66e
* Split the action into two constructs: `Input` and `ExecutableInput`. The former holds all the input configuration, the latter can execute the input based on that configuration (an executable input holds an input)
- This the code clearer to understand and maintain.
- This also enabled to pull some common implementation code into the `ExecutableInput` and by that reduce the implementation details of each executable to the minimum required.
* Also, extracted the `Input.Parser` to its own top level class, and renamed it to - `InputFactory`. The main thing that the factory does is: 1) delegate to the parsing to the `Input` class, 2) construct & wire up the `ExecutableInput`.
* With the introduction of `Input`, we no longer need the `SourceBuilder` for inputs. Instead, we have an `Input.Builder` that help you build an input. This is much more intuitive from the client perspective.
* Changed the `request` xcontent field in the http input result to `sent_request` for clarity
* Changed the `request` xcontent field in the search input result to `executed_request` for clarity
Original commit: elastic/x-pack-elasticsearch@63b93f9c7b
The default mustache template that is supported by elasticsearch doesn't support array/list access. This poses a real limitation for watcher as with `search` input, the hits are returned as an array/list. To bypass this limitation, an extra (tedious) step is required in order to transform the hits to a data structure that is supported by mustache.
This commit adds a new mustache script engine - `xmustache` to elasticsearch that supports array/list access in the form of `array.X` where `X` is the index into the array/list. This enables accessing the search results without using a transform. The following example will fetch the `"key"` field of the 3rd hit in the search result: `ctx.payload.hits.hits.3._source.key`.
This array/list support will be added to elasticsearch, but it'll only be available in later versions. For now, the default template in watcher will therefore be `xmustache`.
Added docs for templates
Fixeselastic/elasticsearch#230
Original commit: elastic/x-pack-elasticsearch@b09cad7f8b
The watch metadata is now accessible to the templates and scripts (under `ctx.metadata`).
Also, changed the default email attachment to include the ctx model as a whole (not just the payload). This provides more context to the watch exectuion (e.g. watch id, execution time, metadata, etc...)
Original commit: elastic/x-pack-elasticsearch@c5dde855d2
The `WatchLockService` was not using fair locks. This could result in out of order execution of queue executions of the same watch.
Fork KeydLock tests from core and make global lock fair.
Fixeselastic/elasticsearch#225
Original commit: elastic/x-pack-elasticsearch@77382e09ca
* Split the action into two constructs: `Condition` and `ExecutableCondition`. The former holds all the condition configuration, the latter can execute the condition based on that configuration (an executable condition holds a condition)
- This the code clearer to understand and maintain.
- This also enabled to pull some common implementation code into the `ExecutableCondition` and by that reduce the implementation details of each executable to the minimum required.
* Also, extracted the `Condition.Parser` to its own top level class, and renamed it to - `ConditionFactory`. The main thing that the factory does is: 1) delegate to the parsing to the `Condition` class, 2) construct & wire up the `ExecutableCondition`.
* With the introduction of `Condition`, we no longer need the `SourceBuilder` for conditions. Instead, we have `Condiiton.Builder` that help you build condition. This is much more intuitive from the client perspective.
* Renamed `always_true` condition type to `always`
* Renamed `always_false` condition type to `never`
* Updated docs (also added docs for the `never` condition)
Original commit: elastic/x-pack-elasticsearch@0f8eb494ca
We weren't properly registering the chain or script transforms so that they were accessable from the Watch.parse and PUT Watch APIs.
Original commit: elastic/x-pack-elasticsearch@2a236d62f3
This change makes the default throttle period 5s.
This default can be controlled from config with the parameter `watcher.throttle.period.default_period`.
Added test and updated docs.
Original commit: elastic/x-pack-elasticsearch@cf8f5de724
This change persists the Watch.Status if needed by marking the Status as `dirty` if the status mutates during watch execution.
If the status is dirty it will be persisted at the end of execution while the execution lock is held.
Change record_in_history to record_execution which also controls weither the status will be updated.
Fixeselastic/elasticsearch#222
Original commit: elastic/x-pack-elasticsearch@25869cabf0
There were a few corner cases where a `HistoryStore.put` operation was underway while we were shutting down since we were not holding any lock during this time.
This change adds a ReadWriteLock and uses the ReadLock for updates and put operations and the WriteLock for shutdown.
Also add a lock around the manual execution of a watch since it could change the status.
Fixeselastic/elasticsearch#204
Original commit: elastic/x-pack-elasticsearch@60ef78eff5
Martijn van Groningen
Brian Murphy
uboness