2017-06-27 20:16:51 -04:00
|
|
|
[role="xpack"]
|
2017-03-29 12:07:55 -04:00
|
|
|
[[watcher-api-execute-watch]]
|
2018-12-20 13:23:28 -05:00
|
|
|
=== Execute watch API
|
|
|
|
++++
|
|
|
|
<titleabbrev>Execute watch</titleabbrev>
|
|
|
|
++++
|
2017-03-29 12:07:55 -04:00
|
|
|
|
|
|
|
The execute watch API forces the execution of a stored watch. It can be used to
|
|
|
|
force execution of the watch outside of its triggering logic, or to simulate the
|
|
|
|
watch execution for debugging purposes.
|
|
|
|
|
2017-09-14 16:01:47 -04:00
|
|
|
[float]
|
|
|
|
==== Request
|
2017-03-29 12:07:55 -04:00
|
|
|
|
2018-12-08 13:57:16 -05:00
|
|
|
`POST _watcher/watch/<watch_id>/_execute` +
|
2017-09-14 16:01:47 -04:00
|
|
|
|
2018-12-08 13:57:16 -05:00
|
|
|
`POST _watcher/watch/_execute`
|
2017-09-14 16:01:47 -04:00
|
|
|
|
|
|
|
[float]
|
|
|
|
==== Description
|
|
|
|
|
|
|
|
For testing and debugging purposes, you also have fine-grained control on how
|
|
|
|
the watch runs. You can execute the watch without executing all of its actions
|
|
|
|
or alternatively by simulating them. You can also force execution by ignoring
|
|
|
|
the watch condition and control whether a watch record would be written to the
|
|
|
|
watch history after execution.
|
|
|
|
|
|
|
|
[float]
|
|
|
|
[[watcher-api-execute-inline-watch]]
|
|
|
|
===== Inline Watch Execution
|
|
|
|
|
|
|
|
You can use the Execute API to execute watches that are not yet registered by
|
|
|
|
specifying the watch definition inline. This serves as great tool for testing
|
|
|
|
and debugging your watches prior to adding them to {watcher}.
|
|
|
|
|
|
|
|
[float]
|
|
|
|
==== Path Parameters
|
|
|
|
|
|
|
|
`watch_id`::
|
|
|
|
(string) Identifier for the watch.
|
|
|
|
|
|
|
|
[float]
|
|
|
|
==== Query Parameters
|
|
|
|
|
|
|
|
`debug`::
|
|
|
|
(boolean) Defines whether the watch runs in debug mode. The default value is
|
|
|
|
`false`.
|
2017-03-29 12:07:55 -04:00
|
|
|
|
2017-09-14 16:01:47 -04:00
|
|
|
[float]
|
|
|
|
==== Request Body
|
2017-03-29 12:07:55 -04:00
|
|
|
|
|
|
|
This API supports the following fields:
|
|
|
|
|
|
|
|
[cols=",^,^,", options="header"]
|
|
|
|
|======
|
|
|
|
| Name | Required | Default | Description
|
|
|
|
|
|
|
|
| `trigger_data` | no | | This structure is parsed as the data of the trigger event
|
|
|
|
that will be used during the watch execution
|
|
|
|
|
|
|
|
| `ignore_condition` | no | false | When set to `true`, the watch execution uses the
|
2018-12-19 17:53:37 -05:00
|
|
|
{stack-ov}/condition-always.html[Always Condition].
|
2018-09-17 15:35:55 -04:00
|
|
|
This can also be specified as an HTTP parameter.
|
2017-03-29 12:07:55 -04:00
|
|
|
|
|
|
|
| `alternative_input` | no | null | When present, the watch uses this object as a payload
|
|
|
|
instead of executing its own input.
|
|
|
|
|
|
|
|
| `action_modes` | no | null | Determines how to handle the watch actions as part of the
|
|
|
|
watch execution. See <<watcher-api-execute-watch-action-mode, Action Execution Modes>>
|
|
|
|
for more information.
|
|
|
|
|
|
|
|
| `record_execution` | no | false | When set to `true`, the watch record representing the watch
|
|
|
|
execution result is persisted to the `.watcher-history`
|
|
|
|
index for the current time. In addition, the status of the
|
|
|
|
watch is updated, possibly throttling subsequent executions.
|
2018-09-17 15:35:55 -04:00
|
|
|
This can also be specified as an HTTP parameter.
|
2017-03-29 12:07:55 -04:00
|
|
|
|
2017-06-27 20:16:51 -04:00
|
|
|
| `watch` | no | null | When present, this
|
2018-12-19 17:53:37 -05:00
|
|
|
{stack-ov}/how-watcher-works.html#watch-definition[watch] is used
|
2017-03-29 12:07:55 -04:00
|
|
|
instead of the one specified in the request. This watch is
|
|
|
|
not persisted to the index and record_execution cannot be set.
|
|
|
|
|======
|
|
|
|
|
2017-09-14 16:01:47 -04:00
|
|
|
[float]
|
|
|
|
[[watcher-api-execute-watch-action-mode]]
|
|
|
|
===== Action Execution Modes
|
|
|
|
|
|
|
|
Action modes define how actions are handled during the watch execution. There
|
|
|
|
are five possible modes an action can be associated with:
|
|
|
|
|
|
|
|
[options="header"]
|
|
|
|
|======
|
|
|
|
| Name | Description
|
|
|
|
|
|
|
|
| `simulate` | The action execution is simulated. Each action type
|
|
|
|
define its own simulation operation mode. For example, the
|
2018-12-19 17:53:37 -05:00
|
|
|
{stack-ov}/actions-email.html[email] action creates
|
2017-09-14 16:01:47 -04:00
|
|
|
the email that would have been sent but does not actually
|
|
|
|
send it. In this mode, the action might be throttled if the
|
|
|
|
current state of the watch indicates it should be.
|
|
|
|
|
|
|
|
| `force_simulate` | Similar to the the `simulate` mode, except the action is
|
|
|
|
not be throttled even if the current state of the watch
|
|
|
|
indicates it should be.
|
|
|
|
|
|
|
|
| `execute` | Executes the action as it would have been executed if the
|
|
|
|
watch would have been triggered by its own trigger. The
|
|
|
|
execution might be throttled if the current state of the
|
|
|
|
watch indicates it should be.
|
|
|
|
|
|
|
|
| `force_execute` | Similar to the `execute` mode, except the action is not
|
|
|
|
throttled even if the current state of the watch indicates
|
|
|
|
it should be.
|
|
|
|
|
|
|
|
| `skip` | The action is skipped and is not executed or simulated.
|
|
|
|
Effectively forces the action to be throttled.
|
|
|
|
|======
|
|
|
|
|
|
|
|
[float]
|
|
|
|
==== Authorization
|
|
|
|
You must have `manage_watcher` cluster privileges to use this API. For more
|
2018-12-19 17:53:37 -05:00
|
|
|
information, see {stack-ov}/security-privileges.html[Security Privileges].
|
2017-09-14 16:01:47 -04:00
|
|
|
|
2017-11-29 08:48:06 -05:00
|
|
|
[float]
|
|
|
|
==== Security Integration
|
|
|
|
|
2018-12-19 17:53:37 -05:00
|
|
|
When {es} {security-features} are enabled on your cluster, watches
|
|
|
|
are executed with the privileges of the user that stored the watches. If your
|
|
|
|
user is allowed to read index `a`, but not index `b`, then the exact same set of
|
2017-11-29 08:48:06 -05:00
|
|
|
rules will apply during execution of a watch.
|
|
|
|
|
|
|
|
When using the execute watch API, the authorization data of the user that
|
|
|
|
called the API will be used as a base, instead of of the information who stored
|
|
|
|
the watch.
|
|
|
|
|
2017-09-14 16:01:47 -04:00
|
|
|
[float]
|
|
|
|
==== Examples
|
|
|
|
|
|
|
|
The following example executes the `my_watch` watch:
|
|
|
|
|
|
|
|
[source,js]
|
|
|
|
--------------------------------------------------
|
2018-12-08 13:57:16 -05:00
|
|
|
POST _watcher/watch/my_watch/_execute
|
2017-09-14 16:01:47 -04:00
|
|
|
--------------------------------------------------
|
|
|
|
// CONSOLE
|
|
|
|
// TEST[setup:my_active_watch]
|
|
|
|
|
2017-03-29 12:07:55 -04:00
|
|
|
The following example shows a comprehensive example of executing the `my-watch` watch:
|
|
|
|
|
|
|
|
[source,js]
|
|
|
|
--------------------------------------------------
|
2018-12-08 13:57:16 -05:00
|
|
|
POST _watcher/watch/my_watch/_execute
|
2017-03-29 12:07:55 -04:00
|
|
|
{
|
|
|
|
"trigger_data" : { <1>
|
|
|
|
"triggered_time" : "now",
|
|
|
|
"scheduled_time" : "now"
|
|
|
|
},
|
|
|
|
"alternative_input" : { <2>
|
|
|
|
"foo" : "bar"
|
|
|
|
},
|
|
|
|
"ignore_condition" : true, <3>
|
|
|
|
"action_modes" : {
|
|
|
|
"my-action" : "force_simulate" <4>
|
|
|
|
},
|
|
|
|
"record_execution" : true <5>
|
|
|
|
}
|
|
|
|
--------------------------------------------------
|
|
|
|
// CONSOLE
|
|
|
|
// TEST[setup:my_active_watch]
|
|
|
|
<1> The triggered and schedule times are provided.
|
|
|
|
<2> The input as defined by the watch is ignored and instead the provided input
|
2017-09-14 16:01:47 -04:00
|
|
|
is used as the execution payload.
|
|
|
|
<3> The condition as defined by the watch is ignored and is assumed to
|
2017-03-29 12:07:55 -04:00
|
|
|
evaluate to `true`.
|
|
|
|
<4> Forces the simulation of `my-action`. Forcing the simulation means that
|
|
|
|
throttling is ignored and the watch is simulated by {watcher} instead of
|
|
|
|
being executed normally.
|
2017-09-14 16:01:47 -04:00
|
|
|
<5> The execution of the watch creates a watch record in the watch history,
|
|
|
|
and the throttling state of the watch is potentially updated accordingly.
|
2017-03-29 12:07:55 -04:00
|
|
|
|
|
|
|
This is an example of the output:
|
|
|
|
|
|
|
|
[source,js]
|
|
|
|
--------------------------------------------------
|
|
|
|
{
|
|
|
|
"_id": "my_watch_0-2015-06-02T23:17:55.124Z", <1>
|
|
|
|
"watch_record": { <2>
|
|
|
|
"watch_id": "my_watch",
|
Watcher: Distributed watch execution (elastic/x-pack-elasticsearch#544)
The distribution of watches now happens on the node which holds the
watches index, instead of on the master node. This requires several
changes to the current implementation.
1. Running on shards and replicas
In order to run watches on the nodes with the watches index on its
primaries and replicas. To ensure that watches do not run twice, there is
a logic which checks the local shards, runs a murmurhash on the id and
runs modulo against the number of shards and replicas, this is the way to
find out, if a watch should run local. Reloading happens
2. Several master node actions moved to a HandledTransportAction, as they
are basically just aliases for indexing actions, among them the
put/delete/get watch actions, the acknowledgement action, the de/activate
actions
3. Stats action moved to a broadcast node action, because we potentially
have to query every node to get watcher statistics
4. Starting/Stopping watcher now is a master node action, which updates
the cluster state and then listeners acts on those. Because of this watches
can be running on two systems, if you those have different cluster state
versions, until the new watcher state is propagated
5. Watcher is started on all nodes now. With the exception of the ticker
schedule engine most classes do not need a lot of resources while running.
However they have to run, because of the execute watch API, which can hit
any node - it does not make sense to find the right shard for this watch
and only then execute (as this also has to work with a watch, that has not
been stored before)
6. By using a indexing operation listener, each storing of a watch now
parses the watch first and only stores on successful parsing
7. Execute watch API now uses the watcher threadpool for execution
8. Getting the number of watches for the stats now simply queries the
different execution engines, how many watches are scheduled, so this is
not doing a search anymore
There will be follow up commits on this one, mainly to ensure BWC compatibility.
Original commit: elastic/x-pack-elasticsearch@0adb46e6589d4ec410739105dc09a26256f51a98
2017-05-02 04:12:46 -04:00
|
|
|
"node": "my_node",
|
2017-03-29 12:07:55 -04:00
|
|
|
"messages": [],
|
|
|
|
"trigger_event": {
|
|
|
|
"type": "manual",
|
|
|
|
"triggered_time": "2015-06-02T23:17:55.124Z",
|
|
|
|
"manual": {
|
|
|
|
"schedule": {
|
|
|
|
"scheduled_time": "2015-06-02T23:17:55.124Z"
|
|
|
|
}
|
|
|
|
}
|
|
|
|
},
|
|
|
|
"state": "executed",
|
2017-05-04 04:08:34 -04:00
|
|
|
"status": {
|
2017-03-29 12:07:55 -04:00
|
|
|
"version": 1,
|
2017-10-10 03:07:33 -04:00
|
|
|
"execution_state": "executed",
|
2017-03-29 12:07:55 -04:00
|
|
|
"state": {
|
|
|
|
"active": true,
|
|
|
|
"timestamp": "2015-06-02T23:17:55.111Z"
|
|
|
|
},
|
|
|
|
"last_checked": "2015-06-02T23:17:55.124Z",
|
|
|
|
"last_met_condition": "2015-06-02T23:17:55.124Z",
|
|
|
|
"actions": {
|
|
|
|
"test_index": {
|
|
|
|
"ack": {
|
|
|
|
"timestamp": "2015-06-02T23:17:55.124Z",
|
|
|
|
"state": "ackable"
|
|
|
|
},
|
|
|
|
"last_execution": {
|
|
|
|
"timestamp": "2015-06-02T23:17:55.124Z",
|
|
|
|
"successful": true
|
|
|
|
},
|
|
|
|
"last_successful_execution": {
|
|
|
|
"timestamp": "2015-06-02T23:17:55.124Z",
|
|
|
|
"successful": true
|
|
|
|
}
|
|
|
|
}
|
|
|
|
}
|
|
|
|
},
|
|
|
|
"input": {
|
|
|
|
"simple": {
|
|
|
|
"payload": {
|
|
|
|
"send": "yes"
|
|
|
|
}
|
|
|
|
}
|
|
|
|
},
|
|
|
|
"condition": {
|
|
|
|
"always": {}
|
|
|
|
},
|
|
|
|
"result": { <3>
|
|
|
|
"execution_time": "2015-06-02T23:17:55.124Z",
|
|
|
|
"execution_duration": 12608,
|
|
|
|
"input": {
|
|
|
|
"type": "simple",
|
|
|
|
"payload": {
|
|
|
|
"foo": "bar"
|
|
|
|
},
|
|
|
|
"status": "success"
|
|
|
|
},
|
|
|
|
"condition": {
|
|
|
|
"type": "always",
|
|
|
|
"met": true,
|
|
|
|
"status": "success"
|
|
|
|
},
|
|
|
|
"actions": [
|
|
|
|
{
|
|
|
|
"id": "test_index",
|
|
|
|
"index": {
|
|
|
|
"response": {
|
|
|
|
"index": "test",
|
|
|
|
"type": "test2",
|
|
|
|
"version": 1,
|
|
|
|
"created": true,
|
|
|
|
"result": "created",
|
|
|
|
"id": "AVSHKzPa9zx62AzUzFXY"
|
|
|
|
}
|
|
|
|
},
|
|
|
|
"status": "success",
|
|
|
|
"type": "index"
|
|
|
|
}
|
|
|
|
]
|
2018-07-16 15:20:16 -04:00
|
|
|
},
|
|
|
|
"user": "test_admin" <4>
|
2017-03-29 12:07:55 -04:00
|
|
|
}
|
|
|
|
}
|
|
|
|
--------------------------------------------------
|
|
|
|
// TESTRESPONSE[s/my_watch_0-2015-06-02T23:17:55.124Z/$body._id/]
|
|
|
|
// TESTRESPONSE[s/"triggered_time": "2015-06-02T23:17:55.124Z"/"triggered_time": "$body.watch_record.trigger_event.triggered_time"/]
|
|
|
|
// TESTRESPONSE[s/"scheduled_time": "2015-06-02T23:17:55.124Z"/"scheduled_time": "$body.watch_record.trigger_event.manual.schedule.scheduled_time"/]
|
|
|
|
// TESTRESPONSE[s/"execution_time": "2015-06-02T23:17:55.124Z"/"execution_time": "$body.watch_record.result.execution_time"/]
|
2017-05-04 04:08:34 -04:00
|
|
|
// TESTRESPONSE[s/"timestamp": "2015-06-02T23:17:55.111Z"/"timestamp": "$body.watch_record.status.state.timestamp"/]
|
|
|
|
// TESTRESPONSE[s/"timestamp": "2015-06-02T23:17:55.124Z"/"timestamp": "$body.watch_record.status.actions.test_index.ack.timestamp"/]
|
|
|
|
// TESTRESPONSE[s/"last_checked": "2015-06-02T23:17:55.124Z"/"last_checked": "$body.watch_record.status.last_checked"/]
|
|
|
|
// TESTRESPONSE[s/"last_met_condition": "2015-06-02T23:17:55.124Z"/"last_met_condition": "$body.watch_record.status.last_met_condition"/]
|
2017-03-29 12:07:55 -04:00
|
|
|
// TESTRESPONSE[s/"execution_duration": 12608/"execution_duration": "$body.watch_record.result.execution_duration"/]
|
|
|
|
// TESTRESPONSE[s/"id": "AVSHKzPa9zx62AzUzFXY"/"id": "$body.watch_record.result.actions.0.index.response.id"/]
|
Watcher: Distributed watch execution (elastic/x-pack-elasticsearch#544)
The distribution of watches now happens on the node which holds the
watches index, instead of on the master node. This requires several
changes to the current implementation.
1. Running on shards and replicas
In order to run watches on the nodes with the watches index on its
primaries and replicas. To ensure that watches do not run twice, there is
a logic which checks the local shards, runs a murmurhash on the id and
runs modulo against the number of shards and replicas, this is the way to
find out, if a watch should run local. Reloading happens
2. Several master node actions moved to a HandledTransportAction, as they
are basically just aliases for indexing actions, among them the
put/delete/get watch actions, the acknowledgement action, the de/activate
actions
3. Stats action moved to a broadcast node action, because we potentially
have to query every node to get watcher statistics
4. Starting/Stopping watcher now is a master node action, which updates
the cluster state and then listeners acts on those. Because of this watches
can be running on two systems, if you those have different cluster state
versions, until the new watcher state is propagated
5. Watcher is started on all nodes now. With the exception of the ticker
schedule engine most classes do not need a lot of resources while running.
However they have to run, because of the execute watch API, which can hit
any node - it does not make sense to find the right shard for this watch
and only then execute (as this also has to work with a watch, that has not
been stored before)
6. By using a indexing operation listener, each storing of a watch now
parses the watch first and only stores on successful parsing
7. Execute watch API now uses the watcher threadpool for execution
8. Getting the number of watches for the stats now simply queries the
different execution engines, how many watches are scheduled, so this is
not doing a search anymore
There will be follow up commits on this one, mainly to ensure BWC compatibility.
Original commit: elastic/x-pack-elasticsearch@0adb46e6589d4ec410739105dc09a26256f51a98
2017-05-02 04:12:46 -04:00
|
|
|
// TESTRESPONSE[s/"node": "my_node"/"node": "$body.watch_record.node"/]
|
2017-03-29 12:07:55 -04:00
|
|
|
<1> The id of the watch record as it would be stored in the `.watcher-history` index.
|
|
|
|
<2> The watch record document as it would be stored in the `.watcher-history` index.
|
|
|
|
<3> The watch execution results.
|
2018-07-16 15:20:16 -04:00
|
|
|
<4> The user used to execute the watch.
|
2017-03-29 12:07:55 -04:00
|
|
|
|
|
|
|
You can set a different execution mode for every action by associating the mode
|
|
|
|
name with the action id:
|
|
|
|
|
|
|
|
[source,js]
|
|
|
|
--------------------------------------------------
|
2018-12-08 13:57:16 -05:00
|
|
|
POST _watcher/watch/my_watch/_execute
|
2017-03-29 12:07:55 -04:00
|
|
|
{
|
|
|
|
"action_modes" : {
|
|
|
|
"action1" : "force_simulate",
|
|
|
|
"action2" : "skip"
|
|
|
|
}
|
|
|
|
}
|
|
|
|
--------------------------------------------------
|
|
|
|
// CONSOLE
|
|
|
|
// TEST[setup:my_active_watch]
|
|
|
|
|
|
|
|
You can also associate a single execution mode with all the actions in the watch
|
|
|
|
using `_all` as the action id:
|
|
|
|
|
|
|
|
[source,js]
|
|
|
|
--------------------------------------------------
|
2018-12-08 13:57:16 -05:00
|
|
|
POST _watcher/watch/my_watch/_execute
|
2017-03-29 12:07:55 -04:00
|
|
|
{
|
|
|
|
"action_modes" : {
|
|
|
|
"_all" : "force_execute"
|
|
|
|
}
|
|
|
|
}
|
|
|
|
--------------------------------------------------
|
|
|
|
// CONSOLE
|
|
|
|
// TEST[setup:my_active_watch]
|
|
|
|
|
|
|
|
The following example shows how to execute a watch inline:
|
|
|
|
|
|
|
|
[source,js]
|
|
|
|
--------------------------------------------------
|
2018-12-08 13:57:16 -05:00
|
|
|
POST _watcher/watch/_execute
|
2017-03-29 12:07:55 -04:00
|
|
|
{
|
|
|
|
"watch" : {
|
|
|
|
"trigger" : { "schedule" : { "interval" : "10s" } },
|
|
|
|
"input" : {
|
|
|
|
"search" : {
|
|
|
|
"request" : {
|
|
|
|
"indices" : [ "logs" ],
|
|
|
|
"body" : {
|
|
|
|
"query" : {
|
|
|
|
"match" : { "message": "error" }
|
|
|
|
}
|
|
|
|
}
|
|
|
|
}
|
|
|
|
}
|
|
|
|
},
|
|
|
|
"condition" : {
|
|
|
|
"compare" : { "ctx.payload.hits.total" : { "gt" : 0 }}
|
|
|
|
},
|
|
|
|
"actions" : {
|
|
|
|
"log_error" : {
|
|
|
|
"logging" : {
|
|
|
|
"text" : "Found {{ctx.payload.hits.total}} errors in the logs"
|
|
|
|
}
|
|
|
|
}
|
|
|
|
}
|
|
|
|
}
|
|
|
|
}
|
|
|
|
--------------------------------------------------
|
|
|
|
// CONSOLE
|
|
|
|
|
|
|
|
All other settings for this API still apply when inlining a watch. In the
|
|
|
|
following snippet, while the inline watch defines a `compare` condition,
|
|
|
|
during the execution this condition will be ignored:
|
|
|
|
|
|
|
|
[source,js]
|
|
|
|
--------------------------------------------------
|
2018-12-08 13:57:16 -05:00
|
|
|
POST _watcher/watch/_execute
|
2017-03-29 12:07:55 -04:00
|
|
|
{
|
|
|
|
"ignore_condition" : true,
|
|
|
|
"watch" : {
|
|
|
|
"trigger" : { "schedule" : { "interval" : "10s" } },
|
|
|
|
"input" : {
|
|
|
|
"search" : {
|
|
|
|
"request" : {
|
|
|
|
"indices" : [ "logs" ],
|
|
|
|
"body" : {
|
|
|
|
"query" : {
|
|
|
|
"match" : { "message": "error" }
|
|
|
|
}
|
|
|
|
}
|
|
|
|
}
|
|
|
|
}
|
|
|
|
},
|
|
|
|
"condition" : {
|
|
|
|
"compare" : { "ctx.payload.hits.total" : { "gt" : 0 }}
|
|
|
|
},
|
|
|
|
"actions" : {
|
|
|
|
"log_error" : {
|
|
|
|
"logging" : {
|
|
|
|
"text" : "Found {{ctx.payload.hits.total}} errors in the logs"
|
|
|
|
}
|
|
|
|
}
|
|
|
|
}
|
|
|
|
}
|
|
|
|
}
|
|
|
|
--------------------------------------------------
|
|
|
|
// CONSOLE
|