178 lines
5.6 KiB
Plaintext
178 lines
5.6 KiB
Plaintext
[[monitoring-watch-execution]]
|
|
[[watch-history]]
|
|
=== Monitoring Watch Execution
|
|
|
|
Whenever a watch is triggered, a `watch_record` document is created and added to the watch history
|
|
index. A new history index is created daily with a name of the form `.watch_history-YYYY.MM.dd`.
|
|
You can search the watch history like any other Elasticsearch index or use Kibana to monitor and
|
|
visualize watch execution.
|
|
|
|
A watch record's `_source` field contains all of the information about the watch execution:
|
|
|
|
`watch_id` :: The name of the watch that was triggered.
|
|
`trigger_event` :: How the watch was triggered (`manual` or `schedule`) and the watch's scheduled
|
|
time and actual trigger time.
|
|
`input` :: The input type (`http`, `search`, or `simple`) and definition.
|
|
`condition` :: The `condition` type (`always`, `never`, or `script`) and definition.
|
|
`state` :: The state of the watch execution (`execution_not_needed`, `executed`,
|
|
`throttled`).
|
|
`result` :: The results of each phase of the watch execution. Shows the input payload,
|
|
condition status, transform status (if defined), and actions status.
|
|
|
|
NOTE: While you can perform read operations on the watch history and manage the daily indices as
|
|
needed, you should never perform write operations on a watch history index. If you have
|
|
Shield installed, we recommend only allowing users read access to the watch history index.
|
|
|
|
[float]
|
|
[[monitoring-watches]]
|
|
==== Monitoring Watches with Kibana
|
|
|
|
You can use Kibana to monitor the watch history and create visualizations of the watches that have
|
|
executed over time.
|
|
|
|
To monitor watches with Kibana:
|
|
|
|
. Go to the Kibana **Settings > Indices** tab. For example,
|
|
`http://localhost:5601/#/settings/indices`.
|
|
. Enter `.watch_history*` in the **Index name or pattern** field.
|
|
. Click in the **Time field name** field and select `trigger_event.triggered_time`.
|
|
. Go to the **Discover** tab to see the most recently executed watches.
|
|
|
|
You can create visualizations and add them to a Kibana dashboard to track what
|
|
watches are being triggered and identify trends.
|
|
|
|
For example you could create a dashboard to:
|
|
|
|
* Track triggered watches over time, broken down by top watch.
|
|
* Identify top senders, priorities, and keywords for email actions.
|
|
* Identify top webhook targets and status codes.
|
|
|
|
image:images/watcher-kibana-dashboard.png[]
|
|
|
|
[float]
|
|
[[searching-watch-history]]
|
|
==== Searching the Watch History
|
|
|
|
To get the watch history for a particular day, search that day's watch history index:
|
|
|
|
[source,js]
|
|
--------------------------------------------------
|
|
GET .watch_history-2015.05.11/_search
|
|
{
|
|
"query" : { "match_all" : {}}
|
|
}
|
|
--------------------------------------------------
|
|
// AUTOSENSE
|
|
|
|
To get all of the watch records that reference a particular watch, search the
|
|
`watch_id` field:
|
|
|
|
[source,js]
|
|
--------------------------------------------------
|
|
GET .watch_history*/_search
|
|
{
|
|
"query" : { "match" : { "watch_id": "rss_watch" }}
|
|
}
|
|
--------------------------------------------------
|
|
// AUTOSENSE
|
|
|
|
To get all of the watch records for watches that were throttled, search the
|
|
`state` field.
|
|
|
|
[source,js]
|
|
--------------------------------------------------
|
|
GET .watch_history*/_search
|
|
{
|
|
"query" : { "match" : { "state": "throttled" }}
|
|
}
|
|
--------------------------------------------------
|
|
// AUTOSENSE
|
|
|
|
To get a date histogram over all triggered watches within a particular
|
|
time range.
|
|
|
|
[source,js]
|
|
--------------------------------------------------
|
|
GET .watch_history*/_search?size=0
|
|
{
|
|
"query": {
|
|
"filtered": {
|
|
"query": {
|
|
"match_all": {}
|
|
},
|
|
"filter": {
|
|
"range": {
|
|
"trigger_event.triggered_time": {
|
|
"gte": 1430438400000,
|
|
"lte": 1431820800000
|
|
}
|
|
}
|
|
}
|
|
}
|
|
},
|
|
"aggs": {
|
|
"records_per_minute": {
|
|
"date_histogram": {
|
|
"field": "trigger_event.triggered_time",
|
|
"interval": "1m",
|
|
"min_doc_count": 0,
|
|
"extended_bounds": {
|
|
"min": 1430438400000,
|
|
"max": 1431820800000
|
|
}
|
|
}
|
|
}
|
|
}
|
|
}
|
|
--------------------------------------------------
|
|
// AUTOSENSE
|
|
|
|
[float]
|
|
[[managing-watch-history]]
|
|
==== Managing Watch History Indexes
|
|
|
|
You should establish a policy for how long you need to keep your watch history indexes. For
|
|
example, you might simply delete the daily history indexes after 30 days. If you need to preserve
|
|
the history but don't need to maintain immediate access to it, you can close the index or take a
|
|
snapshot and then delete it.
|
|
|
|
http://www.elastic.co/guide/en/elasticsearch/client/curator/current/index.html[Elasticsearch Curator]
|
|
provides a convenient CLI for managing time-series indices.
|
|
|
|
You can also set up a watch to manage your watch history indexes. For example, the following watch
|
|
that runs daily and uses a webhook action to delete history indexes older than seven days.
|
|
|
|
[source,js]
|
|
--------------------------------------------------
|
|
PUT _watcher/watch/manage_history
|
|
{
|
|
"metadata": {
|
|
"keep_history_days": 7
|
|
},
|
|
"trigger": {
|
|
"schedule": { "daily": { "at" : "00:01" }}
|
|
},
|
|
"input": {
|
|
"simple": {}
|
|
},
|
|
"condition": {
|
|
"always": {}
|
|
},
|
|
"transform": {
|
|
"script" : "return [ indexToDelete : '/.watch_history-' + ctx.execution_time.minusDays(ctx.metadata.keep_history_days + 1).toString('yyyy.MM.dd') ]"
|
|
},
|
|
"actions": {
|
|
"delete_old_index": {
|
|
"webhook": {
|
|
"method": "DELETE",
|
|
"host": "localhost",
|
|
"port": 9200,
|
|
"path": "{{ctx.payload.indexToDelete}}"
|
|
}
|
|
}
|
|
}
|
|
}
|
|
--------------------------------------------------
|
|
// AUTOSENSE
|
|
|