248 lines
10 KiB
Plaintext
248 lines
10 KiB
Plaintext
|
[[input-http]]
|
||
|
|
=== HTTP Input
|
||
|
|
|
||
|
|
Use the `http` input to submit a request to an HTTP endpoint and load the
|
||
|
|
response into the watch execution context when the watch is triggered. See
|
||
|
|
<<http-input-attributes, Input Attributes>> for all of the supported attributes.
|
||
|
|
|
||
|
|
With the `http` input, you can:
|
||
|
|
|
||
|
|
* Query external Elasticsearch clusters. The `http` input provides a way
|
||
|
|
to submit search requests to clusters other than the one {watcher} is running
|
||
|
|
on. This is useful when you're running a dedicated {watcher} cluster or if you
|
||
|
|
need to search clusters that are running different Elasticsearch versions.
|
||
|
|
|
||
|
|
* Query Elasticsearch APIs other than the search API. For example, you might want
|
||
|
|
to load data from the {ref}/cluster-nodes-stats.html[Nodes Stats],
|
||
|
|
{ref}/cluster-health.html[Cluster Health] or {ref}/cluster-state.html[Cluster
|
||
|
|
State] APIs.
|
||
|
|
|
||
|
|
* Query external web services. The `http` input enables you to load data from
|
||
|
|
any service that exposes an HTTP endpoint. This provides a bridge
|
||
|
|
between Elasticsearch clusters and other systems.
|
||
|
|
|
||
|
|
==== Querying External Elasticsearch Clusters
|
||
|
|
|
||
|
|
To query an external Elasticsearch cluster, you specify the cluster's
|
||
|
|
`host` and `port` attributes and the index's search endpoint as the `path`.
|
||
|
|
If you omit the search body, the request returns all documents in the specified
|
||
|
|
index:
|
||
|
|
|
||
|
|
[source,js]
|
||
|
|
--------------------------------------------------
|
||
|
|
"input" : {
|
||
|
|
"http" : {
|
||
|
|
"request" : {
|
||
|
|
"host" : "example.com",
|
||
|
|
"port" : 9200,
|
||
|
|
"path" : "/idx/_search"
|
||
|
|
}
|
||
|
|
}
|
||
|
|
}
|
||
|
|
--------------------------------------------------
|
||
|
|
|
||
|
|
You can use the full Elasticsearch {ref}/query-dsl.html[Query DSL] to perform
|
||
|
|
more sophisticated searches. For example, the following `http` input retrieves
|
||
|
|
all documents that contain `event` in the `category` field:
|
||
|
|
|
||
|
|
[source,js]
|
||
|
|
--------------------------------------------------
|
||
|
|
"input" : {
|
||
|
|
"http" : {
|
||
|
|
"request" : {
|
||
|
|
"host" : "host.domain",
|
||
|
|
"port" : 9200,
|
||
|
|
"path" : "/idx/_search",
|
||
|
|
"body" : "{\"query\" : { \"match\" : { \"category\" : \"event\"}}}"
|
||
|
|
}
|
||
|
|
}
|
||
|
|
}
|
||
|
|
--------------------------------------------------
|
||
|
|
|
||
|
|
==== Calling Elasticsearch APIs
|
||
|
|
|
||
|
|
To load the data from other Elasticsearch APIs, specify the API
|
||
|
|
endpoint as the `path` attribute. Use the `params` attribute to specify
|
||
|
|
query string parameters. For example, the following `http` input
|
||
|
|
calls the {ref}/cluster-stats.html[Cluster
|
||
|
|
Stats] API and enables the `human` attribute:
|
||
|
|
|
||
|
|
[source,js]
|
||
|
|
--------------------------------------------------
|
||
|
|
"input" : {
|
||
|
|
"http" : {
|
||
|
|
"request" : {
|
||
|
|
"host" : "host.domain",
|
||
|
|
"port" : "9200",
|
||
|
|
"path" : "/_cluster/stats",
|
||
|
|
"params" : {
|
||
|
|
"human" : "true" <1>
|
||
|
|
}
|
||
|
|
}
|
||
|
|
}
|
||
|
|
}
|
||
|
|
--------------------------------------------------
|
||
|
|
<1> Enabling this attribute returns the `bytes` values in the response in human
|
||
|
|
readable format.
|
||
|
|
|
||
|
|
[[input-http-auth-basic-example]]
|
||
|
|
==== Calling External Web Services
|
||
|
|
|
||
|
|
You can use `http` input to get data from any external web service. The `http`
|
||
|
|
input supports basic authentication. For example, the following input provides
|
||
|
|
a username and password to access `myservice`:
|
||
|
|
|
||
|
|
[source,js]
|
||
|
|
--------------------------------------------------
|
||
|
|
"input" : {
|
||
|
|
"http" : {
|
||
|
|
"request" : {
|
||
|
|
"host" : "host.domain",
|
||
|
|
"port" : "9200",
|
||
|
|
"path" : "/myservice",
|
||
|
|
"auth" : {
|
||
|
|
"basic" : {
|
||
|
|
"username" : "user",
|
||
|
|
"password" : "pass"
|
||
|
|
}
|
||
|
|
}
|
||
|
|
}
|
||
|
|
}
|
||
|
|
}
|
||
|
|
--------------------------------------------------
|
||
|
|
|
||
|
|
You can also pass in service-specific API keys and other information
|
||
|
|
through the `params` attribute. For example, the following `http`
|
||
|
|
input loads the current weather forecast for Amsterdam from the
|
||
|
|
http://openweathermap.org/appid[OpenWeatherMap] service:
|
||
|
|
|
||
|
|
[source,js]
|
||
|
|
--------------------------------------------------
|
||
|
|
"input" : {
|
||
|
|
"http" : {
|
||
|
|
"request" : {
|
||
|
|
"url" : "http://api.openweathermap.org/data/2.5/weather",
|
||
|
|
"params" : {
|
||
|
|
"lat" : "52.374031",
|
||
|
|
"lon" : "4.88969",
|
||
|
|
"appid" : "<your openweathermap appid>"
|
||
|
|
}
|
||
|
|
}
|
||
|
|
}
|
||
|
|
}
|
||
|
|
--------------------------------------------------
|
||
|
|
|
||
|
|
==== Using Templates
|
||
|
|
|
||
|
|
The `http` input supports templating. You can use <<templates, templates>> when
|
||
|
|
specifying the `path`, `body`, header values, and parameter values.
|
||
|
|
|
||
|
|
For example, the following snippet uses templates to specify what index to query
|
||
|
|
and restrict the results to documents added within the last five minutes:
|
||
|
|
|
||
|
|
[source,js]
|
||
|
|
--------------------------------------------------
|
||
|
|
"input" : {
|
||
|
|
"http" : {
|
||
|
|
"request" : {
|
||
|
|
"host" : "host.domain",
|
||
|
|
"port" : 9200,
|
||
|
|
"path" : "/{{ctx.watch_id}}/_search",
|
||
|
|
"body" : "{\"query\" : {\"range\": {\"@timestamp\" : {\"from\": \"{{ctx.trigger.triggered_time}}||-5m\",\"to\": \"{{ctx.trigger.triggered_time}}\"}}}}"
|
||
|
|
}
|
||
|
|
}
|
||
|
|
}
|
||
|
|
--------------------------------------------------
|
||
|
|
|
||
|
|
==== Accessing the HTTP Response
|
||
|
|
|
||
|
|
If the response body is formatted in JSON or YAML, it is parsed and loaded into
|
||
|
|
the execution context. If the response body is not formatted in JSON or YAML, it
|
||
|
|
is loaded into the payload's `_value` field.
|
||
|
|
|
||
|
|
Conditions, transforms, and actions access the response data through the
|
||
|
|
execution context. For example, if the response contains a `message`
|
||
|
|
object, you can use `ctx.payload.message` to access the message data.
|
||
|
|
|
||
|
|
In addition all the headers from the response can be accessed using the
|
||
|
|
`ctx.payload._headers` field as well as the HTTP status code of the response using
|
||
|
|
`ctx.payload._status_code`.
|
||
|
|
|
||
|
|
|
||
|
|
[[http-input-attributes]]
|
||
|
|
==== HTTP Input Attributes
|
||
|
|
|
||
|
|
[cols=",^,^,", options="header"]
|
||
|
|
|======
|
||
|
|
| Name | Required | Default | Description
|
||
|
|
|
||
|
|
| `request.scheme` | no | http | Url scheme. Valid values are: `http` or `https`.
|
||
|
|
|
||
|
|
| `request.host` | yes | - | The host to connect to.
|
||
|
|
|
||
|
|
| `request.port` | yes | - | The port the http service is listening on.
|
||
|
|
|
||
|
|
| `request.path` | no | - | The URL path. The path can be static text or contain `mustache`
|
||
|
|
<<templates, templates>>. URL query string parameters must be
|
||
|
|
specified via the `request.params` attribute.
|
||
|
|
|
||
|
|
| `request.method` | no | get | The HTTP method. Supported values are: `head`, `get`, `post`,
|
||
|
|
`put` and `delete`.
|
||
|
|
|
||
|
|
| `request.headers` | no | - | The HTTP request headers. The header values can be static text
|
||
|
|
or include `mustache` <<templates, templates>>.
|
||
|
|
|
||
|
|
| `request.params` | no | - | The URL query string parameters. The parameter values can be
|
||
|
|
static text or contain `mustache` <<templates, templates>>.
|
||
|
|
|
||
|
|
| `request.url` | no | - | Allows you to set `request.scheme`, `request.host`, `request.port`
|
||
|
|
and `request.params` add once by specifying a real URL, like
|
||
|
|
`https://www.example.org:1234/mypath?foo=bar`. May not be combined
|
||
|
|
with on of those four parameters. As those parameters are set,
|
||
|
|
specifying them individually might overwrite them.
|
||
|
|
|
||
|
|
| `request.auth` | no | - | Authentication related HTTP headers. Currently, only basic
|
||
|
|
authentication is supported.
|
||
|
|
|
||
|
|
| `request.proxy.host` | no | - | The proxy host to use when connecting to the host.
|
||
|
|
|
||
|
|
| `request.proxy.port` | no | - | The proxy port to use when connecting to the host.
|
||
|
|
|
||
|
|
| `request.connection_timeout` | no | 10s | The timeout for setting up the http connection. If the connection
|
||
|
|
could not be set up within this time, the input will timeout and
|
||
|
|
fail.
|
||
|
|
|
||
|
|
| `request.read_timeout` | no | 10s | The timeout for reading data from http connection. If no response
|
||
|
|
was received within this time, the input will timeout and fail.
|
||
|
|
|
||
|
|
|
||
|
|
| `request.body` | no | - | The HTTP request body. The body can be static text or include
|
||
|
|
`mustache` <<templates, templates>>.
|
||
|
|
|
||
|
|
| `extract` | no | - | A array of JSON keys to extract from the input response and
|
||
|
|
use as payload. In cases when an input generates a large
|
||
|
|
response this can be used to filter the relevant piece of
|
||
|
|
the response to be used as payload.
|
||
|
|
|
||
|
|
| `response_content_type` | no | json | The expected content type the response body will contain.
|
||
|
|
Supported values are `json`, `yaml` and `text`. If the
|
||
|
|
format is `text` the `extract` attribute cannot exist.
|
||
|
|
Note that this overrides the header that is returned in the
|
||
|
|
HTTP response. If this is set to `text` the body of the
|
||
|
|
response will be assigned and accessible to/via the `_value`
|
||
|
|
variable of the payload.
|
||
|
|
|======
|
||
|
|
|
||
|
|
You can reference the following variables in the execution context when
|
||
|
|
specifying the `path`, `params`, `headers` and `body` values:
|
||
|
|
|
||
|
|
[options="header"]
|
||
|
|
|======
|
||
|
|
| Name | Description
|
||
|
|
| `ctx.watch_id` | The id of the watch that is currently executing.
|
||
|
|
| `ctx.execution_time` | The time execution of this watch started.
|
||
|
|
| `ctx.trigger.triggered_time` | The time this watch was triggered.
|
||
|
|
| `ctx.trigger.scheduled_time` | The time this watch was supposed to be triggered.
|
||
|
|
| `ctx.metadata.*` | Any metadata associated with the watch.
|
||
|
|
|======
|