Until now, if the transform failed (either on the watch or action level), an exception would be thrown and it would be captured globally on the watch execution and in the watch record message
This commit bring the error to the transform result
- A new `status` field was added to the transform result. Can either have `success` or `failure` values. When set to `failure` a `reason` field will hold the failure message.
- The `ExecutionService` changed to enable this functionality. Mainly, instead of relying on exception, during the execution the transform result status is checked and if failed the watch execution is aborted.
Original commit: elastic/x-pack-elasticsearch@65b7f51f00
Until now the email sanitization was fixed and could not be configured. That means that if ppl wanted a feature and our sanitization didn't support it, they were forced to disable sanitizaion all together.
In this commit a new `HtmlSanitizer` construct was introduced that is bound by Guice and can be configured via the settings.
Closeselastic/elasticsearch#586
Original commit: elastic/x-pack-elasticsearch@0081d1bf41
Until today connection and read timeout for http was not directly supported. This means that without setting oracle specific system properties at startup, calling a bad http service would by default hold the watch executing thread forever... is niet goed!!!
This commit introduces connection & read timeouts.
- Connection timeouts are timeouts for setting up the connection
- Read timeouts are timeouts waiting for data to be read
By default both timeouts are set to 10 seconds (overriding the default jdk to indefinite). It is possible to customize the default timeouts by settings `watcher.http.default_connection_timeout` and `watcher.http.default_read_timeout` settings).
It is also possible to override these defaults per http request, meaning, per webhook and http input configuration in the watch.
Original commit: elastic/x-pack-elasticsearch@224f50bc8b
Until now if a condition failed to execute (for whatever reason), an exception would be thrown and the watch would just abort. The problem with that approach is while the error message would have been logged in the watch record message, the result of the condition would have been lost.
This commit moves the condition execution error to the condition result (just like we have it today with actions and inputs).
- A `status` field was added to the condition result (can either be `success` or `failure`)
- A `reason` field is added to the condition result in case its status is `failure`
- If the condition result status is `failure`, the watch execution is aborted
Updated the rest APIs to verify the status & type of both the `input` and `condition` results on execution.
Original commit: elastic/x-pack-elasticsearch@dddca03ff5
If auditing is not enabled, we should not bind the index audit trail as a lifecycle component
since we do not bind all of its dependent classes when auditing is disabled.
Original commit: elastic/x-pack-elasticsearch@39f736ca6a
This changes how the user defines which events should be indexed. Previously, there were
several boolean settings being used. This condenses to an include and exclude setting.
Additionally, the IndexAuditTrail needed to become a lifecycle component since parsing the
enum could throw exceptions. Given this, the IndexBulkProcessor was condensed into the
IndexAuditTrail since it did not make sense to have two lifecycle components for an index
audit trail.
Closeselastic/elasticsearch#900
Original commit: elastic/x-pack-elasticsearch@4b4d824f5e
Today, a realm could throw an exception, which will not be caught and cause the audit
trail to be skipped. This commit catches exceptions thrown by realms and ensures that
the audit trail is called and then re-throws the exceptions.
Closeselastic/elasticsearch#901
Original commit: elastic/x-pack-elasticsearch@d260c00b33
Today, some system actions could be logged by default when the actions performed are
not internal. Additionally for internal actions, we never checked if the user was the system
user.
This adds a check to ensure the user is the System user and that the actions that are being
suppressed are known system actions.
Closeselastic/elasticsearch#902
Original commit: elastic/x-pack-elasticsearch@b107994692
Until now, if the input failed, an exception would be thrown and it would be captured globally on the watch execution and in the watch recod message. The problem with this approach is that the information about the input is lost. In this commit, the failure is returned as part of the input result.
- A new `status` field was added to the input result. Can either have `success` or `failure` values. When set to `failure` a `reason` field will be set with the error message.
- The `ExecutionService` changed to enable this functionality. Mainly, instead of relying on exception, during the execution the input result is checked for its status and the execution is aborted on failure. Also, the two places where the watch execution is handled were consolidated to a single method `execute(WatchExecutionContext)`.
- Also, the watch execution context id (which will end up being the `watch_record` id) was added the the context model (accessible via scripts and templates). This is done mainly for debugging purposes.
Original commit: elastic/x-pack-elasticsearch@e2567deada
There may be current executions still going on during stopping.
We should try to wait for the current executions to have completed.
Otherwise we can run into a situation where we didn't delete the watch from the .triggered_watches index,
but did insert into the history index. Upon start this can lead to DocumentAlreadyExistsException,
because we already stored the history record during shutdown...
Introduced `CurrentExecutions` to handle this synchronization.
(we always first store the watch record and then remove the triggered watch)
Original commit: elastic/x-pack-elasticsearch@89c4a8d8ad
This commit adds support for indexing multiple documents with the `index` action. This is done by introducing a special `_doc` field. During action execution, the `_doc` field will be looked up in the payload. If found, the value of the field will be considered as the document that needs to be indexed. If the value is an array of objects, each object in that array will be treated as a separate document and all the documents in the array will be bulk indexed.
This commit also changes the result of the action to hold `XContentSource` rather than a payload (to avoid Map creation explosions). Th `XContentSource` was also extended to support lists.
Original commit: elastic/x-pack-elasticsearch@86f454b029
The `compare` condition result will now hold all resolved values, keyed by their associated parameter place holder strings.
Original commit: elastic/x-pack-elasticsearch@f930c77d54
Elasticsearch core no longer uses shaded dependencies by default, so this change makes
the plugin use the unshaded dependencies.
Original commit: elastic/x-pack-elasticsearch@495f636266
Elasticsearch core no longer uses shaded dependencies by default, so this change makes
the plugin use the unshaded dependencies.
Original commit: elastic/x-pack-elasticsearch@6f8b8b3feb
This change adds tests to ack a subset of a watch's actions, use a different throttle period per action in a watch, also adds tests to make sure that both the watch level and global level throttle_period are applied correctly.
Also updates the REST tests to make sure that throttle periods can be set at a watch and action level and are returned from the GET API.
Original commit: elastic/x-pack-elasticsearch@4b006c7830
Today, we simply return a SSLSocketFactory from a SSLContext, which provides
the default SSL configuration for sockets. This means that SSLv3 could still be
enabled on these sockets when running in an older JVM. This also means that
the ciphers and protocol settings are not honored for users of this socket factory,
which is currently the LDAP code.
This change returns a custom socket factory that delegates to the default socket
factory and sets the ciphers and protocols on the socket before returning the
socket.
Original commit: elastic/x-pack-elasticsearch@c4cfedfd51
Our current CA configuration creates certificates with only a CN and this caused
the regular expression in the PKI realm to fail. The default regular expression in
the PKI realm has been changed to allow for only a CN, while still maintaining the
ability to extract only the CN when other fields are present in the DN.
Additionally, the CA configuration has been updated so that is will copy any of the
originally specified fields (besides CN and EMAILADDRESS) over to the signed
certificate.
Original commit: elastic/x-pack-elasticsearch@ff27f69781
The IndicesTermsFilter Cache in core can leak data by not authorizing users prior to
retrieving data from the cache. We work around this by ensuring that the cache has
a maximum size of 0, effectively disabling it.
A test is also added to ensure that data is not leaked by this cache or the cache used by
the ScriptService in core.
Closeselastic/elasticsearch#854
Original commit: elastic/x-pack-elasticsearch@8a48bdad98
Today, we only have the option to require client authentication or not require it. A third case
exists where the server can want client authentication, but if the client does not have credentials
or wish to send credentials the connection will still be allowed.
This is beneficial for the PKI realm because it allows some clients to authenticate with PKI and
others to use another authentication token such as username and password authentication.
Closeselastic/elasticsearch#858
Original commit: elastic/x-pack-elasticsearch@e7cbc20bca
Until today it was required to provide a trigger event as part of the execute API. There are two issues with it:
1. It's not user friendly (you'd expect that the execute API would just work with just pointing to the watch)
2. The API could expose inconsistencies where on one hand it points to a watch (with a well defined trigger) on the other it enabled the user to provide completely different trigger (of a different type)
This change enable supporting default triggers by enabling the trigger engine to create a simulated trigger event. This enables the execute API to look up the trigger type of the pointed watch, and ask the trigger service to simulate an event for it. It is still possible to override the trigger event data by providing it via the `trigger_data` parameter in the API.
This simplifies the execute API and prepares it for future trigger types as well.
- This commit add missing integration tests for the execute API
- Also, removed unused `setIgnoreThrottling` from the execute request/builder.
Original commit: elastic/x-pack-elasticsearch@b494ae62e6
This change allows the httpinput to receive non json formatted data from a http endpoint
(such as the elasticsearch _cat apis). If non json is read it will be stored in the `payload._value` in the same
way that the `ScriptTransform` handles non map/json data returned by transforming scripts.
Added response_content_type to http input so that the expected content type in the response can be configured. This accepts `yaml`, `json` and `text` but will be overridden by the http headers.
Original commit: elastic/x-pack-elasticsearch@753d37f14e
Now that the watch record is write once and never read/parsed. We can de-normalize its format (the structure of the `watch_record` document) such it'd be optimal for searching and aggregating.
- `execution_result` renamed to `result` (for minimal verbosity)
- the structure of `trigger_event` changed to:
```
{
"trigger_event" : {
"type" : "<trigger_type>",
"triggered_time" : "<datetime>",
"<trigger_type>" : { // type specific data (optional) }
}
}
```
- the structure of `input` results changed to:
```
{
"result" : {
"input" : {
"type" : "<input_type>",
"payload" : { // the payload },
"<input_type>" : { // custom result fields per type (optional) }
},
...
}
}
```
- the structure of `condition` results changed to:
```
{
"result" : {
...
"condition" : {
"type" : "<condition_type>",
"met" : true | false,
"<condition_type>" : { // custom result fields per type (optional) }
},
...
}
}
```
- the structure of `transform` results changed to:
```
{
"result" : {
...
"transform" : {
"type" : "<transform_type>",
"payload" : { // the transformed payload }
"<transform_type>" : { // custom result fields per type (optional) }
},
...
}
}
```
- the structure of `actions` results changed to:
```
{
"result" : {
...
"actions" : [
{
"id" : "<action_id>"
"type" : "<action_type>",
"status" : "success" | "failure" | "simulated" | "throttled",
"reason" : "holds the reasoning if status is either success or throttled",
"transform" : { // action level transform result (if applicable)}
"<action_type>" : { // custom result fields per type (optional) }
},
...
]
}
}
```
Original commit: elastic/x-pack-elasticsearch@98466d4b83
Today, an exception is thrown when calls to the encrypt and decrypt methods are
made without a key being present. For now, we will not require the system key and
this behavior is undesirable.
This commit changes the behavior to just return the provided characters or bytes
when no key is present. Additionally, a method has been added for callers to see
if encryption is supported. Finally, the listener interface has been made public and
expanded to provide the old keys when the keys are changed. This allows
consumers to decrypt with the old key and re-encrypt with the new key.
Original commit: elastic/x-pack-elasticsearch@de3d5b6180
Today, we always throw an AuthorizationException for an unauthorized user. This is
problematic when anonymous access is enabled and the HTTP client being used
does not support preemptive basic authentication as only the anonymous user
will be used by such a client.
This change adds a setting to allow an AuthenticationException to be thrown for
anonymous users. This will clients such as browsers to work with anonymous
access and authenticated access.
Closeselastic/elasticsearch#853
Original commit: elastic/x-pack-elasticsearch@d338b468c7
The esusers tool reads the list of roles to provide validation feedback to the user,
however since we have added custom roles the tool doesn't know about these roles
as they come from outside of Shield. When a custom role was found, a warning
was printed that can be confusing to users.
Now when validating roles, we only read the names from the roles.yml file.
Closeselastic/elasticsearch#835
Original commit: elastic/x-pack-elasticsearch@89d0e3efce
uboness
Simon Willnauer
jaymode
Chris Cowan
Konrad Beiske
Martijn van Groningen
aleph-zero
Brian Murphy
Areek Zillur