Commit Graph

1553 Commits

Author SHA1 Message Date
Simon Willnauer 64fce2b360 Cleanup pom.xml after using elasticsearch-parent pom
There are a lot of settigns which we don't need at all or are already
defined in the parent. For instance dependencies are not needed if
they are included in elasticsearch-parent.

This commit also removes the shading which we don't do anymore in
core and we include guava already.

Original commit: elastic/x-pack-elasticsearch@c4f951a751
2015-06-18 10:19:06 +02:00
Simon Willnauer e96eab8ba0 Enable REST tests
Enable REST tests again after core support for 3rd party REST tests
was added back.

Closes elastic/elasticsearch#599

Original commit: elastic/x-pack-elasticsearch@a5fffbca6e
2015-06-18 09:25:13 +02:00
Boaz Leskes 3394c2e883 Add a debug log to IndexAuditTrail
Original commit: elastic/x-pack-elasticsearch@ccff079b12
2015-06-18 08:55:34 +02:00
Simon Willnauer 483fc360f4 [BUILD] Remove dead files from ancient build system
Original commit: elastic/x-pack-elasticsearch@98aa655945
2015-06-17 21:14:19 +02:00
Simon Willnauer 6d255c0c49 [TEST] Use temp directory to fix script loading on windows
Original commit: elastic/x-pack-elasticsearch@6fc6c29718
2015-06-17 21:06:33 +02:00
uboness 110a9ed6a0 Added missing mail jar in the distribution
Original commit: elastic/x-pack-elasticsearch@d05af66792
2015-06-17 20:23:15 +02:00
Simon Willnauer d730917b53 Merge pull request elastic/elasticsearch#594 from elastic/upgrade_to_es_20
Upgrade to Elasticsearch 2.0.0-SNAPSHOT

This moves the master branch to follow Elasticsearch 2.0.0-SNAPSHOT and fixes most problems that occurred during the upgrade. The remaining issues  not yet fixed are:

 * `HttpClient` and the `Account` used for Email support need to install security manager which is not supported by the elasticsearch security policy. This is not yet resolved an requires fundamental changes and/or a rule in the core policy file. See elastic/elasticsearch#597
 * Due to changes to the way Time/Byte settings are parsed settings without a unit must be upgraded.  See elastic/elasticsearch#598
 * REST tests are currently disabled due to some limitations from Elasticsearch core that don't allow to run 3rd party REST tests. See elastic/elasticsearch#599

Watcher now also inherits the elaticsearch-parent pom file and all it's properties.

Original commit: elastic/x-pack-elasticsearch@1e03234e3e
2015-06-17 17:30:40 +02:00
Simon Willnauer de39879558 Merge branch 'master' into upgrade_to_es_20
Conflicts:
	src/main/java/org/elasticsearch/watcher/execution/CurrentExecutions.java
	src/main/java/org/elasticsearch/watcher/watch/WatchLockService.java
	src/test/java/org/elasticsearch/watcher/actions/throttler/ActionThrottleTests.java

Original commit: elastic/x-pack-elasticsearch@ab51f0104f
2015-06-17 15:56:45 +02:00
Simon Willnauer d5c94148a5 remove nocommits
Original commit: elastic/x-pack-elasticsearch@8e0f99ea0e
2015-06-17 15:53:19 +02:00
Simon Willnauer b0986aec1c disable REST
Original commit: elastic/x-pack-elasticsearch@c15b03e388
2015-06-17 15:47:40 +02:00
Simon Willnauer ef7d216b4e make the rest pass
Original commit: elastic/x-pack-elasticsearch@3d60f13994
2015-06-17 15:47:39 +02:00
jaymode 853f8d50ef Test: assert that the current thread is not already interrupted
Original commit: elastic/x-pack-elasticsearch@acc517ca53
2015-06-17 09:38:50 -04:00
Robert Muir 9886ecab5c fix maven to work at all
Original commit: elastic/x-pack-elasticsearch@28f61700e6
2015-06-17 09:15:17 -04:00
Martijn van Groningen b82fca3379 We should cope with the situation when a watch is going to be executed whilst shutting down.
The execution service will throw an exection if we acquire a watch lock or add a watch to the current executions if Watcher has stopped or is stopping. We should deal with this sitation by properly catching those failures. FYI if a watch managed to put itself to the current executions a shutdown will wait until a watch execution is finished.

Original commit: elastic/x-pack-elasticsearch@5692316072
2015-06-17 13:38:38 +02:00
jaymode d036fc505d Test: override beforeIndexDeletion for audit trail tests
The beforeIndexDeletion method expects that nothing is still indexing when it runs
but this is not the case as the index audit trail will continue indexing events that
occur as checks are being performed in the cluster.

Original commit: elastic/x-pack-elasticsearch@02001a5222
2015-06-17 06:26:12 -04:00
Alexander Reelsen 22c77805bd Test: Removed river handler due to removal in master
Original commit: elastic/x-pack-elasticsearch@c6a3372ce0
2015-06-17 12:07:02 +02:00
Martijn van Groningen 2f14240915 test: increased time throttle timeout to deal with timing issues
Original commit: elastic/x-pack-elasticsearch@90fd3c9cd9
2015-06-17 09:17:13 +02:00
jaymode 38837b310c [Test] disable random dynamic templates for audit index
Original commit: elastic/x-pack-elasticsearch@914e0990ef
2015-06-16 15:43:58 -04:00
Simon Willnauer c5bf865543 Merge branch 'master' into upgrade_to_es_20
Conflicts:
	src/test/java/org/elasticsearch/watcher/history/HistoryStoreSettingsTests.java

Original commit: elastic/x-pack-elasticsearch@019d0bcd19
2015-06-16 20:12:05 +02:00
jaymode cb80e9ccbd add mapping and configurable settings for audit index output
Previously we relied on the default dynamic mapping for the audit index output, which did
not create an ideal mapping. This change adds a mapping file with default settings and
dynamic mapping disabled for the audit indexes.

Additionally, the ability to override settings for the audit indexes has been provided so that
users can customize the number of shards and replicas to meet their needs.

In order to implement these changes, the index audit service had to be moved from a lifecycle
component to an abstract component that had its own custom lifecycle on top of the cluster
state. A ShieldLifecycleService class was added to accomplish this. In the future, this class
can be used for other services that need to perform index based operations.

Closes elastic/elasticsearch#913

Original commit: elastic/x-pack-elasticsearch@231740c1cc
2015-06-16 13:33:03 -04:00
Martijn van Groningen f72f4807aa test: increased logging and ensure update requests are acked
Original commit: elastic/x-pack-elasticsearch@b4cab02148
2015-06-16 18:01:39 +02:00
Simon Willnauer 10c5ac6353 Merge branch 'master' into upgrade_to_es_20
Conflicts:
	src/main/java/org/elasticsearch/watcher/support/http/HttpClient.java
	src/main/java/org/elasticsearch/watcher/support/http/HttpRequestTemplate.java
	src/main/java/org/elasticsearch/watcher/transport/actions/execute/ExecuteWatchRequestBuilder.java
	src/main/java/org/elasticsearch/watcher/transport/actions/execute/TransportExecuteWatchAction.java
	src/test/java/org/elasticsearch/watcher/actions/email/EmailActionTests.java
	src/test/java/org/elasticsearch/watcher/actions/email/service/HtmlSanitizeTests.java
	src/test/java/org/elasticsearch/watcher/actions/webhook/WebhookHttpsIntegrationTests.java
	src/test/java/org/elasticsearch/watcher/execution/ManualExecutionTests.java
	src/test/java/org/elasticsearch/watcher/support/http/HttpClientTests.java
	src/test/java/org/elasticsearch/watcher/support/http/HttpRequestTemplateTests.java

Original commit: elastic/x-pack-elasticsearch@61885ed389
2015-06-16 17:48:06 +02:00
Simon Willnauer d071e7ebf7 randomly use shield
Original commit: elastic/x-pack-elasticsearch@e3729112e3
2015-06-16 17:24:13 +02:00
Simon Willnauer 501fd2164d make all rest tests pass
Original commit: elastic/x-pack-elasticsearch@b7a108b077
2015-06-16 17:23:47 +02:00
Simon Willnauer d175a75cfc no sec manager for now
Original commit: elastic/x-pack-elasticsearch@36329494ea
2015-06-16 16:28:47 +02:00
Martijn van Groningen e29df8dd60 Added inline watch support to _execute API
This change allows the specification of a watch inline to the `_execute` API.
This watch id will not be persisted to the index and if record_execution is set to true it will result in an error.
The internal id `_anonymous_` will be used for the watch id and will be the watch id in the watch record.

Original commit: elastic/x-pack-elasticsearch@00e32c3838
2015-06-16 16:10:07 +02:00
Simon Willnauer 4cfdf620e5 make tests pass witout security manager
Original commit: elastic/x-pack-elasticsearch@59931382d5
2015-06-16 15:16:21 +02:00
Simon Willnauer 9019c979f3 fix permissions for REST tests
Original commit: elastic/x-pack-elasticsearch@a157980a47
2015-06-16 14:10:15 +02:00
Simon Willnauer 69b65f613a don't use DNS resolution to test unknown domains
Original commit: elastic/x-pack-elasticsearch@953f840a99
2015-06-16 14:04:43 +02:00
Simon Willnauer 0d757413d5 make other slow=false tests pass
Original commit: elastic/x-pack-elasticsearch@1aee8b7744
2015-06-16 13:53:15 +02:00
Simon Willnauer 491e8fc167 more test fixes - more nocommits
Original commit: elastic/x-pack-elasticsearch@9d230d3b71
2015-06-16 10:08:24 +02:00
Martijn van Groningen d3bd643bf4 Fix HttpClientTests, TriggeredWatchStoreTests and WatchStoreTests.
Original commit: elastic/x-pack-elasticsearch@efd0291a09
2015-06-15 23:08:48 +02:00
Martijn van Groningen 2bb03cba89 Fix rest tests
Original commit: elastic/x-pack-elasticsearch@e92139512b
2015-06-15 23:05:01 +02:00
Martijn van Groningen 72fed1be7f Fix Shield related errors
Original commit: elastic/x-pack-elasticsearch@8a2f1e0586
2015-06-15 23:04:40 +02:00
Simon Willnauer e03b4c1b83 some progress while shield is still fucked up
Original commit: elastic/x-pack-elasticsearch@90402b7ba8
2015-06-15 18:43:16 +02:00
uboness 03b704f79b Moved transform errors to the tranform result
Until now, if the transform failed (either on the watch or action level), an exception would be thrown and it would be captured globally on the watch execution and in the watch record message

This commit bring the error to the transform result

- A new `status` field was added to the transform result. Can either have `success` or `failure` values. When set to `failure` a `reason` field will hold the failure message.
- The `ExecutionService` changed to enable this functionality. Mainly, instead of relying on exception, during the execution the transform result status is checked and if failed the watch execution is aborted.

Original commit: elastic/x-pack-elasticsearch@65b7f51f00
2015-06-15 17:28:52 +02:00
uboness 09fcecc069 Made email html body sanitization configurable
Until now the email sanitization was fixed and could not be configured. That means that if ppl wanted a feature and our sanitization didn't support it, they were forced to disable sanitizaion all together.

In this commit a new `HtmlSanitizer` construct was introduced that is bound by Guice and can be configured via the settings.

Closes elastic/elasticsearch#586

Original commit: elastic/x-pack-elasticsearch@0081d1bf41
2015-06-15 16:44:07 +02:00
Simon Willnauer 78c0159949 make tests pass forbidden APIs
Original commit: elastic/x-pack-elasticsearch@985fc6d2ff
2015-06-15 15:45:35 +02:00
uboness b6e8df6a32 Added connection/read timeout support for HTTP
Until today connection and read timeout for http was not directly supported. This means that without setting oracle specific system properties at startup, calling a bad http service would by default hold the watch executing thread forever... is niet goed!!!

This commit introduces connection & read timeouts.

- Connection timeouts are timeouts for setting up the connection
- Read timeouts are timeouts waiting for data to be read

By default both timeouts are set to 10 seconds (overriding the default jdk to indefinite). It is possible to customize the default timeouts by settings `watcher.http.default_connection_timeout` and `watcher.http.default_read_timeout` settings).

It is also possible to override these defaults per http request, meaning, per webhook and http input configuration in the watch.

Original commit: elastic/x-pack-elasticsearch@224f50bc8b
2015-06-15 15:33:40 +02:00
Simon Willnauer fc95323904 first cut - everything compiles
Original commit: elastic/x-pack-elasticsearch@f29fd6de3f
2015-06-15 14:53:03 +02:00
uboness a9448ab2a4 Moved condition errors to their result
Until now if a condition failed to execute (for whatever reason), an exception would be thrown and the watch would just abort. The problem with that approach is while the error message would have been logged in the watch record message, the result of the condition would have been lost.

This commit moves the condition execution error to the condition result (just like we have it today with actions and inputs).

- A `status` field was added to the condition result (can either be `success` or `failure`)
- A `reason` field is added to the condition result in case its status is `failure`
- If the condition result status is `failure`, the watch execution is aborted

Updated the rest APIs to verify the status & type of both the `input` and `condition` results on execution.

Original commit: elastic/x-pack-elasticsearch@dddca03ff5
2015-06-13 04:22:31 +02:00
jaymode f4ed6282fd only bind the index audit trail if auditing is enabled
If auditing is not enabled, we should not bind the index audit trail as a lifecycle component
since we do not bind all of its dependent classes when auditing is disabled.

Original commit: elastic/x-pack-elasticsearch@39f736ca6a
2015-06-12 17:08:31 -04:00
jaymode 95ad77a778 allow periods in esusers usernames and role names
This change allows periods in the usernames and role names in the esusers realm.

Closes elastic/elasticsearch#905

Original commit: elastic/x-pack-elasticsearch@64b4f02ee5
2015-06-12 16:39:29 -04:00
uboness 0ac5dc8271 [lib] - upgraded java mail to 1.5.3
Original commit: elastic/x-pack-elasticsearch@15142f2c79
2015-06-12 21:38:02 +02:00
jaymode 9a97b046d5 control index audit events through an include/excludes mechanism
This changes how the user defines which events should be indexed. Previously, there were
several boolean settings being used. This condenses to an include and exclude setting.

Additionally, the IndexAuditTrail needed to become a lifecycle component since parsing the
enum could throw exceptions. Given this, the IndexBulkProcessor was condensed into the
IndexAuditTrail since it did not make sense to have two lifecycle components for an index
audit trail.

Closes elastic/elasticsearch#900

Original commit: elastic/x-pack-elasticsearch@4b4d824f5e
2015-06-12 14:17:38 -04:00
Chris Cowan 72730d3d9c Merge branch 'master' of github.com:elasticsearch/elasticsearch-marvel
Original commit: elastic/x-pack-elasticsearch@00b1faab97
2015-06-12 10:24:28 -07:00
jaymode c82816dd49 ensure we call authentication failed even if an exception is thrown
Today, a realm could throw an exception, which will not be caught and cause the audit
trail to be skipped. This commit catches exceptions thrown by realms and ensures that
the audit trail is called and then re-throws the exceptions.

Closes elastic/elasticsearch#901

Original commit: elastic/x-pack-elasticsearch@d260c00b33
2015-06-12 12:20:09 -04:00
jaymode 7c67b49aa5 system actions are only logged at the correct audit level
Today, some system actions could be logged by default when the actions performed are
not internal. Additionally for internal actions, we never checked if the user was the system
user.

This adds a check to ensure the user is the System user and that the actions that are being
suppressed are known system actions.

Closes elastic/elasticsearch#902

Original commit: elastic/x-pack-elasticsearch@b107994692
2015-06-12 08:39:43 -04:00
uboness 288ce368d5 Moved input errors to the input result
Until now, if the input failed, an exception would be thrown and it would be captured globally on the watch execution and in the watch recod message. The problem with this approach is that the information about the input is lost. In this commit, the failure is returned as part of the input result.

- A new `status` field was added to the input result. Can either have `success` or `failure` values. When set to `failure` a `reason` field will be set with the error message.
- The `ExecutionService` changed to enable this functionality. Mainly, instead of relying on exception, during the execution the input result is checked for its status and the execution is aborted on failure. Also, the two places where the watch execution is handled were consolidated to a single method `execute(WatchExecutionContext)`.
- Also, the watch execution context id (which will end up being the `watch_record` id) was added the the context model (accessible via scripts and templates). This is done mainly for debugging purposes.

Original commit: elastic/x-pack-elasticsearch@e2567deada
2015-06-12 14:08:07 +02:00
jaymode 5a6dcfa0ac fix compilation due to change in snapshot state enum
Original commit: elastic/x-pack-elasticsearch@771333594d
2015-06-12 06:34:54 -04:00