Commit Graph

705 Commits

Author SHA1 Message Date
uboness fcad80b973 fixed typo in NOTICE.txt
Original commit: elastic/x-pack-elasticsearch@90bded37ea
2015-06-26 17:02:15 +03:00
uboness a459c753ad Added NOTICE.txt
- added both the `NOTICE.txt` and `LICENSE.txt` to the distribution

Original commit: elastic/x-pack-elasticsearch@5acfd50874
2015-06-26 16:49:28 +03:00
uboness 266dfbb3c5 Added version 1.0.0 to WatcherVersion
Original commit: elastic/x-pack-elasticsearch@f7a0996ddf
2015-06-25 16:54:20 +02:00
uboness 39f9f53a0a Renamed the status field in the stored watch
was renamed to `_status` so it'll distinct from the other "formal" fields of the watch and also be aligned with the `_status` field that is returned by the Get API.

Also updated the installation docs

Original commit: elastic/x-pack-elasticsearch@1081b75d13
2015-06-25 15:39:19 +02:00
Martijn van Groningen 9cb7232075 test: Use utc time to create index name in test, since that is what internally is used too.
Original commit: elastic/x-pack-elasticsearch@48f300f657
2015-06-25 00:36:47 +02:00
Martijn van Groningen 193d8092db Added timeout option to the `search` transform, `search` input and `index` action.
Added default timeout setting:
* `watcher.action.index.default_timeout` sets the timeout for `index` action, if no timeout is defined in the `index` action itself.
* `watcher.input.search.default_timeout` sets the timeout for `search` input, if no timeout is defined in the `search` input itself.
* `watcher.transform.search.default_timeout`  sets the timeout for `search` transform, if no timeout is defined in the `search` transform itself.

Added general Watcher wide default timeout settings to the operations exposed in the client proxy:
* `watcher.internal.ops.search.default_timeout` for search related operations.
* `watcher.internal.ops.index.default_timeout` for index operations.
* `watcher.internal.ops.bulk.default_timeout` for bulk operations.

Original commit: elastic/x-pack-elasticsearch@5a3ef35a9d
2015-06-25 00:10:24 +02:00
Martijn van Groningen 591ea500f2 test: Fetch the status from the get response itself instead of the watch source
Original commit: elastic/x-pack-elasticsearch@ab0f4b8fe9
2015-06-24 22:29:17 +02:00
Martijn van Groningen ba17333523 get watch api: Return the watch status separate from the watch source.
The status isn't maintained by the user but rather by Watcher itself. The idea here is that the get watch api should return the watch as was provided to Watcher via the put watch api. The status will be reported under the top level `_status` field.

Original commit: elastic/x-pack-elasticsearch@54e2452493
2015-06-24 21:31:52 +02:00
Martijn van Groningen 67fdad6357 If starting watcher fails, watcher should set itself back to the stopped state.
At the moment if the starting fails watcher will hang in the `starting` phase. This is bad because any subsequent start will be ignored, because the the watcher state isn't `stopped`.

Original commit: elastic/x-pack-elasticsearch@5cbc1d2a13
2015-06-24 21:06:57 +02:00
uboness b8e80773d9 Fixed the stats API
- now reflects the right stats around the execution threadpool

Original commit: elastic/x-pack-elasticsearch@ecf557c6be
2015-06-24 15:53:06 +02:00
Martijn van Groningen be33ee6323 fixed compile error due to a recent upstream change in core
Original commit: elastic/x-pack-elasticsearch@f09b84c928
2015-06-24 15:11:49 +02:00
Martijn van Groningen d1bc1e1f68 fixed compile error
Original commit: elastic/x-pack-elasticsearch@9675f254c3
2015-06-24 14:04:22 +02:00
uboness bbabf8ba85 fixed build failure
failed due changes in core master

Original commit: elastic/x-pack-elasticsearch@43f6badffe
2015-06-24 12:20:56 +02:00
Martijn van Groningen 26a72e4d21 lifecycle: upon node shutdown make clear it is a manual stop, so that potential cluster states that come in after the LifyCycle#beforeStop can't start watcher.
Original commit: elastic/x-pack-elasticsearch@e2522a21fc
2015-06-24 06:56:01 +02:00
uboness c4dbb8d794 [cleanup] index names parser
Was bound in guice without a reason.

Original commit: elastic/x-pack-elasticsearch@75a9b66946
2015-06-23 17:58:45 +02:00
Martijn van Groningen d46dad18d0 Added logging in case state is unexpected when starting or stopping.
Also increased test logging in BootStrapTests.

Original commit: elastic/x-pack-elasticsearch@6b8141dbc0
2015-06-23 16:15:50 +02:00
uboness 71aa3a8059 Added Dynamic Index Names Support
The search input/transform rely on users configuring the search requests. Sometimes (often), these search requests are executed on time-based indices. The problem the

Until now, there's no way to define dynamic index names that are bound to time, which forces the request to search all the indices (for example, the Marvel watches se

This commit adds dynamic index name resolution. It works in the following way:

- and index name can be a simple string (indicating the static/absolute index name you're searching, incl. wildcards)
- an index name can also be a template. The template is enclosed within `<` and `>` (these are officially illegal characters for index names, so these are safe to use)
- the template can have both static parts to it and place holder parts. The place holders are enclosed within `{` and `}`. The place holder holds `date math` expression
 * `"<.marvel-{now}>"` will resolve to `".marvel-2022.03.03"` (the default date format is `YYYY.MM.dd`)
 * `"<.marvel-{now/M}>"` will resolve to `".marvel-2022.03.01"`
 * `"<.marvel-{now{YYYY.MM}}>"` will resolve to `".marvel-2022.03"` (this one has a custom date format - `YYYY.MM`)
 * `"<.marvel-{now/M-1M{YYYY.MM}}>"` will resolve to `".marvel-2022.02"`

The following is an example of a search input that searches marvel indices for the last 3 days (relies on the default Marvel indices format - `.marvel-YYYY.MM.dd`):

```
{
    ...
    "input" : {
            "search" : {
                    "request" : {
                            "indices" : [
                                    "<.marvel-{now/d-2d}>",
                                    "<.marvel-{now/d-1d}>",
                                    "<.marvel-{now/d}>"
                            ],
                            ...
                    }
            }
    }
    ...
}
```

- `index` action was also updated to work with a dynamic index name (e.g. it's possible to index into daily indices by setting the index name to `<idx-{now}>`)

Original commit: elastic/x-pack-elasticsearch@9c15a96029
2015-06-23 15:54:45 +02:00
Martijn van Groningen d2d1c44d07 test: increased test logging
Original commit: elastic/x-pack-elasticsearch@455892c628
2015-06-23 15:31:06 +02:00
Martijn van Groningen 075e6caba0 logging: if starting watcher fails this should be more visible
Original commit: elastic/x-pack-elasticsearch@26da9d7661
2015-06-23 08:42:56 +02:00
Simon Willnauer 58bd0cc509 [TEST] Remove awaits-fix, the issue is resovled in es-core master
Original commit: elastic/x-pack-elasticsearch@eb813e83dd
2015-06-22 12:54:03 +02:00
uboness 3cf6b32f6a [cleanup] XContentSource now requires XContentType
We need this as the `XContentSource` supports all xcontent constructs as the root construct, while xcontent in core only supports objects. For this reason, we can't rely on xcontent auto-detection of the xcontent type. We need to be explicit about it.

Original commit: elastic/x-pack-elasticsearch@a2ed944a21
2015-06-22 12:24:56 +02:00
Simon Willnauer 797945b586 [TEST] Beef up test to sometimes pass a plain number as a string
Original commit: elastic/x-pack-elasticsearch@f42662c719
2015-06-22 12:11:17 +02:00
Simon Willnauer 5ea2d0528c Require units for all time values
This commit is a backwards compatibilty break for all watcher indices
that had a `throttel_period` set on their watches. `throttle_period` used
to be a numeric value but now is stored as a string AND requires a unit
like seconds or minutes etc. to prevent errors. All other time valiues like
http timeouts also require units now.

Closes elastic/elasticsearch#598

Original commit: elastic/x-pack-elasticsearch@e3b2c2a4af
2015-06-22 11:47:37 +02:00
uboness 5766c64f94 added support for 1.0.0-rc1 in WatcherVersion
Original commit: elastic/x-pack-elasticsearch@02c018c0b9
2015-06-19 15:51:22 +02:00
uboness fa57b2c78e removed javax.mail from test dependencies
Original commit: elastic/x-pack-elasticsearch@294f461456
2015-06-19 11:45:57 +02:00
uboness fad95315fb remove @Repeat annotation from test
Original commit: elastic/x-pack-elasticsearch@d5414a32e8
2015-06-19 01:46:32 +02:00
uboness d41815ca18 fixed version scheme in WatcherVersion
Original commit: elastic/x-pack-elasticsearch@1007cd71a3
2015-06-18 21:50:39 +02:00
Simon Willnauer bc4ce6f153 Remove @Repeat - don't commit hardcoded repeats
Original commit: elastic/x-pack-elasticsearch@e9de83bb3a
2015-06-18 20:59:46 +02:00
Simon Willnauer 11e928bbe4 Reenable Security Manager
Closes elastic/elasticsearch#597

Original commit: elastic/x-pack-elasticsearch@d7737e6f2f
2015-06-18 20:23:10 +02:00
uboness 38f3c8b607 Fixed the version
- moved to 2.0.0-beta1
- moved the min license version to 2.0.0
- moved to min shield version to 2.0.0
- lowercased the "beta" and "rc" part of the version

Original commit: elastic/x-pack-elasticsearch@fab1983bbb
2015-06-18 17:06:44 +02:00
Martijn van Groningen 7a55a957d3 removed retry start watcher mechanism
If nodes drop and .watches / .triggered_watches shards are available after those shards were started a new cluster state update will come along that triggers the start watcher logic.

Original commit: elastic/x-pack-elasticsearch@af36f8b078
2015-06-18 15:48:16 +02:00
Martijn van Groningen ca58e93150 applied feedback
Original commit: elastic/x-pack-elasticsearch@4767935457
2015-06-18 15:26:14 +02:00
Martijn van Groningen c4cb85104e lifecycle: Verify whether Watcher was stopped manually under a lock
Before if timing permitted it a start and stop could be executed very close to each other time wise. This could make the manual stopped check seem to be valid, because the stop hadn't yet been performed.

Original commit: elastic/x-pack-elasticsearch@c1865c1069
2015-06-18 13:45:55 +02:00
uboness 8af461daaf removed all timezone'less DateTime constructions
Now as this rule was added to the forbidden APIs in core, this is required for the build to run.

Original commit: elastic/x-pack-elasticsearch@a3b570e7f1
2015-06-18 13:28:25 +02:00
uboness aef6ad5c12 Added dynamic variables registry to the execution context
This enables different constructs (primarily scripts) to set variables that can be access by subsequent constructs throughout the wathc execution. These variables are scoped to a single execution, that is, they are not persisted across multiple executions of the same watch.

Closes elastic/elasticsearch#589

Original commit: elastic/x-pack-elasticsearch@34223d1991
2015-06-18 12:17:07 +02:00
Martijn van Groningen bcb7428868 Renamed `email.email` to `email.message` in the email action result.
Closes elastic/elasticsearch#592

Original commit: elastic/x-pack-elasticsearch@043a4084e6
2015-06-18 11:20:55 +02:00
Simon Willnauer 4c6d709bca Add back lost tests.timewarp settings
This setting was lost during the upgrade to Elasticsearch 2.0

Original commit: elastic/x-pack-elasticsearch@d0e3b0e080
2015-06-18 10:24:18 +02:00
Simon Willnauer 64fce2b360 Cleanup pom.xml after using elasticsearch-parent pom
There are a lot of settigns which we don't need at all or are already
defined in the parent. For instance dependencies are not needed if
they are included in elasticsearch-parent.

This commit also removes the shading which we don't do anymore in
core and we include guava already.

Original commit: elastic/x-pack-elasticsearch@c4f951a751
2015-06-18 10:19:06 +02:00
Simon Willnauer e96eab8ba0 Enable REST tests
Enable REST tests again after core support for 3rd party REST tests
was added back.

Closes elastic/elasticsearch#599

Original commit: elastic/x-pack-elasticsearch@a5fffbca6e
2015-06-18 09:25:13 +02:00
Simon Willnauer 483fc360f4 [BUILD] Remove dead files from ancient build system
Original commit: elastic/x-pack-elasticsearch@98aa655945
2015-06-17 21:14:19 +02:00
Simon Willnauer 6d255c0c49 [TEST] Use temp directory to fix script loading on windows
Original commit: elastic/x-pack-elasticsearch@6fc6c29718
2015-06-17 21:06:33 +02:00
uboness 110a9ed6a0 Added missing mail jar in the distribution
Original commit: elastic/x-pack-elasticsearch@d05af66792
2015-06-17 20:23:15 +02:00
Simon Willnauer d730917b53 Merge pull request elastic/elasticsearch#594 from elastic/upgrade_to_es_20
Upgrade to Elasticsearch 2.0.0-SNAPSHOT

This moves the master branch to follow Elasticsearch 2.0.0-SNAPSHOT and fixes most problems that occurred during the upgrade. The remaining issues  not yet fixed are:

 * `HttpClient` and the `Account` used for Email support need to install security manager which is not supported by the elasticsearch security policy. This is not yet resolved an requires fundamental changes and/or a rule in the core policy file. See elastic/elasticsearch#597
 * Due to changes to the way Time/Byte settings are parsed settings without a unit must be upgraded.  See elastic/elasticsearch#598
 * REST tests are currently disabled due to some limitations from Elasticsearch core that don't allow to run 3rd party REST tests. See elastic/elasticsearch#599

Watcher now also inherits the elaticsearch-parent pom file and all it's properties.

Original commit: elastic/x-pack-elasticsearch@1e03234e3e
2015-06-17 17:30:40 +02:00
Simon Willnauer de39879558 Merge branch 'master' into upgrade_to_es_20
Conflicts:
	src/main/java/org/elasticsearch/watcher/execution/CurrentExecutions.java
	src/main/java/org/elasticsearch/watcher/watch/WatchLockService.java
	src/test/java/org/elasticsearch/watcher/actions/throttler/ActionThrottleTests.java

Original commit: elastic/x-pack-elasticsearch@ab51f0104f
2015-06-17 15:56:45 +02:00
Simon Willnauer d5c94148a5 remove nocommits
Original commit: elastic/x-pack-elasticsearch@8e0f99ea0e
2015-06-17 15:53:19 +02:00
Simon Willnauer b0986aec1c disable REST
Original commit: elastic/x-pack-elasticsearch@c15b03e388
2015-06-17 15:47:40 +02:00
Simon Willnauer ef7d216b4e make the rest pass
Original commit: elastic/x-pack-elasticsearch@3d60f13994
2015-06-17 15:47:39 +02:00
Robert Muir 9886ecab5c fix maven to work at all
Original commit: elastic/x-pack-elasticsearch@28f61700e6
2015-06-17 09:15:17 -04:00
Martijn van Groningen b82fca3379 We should cope with the situation when a watch is going to be executed whilst shutting down.
The execution service will throw an exection if we acquire a watch lock or add a watch to the current executions if Watcher has stopped or is stopping. We should deal with this sitation by properly catching those failures. FYI if a watch managed to put itself to the current executions a shutdown will wait until a watch execution is finished.

Original commit: elastic/x-pack-elasticsearch@5692316072
2015-06-17 13:38:38 +02:00
Martijn van Groningen 2f14240915 test: increased time throttle timeout to deal with timing issues
Original commit: elastic/x-pack-elasticsearch@90fd3c9cd9
2015-06-17 09:17:13 +02:00