Commit Graph

557 Commits

Author SHA1 Message Date
Shay Banon 7ff92c8f96 Merge pull request elastic/elasticsearch#445 from kimchy/remove_content_unsafe
Remove rest content unsafe usage

Original commit: elastic/x-pack-elasticsearch@c290326f3f
2015-05-09 16:26:11 +02:00
Shay Banon 02ad78743e Remove rest content unsafe usage
Original commit: elastic/x-pack-elasticsearch@4725a3de22
2015-05-09 16:21:45 +02:00
Brian Murphy 9a5455f573 [TESTS] Fix Inline template tests
This change adds a method to WatcherTestUtils to compare two json structures since order is not guaranteed in template parameters.
The Inline template tests in SearchInputTests and SearchTransformTests now use this instead of string comparision.

Original commit: elastic/x-pack-elasticsearch@c433545128
2015-05-09 10:18:45 -04:00
Martijn van Groningen 8fad9937f7 Changed search template format for search input and search transform to the new Watcher template format.
Also the WatcherUtils now always use the search template support from core and never uses ScriptService:
* A template is re-parsed and extended with the watch context variables.
* A normal request body is converted into a search template and watch context variables are used as template params.

Original commit: elastic/x-pack-elasticsearch@16bacaf094
2015-05-09 13:17:25 +02:00
uboness 8ddb0a65c4 Added a setting to disable watcher
The following node setting will disable watcher (it's enabled by default).

```
watcher.enabled: false
```

Original commit: elastic/x-pack-elasticsearch@1d0541a924
2015-05-09 01:33:00 +02:00
Brian Murphy eef3a1b1e8 Add force parameter to delete to force a delete
This change adds a force paramter to delete to allow the forcing of a delete to happen ignoring locking.
This means that watch executions may fail when they go to update the watch. Watches executing from the scheduler will fail fast if the watch they are supposed to execute has been removed.
Also move the history store updates outside of the watch locks.

Fixes: elastic/elasticsearch#405

Original commit: elastic/x-pack-elasticsearch@57561b6f85
2015-05-08 19:07:10 -04:00
uboness 87d4f67b10 Changed the `body` settings of the email
moved from `text_body` and `html_body` to a more structured `body` object as follows:

```
{
  "body" : {
    "text" : "the text body",
    "html" : "the html body"
  }
}
```
`body` can also accpet a string, in which case it will default to the text body of the email:

```
{
  "body" : "the text body of the email"
}
```
the above is a syntactic sugar for the following:

```
{
  "body" : {
    "text" : "the text body of the email"
  }
}
```

Original commit: elastic/x-pack-elasticsearch@92406ac2a1
2015-05-08 19:12:06 +02:00
Brian Murphy 45f39ca2a2 [REST-TEST] forgot to add this file.
Original commit: elastic/x-pack-elasticsearch@23de354192
2015-05-08 11:14:22 -04:00
Brian Murphy 4c939a7c3d Hijack the DELETE .watches index delete endpoint.
This change disables the DELETE .watches index endpoint.

Fixes: elastic/elasticsearch#439

Original commit: elastic/x-pack-elasticsearch@2e66cd55fa
2015-05-08 10:39:06 -04:00
Brian Murphy 8bc238e10a [REST-TEST] Use catch: missing instead of ignore: 404
Some of the non java clients don't return the body on an ignore: 404 so we must use catch missing instead.

Original commit: elastic/x-pack-elasticsearch@a10582691c
2015-05-08 10:11:43 -04:00
uboness 3289ee9029 Changed `attach_data` in emails to be more flexible
Bye default, when `attach_data` is set to `true` we attache the watch execution data as a `yaml` file. Now it's possible to configure the format of the file by setting the `attach_data` to an object with a `format` field (can either be set to `yaml` or `json`). Example:

```
"attach_data" : {
  "format" : "json"
}
```

Original commit: elastic/x-pack-elasticsearch@5cab59a676
2015-05-08 14:37:38 +02:00
Martijn van Groningen 97413eba38 test: also have trace logging for the other tests
Original commit: elastic/x-pack-elasticsearch@84584cd158
2015-05-08 12:19:30 +02:00
Martijn van Groningen 46838d3477 test: Don't run check index on shard close.
Original commit: elastic/x-pack-elasticsearch@38404e8994
2015-05-08 11:43:54 +02:00
uboness a06303a894 Change Script & Template configuration
Scripts and Templates are very much alike. There are only two main differences between them when it comes to watcher:

1. The template doesn't have a language type - it's always mustache.
2. Templates are often used to render xcontent (json/yaml). So there's build in support for  content type in it

Traditionally, in es core, the configuration of the scripts and templates was always a bit of a mess. There was no consistency between the two and when it comes to the templates configuration, they're even not future proof (future additions to the search requests body can break the templates).

After a long discussion, we've decided that we need to change the way users configure templates & scripts in es core... and align the two. These changes will only come in es 2.0. That said, it's important that watcher will be aligned with es core in how templates and scripts are configured and therefore we need to change it for that purpose.

Watcher will come with support for the scripts & templates configuration format of es 2.0. We want to future proof the configuration from day one.

here are the configuration:

Inline:

```
{
  "inline" : "script here",
  "lang" : "lang here",
  "params" : { parameters here }
}
```

File:

```
{
  "file" : "file name here",
  "lang" : "language here",
  "params" : { parameters here }
}
```

Indexed:

```
{
  "id" : "template id here",
  "lang" : "language here",
  "params" : { parameters here }
}
```

For templates it's the same configuration except there is no `lang` field. Also, for templates, there's native support for xcontent:

```
{
  "inline" : { "key" : "{{param1}}" },
  "params" : { "param1" : "value1" }
  }
}
```

Original commit: elastic/x-pack-elasticsearch@4a31114f35
2015-05-08 09:27:18 +02:00
uboness ddb5efeb25 Throw appropriate error when input is empty
When the watch `input` is an empty object, we need to throw an appropriate error indicating the problem (instead of the NPE that is currently thrown)

Closes elastic/elasticsearch#427

Original commit: elastic/x-pack-elasticsearch@c8b27e6653
2015-05-08 03:28:22 +02:00
Brian Murphy f78bc8dcb2 Add DateMath support to ScheduledTriggerEvent
This change changes how the ScheduledTriggerEvent is parsed to parse using DateMath instead of just as a date. This will allow the execute API to use such constructs as
```
POST _watcher/watch/test_watch/_execute
{
  "ignore_throttle" : true,
  "trigger_event" : {
    "schedule" : {
        "triggered_time": "now-5h",
        "scheduled_time": "now"
    }
  }
}
```

Fixes: elastic/elasticsearch#374

Original commit: elastic/x-pack-elasticsearch@fa286b217e
2015-05-07 19:23:33 -04:00
Brian Murphy 6f2429afb4 [REST-TEST] Add master_timeout to the rest API specs
This change adds the parameter `master_timeout` to the rest requests that can write to a watch PUT, ACK and DELETE.
Also add support in the rest tests to verify that this is accepted.

Fixes: elastic/elasticsearch#416

Original commit: elastic/x-pack-elasticsearch@4cc1c50f2c
2015-05-07 19:19:49 -04:00
Brian Murphy 2e787fff2b Throw an WatcherSettingsException when parsing an invalid hourly schedule.
```
   "schedule" : {
       "hourly" :  [0]
}
```
Was throwing an NPE now it gives a WatcherSettingsException.

Fixes: elastic/elasticsearch#419

Original commit: elastic/x-pack-elasticsearch@f334712f96
2015-05-07 17:23:27 -04:00
Brian Murphy f5c50c44de Fix NPE in Rest Layer when GET missing watch.
The REST GET API was trying to render a	null watch on GET which	was causing an NPE. Don't render the watch if it's not found and add a test for this case.

Fixes: elastic/elasticsearch#202

Original commit: elastic/x-pack-elasticsearch@0c8afa63ba
2015-05-07 17:12:57 -04:00
Martijn van Groningen a90d5a581e fix wrong parse field
Original commit: elastic/x-pack-elasticsearch@d017fb3e63
2015-05-07 16:50:52 +02:00
Martijn van Groningen f198b1bbae Use parse field instead of a plain string
Original commit: elastic/x-pack-elasticsearch@4c58dbdbb1
2015-05-07 16:46:10 +02:00
Martijn van Groningen 16910bb7d8 removed unused method
Original commit: elastic/x-pack-elasticsearch@8b5568c4e8
2015-05-07 16:45:37 +02:00
Clinton Gormley f5fe774340 Merge pull request elastic/elasticsearch#412 from clintongormley/real_bulk_body
REST tests: Add a real bulk body to the hijack tests

Original commit: elastic/x-pack-elasticsearch@b17e49a3d5
2015-05-07 16:41:10 +02:00
Clinton Gormley 00efe8593e REST tests: Add a real bulk body to the hijack tests
The Java REST runner expects a real body for the bulk API, rather than
an empty array

Original commit: elastic/x-pack-elasticsearch@86dc149ff7
2015-05-07 16:36:55 +02:00
Clinton Gormley d76df0d590 Merge pull request elastic/elasticsearch#411 from clintongormley/wait_for_green
REST tests: Wait for yellow, not green

Original commit: elastic/x-pack-elasticsearch@296b312e17
2015-05-07 16:07:08 +02:00
Clinton Gormley 2aab635792 Merge pull request elastic/elasticsearch#409 from clintongormley/bulk_format
REST tests: Bulk API expects an array not a hash

Original commit: elastic/x-pack-elasticsearch@82e39629fa
2015-05-07 16:06:32 +02:00
Brian Murphy 9bf00d80ba [TEST] Make hijack test description more descriptive.
This change renames the hijack test name from "Test Hijack api" to "Test Hijack and Block Direct Access to Internal Indices".

Original commit: elastic/x-pack-elasticsearch@118ac18132
2015-05-07 10:05:08 -04:00
Clinton Gormley 372c15e0c1 Merge pull request elastic/elasticsearch#410 from clintongormley/bad_yaml
REST tests: Fix bad YAML

Original commit: elastic/x-pack-elasticsearch@9e06863983
2015-05-07 16:04:00 +02:00
Clinton Gormley 41b974b784 REST tests: Wait for yellow, not green
When testing on a single node, waiting for green times out

Original commit: elastic/x-pack-elasticsearch@ca91bbbbe6
2015-05-07 15:43:09 +02:00
Clinton Gormley 85ac38c622 REST tests: Bulk API expects an array not a hash
Original commit: elastic/x-pack-elasticsearch@20ee1d38e7
2015-05-07 15:42:37 +02:00
Clinton Gormley 35d2d86beb REST tests: Fix bad YAML
Original commit: elastic/x-pack-elasticsearch@362aac9c96
2015-05-07 15:41:33 +02:00
Joshua Rich bd1f67ec99 Merge pull request elastic/elasticsearch#402 from joshuar/master
Fix type: index to type: indexed.

Original commit: elastic/x-pack-elasticsearch@73a8d0ef2e
2015-05-07 10:57:55 +10:00
Martijn van Groningen 34a0de4b38 test: after first elected master node has been stopped assert that watcher is started and only running once.
Original commit: elastic/x-pack-elasticsearch@e193b80215
2015-05-07 00:54:44 +02:00
Martijn van Groningen 5e7ef773b3 test: change no master block assert and if it fails print which node doesn't have a master block.
Original commit: elastic/x-pack-elasticsearch@656a1bc3df
2015-05-07 00:40:39 +02:00
Martijn van Groningen e1beb6e9b3 Fixes a bug that prevents the actual job in the schedule engine not being adjusted if the watch's interval is being updated with the same interval unit.
Closes elastic/elasticsearch#388

Original commit: elastic/x-pack-elasticsearch@ecf991f42c
2015-05-07 00:11:53 +02:00
Brian Murphy cf23d54aed Fix execute REST API call
We weren't assigning the trigger event in the execute REST API

Original commit: elastic/x-pack-elasticsearch@cc8a6d043c
2015-05-06 17:05:43 -04:00
Brian Murphy 7d98358012 Change the `ManualTriggerEvent` to wrap a `TriggerEvent`
This change changes the manual trigger event to be a wrapper for another `TriggerEvent`.
TriggerEvent is required field for the execute API either via setTriggerEvent on the request or requestBuilder in the java API or `trigger_event` in the REST API

Original commit: elastic/x-pack-elasticsearch@790bdb96a5
2015-05-06 16:59:26 -04:00
Brian Murphy bd9bf2810d Make log4j dependency a test only dependency
Original commit: elastic/x-pack-elasticsearch@cc8936ad8f
2015-05-06 15:59:06 -04:00
Brian Murphy 42fc43451c Unshade the javax.mail and javax.activation jars
This change removes these jars from shading and changes the assemblies file so that they are included as is.

Original commit: elastic/x-pack-elasticsearch@695f5e2b20
2015-05-06 15:59:05 -04:00
Martijn van Groningen 3f2070cc0a index template: Added support to add additional settings to the index templates.
Also removed the version checking from the index template support, so that if the settings are changed these settings always get stored in the index template. This means we always put two index templates once watcher started.

And also shutdown watcher after the cluster service has been stopped.

This was done caused deadlock in the integration test framework when we are stopping the test cluster.
If multiple nodes are stopped, a node may briefly try to or run Watcher during the test cluster shutdown.
As part if starting we always put an index template, which will block any other start or stop calls for the WatcherService.
A node was running a put index template call as part of the start procedure while the start framework would stop the node.
The stop call would wait because it doesn't own the lock yet.
The put index template call didn't return, which caused the stop call the keep waiting.

The reason why put index template was hanging is that it failed to return anything while it the node was stopped.
Likely the put index template request got queued up waiting for a change in the cluster state, the cluster service
got stopped and the request got lost, but the watcher start procedure was still waiting.

By stopping Watcher before we stop the cluster service we avoid the scenario described above.

Original commit: elastic/x-pack-elasticsearch@db94b2279e
2015-05-06 21:22:47 +02:00
uboness 1a9db6140a fixed Java API docs
- wrote an intro where we tell how to get the watcher jar and we tall about the `WatcherClient`
- cleaned up the code snippets
- removed a reference to the `Watch.Parser`

- Also renamed `WatcherStatsResponse#getWatchServiceState` to `watcherStatsResponse.getWatcherState`

Original commit: elastic/x-pack-elasticsearch@814329cba7
2015-05-06 20:50:21 +02:00
uboness 5ecf1dbdcd Introduced `XContentSource`
An absraction aover XContent byte reference that:

- implements `ToXContent`
- can resolve values given a dot-notation path
- Used in `GetWatchResponse` and `ExecuteWatchResponse`

Also

- Moved `WatchExecutionResult` to the `execution` package

Original commit: elastic/x-pack-elasticsearch@0b41f53f38
2015-05-06 18:37:00 +02:00
uboness 8730b066c1 Refactored `WatcherService.State`
- Moved it to its own top level enum `WatcherState`
- Also changed the key holding the current state in the `stats` API to `watcher_state`

Original commit: elastic/x-pack-elasticsearch@2e5ca90f57
2015-05-06 12:29:59 +02:00
Martijn van Groningen 70cc41d710 Changed internal indices naming pattern
The `.watch_history` indices names should be aligned with logstash and marvel indices patterns so changing them to `.watch_history-YY.MM.dd`

Closes elastic/elasticsearch#387

Original commit: elastic/x-pack-elasticsearch@20baa1018f
2015-05-06 11:16:14 +02:00
uboness 5687c91bf6 Update README.asciidoc
Original commit: elastic/x-pack-elasticsearch@f61bd59d86
2015-05-05 23:38:55 +02:00
uboness 690641d0c9 Cleaned up the `Script` and `Template` constructs
Change them to consistently serialize themselves the same way they deserialize themselves.

If the script (or template) is read from a `string` value, it will serialize it self to xcontent as a `string` value as well. Otherwise it will serialize as an object, holding only those fields that where configured in the first place.

Original commit: elastic/x-pack-elasticsearch@52a82e0bbe
2015-05-05 23:25:14 +02:00
Brian Murphy 75bdf7de0d [TEST] Change execute test watches to have cron schedules that trigger in 2099.
Original commit: elastic/x-pack-elasticsearch@dde1dbae86
2015-05-05 12:37:44 -04:00
Brian Murphy b5f5862146 Clean up Watch parsing to disallow arrays.
This change cleans up the watch parsing which had allowed arrays in actions as a hold over from when actions was an array.

Fixes elastic/elasticsearch#367

Original commit: elastic/x-pack-elasticsearch@3f4fb82124
2015-05-05 10:44:28 -04:00
Brian Murphy 2ff8e2fb4e [TEST] Fix ack watch test to not trigger while test is running.
Original commit: elastic/x-pack-elasticsearch@d0261b355d
2015-05-04 16:19:41 -04:00
Brian Murphy b0fd2f9f59 Relocate guava and owasp under org.elasticsearch.watcher.common/common.owasp.
This change relocates the shaded jars for guava and owasp under org.elasticsearch.watcher.common and org.elasticsearch.watcher.common.owasp respectively.

Original commit: elastic/x-pack-elasticsearch@0e2d6303e0
2015-05-04 12:59:34 -04:00