Commit Graph

352 Commits

Author SHA1 Message Date
Martijn van Groningen 6277a32b91 Set minimum binary compatibility to ES 1.5
Closes elastic/elasticsearch#160

Original commit: elastic/x-pack-elasticsearch@10da3efb1a
2015-03-26 18:20:39 +01:00
Martijn van Groningen 9994326f11 Renamed AlertsBenchmark to WatcherBenchmark
Original commit: elastic/x-pack-elasticsearch@3c8000e7bf
2015-03-26 11:19:47 +01:00
uboness 41832b6f5b Change project name to elasticsearch-watcher
- `alerts` package is now `watcher` package
- we no longer use the term `Alert`, but instead we use `Watch`
- documentation still needs to be updated

Original commit: elastic/x-pack-elasticsearch@1225edf7e3
2015-03-24 13:46:01 +01:00
Martijn van Groningen a6bdbf0b0b Added simple benchmark to test fired alert throughput
Original commit: elastic/x-pack-elasticsearch@186b6305ff
2015-03-19 13:09:05 -07:00
Brian Murphy 100b5b922c Fix SearchInput tests.
SearchInput tests were failing locally for me in DEBUG because they weren't setting a valid alert on the ExecutionContext which was causing
the debug log messages in the SearchInput.execute method to NPE. I've added a dummy Alert to the context to fix things.
`

Original commit: elastic/x-pack-elasticsearch@2d744fdb7b
2015-03-19 15:56:19 -04:00
uboness 47b0fb4ea1 [docs] added docs for input, schedule and condition
Also:

- removed `throttle` file (throttling is not covered in `alert-anatomy` page)
- fixed ctx model parameters in templates/scripts to use the `ctx` prefix.
- added a dediated section in alert-anatomy about execution context as template/script model
- wrote first page of `transform` section

Original commit: elastic/x-pack-elasticsearch@471ca7f0d9
2015-03-19 12:07:20 -07:00
Martijn van Groningen 5dfe02ec64 Core: Make sure that the queue size is set on the alert thread pools
Also removed hack leftover.

Original commit: elastic/x-pack-elasticsearch@44fd070f5d
2015-03-18 16:40:22 -07:00
Martijn van Groningen 2377d1525b Tests: added tests for AlertStore, AlertService and AlertLockService
Changed ClientProxy to be return responses instead of ActionFutures and removed builders. This helps with mocking.

Original commit: elastic/x-pack-elasticsearch@bfc36d9405
2015-03-18 10:08:28 -07:00
Martijn van Groningen e6445a9d2e Iternal: Removed Payload.ActionResponse in favour of an extra constructor to Payload.XContent
Tests: Write more tests for AlertUtils

Closes elastic/elasticsearch#132

Original commit: elastic/x-pack-elasticsearch@b0e675b89c
2015-03-06 10:36:54 +01:00
uboness 50f4a1c0e3 Introducing TimeWarp mode for tests
The idea behind a time warp mode is that while it's enabled the time related constructs in the alerts module are replaced with a mock test friendly version.. so we'll be able to control time and therefore avoid sleeping the threads.

 In time warp mode:

-  The `SchedulerMock` is used to manually fire jobs
-  The `ClockMock` is used to set and fast forward time
-  The alerts are executed on the same thread as the scheduler mock... so we don't have to deal with async nature at all. This is accomplished by the added `AlertsExecutor` abstraction.

By default, the time warp mode is enabled and tests run in it. If a test must not use the time warp mode, it is possible to add `@TimeWarped(false)` annotation to the test and it will then run with the standard scheduler & clock. It is also possible to disable this mode all together by running the tests with `-Dtests.timewarp=false`.

All the updated tests now work in both modes (whether the time warp mode is dis/enabled). This is important as on the server we would like to run the tests outside of this mode as well, but locally we'd like to run them with time warped enabled (so they'll be faster)

Also, cleaned up the tests.. we now only do `assertThat(...)` calls (no `assertTrue` or `assertEquals`... for consistency sake)

Original commit: elastic/x-pack-elasticsearch@11e09f6dea
2015-03-06 08:33:21 +01:00
Martijn van Groningen ffdf23b411 Tests: Added tests for HistoryStore
Closes elastic/elasticsearch#129

Original commit: elastic/x-pack-elasticsearch@16043bcec2
2015-03-05 20:49:22 +01:00
uboness 6eb27e2519 Introduced Clock as an abstraction over the system clock
Use in:

- Determining the execution time of an alert
- The period throttler to determine the time passed since last execution

Original commit: elastic/x-pack-elasticsearch@9197b86b68
2015-03-04 23:26:56 +01:00
Martijn van Groningen f28dee2e44 Tests: Made sure that resources are properly cleaned up after test classes extending from AbstractAlertsSingleNodeTests have been completed
Original commit: elastic/x-pack-elasticsearch@4621b76fe1
2015-03-04 12:11:21 +01:00
uboness 5903ea7493 Fixed NPE when email action has no subject or text body
Fixes elastic/elasticsearch#137

Original commit: elastic/x-pack-elasticsearch@7f575657f3
2015-03-04 11:03:13 +01:00
uboness f5c48576bc Cleaned up the use of execution context mocking in the tests
Added helper methods in `AlertsTestUtils` to mock `ExecutionContext` and create a simple payload

Original commit: elastic/x-pack-elasticsearch@eac6b63e62
2015-03-04 10:37:38 +01:00
uboness 4874fa2f1b change the SES email port to 587
Original commit: elastic/x-pack-elasticsearch@d69d02236b
2015-03-04 10:03:07 +01:00
uboness 01145b8025 Aligned Transform.Result with all other result constructs
- the transform parser and registry can now parse transform results
- every transform result may have its own format
- the chain transform result outputs the "transformation trail" of all the chained transforms (great tool for debugging)
- removed `Transform.NOOP` - was redundant, when no transform is defined the alert's transform is `null`
- removed `payload` from the `AlertExecution`. Instead, the execution holds the input result, transform result and potentially the transform results of the actions
- changed the xcontent representing a transofrm result to `{ "transform_type" : { "payload" : {...}, ... } }`
- with this change, the `Action` interface is cleaned up (`execute` only accepts the execution context)

Original commit: elastic/x-pack-elasticsearch@6ecf7f2c92
2015-03-04 09:52:31 +01:00
Martijn van Groningen 167f6814ab Fix tests
Original commit: elastic/x-pack-elasticsearch@ad7b4ae1ef
2015-03-04 09:42:57 +01:00
Martijn van Groningen 0382310cae Moved testAlertsWithDifferentSearchType and testWithAggregations to more unit like tests.
Original commit: elastic/x-pack-elasticsearch@115e361c42
2015-03-04 09:15:35 +01:00
uboness f6c17bd802 Added a transform construct to actions
An alert may have multiple actions associated with it, but each action may need to have different view of the data. For example, the email may need a certain model for its templates while the index or webhook actions may need a completely different data structure.

Until now, there's only an option to defina a single `transform` on the alert level that would have applied to all actions. This commit adds
 the ability to associate a transform with each action. We still keep the tranform on the alert level (in case all actions need the same transformation, in which case we'd like to avoid repetition).

Original commit: elastic/x-pack-elasticsearch@5493a2179b
2015-03-04 03:20:17 +01:00
uboness d25bf008b3 Added Amazon SES to ManualPublicSmtpServersTests
Original commit: elastic/x-pack-elasticsearch@d7f8180d1a
2015-03-03 23:41:56 +01:00
uboness dfc8249d4b added support for template params in search requests
Original commit: elastic/x-pack-elasticsearch@06e7dab935
2015-03-03 12:01:45 +02:00
uboness f9765fd393 standardized script/template model variables
Now `Variables.createCtxModel` is responsible for creating the model for scripts & templates across the board. Accessing the payload, alert name, fired time and scheduled fire time is done via the `ctx.` prefix.

Original commit: elastic/x-pack-elasticsearch@443ac17579
2015-03-03 10:27:47 +02:00
uboness 43f11cbb3d [fix] added the script params to the condition script execution
Now the script's params in the `script` condition are merged with the payload data into a single variable context to the script execution. The payload data is now accessed using the `payload.` prefix.

Original commit: elastic/x-pack-elasticsearch@e313a6301c
2015-03-02 22:01:02 +02:00
Martijn van Groningen 2aa91e84ed Now fix test for real
Original commit: elastic/x-pack-elasticsearch@57e5a401e7
2015-03-02 14:40:05 +01:00
Martijn van Groningen 14b1f6c3d2 Test: fixed test issue
Original commit: elastic/x-pack-elasticsearch@7fbce014d9
2015-03-02 14:31:37 +01:00
Martijn van Groningen 96d80597c4 Test: fixed test timing issue
Original commit: elastic/x-pack-elasticsearch@ccdac702f4
2015-03-02 14:17:37 +01:00
Martijn van Groningen 803fa4c4df Added tests for HistoryService#execute(...)
Original commit: elastic/x-pack-elasticsearch@b5c5fcf7f3
2015-03-02 11:18:18 +01:00
Martijn van Groningen a2b71a94b7 Increased logging to see why history index isn't deleted
Original commit: elastic/x-pack-elasticsearch@eb0f4e9e46
2015-03-02 10:56:15 +01:00
Martijn van Groningen 9e8405e619 applied feedback
Original commit: elastic/x-pack-elasticsearch@115429ae67
2015-02-27 11:00:10 +01:00
Martijn van Groningen 127aee514c Added unit test
Renamed AlertBootstrap to AlertsLifeCycleService

Original commit: elastic/x-pack-elasticsearch@5dedefe196
2015-02-27 10:14:53 +01:00
Martijn van Groningen bebfbf9664 Move the bootstrap logic out of AlertService which will make it easier to test.
Original commit: elastic/x-pack-elasticsearch@bf5c47dd5f
2015-02-27 10:14:52 +01:00
uboness aae6ff834f [client] reorganized tests and added alert source builder
The `AlertSourceBuilder` along with a set of source builder for all the different constructs that make an alert (condition, input, transform and action), provides a structured approach for building an alert from the client side (instead of forcing the clients to use xcontent directory)

- fixed some of the tests to already use these builders (I reckon there are still quite a few that need to be converted.. but we'll do that over time).
- moved all integration tests under `test/integration` package.
- changed the `AlertsTests` to **not** be an integration test... it randomizes the alert structure and makes sure that it can serialize & deserialize itself to/from xcontent.
- fixed small bugs found by the tests

Original commit: elastic/x-pack-elasticsearch@94b76b6fc7
2015-02-27 08:01:10 +02:00
Martijn van Groningen b76b0e7129 Test: change assertion to make sense
Original commit: elastic/x-pack-elasticsearch@8f01dcff03
2015-02-26 19:26:07 +01:00
Martijn van Groningen 4363acb09b Internal: Only clear the alert execution threadpool's queue when alerts plugin stops.
Before we shutdown the alert execution threadpool, which caused us to use a hacky workaround to get the thread pool started again when alerts is going to run again.
Clearing the threadpool's queue is sufficient for stopping fired alerts from being ran. Only fired alerts already being executed by TP will won't be stopped.

Also removed the volatile previousFiredAlerts field, because execution the fired alert doesn't need the AlertService anymore the purpose of this field doesn't exist any more.

Original commit: elastic/x-pack-elasticsearch@6a622b5579
2015-02-26 14:30:37 +01:00
Martijn van Groningen 469acfa551 Test: Fix SearchTransformTests suite error
Original commit: elastic/x-pack-elasticsearch@03568dce09
2015-02-26 12:43:25 +01:00
uboness 0f7dc295b3 [script] utilize Script in ScriptTemplate & ScriptCondition
- also changed the `AbstractAlertsSingleNodeTests` to not reset after each test

Original commit: elastic/x-pack-elasticsearch@14377498e8
2015-02-26 13:35:51 +02:00
Martijn van Groningen de681481d1 Tests: wait for alerting to be stopped and added a TODO about using a start/stop lock
Original commit: elastic/x-pack-elasticsearch@7404bb56b0
2015-02-26 11:56:27 +01:00
Martijn van Groningen 42716a5adb Test: pass down parameters
Original commit: elastic/x-pack-elasticsearch@2210246872
2015-02-26 11:05:52 +01:00
Martijn van Groningen 4934267789 test: increase timeout
Original commit: elastic/x-pack-elasticsearch@abf2297f87
2015-02-26 08:36:49 +01:00
Martijn van Groningen c016516d7e test: evaluate if this prevents the data loss that causes no alerts being loaded.
Original commit: elastic/x-pack-elasticsearch@611f3df816
2015-02-25 23:31:11 +01:00
Martijn van Groningen 1aad9fd390 typo
Original commit: elastic/x-pack-elasticsearch@3ae5403f39
2015-02-25 22:33:44 +01:00
Martijn van Groningen 57d4fd0d50 test: increased logging
Original commit: elastic/x-pack-elasticsearch@b1cfced485
2015-02-25 22:27:05 +01:00
Martijn van Groningen ab43bde67f adjust log level
Original commit: elastic/x-pack-elasticsearch@ddbe44ed25
2015-02-25 22:27:05 +01:00
uboness df491d036f [transform] added a new chain transform
Enables chaining multiple transforms

Original commit: elastic/x-pack-elasticsearch@312b7330df
2015-02-25 17:33:23 +02:00
uboness 46f6572756 [transform] added a new script transform
Enables manipulating existing payload and building a new payload based on a script

Original commit: elastic/x-pack-elasticsearch@912dafe709
2015-02-25 17:22:24 +02:00
uboness 7ab8271692 [test] introduced AlertsSingleNodeTests
- changed SearchTransformTests to extend it

Original commit: elastic/x-pack-elasticsearch@19d1d32d1a
2015-02-25 15:36:35 +02:00
uboness 3aa988472c [transform] fixed and added unit tests for SearchTransform
- Also, the search template/script are not populated not just by the fired/scheduled time, but also by the payload

Original commit: elastic/x-pack-elasticsearch@7ca8331a1c
2015-02-25 14:52:19 +02:00
Brian Murphy 46cefe261a Now input is separate from condition and condition just contains the decision logic.
```
    "input": {
        "search": {
            "request": {
                "body": {
                    "query": {
                        "match_all": {}
                    }
                }
            }
        }
    },
    "condition": {
        "script": {
            "script": "return true"
        }
    },
```
The result of this in the `alert_execution` looks like :
```
        "input_result": {
            "search": {
                "payload": {
                    "hits": {
                        "total": 1,
                        "hits": [
                            {
                                "_type": "my-type",
                                "_source": {
                                    "field": "value"
                                },
                                "_id": "AUujS61M4FTW2U3Ztz5U",
                                "_index": "my-index",
                                "_score": 0.30685282
                            }
                        ],
                        "max_score": 0.30685282
                    },
                    "_shards": {
                        "total": 5,
                        "failed": 0,
                        "successful": 5
                    },
                    "timed_out": false,
                    "took": 1823
                },
                "request": {
                    "body": {
                        "query": {
                            "match_all": {}
                        }
                   }
                }
            }
      }
      "condition_result": {
        "script": {
            "met": true
        }
      }
```
There are two Inputs currently the `SearchInput` as shown above and a `SimpleInput` that just contains a payload that will be returned in the result.
There are three conditions, the `ScriptCondition` as shown above and an `AlwaysTrueCondition` and AlwaysFalseCondition` condition.

Original commit: elastic/x-pack-elasticsearch@0d8ac24c5a
2015-02-24 12:05:30 -05:00
uboness fa02c150b4 [throttler] added unit tests for all throttlers
Original commit: elastic/x-pack-elasticsearch@3014d2b723
2015-02-24 14:45:02 +02:00