Commit Graph

338 Commits

Author SHA1 Message Date
uboness f5c48576bc Cleaned up the use of execution context mocking in the tests
Added helper methods in `AlertsTestUtils` to mock `ExecutionContext` and create a simple payload

Original commit: elastic/x-pack-elasticsearch@eac6b63e62
2015-03-04 10:37:38 +01:00
uboness 4874fa2f1b change the SES email port to 587
Original commit: elastic/x-pack-elasticsearch@d69d02236b
2015-03-04 10:03:07 +01:00
uboness 01145b8025 Aligned Transform.Result with all other result constructs
- the transform parser and registry can now parse transform results
- every transform result may have its own format
- the chain transform result outputs the "transformation trail" of all the chained transforms (great tool for debugging)
- removed `Transform.NOOP` - was redundant, when no transform is defined the alert's transform is `null`
- removed `payload` from the `AlertExecution`. Instead, the execution holds the input result, transform result and potentially the transform results of the actions
- changed the xcontent representing a transofrm result to `{ "transform_type" : { "payload" : {...}, ... } }`
- with this change, the `Action` interface is cleaned up (`execute` only accepts the execution context)

Original commit: elastic/x-pack-elasticsearch@6ecf7f2c92
2015-03-04 09:52:31 +01:00
Martijn van Groningen 167f6814ab Fix tests
Original commit: elastic/x-pack-elasticsearch@ad7b4ae1ef
2015-03-04 09:42:57 +01:00
Martijn van Groningen 0382310cae Moved testAlertsWithDifferentSearchType and testWithAggregations to more unit like tests.
Original commit: elastic/x-pack-elasticsearch@115e361c42
2015-03-04 09:15:35 +01:00
uboness f6c17bd802 Added a transform construct to actions
An alert may have multiple actions associated with it, but each action may need to have different view of the data. For example, the email may need a certain model for its templates while the index or webhook actions may need a completely different data structure.

Until now, there's only an option to defina a single `transform` on the alert level that would have applied to all actions. This commit adds
 the ability to associate a transform with each action. We still keep the tranform on the alert level (in case all actions need the same transformation, in which case we'd like to avoid repetition).

Original commit: elastic/x-pack-elasticsearch@5493a2179b
2015-03-04 03:20:17 +01:00
uboness d25bf008b3 Added Amazon SES to ManualPublicSmtpServersTests
Original commit: elastic/x-pack-elasticsearch@d7f8180d1a
2015-03-03 23:41:56 +01:00
uboness dfc8249d4b added support for template params in search requests
Original commit: elastic/x-pack-elasticsearch@06e7dab935
2015-03-03 12:01:45 +02:00
uboness f9765fd393 standardized script/template model variables
Now `Variables.createCtxModel` is responsible for creating the model for scripts & templates across the board. Accessing the payload, alert name, fired time and scheduled fire time is done via the `ctx.` prefix.

Original commit: elastic/x-pack-elasticsearch@443ac17579
2015-03-03 10:27:47 +02:00
uboness 43f11cbb3d [fix] added the script params to the condition script execution
Now the script's params in the `script` condition are merged with the payload data into a single variable context to the script execution. The payload data is now accessed using the `payload.` prefix.

Original commit: elastic/x-pack-elasticsearch@e313a6301c
2015-03-02 22:01:02 +02:00
Martijn van Groningen 2aa91e84ed Now fix test for real
Original commit: elastic/x-pack-elasticsearch@57e5a401e7
2015-03-02 14:40:05 +01:00
Martijn van Groningen 14b1f6c3d2 Test: fixed test issue
Original commit: elastic/x-pack-elasticsearch@7fbce014d9
2015-03-02 14:31:37 +01:00
Martijn van Groningen 96d80597c4 Test: fixed test timing issue
Original commit: elastic/x-pack-elasticsearch@ccdac702f4
2015-03-02 14:17:37 +01:00
Martijn van Groningen 803fa4c4df Added tests for HistoryService#execute(...)
Original commit: elastic/x-pack-elasticsearch@b5c5fcf7f3
2015-03-02 11:18:18 +01:00
Martijn van Groningen a2b71a94b7 Increased logging to see why history index isn't deleted
Original commit: elastic/x-pack-elasticsearch@eb0f4e9e46
2015-03-02 10:56:15 +01:00
Martijn van Groningen 9e8405e619 applied feedback
Original commit: elastic/x-pack-elasticsearch@115429ae67
2015-02-27 11:00:10 +01:00
Martijn van Groningen 127aee514c Added unit test
Renamed AlertBootstrap to AlertsLifeCycleService

Original commit: elastic/x-pack-elasticsearch@5dedefe196
2015-02-27 10:14:53 +01:00
Martijn van Groningen bebfbf9664 Move the bootstrap logic out of AlertService which will make it easier to test.
Original commit: elastic/x-pack-elasticsearch@bf5c47dd5f
2015-02-27 10:14:52 +01:00
uboness aae6ff834f [client] reorganized tests and added alert source builder
The `AlertSourceBuilder` along with a set of source builder for all the different constructs that make an alert (condition, input, transform and action), provides a structured approach for building an alert from the client side (instead of forcing the clients to use xcontent directory)

- fixed some of the tests to already use these builders (I reckon there are still quite a few that need to be converted.. but we'll do that over time).
- moved all integration tests under `test/integration` package.
- changed the `AlertsTests` to **not** be an integration test... it randomizes the alert structure and makes sure that it can serialize & deserialize itself to/from xcontent.
- fixed small bugs found by the tests

Original commit: elastic/x-pack-elasticsearch@94b76b6fc7
2015-02-27 08:01:10 +02:00
Martijn van Groningen b76b0e7129 Test: change assertion to make sense
Original commit: elastic/x-pack-elasticsearch@8f01dcff03
2015-02-26 19:26:07 +01:00
Martijn van Groningen 4363acb09b Internal: Only clear the alert execution threadpool's queue when alerts plugin stops.
Before we shutdown the alert execution threadpool, which caused us to use a hacky workaround to get the thread pool started again when alerts is going to run again.
Clearing the threadpool's queue is sufficient for stopping fired alerts from being ran. Only fired alerts already being executed by TP will won't be stopped.

Also removed the volatile previousFiredAlerts field, because execution the fired alert doesn't need the AlertService anymore the purpose of this field doesn't exist any more.

Original commit: elastic/x-pack-elasticsearch@6a622b5579
2015-02-26 14:30:37 +01:00
Martijn van Groningen 469acfa551 Test: Fix SearchTransformTests suite error
Original commit: elastic/x-pack-elasticsearch@03568dce09
2015-02-26 12:43:25 +01:00
uboness 0f7dc295b3 [script] utilize Script in ScriptTemplate & ScriptCondition
- also changed the `AbstractAlertsSingleNodeTests` to not reset after each test

Original commit: elastic/x-pack-elasticsearch@14377498e8
2015-02-26 13:35:51 +02:00
Martijn van Groningen de681481d1 Tests: wait for alerting to be stopped and added a TODO about using a start/stop lock
Original commit: elastic/x-pack-elasticsearch@7404bb56b0
2015-02-26 11:56:27 +01:00
Martijn van Groningen 42716a5adb Test: pass down parameters
Original commit: elastic/x-pack-elasticsearch@2210246872
2015-02-26 11:05:52 +01:00
Martijn van Groningen 4934267789 test: increase timeout
Original commit: elastic/x-pack-elasticsearch@abf2297f87
2015-02-26 08:36:49 +01:00
Martijn van Groningen c016516d7e test: evaluate if this prevents the data loss that causes no alerts being loaded.
Original commit: elastic/x-pack-elasticsearch@611f3df816
2015-02-25 23:31:11 +01:00
Martijn van Groningen 1aad9fd390 typo
Original commit: elastic/x-pack-elasticsearch@3ae5403f39
2015-02-25 22:33:44 +01:00
Martijn van Groningen 57d4fd0d50 test: increased logging
Original commit: elastic/x-pack-elasticsearch@b1cfced485
2015-02-25 22:27:05 +01:00
Martijn van Groningen ab43bde67f adjust log level
Original commit: elastic/x-pack-elasticsearch@ddbe44ed25
2015-02-25 22:27:05 +01:00
uboness df491d036f [transform] added a new chain transform
Enables chaining multiple transforms

Original commit: elastic/x-pack-elasticsearch@312b7330df
2015-02-25 17:33:23 +02:00
uboness 46f6572756 [transform] added a new script transform
Enables manipulating existing payload and building a new payload based on a script

Original commit: elastic/x-pack-elasticsearch@912dafe709
2015-02-25 17:22:24 +02:00
uboness 7ab8271692 [test] introduced AlertsSingleNodeTests
- changed SearchTransformTests to extend it

Original commit: elastic/x-pack-elasticsearch@19d1d32d1a
2015-02-25 15:36:35 +02:00
uboness 3aa988472c [transform] fixed and added unit tests for SearchTransform
- Also, the search template/script are not populated not just by the fired/scheduled time, but also by the payload

Original commit: elastic/x-pack-elasticsearch@7ca8331a1c
2015-02-25 14:52:19 +02:00
Brian Murphy 46cefe261a Now input is separate from condition and condition just contains the decision logic.
```
    "input": {
        "search": {
            "request": {
                "body": {
                    "query": {
                        "match_all": {}
                    }
                }
            }
        }
    },
    "condition": {
        "script": {
            "script": "return true"
        }
    },
```
The result of this in the `alert_execution` looks like :
```
        "input_result": {
            "search": {
                "payload": {
                    "hits": {
                        "total": 1,
                        "hits": [
                            {
                                "_type": "my-type",
                                "_source": {
                                    "field": "value"
                                },
                                "_id": "AUujS61M4FTW2U3Ztz5U",
                                "_index": "my-index",
                                "_score": 0.30685282
                            }
                        ],
                        "max_score": 0.30685282
                    },
                    "_shards": {
                        "total": 5,
                        "failed": 0,
                        "successful": 5
                    },
                    "timed_out": false,
                    "took": 1823
                },
                "request": {
                    "body": {
                        "query": {
                            "match_all": {}
                        }
                   }
                }
            }
      }
      "condition_result": {
        "script": {
            "met": true
        }
      }
```
There are two Inputs currently the `SearchInput` as shown above and a `SimpleInput` that just contains a payload that will be returned in the result.
There are three conditions, the `ScriptCondition` as shown above and an `AlwaysTrueCondition` and AlwaysFalseCondition` condition.

Original commit: elastic/x-pack-elasticsearch@0d8ac24c5a
2015-02-24 12:05:30 -05:00
uboness fa02c150b4 [throttler] added unit tests for all throttlers
Original commit: elastic/x-pack-elasticsearch@3014d2b723
2015-02-24 14:45:02 +02:00
uboness 6fdd6d49e2 [throttler] changed the Throttler interface
Removed the passed in `Condition.Result` from the `apply` method

Original commit: elastic/x-pack-elasticsearch@23101b028d
2015-02-24 14:45:02 +02:00
uboness b292051a13 [email] fixed bugs and added unit tests
Original commit: elastic/x-pack-elasticsearch@3b5406d4c8
2015-02-24 12:27:13 +02:00
Martijn van Groningen d916f99800 The lock used during executing a fired alert should also encapsulate the updating of the fired alert.
During stopping alerts we wait until all ongoing operations are completed, such as adding, deleting alerts and executing fired alerts.
For fired alerts we should also let the update of fired alert happen under a lock. We could miss an ongoing operation if we would shutdown.

This can be a reason why BasicAlertingTest.testDeleteAlert failed: https://build.elasticsearch.com/job/es_alerting/187/testReport/junit/org.elasticsearch.alerts/BasicAlertingTest/testDeleteAlert/

Original commit: elastic/x-pack-elasticsearch@732b213cf2
2015-02-23 23:24:28 +01:00
Martijn van Groningen a3825aa242 Cleanup: removed unused method.
Original commit: elastic/x-pack-elasticsearch@5dd9126d71
2015-02-23 15:29:36 +01:00
uboness 6a5c86e46c Added a SchedulerMock to be used for unit tests
Original commit: elastic/x-pack-elasticsearch@d4dfb37431
2015-02-23 15:44:15 +02:00
uboness 8dffa5afd1 Enabled tests @Slow annotation
Original commit: elastic/x-pack-elasticsearch@4cabffcdf4
2015-02-23 15:44:15 +02:00
uboness 89b7d085e1 [scheduler] cleaned up and extended scheduler support
- Added additional user friendly schedules
 - `hourly` - a simple to configure schedule that will fire every hour on one or more specific minutes in the hour
 - `daily` - a simple to configure schedule that will fire every day on one or more specific times in the day
 - `weekly` - a simple to configure schedule that will fire every week on one or more specific days + times in the week
 - `monthly` - a simple to configure schedule that will fire every month on one or more specific days + times in the month
 - `yearly` - a simple to configure schedule that will fire every year on one or more specific months + days + times in the year
 - `interval` - a simple interval based schedule that will fire every fixed configurable interval (supported units are: seconds, minutes, hours, days and weeks)

- Added unit tests to all the schedules and the schedule registry
- Introduced `Scheduler` as an interface and `InternalScheduler` for the quartz implementation. This will help unit testing other dependent services
- `Scheduler` is now independent of `Alert`. It works with `Job` constructs (`Alert` now implements a `Job`).
- Introduced `SchedulerMock` as a simple `Scheduler` implementation that can be used for unit tests - enables manual triggering of jobs.

- introduced `@Slow` test annotation support in the `pom.xml`

Original commit: elastic/x-pack-elasticsearch@94a8f5ddea
2015-02-23 15:44:14 +02:00
Martijn van Groningen 59f0883721 Cleanup: Removed config service infrastructure
Original commit: elastic/x-pack-elasticsearch@3f99c357c8
2015-02-23 14:36:17 +01:00
uboness 2136210711 [template] formalized the template definition
- Introduced `Template` & `Template.Parser` interfaces
- There main template implementation is the `ScriptTemplate` and its parser is bound to `Template.Parser`
- There are also xContent templates - YAML & JSON that just render the model as xContent. (used as a fallback in webhook action)
- updated all actions to use the new template infrastructure

Also

- introduced mockito for unit testing
- removed `WebhookTest` as it was effectively testing the template functionality... we'll add a proper test for teh webhook action in a later commit

Original commit: elastic/x-pack-elasticsearch@34a90e8c2f
2015-02-23 12:30:31 +01:00
uboness 48bf4e8a8b Update README.asciidoc
Original commit: elastic/x-pack-elasticsearch@ec5c9c23ad
2015-02-21 13:03:13 +01:00
Martijn van Groningen c0c5b3cf6b Test: just create json based alerts
Original commit: elastic/x-pack-elasticsearch@6aa88ac5a1
2015-02-18 20:50:44 +01:00
Martijn van Groningen b4a465496a Test: increased logging
Original commit: elastic/x-pack-elasticsearch@8f95a60683
2015-02-18 20:11:51 +01:00
uboness aa529f1c3b update core constructs definition
Original commit: elastic/x-pack-elasticsearch@d550e0abd1
2015-02-17 22:35:00 +01:00
Martijn van Groningen 69bbea6985 Changed the initialization retry logic of the AlertService to be event based instead of blocking (actively polling for a new cluster state).
This avoids that a single thread will be busy during the time that not all primary shards of the alerts and alert history indices are started.

Also the execution of alert history items that were loaded during initialization will be executed once the AlertService goes into started state, before this was executed once the AlertActionService has started, which could load to failures, because there was a small window of time where the alert manager wasn't started. Executing alert history items with the state search_needed requires the alert manager to be started and that isn't yet the case when the AlertActionService has started.

Closes elastic/elasticsearch#75
Closes elastic/elasticsearch#76

Original commit: elastic/x-pack-elasticsearch@a799bc34e3
2015-02-17 19:35:18 +01:00