Commit Graph

236 Commits

Author SHA1 Message Date
Brian Murphy e79a98c568 REST : Fix rendering on GetAlerts
Original commit: elastic/x-pack-elasticsearch@81fc5d86df
2014-11-13 18:23:11 +00:00
Martijn van Groningen cabdcd862b Core: Use reading/writing search request infra also in alert action
Original commit: elastic/x-pack-elasticsearch@14dbbfe164
2014-11-13 19:22:41 +01:00
Martijn van Groningen eb3f123ad6 Core: Better infra for reading/writing search request in alert
Original commit: elastic/x-pack-elasticsearch@febd43058b
2014-11-13 19:14:36 +01:00
Martijn van Groningen 6a5a1710d0 Core: Change <<< placeholder to {{
Original commit: elastic/x-pack-elasticsearch@39d2de2dd8
2014-11-13 18:39:23 +01:00
Martijn van Groningen f58b36b70a Core: fix trigger search for templates
Original commit: elastic/x-pack-elasticsearch@11cdc8a396
2014-11-13 18:36:07 +01:00
Brian Murphy c070e932c3 CORE : add keyed lock.
This commit adds a keyed lock to prevent concurrent modification of the alert store and alert index.

Original commit: elastic/x-pack-elasticsearch@416351c06d
2014-11-13 16:38:49 +00:00
Martijn van Groningen 1e593a4075 Build: Added mail and activation to depdency sets for the assembly plugin
Original commit: elastic/x-pack-elasticsearch@3af4f1b8d9
2014-11-13 15:55:16 +01:00
Brian Murphy d20bd52a27 REST : actually add the handlers.
Original commit: elastic/x-pack-elasticsearch@fce0de9dea
2014-11-13 14:27:25 +00:00
Brian Murphy e2340a9645 Merge branch 'master' of https://github.com/elasticsearch/elasticsearch-alerts
Original commit: elastic/x-pack-elasticsearch@9169ddaabe
2014-11-13 14:21:11 +00:00
Brian Murphy 31baa2aa47 REST : Add stats and get alert rest handlers.
Original commit: elastic/x-pack-elasticsearch@f402c13fc5
2014-11-13 14:20:05 +00:00
Martijn van Groningen d801537c9e Test: Added trigger search test
Core: Modified TriggerManager#prepareTriggerSearch to always produce a new instance.

Original commit: elastic/x-pack-elasticsearch@4041898645
2014-11-13 13:42:51 +01:00
Brian Murphy 8b2c6ba39d CORE : Add alerts stats.
This commit adds the alerts stats actions.

Original commit: elastic/x-pack-elasticsearch@c3c471de74
2014-11-13 11:25:28 +00:00
Martijn van Groningen b731f84d68 Core: Removed unused methods.
Original commit: elastic/x-pack-elasticsearch@1aac9a440e
2014-11-12 22:11:45 +01:00
Martijn van Groningen cd2c74a6a0 Core: Simplify the template helper by using the actual put index template api instead of the MetaDataIndexTemplateService directly which allows for code reuse.
Original commit: elastic/x-pack-elasticsearch@1c646f3448
2014-11-12 22:04:41 +01:00
Martijn van Groningen 4d581876bd Cleanup: removed unused field and method
Original commit: elastic/x-pack-elasticsearch@058e57f6c4
2014-11-12 18:03:17 +01:00
Martijn van Groningen 3bed2c92e2 Core: Retry shouldn't cause a new cluster state, but rather should be done in a forked thread.
Original commit: elastic/x-pack-elasticsearch@cb944ddce7
2014-11-12 18:02:15 +01:00
Brian Murphy 36dc82bcc5 AlertActions : Change the AlertActions to be simple POJOs and do the actions in the factories.
With this commit alert actions are simple pojos with the action descriptions. The actual actions are undertaken by the AlertActionFactories.

Original commit: elastic/x-pack-elasticsearch@d47b453018
2014-11-12 11:06:35 +00:00
Martijn van Groningen e87b0a980c Test: Fixed local discovery issue when running tests for plugins
Core: synchronized initializing and stopping the scheduler to avoid scheduler stops leaving leaking threads behind

Original commit: elastic/x-pack-elasticsearch@b845651430
2014-11-09 22:21:51 +00:00
Martijn van Groningen 15c1b4c56d Core: Changed the way index templates are installed
Before the index templates were installed via json files in the config directory, this commit installs templates from the classpath via the put index template api.
Also template versioning is enforced.

Original commit: elastic/x-pack-elasticsearch@22f6596015
2014-11-09 20:50:03 +01:00
Martijn van Groningen b62da0691b Core: Make sure deletes are unscheduled and no npe is returned when a alert is deleted twice
Core: When an alert searches in an index that doesn't exist it shouldn't fail

Original commit: elastic/x-pack-elasticsearch@fc4ab8f823
2014-11-08 02:13:59 +01:00
Martijn van Groningen 4b089cb157 Build: include dependencies in plugin assembly
Original commit: elastic/x-pack-elasticsearch@276e87d8f8
2014-11-08 01:47:01 +01:00
Martijn van Groningen 94cf006b8f Replaced quartz thread pool by an impl that is backed by ES thread pool
Original commit: elastic/x-pack-elasticsearch@24a055dc00
2014-11-07 23:49:44 +01:00
Martijn van Groningen b11f0bf6df Test: changed default node settings
Core: AlertManager sets itself now

Original commit: elastic/x-pack-elasticsearch@fad2318528
2014-11-07 21:47:02 +01:00
Brian Murphy 7a23074c7c Actually add the new TriggerManager classes.
These should have been added in the previous commit.

Original commit: elastic/x-pack-elasticsearch@9efecc7ace
2014-11-07 17:42:43 +00:00
Brian Murphy 5d8f43225a Alerts update trigger manager.
This change changes the trigger manager to be pluggable.
Also removes the SimpleTrigger class, for now all triggers should be scripts.

Original commit: elastic/x-pack-elasticsearch@f7d0cb77e7
2014-11-07 17:40:37 +00:00
Martijn van Groningen f2453f53ac Test: Added no master node test
Original commit: elastic/x-pack-elasticsearch@0edeaba3a7
2014-11-07 13:21:15 +01:00
Brian Murphy 7efeffd2c2 Add License header to all files and javadocs for AlertClientInterface.
This commit adds the license header to all java files and enforces the license check on compile.
It also adds javadocs for all the methods in the AlertClientInterface

Original commit: elastic/x-pack-elasticsearch@2ec6f89b4b
2014-11-07 10:38:43 +00:00
Martijn van Groningen 0f0436e128 Cleaned up rest layer
Original commit: elastic/x-pack-elasticsearch@cdcdc98383
2014-11-06 23:49:50 +01:00
Martijn van Groningen 22cf7f68f4 Make tests work with multiple nodes.
Original commit: elastic/x-pack-elasticsearch@68004e1d6c
2014-11-06 22:52:44 +01:00
Martijn van Groningen 8f0e454ac5 Clear alerts after each test run, so that we don't run into exceptions between wipe indices and node closing.
Improved stop logging

Original commit: elastic/x-pack-elasticsearch@195a4cb75d
2014-11-06 22:15:11 +01:00
Martijn van Groningen 5be1c5964c on going work for improving test coverage
Original commit: elastic/x-pack-elasticsearch@1ccb9c9d9b
2014-11-06 21:25:53 +01:00
Brian Murphy 89dd5e2599 Move alert execution out of scheduler thread.
This commit moves the execution of the alert trigger to the AlertActions queued thread.
Also change the states of AlertActionEntry to include SEARCH_NEEDED and ERROR and remove now uneeded states.
AlertActions now take a TriggerResult instead of an alert action entry.

Original commit: elastic/x-pack-elasticsearch@2b650ca4c1
2014-11-06 13:34:48 +00:00
Brian Murphy f86544497a Remove extraneous methods and change IndexAlertAction.NAME
This commit removes the uneeded addAlert methods in AlertManager and AlertsStore and
also changes the IndexAlertAction.NAME to "indices:data/write/alert/index"

Original commit: elastic/x-pack-elasticsearch@19013dcf3e
2014-11-06 11:14:58 +00:00
Brian Murphy bce97abc1d Refactor and rename transport layer
This commit removes the update action and renames the create action to index.
It also hides the Alert objects behind the interface rather than exposing them.
This means that all interactions should be via BytesReferences.

Original commit: elastic/x-pack-elasticsearch@87ac377d3e
2014-11-06 11:08:57 +00:00
Martijn van Groningen 0f78028190 * Update the alert version on each write
* Make AlertStore#addAlert() return a tuple<Alert, IndexResponse>

Original commit: elastic/x-pack-elasticsearch@b2f270caaf
2014-11-06 11:30:52 +01:00
Martijn van Groningen 9c81a2ab87 * Simplified add alert action logic.
* Reduce needed code to index Alert and AlertActionEntry
* Added scroll for loading action needed alert action entries

Original commit: elastic/x-pack-elasticsearch@a7b790ecc4
2014-11-06 09:40:44 +01:00
Brian Murphy 6422e8b5d5 Add IndexResponse to Update and Create Alert Responses.
This commit adds the IndexResponse if available to the Update and Create
Alert Response objects.

Original commit: elastic/x-pack-elasticsearch@5fed850c61
2014-11-05 18:08:22 +00:00
Brian Murphy 5d52c374b3 Merge branch 'master' of https://github.com/elasticsearch/elasticsearch-alerts
Original commit: elastic/x-pack-elasticsearch@d91926ebdf
2014-11-05 17:50:54 +00:00
Brian Murphy 41821c39c6 DeleteAlert : Add the delete response to the DeleteAlertResponse
This commit adds the delete response to the DeleteAlertResponse.

Original commit: elastic/x-pack-elasticsearch@a5cc31f321
2014-11-05 17:50:05 +00:00
Martijn van Groningen b07b0d497d AlertManager#stop() didn't stop, the expected and update was reversed in compareAndSet
Original commit: elastic/x-pack-elasticsearch@5b001af292
2014-11-05 16:13:22 +01:00
Martijn van Groningen 5a957ed678 Introduced dedicated alerting thread pool, which executes alert actions
Original commit: elastic/x-pack-elasticsearch@e49ecb3f7c
2014-11-05 16:08:27 +01:00
Brian Murphy 47a78648df Alerts : Cleanup
This commit :
1. Ensures an alert will run even if the lastFireTime is null (first time we see an alert)
2. Removes the incorrect license that existed in some of the files.
3. Cuts the BasicTest over to use the AlertsClient and deletes the alert at the end of the test
to prevent it running in a race condition.
4. Moves the execute method signatures in AlertsClient to be on a single line.

Original commit: elastic/x-pack-elasticsearch@9c3aec0127
2014-11-05 10:05:17 +00:00
Brian Murphy 9a2e594647 Merge branch 'master' of https://github.com/elasticsearch/elasticsearch-alerts
Original commit: elastic/x-pack-elasticsearch@258c047506
2014-11-04 17:24:57 +00:00
Brian Murphy 4f58b9c5ff Alerts : Forgot to add the AlertsClient package.
Sorry.

Original commit: elastic/x-pack-elasticsearch@8231a3845b
2014-11-04 17:24:18 +00:00
Brian Murphy 9f3f23698f Merge branch 'add-transport-actions' of https://github.com/elasticsearch/elasticsearch-alerts into add-transport-actions
Conflicts:
	src/main/java/org/elasticsearch/alerts/AlertManager.java
	src/main/java/org/elasticsearch/alerts/AlertsStore.java
	src/main/java/org/elasticsearch/alerts/transport/actions/update/TransportUpdateAlertAction.java
	src/main/java/org/elasticsearch/alerts/transport/actions/update/UpdateAlertResponse.java
	src/test/java/org/elasticsearch/alerts/actions/AlertActionsTest.java

Original commit: elastic/x-pack-elasticsearch@3f1b58a2ee
2014-11-04 17:09:40 +00:00
Brian Murphy 85655ac16d Alerts : Alert Transport Layer
This commit fixes the test and tests to make sure the alert we get back from
the alertsstore isn't null in execute alert since it may have been deleted.

Original commit: elastic/x-pack-elasticsearch@7a57dd090a
2014-11-04 17:03:03 +00:00
Brian Murphy a05fc88f0d Alerts: Transport actions
Add the UpdateAction (oops) and fix the test.

Original commit: elastic/x-pack-elasticsearch@ead7d446d7
2014-11-04 16:49:56 +00:00
Brian Murphy 75ce20ecff Alerting transport layer.
This commit adds the transport layer to the alerting plugin.

Original commit: elastic/x-pack-elasticsearch@a99dce9411
2014-11-04 16:49:56 +00:00
Martijn van Groningen 669203b2a1 Removed checks for creating alert & history index from code and use index templates instead
Original commit: elastic/x-pack-elasticsearch@c81134701f
2014-11-04 17:23:05 +01:00
Brian Murphy 69942c6c79 Alerting transport layer.
This commit adds the transport layer to the alerting plugin.

Original commit: elastic/x-pack-elasticsearch@0b6cb4760f
2014-11-04 16:21:57 +00:00
Martijn van Groningen c8c30f9b28 Properly resolve the place holders in the request source
Original commit: elastic/x-pack-elasticsearch@0d19c920d1
2014-11-04 16:01:20 +01:00
Martijn van Groningen 8ba9c1c609 Cleared up the alert / alert action entry classes and how the search request in the trigger manager gets prepared.
Original commit: elastic/x-pack-elasticsearch@52b16abb53
2014-11-04 11:27:58 +01:00
Martijn van Groningen 4373330a28 Changed the alert manager starting logic to move the loading logic down to alert store and alert action manager
Moved logic around from alert scheduler to action manager and trigger manager.

Original commit: elastic/x-pack-elasticsearch@8cba72b005
2014-10-31 10:28:42 +01:00
Martijn van Groningen 3e45310877 make more readable
Original commit: elastic/x-pack-elasticsearch@c7727618ac
2014-10-29 14:08:06 +01:00
Martijn van Groningen 12a6de0a57 Moved alert loading to use scan scroll instead of a single normal search
Original commit: elastic/x-pack-elasticsearch@06e70836ec
2014-10-29 10:20:36 +01:00
Martijn van Groningen 1e7fc84f06 Forgot to parse field
Original commit: elastic/x-pack-elasticsearch@23512bce74
2014-10-29 09:46:39 +01:00
Martijn van Groningen 3625b5bc91 Moved over the streaming parsing.
Original commit: elastic/x-pack-elasticsearch@10bd127df5
2014-10-29 00:54:18 +01:00
Martijn van Groningen 6b2fbe400e Introduced AlertsStore that is responsible for maintaining / storing / parsing etc of alerts
Original commit: elastic/x-pack-elasticsearch@40aae7dc30
2014-10-28 18:49:23 +01:00
Martijn van Groningen 34c359281a Added a todo
Original commit: elastic/x-pack-elasticsearch@56f5b1326b
2014-10-28 16:35:41 +01:00
Brian Murphy 4da8f9fec7 AlertActionManager: Add alert action job queue
This change adds the AlertActionManager and AlertActionEntry. The old AlertActionManager has become the AlertActionRegistry.
This means that now the results of Alerts are queued up in a job queue and executed in separate threads.
The AlertActionManager is a composite member of the AlertManager.
Change the BasicTest to just run on a single node to fix the action registration if the action happens on a different node.
Threads are not directly constructed but now the threadpool is used.
The ClusterStateListener in AlertManager is responsible now for starting the job queue.

Original commit: elastic/x-pack-elasticsearch@a73c6b60f8
2014-10-28 13:49:04 +00:00
Martijn van Groningen a23487cd38 Fixed build by:
* Using cluster state listener to clear alerts when .alerts index is removed. (when running on multiple nodes the .alerts index is scattered so indices listener doesn't work).
* Remove the starting / loading thread and move the initial loading to cluster state listener.

Original commit: elastic/x-pack-elasticsearch@b8f41db2ea
2014-10-25 23:37:05 +02:00
Martijn van Groningen 449edcda1d * Throw a understandle error if an alert action doesn't exist
* Moved over to a copy on write map instead of a hash map that is protected by synchronized blocks

Original commit: elastic/x-pack-elasticsearch@285515d585
2014-10-24 18:50:47 +02:00
Martijn van Groningen 1a32243781 Changes to AlertManager:
* Made use of IndicesLifecycle, to catch when the .alerts index gets deleted, so we can clear the alerts
* Moved to concurrent hashmap in favour over normal hashmap with synchronized blocks

Original commit: elastic/x-pack-elasticsearch@5599d01c78
2014-10-24 16:51:12 +02:00
Martijn van Groningen 16a7991d6d Rename and move the plugin class.
Original commit: elastic/x-pack-elasticsearch@b6f7a0490e
2014-10-24 14:31:20 +02:00
Brian Murphy ac979c880d Alerting : rename alerting package to alerts and fix test.
This commit renames the alerting package to alerts and will create the
alerts index on addAlert if needed.

Original commit: elastic/x-pack-elasticsearch@7cd691bd9b
2014-10-24 13:01:45 +01:00
Brian Murphy db9fae8021 Alerting: Split alerting into packages
This commit adds separate packages for actions,triggers,rest, and the scheduler.

Original commit: elastic/x-pack-elasticsearch@e104bbc521
2014-10-24 12:32:18 +01:00
Martijn van Groningen 75ef2dc3b3 Initial step to running alerts on master and added a very simple test.
Original commit: elastic/x-pack-elasticsearch@480f6bd44b
2014-10-24 12:49:33 +02:00
Brian Murphy 8c623534c2 Alerting : Cleanup and fixes.
These changes fix the alert throttling during the time period, move alert history it's own
index.

Original commit: elastic/x-pack-elasticsearch@5130637824
2014-08-21 16:33:11 +01:00
Brian Murphy 47e1e77b58 Alerting : Start of multiprocess coordination work
On alert claim time, query index to make sure alert is still valid and enabled and is the same alert.

Original commit: elastic/x-pack-elasticsearch@72b816b7d1
2014-08-19 13:31:01 +01:00
Brian Murphy 0eea73dd72 Alerting: Scripted triggers and support for aggregations in searches.
This commit adds support for triggers that are scripts:

Query :
````
POST /_search/template/testFilteredAgg
{
  "query" : {
    "filtered" : {
      "query" : {
        "match_all" : {}
     },
     "filter": {
       "range" : {
         "@timestamp" : {
             "gte" : "{{from}}",
             "lt" : "{{to}}"
         }
       }
     }
    }
  },
    "aggs" : {
      "response" : {
        "terms" : {
          "field" : "response",
          "size" : 100
        }
      }
}, "size" : 0  }
````

Trigger Script:
````
POST /_scripts/groovy/testScript
{
  "script" : "ok_count = 0.0;error_count = 0.0;for(bucket in aggregations.response.buckets) {if (bucket.key < 400){ok_count += bucket.doc_count;} else {error_count += bucket.doc_count;}}; return error_count/(ok_count+1) > 0.1;"
}
````

Alert:
````
POST /_alerting/_create/myScriptedAlert
{
    "query" : "testFilteredAgg",
    "schedule" : "05 * * * * ?",
    "trigger" : {
         "script" : {
           "script" : "testScript",
           "script_lang" : "groovy",
           "script_type" : "INDEXED"
         }
     },
    "timeperiod" : "300s",
     "action" : {
         "index" : {
           "index" : "weberrorhistory",
           "type" : "weberrorresult"
         }
     },
    "indices" : [ "logstash*" ],
    "enabled" : true,
    "simple" : false
}
````

If you want to use aggs with your alert you must create a search that contains the timefilter with the params ````{{from}}```` and ````{{to}}```` and set the ````simple```` flag to ````true````.

Original commit: elastic/x-pack-elasticsearch@0430a1bf40
2014-08-18 16:59:49 +01:00
Brian Murphy 4216491824 Alerting : Improve error handling in the RestInterface
This commit improves error handling in the RestHandler. Errors are now
returned to the client.

Original commit: elastic/x-pack-elasticsearch@433e8a57bc
2014-08-18 12:28:48 +01:00
Brian Murphy 14d0bcd589 Alerting add TODO
Original commit: elastic/x-pack-elasticsearch@23cf5fce8b
2014-08-18 12:22:13 +01:00
Brian Murphy 418b9f1a31 Alerting : Add scripted triggers and alert disabling.
This commit adds support for disabling alerts.
This commit adds preliminary support for scripted triggers.

Original commit: elastic/x-pack-elasticsearch@e14a56dbeb
2014-08-18 12:13:39 +01:00
Brian Murphy 4c1c502f80 Alerting : Fix IndexAction
This commit contains fixes to the IndexAlertAction and normalizes the serialization of
Alerts using ToXContent interface.

Original commit: elastic/x-pack-elasticsearch@47325bdd01
2014-08-18 11:23:45 +01:00
Brian Murphy 26e053eaed Alerting : Claim alert runs
This commit add functionality so that multiple nodes running alert plugins won't stand on
each other trying to run the same alerts at the same time.

Original commit: elastic/x-pack-elasticsearch@9c350c0114
2014-08-18 10:27:07 +01:00
Brian Murphy 83287e009a Alerting : Add index action.
This commit adds support for indexing on alert trigger.

Original commit: elastic/x-pack-elasticsearch@d6508ce16d
2014-08-17 13:49:29 +01:00
Brian Murphy d1af9f15fe Alerting : Small fixes.
Fix the constructor for the AlertResult.

Original commit: elastic/x-pack-elasticsearch@0820edaa6a
2014-08-16 17:44:44 +01:00
Brian Murphy 019cdb37d4 Alerting : Add support for history and fix issues around parsing lastRun.
This commit adds a history log for alerts and updates lastRan with the correct time.

Original commit: elastic/x-pack-elasticsearch@49c77f1ef8
2014-08-15 16:46:41 +01:00
Brian Murphy 4c47c8ba9a Alerting : REST endpoints
This commit adds REST endpoints for list/delete/create and fixes refresh.

Original commit: elastic/x-pack-elasticsearch@36e2c15753
2014-08-15 11:45:45 +01:00
Brian Murphy b5cb8b3d04 Alerting : Add RestHandler
This commit adds the rest handler and supports /_alerting/_refresh to reload all alerts.

Original commit: elastic/x-pack-elasticsearch@14fadbeeee
2014-08-13 18:00:36 +01:00
Brian Murphy 0d9061b838 Alerting : better email formatting
This commit enables better email formatting.

Original commit: elastic/x-pack-elasticsearch@8be3e3b6d1
2014-08-13 17:05:25 +01:00
Brian Murphy e3250c0366 Alerting : Add time clamping.
This commit adds timestamp clamping on the timeperiod to alerting.
Also adds index setting on alerts.

Original commit: elastic/x-pack-elasticsearch@222cd6eaef
2014-08-13 15:21:55 +01:00
Brian Murphy 88ed7a5624 Alerting : Add parsing and indices.
This commit adds parsing of the triggers and actions, and allows alerts to be restricted to indices.

Original commit: elastic/x-pack-elasticsearch@44a6a51b17
2014-08-13 10:11:51 +01:00
Brian Murphy 940d6402ae Alerting : connect email
The last commit ommited a line that actually sends the email on alert trigger.

Original commit: elastic/x-pack-elasticsearch@24b5c01db8
2014-08-12 19:54:13 +01:00
Brian Murphy 5f84596c92 Alerting : Add support for quartz scheduler.
This commit enables loading of alerts from the .alerts index and adds the
Quartz scheduler.
You can add the following alert :
````
curl -XPOST http://localhost:9200/.alerts/alert/myTestAlert -d '{
    "query" : "myAlertQuery",
    "schedule" : "00 * * * * ?",
    "trigger" : {
         "numberOfEvents" : ">1"
     },
    "timeperiod" : 300,
     "action" : {
         "email" : [ "brian.murphy@elasticsearch.com" ]
     },
    "version" : 1,
    "lastRan" : "2014-05-05T12:12:12.123Z"
}
````
With the following search template:
````
curl -XPOST localhost:9200/_search/template/myAlertQuery -d '{ "template" : { "query" : { "match_all" : {} } } }'
````
This will execute the search every minute and trigger if there is more than one match (which there will be).

Original commit: elastic/x-pack-elasticsearch@708f927914
2014-08-12 18:57:28 +01:00
Brian Murphy 1e6d6b58c9 Alerting Plugin : First commit
Very rough non-functional commit of the alerting plugin.
Please be gentle.

Original commit: elastic/x-pack-elasticsearch@98870d0778
2014-08-12 13:55:10 +01:00