Commit Graph

4899 Commits

Author SHA1 Message Date
uboness 9c55be1530 Added transport filtering
Requests are now auth'ed on the transport level as well (in addition to the transport action level). This is required as some internal requests are not executed as actions, thus not going through the auth process in the transport action. Since we have n2n authentication, we also assume here that requests that are not associated with an authentication token are internal system calls. We then, auth the request as a system user.

Also Added a system realm (to handle system requests)

Original commit: elastic/x-pack-elasticsearch@2c917318f0
2014-08-18 14:01:11 -07:00
Brian Murphy 0eea73dd72 Alerting: Scripted triggers and support for aggregations in searches.
This commit adds support for triggers that are scripts:

Query :
````
POST /_search/template/testFilteredAgg
{
  "query" : {
    "filtered" : {
      "query" : {
        "match_all" : {}
     },
     "filter": {
       "range" : {
         "@timestamp" : {
             "gte" : "{{from}}",
             "lt" : "{{to}}"
         }
       }
     }
    }
  },
    "aggs" : {
      "response" : {
        "terms" : {
          "field" : "response",
          "size" : 100
        }
      }
}, "size" : 0  }
````

Trigger Script:
````
POST /_scripts/groovy/testScript
{
  "script" : "ok_count = 0.0;error_count = 0.0;for(bucket in aggregations.response.buckets) {if (bucket.key < 400){ok_count += bucket.doc_count;} else {error_count += bucket.doc_count;}}; return error_count/(ok_count+1) > 0.1;"
}
````

Alert:
````
POST /_alerting/_create/myScriptedAlert
{
    "query" : "testFilteredAgg",
    "schedule" : "05 * * * * ?",
    "trigger" : {
         "script" : {
           "script" : "testScript",
           "script_lang" : "groovy",
           "script_type" : "INDEXED"
         }
     },
    "timeperiod" : "300s",
     "action" : {
         "index" : {
           "index" : "weberrorhistory",
           "type" : "weberrorresult"
         }
     },
    "indices" : [ "logstash*" ],
    "enabled" : true,
    "simple" : false
}
````

If you want to use aggs with your alert you must create a search that contains the timefilter with the params ````{{from}}```` and ````{{to}}```` and set the ````simple```` flag to ````true````.

Original commit: elastic/x-pack-elasticsearch@0430a1bf40
2014-08-18 16:59:49 +01:00
Alexander Reelsen 452367b674 Return correct HTTP headers on security exceptions
* Changed Authentication/AuthorizationException to always return RestStatus.UNAUTHORIZED
* Added the WWW-Authenticate header, which results in user/pass input in browsers
* Added tests and own assertions class

Closes elastic/elasticsearch#18

Original commit: elastic/x-pack-elasticsearch@c6ce084692
2014-08-18 15:15:28 +02:00
Brian Murphy 4216491824 Alerting : Improve error handling in the RestInterface
This commit improves error handling in the RestHandler. Errors are now
returned to the client.

Original commit: elastic/x-pack-elasticsearch@433e8a57bc
2014-08-18 12:28:48 +01:00
Brian Murphy 14d0bcd589 Alerting add TODO
Original commit: elastic/x-pack-elasticsearch@23cf5fce8b
2014-08-18 12:22:13 +01:00
Brian Murphy 418b9f1a31 Alerting : Add scripted triggers and alert disabling.
This commit adds support for disabling alerts.
This commit adds preliminary support for scripted triggers.

Original commit: elastic/x-pack-elasticsearch@e14a56dbeb
2014-08-18 12:13:39 +01:00
Brian Murphy 4c1c502f80 Alerting : Fix IndexAction
This commit contains fixes to the IndexAlertAction and normalizes the serialization of
Alerts using ToXContent interface.

Original commit: elastic/x-pack-elasticsearch@47325bdd01
2014-08-18 11:23:45 +01:00
Brian Murphy 26e053eaed Alerting : Claim alert runs
This commit add functionality so that multiple nodes running alert plugins won't stand on
each other trying to run the same alerts at the same time.

Original commit: elastic/x-pack-elasticsearch@9c350c0114
2014-08-18 10:27:07 +01:00
Brian Murphy 83287e009a Alerting : Add index action.
This commit adds support for indexing on alert trigger.

Original commit: elastic/x-pack-elasticsearch@d6508ce16d
2014-08-17 13:49:29 +01:00
Brian Murphy d1af9f15fe Alerting : Small fixes.
Fix the constructor for the AlertResult.

Original commit: elastic/x-pack-elasticsearch@0820edaa6a
2014-08-16 17:44:44 +01:00
Brian Murphy 019cdb37d4 Alerting : Add support for history and fix issues around parsing lastRun.
This commit adds a history log for alerts and updates lastRan with the correct time.

Original commit: elastic/x-pack-elasticsearch@49c77f1ef8
2014-08-15 16:46:41 +01:00
Brian Murphy 4c47c8ba9a Alerting : REST endpoints
This commit adds REST endpoints for list/delete/create and fixes refresh.

Original commit: elastic/x-pack-elasticsearch@36e2c15753
2014-08-15 11:45:45 +01:00
Brian Murphy e648cc7f82 Alerting : Update README.md
Add sample alert to README.md and describe the email output.

Original commit: elastic/x-pack-elasticsearch@2b78e26dfd
2014-08-13 18:05:56 +01:00
Brian Murphy b5cb8b3d04 Alerting : Add RestHandler
This commit adds the rest handler and supports /_alerting/_refresh to reload all alerts.

Original commit: elastic/x-pack-elasticsearch@14fadbeeee
2014-08-13 18:00:36 +01:00
Brian Murphy 0d9061b838 Alerting : better email formatting
This commit enables better email formatting.

Original commit: elastic/x-pack-elasticsearch@8be3e3b6d1
2014-08-13 17:05:25 +01:00
Brian Murphy e3250c0366 Alerting : Add time clamping.
This commit adds timestamp clamping on the timeperiod to alerting.
Also adds index setting on alerts.

Original commit: elastic/x-pack-elasticsearch@222cd6eaef
2014-08-13 15:21:55 +01:00
uboness 1f5f3f21f9 Changed the contract of AuthenticationService#token to throw an authentication exception when no token found
- Also added an overloaded version of the AuthenticationService#token method that accepts a default token and that doesn't thrown an authentication exception
- Added AuditTrail#authenticationFailed method at a higher level than the realm version. Now the realm authc failure will only be logged when trace is enabled. With this change, the audit trail logging is more consistent in its terminology (anonymous is now logged when no auth token is found)
- extended the level of audit trail logging tuning (now using all logging levels, incl. error & warn)
- Added tests for audit trails
- Added tests for authentication service
- Added mockito as a test lib (will serve as our mocking framework)

Original commit: elastic/x-pack-elasticsearch@8d21ab7484
2014-08-13 14:23:28 +02:00
Brian Murphy 88ed7a5624 Alerting : Add parsing and indices.
This commit adds parsing of the triggers and actions, and allows alerts to be restricted to indices.

Original commit: elastic/x-pack-elasticsearch@44a6a51b17
2014-08-13 10:11:51 +01:00
Brian Murphy 940d6402ae Alerting : connect email
The last commit ommited a line that actually sends the email on alert trigger.

Original commit: elastic/x-pack-elasticsearch@24b5c01db8
2014-08-12 19:54:13 +01:00
Brian Murphy 5f84596c92 Alerting : Add support for quartz scheduler.
This commit enables loading of alerts from the .alerts index and adds the
Quartz scheduler.
You can add the following alert :
````
curl -XPOST http://localhost:9200/.alerts/alert/myTestAlert -d '{
    "query" : "myAlertQuery",
    "schedule" : "00 * * * * ?",
    "trigger" : {
         "numberOfEvents" : ">1"
     },
    "timeperiod" : 300,
     "action" : {
         "email" : [ "brian.murphy@elasticsearch.com" ]
     },
    "version" : 1,
    "lastRan" : "2014-05-05T12:12:12.123Z"
}
````
With the following search template:
````
curl -XPOST localhost:9200/_search/template/myAlertQuery -d '{ "template" : { "query" : { "match_all" : {} } } }'
````
This will execute the search every minute and trigger if there is more than one match (which there will be).

Original commit: elastic/x-pack-elasticsearch@708f927914
2014-08-12 18:57:28 +01:00
uboness 2d62aee42b Change the authentication workflow
- The authentication service now exposes a token() method to extract the token from the message
- The AuthenticationService#authenticate now accepts the token (extracted using the AuthenticationService#token)
- The Realm now exposes a support(AuthenticationToken) method
- The authc service will now consult all the realms that support a specific token, and the first realm that successfully authenticate will "win".
- Removed the SecurityActionFilter class - it wasn't tested or used anywhere. We'll add a new action filter in a separate commit

Original commit: elastic/x-pack-elasticsearch@e4dd36175f
2014-08-12 16:29:33 +02:00
Brian Murphy 1e6d6b58c9 Alerting Plugin : First commit
Very rough non-functional commit of the alerting plugin.
Please be gentle.

Original commit: elastic/x-pack-elasticsearch@98870d0778
2014-08-12 13:55:10 +01:00
uboness 5c2c8d04e9 Fixed ShieldVersion to be a snapshot
Original commit: elastic/x-pack-elasticsearch@cc9df41c6a
2014-08-12 13:57:06 +02:00
uboness fe4571da43 Changed the version to 1.0.0-SNAPSHOT
Also added ShieldVersion to track version changes in the codebase

Original commit: elastic/x-pack-elasticsearch@3282329ee9
2014-08-12 13:34:46 +02:00
Brian Murphy d19f4b5954 Initial commit
Original commit: elastic/x-pack-elasticsearch@c4b218b393
2014-08-11 17:11:03 +01:00
uboness 7ba3c3594e Changed how privileges and permissions work
- changed Privileges class (moved away from enum and use constants instead), now implies(...) can only work on same type privilege
 - introduces Permissions.Global (replacing the generic Compound)
 - Fixed FileRolesStore to use new privileges & permsissions
 - Changed the file format for FileRolesStore. Now instead of the `indices::privileges` array, the indices permissions are defined as an object where the indices are the keys and the privileges are the values. This moves the configuration to pure yaml format.
 - Added an option on the authorization service to extract all indices & aliases permitted for a specific user+privilege

Original commit: elastic/x-pack-elasticsearch@77afce767c
2014-08-11 16:46:46 +02:00
Alexander Reelsen ed959d684a SSL/TLS: Properly test for client auth and allow to require
* Fixed issue, where client auth was never needed for HTTP.
* Changed parameter name to `require.client.auth`
* Added tests, removed useless code in SslIntegrationTests

Original commit: elastic/x-pack-elasticsearch@ea424e0ae4
2014-08-11 15:09:04 +02:00
Alexander Reelsen 8e22ffa1fd Test: Adapt tests to latest changes in core
The classes which fake a CLI terminal have been renamed.

Original commit: elastic/x-pack-elasticsearch@e2b3cb8f14
2014-08-11 15:04:49 +02:00
Alexander Reelsen 91a4b74037 Packaging: Add commons codec jar to packaged zip
In order to use the `esusers` tool and to crypt passwords, one needs
to have the jar packaged in the zip release. This adds the depdendency
to the assembly descriptor.

Original commit: elastic/x-pack-elasticsearch@82ae451892
2014-08-11 10:11:26 +02:00
uboness 2c71ece598 [cleanup] Moved to TransportMessage where possible
Original commit: elastic/x-pack-elasticsearch@46535f7818
2014-08-08 21:23:26 +02:00
uboness ad02ec4609 Added the realm type to the audit trail logs for failed authentication
- Also introduced an option to disable esusers auth (such that if users configured ldap, the audit trails won't get cluttered by failed authentication in esusers
 - Moved the realms initialization to a dedicated Realms class
 - Also introduce an option to completely disable shield while keeping the installed plugin and its settings

Original commit: elastic/x-pack-elasticsearch@b554ad5ba7
2014-08-08 19:56:31 +02:00
uboness bf73ba3665 Updated the code base to use the latest changes in TransportMessage on es-core
Now the UsernamePasswordToken, once resolved, is placed in the (new) request context as is (avoids the need to re-parse headers)

Original commit: elastic/x-pack-elasticsearch@f898a94157
2014-08-08 18:07:25 +02:00
Alexander Reelsen 3ee4d527f3 CmdTool: Added options for editing roles and listing users/roles
Two new commands to the command line tool have been added

* esusers list: Allows to list all users with their roles or only a single one
* esusers roles: Allows to add or remove roles

Roles have been configured as to only consist of lowercase alphanumeric characters.

Original commit: elastic/x-pack-elasticsearch@6fcb4c56e4
2014-08-08 17:36:29 +02:00
Alexander Reelsen 75cf637fed Build: Do not depend on guava in compile scope
Changed the scope of the guava dependency to prevent wrong imports and always use the shaded ones.
This required a change in the forbidden API signatures, as that tool alwyas try load the class and fail
if the class cannot be found.

Original commit: elastic/x-pack-elasticsearch@90a245423a
2014-08-08 12:57:28 +02:00
Alexander Reelsen f4824d78e4 Compilation: Fix compilation issue with latest 1.4-SNAPSHOT
Original commit: elastic/x-pack-elasticsearch@66624a104f
2014-08-08 08:58:12 +02:00
Alexander Reelsen 44a549fd12 Test: Removed network.mode from test to prevent slowdown
Original commit: elastic/x-pack-elasticsearch@69b9296928
2014-08-07 14:08:51 +02:00
Alexander Reelsen b78262046e Packaging: Fix broken paths, adapt to elasticsearch packaging
* Add config and bin directory to assembly
* Add main method to ESUsersTool so it starts
* Fix bin/esusers to actually start
* Fix ESUsersTool to write files in config/shield by default as each plugin has its own directory after installation
* Changed bin/ and config/ directory to reflect path after installation in ES

Original commit: elastic/x-pack-elasticsearch@7d4165e389
2014-08-07 14:06:14 +02:00
Alexander Reelsen 9dd52bfb5f Packaging: Add files from bin/ and config/ directories
Original commit: elastic/x-pack-elasticsearch@76d389966e
2014-08-06 17:05:28 +02:00
uboness 09bdfc3da8 Added support class for caching username/password realm
Original commit: elastic/x-pack-elasticsearch@493234a0a5
2014-08-06 03:23:10 +02:00
uboness 605aee5a54 Moved ESUsersTool to cores cli tool infra
Also removed the cli tool infra from this repo (as it was moved to core)

Original commit: elastic/x-pack-elasticsearch@0bf7f84ab6
2014-08-03 23:08:28 +02:00
Alexander Reelsen 86546e80ad Netty: Added ip filter capabilties to pipeline
Used the existing infra structure to filter by ip in the netty pipeline
before any other handler is hit, in order to reject as soon as possible.
Right now the connection is simply closed.

The configuration is a simple YAML file which uses allow/deny rules

Original commit: elastic/x-pack-elasticsearch@000e44f8cc
2014-07-30 14:13:22 +02:00
Alexander Reelsen a6bf836ae8 SSL: Renamed settings, added SSLConfig tests
Settings for SSL now all start with `shield` as well. Changed documentation and tests to reflect this.

Original commit: elastic/x-pack-elasticsearch@9dd3bc865e
2014-07-30 11:14:22 +02:00
javanna 3a4af4c7bc uncommented security filter now that the action filter chain is in
Original commit: elastic/x-pack-elasticsearch@5b96b80409
2014-07-28 22:56:45 +02:00
javanna 0b751b12f1 integrated indices related request change added to es core and added wildcard expansion
Original commit: elastic/x-pack-elasticsearch@1192d0e7f8
2014-07-25 11:39:30 +02:00
Alexander Reelsen 9bb9fb478d Fix all guava imports to use shaded classes
This is required for the plugin to run externally

Original commit: elastic/x-pack-elasticsearch@de16c825f1
2014-07-25 08:24:02 +02:00
Alexander Reelsen ff820ce956 Docs: Added SSL certification/CA creation docs
Original commit: elastic/x-pack-elasticsearch@f38f2806b3
2014-07-23 15:42:10 +02:00
Alexander Reelsen d0673b0cfb Added SSL support in netty
This introduces the possibility to have all communications (transport
and HTTP) to run over SSL.

Original commit: elastic/x-pack-elasticsearch@c816a65f53
2014-07-22 11:41:17 +02:00
Alexander Reelsen bc0e233589 Removed netty dependency. Use shaded classes as imports
Original commit: elastic/x-pack-elasticsearch@46483aa44f
2014-07-22 11:20:41 +02:00
uboness 9b3160b7ac Added more unit tests, re-implemented & added tests for ESUsersTool
- Added CliTool infrastructure (should eventually be moved to core and removed from this repo)

Original commit: elastic/x-pack-elasticsearch@ba498163f5
2014-07-21 04:42:04 +02:00
uboness f727e29066 Initial commit of the infrastructure codebase for security
There are four modules:
 - authc: realm based authentication module
 - authz: role based privileges & permissions authorization module
 - n2n: node to node authentication module (incl. IP filtering auth)
 - audit: audit trail module (only includes log file audit trails for now)

Original commit: elastic/x-pack-elasticsearch@b1ec9e2923
2014-07-17 17:58:36 +02:00