Commit Graph

5072 Commits

Author SHA1 Message Date
uboness 956aeb53f4 Modified the initial authentication logic
When a http request arrives, we first verify that it carries an authentication token (if it doesn't we throw an authentication exception). Beyond that, any action request that arrives, if it doesn't have an authentication token we assume system user identity. The rationale behind it is that  if a request comes in via the transport, then the sending peer authenticated with a client auth cert (the cert acts as the guarantee here that the actor can be assumed as System)... otherwise, the request can come from the local node and triggered by the system (e.g. gateway recovery)

The System user only has permissions to internal apis (it doesn't have full access/permission to all the apis). when a System identity is assumed, the authorization service will grant/deny the request based on whether the request is an internal api or not.

Aso fixed the known actions (to be insync with 1.x branch)

Closes elastic/elasticsearch#45

Original commit: elastic/x-pack-elasticsearch@be27cb0e1b
2014-08-29 15:35:01 -07:00
Alexander Reelsen eb29414077 incorporating review comments as parseFile methods already return immutable maps
Original commit: elastic/x-pack-elasticsearch@5baaad0971
2014-08-29 10:23:58 +02:00
Alexander Reelsen c6b7be5596 Cleanup: Incorporating findbugs/pmd findings
Original commit: elastic/x-pack-elasticsearch@5567fae416
2014-08-29 10:15:03 +02:00
Alexander Reelsen 25d2480e78 Ip filtering: Check correct path for default file
Also added logging, so that on start up a message is logged, if all
connections are rejected or the config file is not found.

Closes elastic/elasticsearch#48

Original commit: elastic/x-pack-elasticsearch@51f16d75ba
2014-08-29 09:11:28 +02:00
uboness c17c140cd2 Small fixes
- throw an error on initialization if the audit trail is configured with an unknown output
- removed unnecessary null checks

Original commit: elastic/x-pack-elasticsearch@8d26723553
2014-08-28 16:05:04 -07:00
Alexander Reelsen f73645054b Ip Filter: Allow to disable via node configuration
Added `shield.n2n.enabled` property to disable this feature, so
it does not get added to the netty pipeline.

Original commit: elastic/x-pack-elasticsearch@d73d27619c
2014-08-28 16:34:27 +02:00
Alexander Reelsen c9379b3875 CliTool: Do not leave invalid lines in roles file
Removing all roles from a user result in an invalid line left in the
roles file. This commit simply removes the user from the roles file
in that case.

Original commit: elastic/x-pack-elasticsearch@c1f4a961fd
2014-08-28 15:10:24 +02:00
Alexander Reelsen 52ba1ceff3 CliTool: Fixed adding of roles for existing users
The roles file had been checked instead of the users file when checking
if the user already exists. This lead to wrong "user not found" error
messages.

Original commit: elastic/x-pack-elasticsearch@6fc5646ce7
2014-08-28 14:50:08 +02:00
javanna 929e12e9cb [TEST] Removed discovery configuration from transport clients settings
Also removed unused copyFile method

Original commit: elastic/x-pack-elasticsearch@2d96e05ded
2014-08-28 13:57:06 +02:00
Alexander Reelsen d0e377ff76 Testing: Create ShieldIntegrationTest that other test inherit from
This integration test contains all of the configuration needed to run with shield
and will be the base for further improvements.

Closes elastic/elasticsearch#36
Relates elastic/elasticsearch#31

Original commit: elastic/x-pack-elasticsearch@fe77f4d32b
2014-08-28 09:30:51 +02:00
uboness fcd3a89a3d We've decided that for now (phase1) index template APIs will be under cluster management privilege
Original commit: elastic/x-pack-elasticsearch@390d55e859
2014-08-27 12:28:51 -07:00
Alexander Reelsen b887fad51a Config file parsing: Improved parsing for edge cases
Created some tests to improve parsing of files, most importantly fixed an endless loop
in YAML parsing and made the non-yaml parsers a bit more error resistant.

Closes elastic/elasticsearch#40

Original commit: elastic/x-pack-elasticsearch@8873138d0c
2014-08-26 10:49:48 +02:00
c-a-m 9cd397727f LDAP: Implements an LDAP realm
The LDAP realm is a bind-per-user strategy and the group lookup strategy is configurable.
If a role mapping file is not defined, groups names are taken as role names.
Special configuration for active directory simplifies the configuration.

Integration Tests are using an embedded apache DS LDAP Server.

Original commit: elastic/x-pack-elasticsearch@ce20e1b3be
2014-08-25 15:36:31 -06:00
Brian Murphy f57ca58605 Alerting : Update the readme with right syntax for the commands
Original commit: elastic/x-pack-elasticsearch@bd78e7a2e9
2014-08-21 17:47:45 +01:00
Brian Murphy 8c623534c2 Alerting : Cleanup and fixes.
These changes fix the alert throttling during the time period, move alert history it's own
index.

Original commit: elastic/x-pack-elasticsearch@5130637824
2014-08-21 16:33:11 +01:00
c-a-m cf40c76e15 User Caching Issue Fix
Fixes Caching issue with passwords hashing with unique salts.

Original commit: elastic/x-pack-elasticsearch@e0ee1440fa
2014-08-20 12:26:46 -07:00
uboness f040d895b3 Added the notion of IndicesResolver
A strategy for extracting the related indices from a request. The default strategy relies on the request to either be `IndicesRequest` or `CompositeIndicesRequest`. But depending on the request type, there might be other strategies as well. For example, `GetIndexTemplatesRequest` only holds the template name and therefore requires a different strategy where the template is looked up in the cluster metadata and the indices are resolved from there.

This also cleans up the Permission class and makes sure that the indices are resolved once per request (and not once per configured group permission)

Original commit: elastic/x-pack-elasticsearch@95192ccdff
2014-08-20 10:42:51 -07:00
uboness 1154f13345 Allow configuring cluster & indices permissions on specific actions
- It is now possible to assign index & cluster permission on an action level (not just the fixed privileges we defined).

 - also added a test to check that all the actions elasticsearch has are indeed known to shield. So whenever a new action is introduced in elasticsearch, and shield is not aware of it, the build will fail. This will help us ensure that all actions in elasticsearch are 1) well formatted/categorized, 2) secured and "permissible"

Closes elastic/elasticsearch#19

Original commit: elastic/x-pack-elasticsearch@170c3b9185
2014-08-20 09:27:52 -07:00
Alexander Reelsen 0d5c83d0f0 Testing: Disable integration tests until infrastructure is created
Relates elastic/elasticsearch#36

Original commit: elastic/x-pack-elasticsearch@cc90d83111
2014-08-20 16:56:18 +02:00
Alexander Reelsen 22862cd416 Fix loading of secured transports
Load transports in plugin. No need to specify any transport modules anymore.
Removed the transport modules.

Original commit: elastic/x-pack-elasticsearch@45f3245361
2014-08-20 15:47:38 +02:00
uboness 1728c2a381 Fixed comments on Transport service filter commit
- Fixed a bug in the logic of applying the indices group permission check
- For the rest, mainly cosmetics refactoring (renaming & redundant code removal)

Original commit: elastic/x-pack-elasticsearch@748575dc2e
2014-08-19 07:45:54 -07:00
Alexander Reelsen 6a7c537ad6 Docs: Initial documentation
Created an initial set of documentation which can also be built
using `build_docs.pl` without errors.

Original commit: elastic/x-pack-elasticsearch@1a3ccf4eb9
2014-08-19 15:40:20 +02:00
Brian Murphy 47e1e77b58 Alerting : Start of multiprocess coordination work
On alert claim time, query index to make sure alert is still valid and enabled and is the same alert.

Original commit: elastic/x-pack-elasticsearch@72b816b7d1
2014-08-19 13:31:01 +01:00
uboness 9c55be1530 Added transport filtering
Requests are now auth'ed on the transport level as well (in addition to the transport action level). This is required as some internal requests are not executed as actions, thus not going through the auth process in the transport action. Since we have n2n authentication, we also assume here that requests that are not associated with an authentication token are internal system calls. We then, auth the request as a system user.

Also Added a system realm (to handle system requests)

Original commit: elastic/x-pack-elasticsearch@2c917318f0
2014-08-18 14:01:11 -07:00
Brian Murphy 0eea73dd72 Alerting: Scripted triggers and support for aggregations in searches.
This commit adds support for triggers that are scripts:

Query :
````
POST /_search/template/testFilteredAgg
{
  "query" : {
    "filtered" : {
      "query" : {
        "match_all" : {}
     },
     "filter": {
       "range" : {
         "@timestamp" : {
             "gte" : "{{from}}",
             "lt" : "{{to}}"
         }
       }
     }
    }
  },
    "aggs" : {
      "response" : {
        "terms" : {
          "field" : "response",
          "size" : 100
        }
      }
}, "size" : 0  }
````

Trigger Script:
````
POST /_scripts/groovy/testScript
{
  "script" : "ok_count = 0.0;error_count = 0.0;for(bucket in aggregations.response.buckets) {if (bucket.key < 400){ok_count += bucket.doc_count;} else {error_count += bucket.doc_count;}}; return error_count/(ok_count+1) > 0.1;"
}
````

Alert:
````
POST /_alerting/_create/myScriptedAlert
{
    "query" : "testFilteredAgg",
    "schedule" : "05 * * * * ?",
    "trigger" : {
         "script" : {
           "script" : "testScript",
           "script_lang" : "groovy",
           "script_type" : "INDEXED"
         }
     },
    "timeperiod" : "300s",
     "action" : {
         "index" : {
           "index" : "weberrorhistory",
           "type" : "weberrorresult"
         }
     },
    "indices" : [ "logstash*" ],
    "enabled" : true,
    "simple" : false
}
````

If you want to use aggs with your alert you must create a search that contains the timefilter with the params ````{{from}}```` and ````{{to}}```` and set the ````simple```` flag to ````true````.

Original commit: elastic/x-pack-elasticsearch@0430a1bf40
2014-08-18 16:59:49 +01:00
Alexander Reelsen 452367b674 Return correct HTTP headers on security exceptions
* Changed Authentication/AuthorizationException to always return RestStatus.UNAUTHORIZED
* Added the WWW-Authenticate header, which results in user/pass input in browsers
* Added tests and own assertions class

Closes elastic/elasticsearch#18

Original commit: elastic/x-pack-elasticsearch@c6ce084692
2014-08-18 15:15:28 +02:00
Brian Murphy 4216491824 Alerting : Improve error handling in the RestInterface
This commit improves error handling in the RestHandler. Errors are now
returned to the client.

Original commit: elastic/x-pack-elasticsearch@433e8a57bc
2014-08-18 12:28:48 +01:00
Brian Murphy 14d0bcd589 Alerting add TODO
Original commit: elastic/x-pack-elasticsearch@23cf5fce8b
2014-08-18 12:22:13 +01:00
Brian Murphy 418b9f1a31 Alerting : Add scripted triggers and alert disabling.
This commit adds support for disabling alerts.
This commit adds preliminary support for scripted triggers.

Original commit: elastic/x-pack-elasticsearch@e14a56dbeb
2014-08-18 12:13:39 +01:00
Brian Murphy 4c1c502f80 Alerting : Fix IndexAction
This commit contains fixes to the IndexAlertAction and normalizes the serialization of
Alerts using ToXContent interface.

Original commit: elastic/x-pack-elasticsearch@47325bdd01
2014-08-18 11:23:45 +01:00
Brian Murphy 26e053eaed Alerting : Claim alert runs
This commit add functionality so that multiple nodes running alert plugins won't stand on
each other trying to run the same alerts at the same time.

Original commit: elastic/x-pack-elasticsearch@9c350c0114
2014-08-18 10:27:07 +01:00
Brian Murphy 83287e009a Alerting : Add index action.
This commit adds support for indexing on alert trigger.

Original commit: elastic/x-pack-elasticsearch@d6508ce16d
2014-08-17 13:49:29 +01:00
Brian Murphy d1af9f15fe Alerting : Small fixes.
Fix the constructor for the AlertResult.

Original commit: elastic/x-pack-elasticsearch@0820edaa6a
2014-08-16 17:44:44 +01:00
Brian Murphy 019cdb37d4 Alerting : Add support for history and fix issues around parsing lastRun.
This commit adds a history log for alerts and updates lastRan with the correct time.

Original commit: elastic/x-pack-elasticsearch@49c77f1ef8
2014-08-15 16:46:41 +01:00
Brian Murphy 4c47c8ba9a Alerting : REST endpoints
This commit adds REST endpoints for list/delete/create and fixes refresh.

Original commit: elastic/x-pack-elasticsearch@36e2c15753
2014-08-15 11:45:45 +01:00
Brian Murphy e648cc7f82 Alerting : Update README.md
Add sample alert to README.md and describe the email output.

Original commit: elastic/x-pack-elasticsearch@2b78e26dfd
2014-08-13 18:05:56 +01:00
Brian Murphy b5cb8b3d04 Alerting : Add RestHandler
This commit adds the rest handler and supports /_alerting/_refresh to reload all alerts.

Original commit: elastic/x-pack-elasticsearch@14fadbeeee
2014-08-13 18:00:36 +01:00
Brian Murphy 0d9061b838 Alerting : better email formatting
This commit enables better email formatting.

Original commit: elastic/x-pack-elasticsearch@8be3e3b6d1
2014-08-13 17:05:25 +01:00
Brian Murphy e3250c0366 Alerting : Add time clamping.
This commit adds timestamp clamping on the timeperiod to alerting.
Also adds index setting on alerts.

Original commit: elastic/x-pack-elasticsearch@222cd6eaef
2014-08-13 15:21:55 +01:00
uboness 1f5f3f21f9 Changed the contract of AuthenticationService#token to throw an authentication exception when no token found
- Also added an overloaded version of the AuthenticationService#token method that accepts a default token and that doesn't thrown an authentication exception
- Added AuditTrail#authenticationFailed method at a higher level than the realm version. Now the realm authc failure will only be logged when trace is enabled. With this change, the audit trail logging is more consistent in its terminology (anonymous is now logged when no auth token is found)
- extended the level of audit trail logging tuning (now using all logging levels, incl. error & warn)
- Added tests for audit trails
- Added tests for authentication service
- Added mockito as a test lib (will serve as our mocking framework)

Original commit: elastic/x-pack-elasticsearch@8d21ab7484
2014-08-13 14:23:28 +02:00
Brian Murphy 88ed7a5624 Alerting : Add parsing and indices.
This commit adds parsing of the triggers and actions, and allows alerts to be restricted to indices.

Original commit: elastic/x-pack-elasticsearch@44a6a51b17
2014-08-13 10:11:51 +01:00
Brian Murphy 940d6402ae Alerting : connect email
The last commit ommited a line that actually sends the email on alert trigger.

Original commit: elastic/x-pack-elasticsearch@24b5c01db8
2014-08-12 19:54:13 +01:00
Brian Murphy 5f84596c92 Alerting : Add support for quartz scheduler.
This commit enables loading of alerts from the .alerts index and adds the
Quartz scheduler.
You can add the following alert :
````
curl -XPOST http://localhost:9200/.alerts/alert/myTestAlert -d '{
    "query" : "myAlertQuery",
    "schedule" : "00 * * * * ?",
    "trigger" : {
         "numberOfEvents" : ">1"
     },
    "timeperiod" : 300,
     "action" : {
         "email" : [ "brian.murphy@elasticsearch.com" ]
     },
    "version" : 1,
    "lastRan" : "2014-05-05T12:12:12.123Z"
}
````
With the following search template:
````
curl -XPOST localhost:9200/_search/template/myAlertQuery -d '{ "template" : { "query" : { "match_all" : {} } } }'
````
This will execute the search every minute and trigger if there is more than one match (which there will be).

Original commit: elastic/x-pack-elasticsearch@708f927914
2014-08-12 18:57:28 +01:00
uboness 2d62aee42b Change the authentication workflow
- The authentication service now exposes a token() method to extract the token from the message
- The AuthenticationService#authenticate now accepts the token (extracted using the AuthenticationService#token)
- The Realm now exposes a support(AuthenticationToken) method
- The authc service will now consult all the realms that support a specific token, and the first realm that successfully authenticate will "win".
- Removed the SecurityActionFilter class - it wasn't tested or used anywhere. We'll add a new action filter in a separate commit

Original commit: elastic/x-pack-elasticsearch@e4dd36175f
2014-08-12 16:29:33 +02:00
Brian Murphy 1e6d6b58c9 Alerting Plugin : First commit
Very rough non-functional commit of the alerting plugin.
Please be gentle.

Original commit: elastic/x-pack-elasticsearch@98870d0778
2014-08-12 13:55:10 +01:00
uboness 5c2c8d04e9 Fixed ShieldVersion to be a snapshot
Original commit: elastic/x-pack-elasticsearch@cc9df41c6a
2014-08-12 13:57:06 +02:00
uboness fe4571da43 Changed the version to 1.0.0-SNAPSHOT
Also added ShieldVersion to track version changes in the codebase

Original commit: elastic/x-pack-elasticsearch@3282329ee9
2014-08-12 13:34:46 +02:00
Brian Murphy d19f4b5954 Initial commit
Original commit: elastic/x-pack-elasticsearch@c4b218b393
2014-08-11 17:11:03 +01:00
uboness 7ba3c3594e Changed how privileges and permissions work
- changed Privileges class (moved away from enum and use constants instead), now implies(...) can only work on same type privilege
 - introduces Permissions.Global (replacing the generic Compound)
 - Fixed FileRolesStore to use new privileges & permsissions
 - Changed the file format for FileRolesStore. Now instead of the `indices::privileges` array, the indices permissions are defined as an object where the indices are the keys and the privileges are the values. This moves the configuration to pure yaml format.
 - Added an option on the authorization service to extract all indices & aliases permitted for a specific user+privilege

Original commit: elastic/x-pack-elasticsearch@77afce767c
2014-08-11 16:46:46 +02:00
Alexander Reelsen ed959d684a SSL/TLS: Properly test for client auth and allow to require
* Fixed issue, where client auth was never needed for HTTP.
* Changed parameter name to `require.client.auth`
* Added tests, removed useless code in SslIntegrationTests

Original commit: elastic/x-pack-elasticsearch@ea424e0ae4
2014-08-11 15:09:04 +02:00