Commit Graph

5732 Commits

Author SHA1 Message Date
Areek Zillur 83651e3314 [TEST] ignore flacky test
Original commit: elastic/x-pack-elasticsearch@61000d13b6
2015-01-23 01:32:06 -05:00
Areek Zillur 246879aebf [TEST] change notification event time
Original commit: elastic/x-pack-elasticsearch@41d038e074
2015-01-23 01:13:55 -05:00
Areek Zillur e45d6364ea [TEST] re-structure notification event test
Original commit: elastic/x-pack-elasticsearch@1270f14571
2015-01-23 00:55:59 -05:00
Areek Zillur c12bc0ac3d [TEST] increase license expiry
Original commit: elastic/x-pack-elasticsearch@ade53101db
2015-01-23 00:38:28 -05:00
Areek Zillur f32d49369c [TEST] increase trial license expiry
Original commit: elastic/x-pack-elasticsearch@dff2b30197
2015-01-23 00:23:58 -05:00
Areek Zillur 57ec891db0 [TEST] : take into account inclusive range in Pre/Post ExpirationCallbacks
Original commit: elastic/x-pack-elasticsearch@346b9f3b4c
2015-01-23 00:01:03 -05:00
uboness 91881f8c04 Updated the pom.xml
Added the new profiles with the new distribution management

Original commit: elastic/x-pack-elasticsearch@879e02211a
2015-01-23 05:41:13 +01:00
Areek Zillur 5be5b1915b Add support for License Expiration event triggers
This enhancement allows consumer plugins to configure event notifications from the licensing plugin relative to its license expiry.

Original commit: elastic/x-pack-elasticsearch@11b53dd78d
2015-01-22 23:17:34 -05:00
uboness dd4a66bd6c Changed search and get privileges
- separated `get` privilege from `search`. This should make it simpler for users to only allow search (and not get) when working with filtered aliases
- added multi search under the `search` privilege
- added the multi get under the `get` privilege

Original commit: elastic/x-pack-elasticsearch@6fafb08a2c
2015-01-22 21:10:54 +01:00
uboness a25d603b93 Adds SUGGEST index privilege
The `suggest` action was also added to the `SEARCH` privilege as one can execute suggestions under the `_search` API as well.

 Closes elastic/elasticsearch#24

Original commit: elastic/x-pack-elasticsearch@672809e199
2015-01-22 19:39:12 +01:00
jaymode 97f229f667 SSL/TLS: Do not require keystore or truststore on for clients
This commit removes the requirement that a client using the SSLService must
have defined a keystore. Now for clients both the keystore and truststore are
optional; if neither are defined the system default trust managers will be used.

Closes elastic/elasticsearch#613

Original commit: elastic/x-pack-elasticsearch@1055a9666a
2015-01-22 13:22:22 -05:00
Alexander Reelsen 2986502984 CLI Tools: Add command to check for same permissions and owners after run
In case the creation of files changed the owner, group or the permissions, this command
will write an error message to the console.

Relates elastic/elasticsearch#517

Original commit: elastic/x-pack-elasticsearch@49aab5f712
2015-01-22 19:13:45 +01:00
jaymode c5028f7384 SSL/TLS: Allow control of SSL per profile
SSL can now be enabled or disabled per profile. This allows to have both
secured and unsecured client connections.

Closes elastic/elasticsearch#612

Original commit: elastic/x-pack-elasticsearch@53a7efa5b1
2015-01-22 07:39:37 -05:00
uboness 2c55d85aa5 [Audit] Changed the log entry prefix configuration
Prepended `emit_` to each of the settings to make it clearer what they're all about.

Original commit: elastic/x-pack-elasticsearch@8e648eee23
2015-01-22 02:57:05 +01:00
uboness 2c687271d4 [Audit] Renamed anonymous_access to anonymous_access_denied
- The `anonymous_access_denied` clearly indicates that the requests were denied.
- In the future, if/when we add anonymous realm, we'll add another event type - `anonymous_access_granted` - plays nice with this change

Original commit: elastic/x-pack-elasticsearch@1fead24a0d
2015-01-22 02:29:11 +01:00
javanna 14699d6610 Indices resolution: empty aliases to be treated same as _all in GetAliasesRequest
While IndicesAliasesRequest doesn't support empty aliases, thus only explicit _all needs to resolved to all existing authorized aliases, GetAliasesRequest does support empty aliases, thus we have to treat them the same as _all.

Closes elastic/elasticsearch#606

Original commit: elastic/x-pack-elasticsearch@3e993ea2bd
2015-01-21 23:45:07 +01:00
c-a-m a01c271460 ldap: changes default AD URL to be clear-text
If no URL is set, it is derived from the URL.  Now it will default to clear-text and port 389

Original commit: elastic/x-pack-elasticsearch@6d1b9d3e42
2015-01-21 14:15:25 -07:00
c-a-m b3630c7ea9 tests: Ldap unit tests for GroupResolvers
This adds unit tests for the three new GroupResolvers.

Original commit: elastic/x-pack-elasticsearch@d303388696
2015-01-21 13:46:53 -07:00
javanna 4a7731099b [TEST] Remove current locale log line from ShieldRestTests
Original commit: elastic/x-pack-elasticsearch@b26badd740
2015-01-21 19:14:52 +01:00
jaymode e8a17d9ccd SSL/TLS: Add options to configure session cache size and timeout
The default settings for the SSL session cache is unbounded with a timeout of
24 hours. This can lead to memory issues when clients do not resume connections.
This adds a default limit of 1000 sessions in the cache in addition to exposing
settings to control these values.

Closes elastic/elasticsearch#602

Original commit: elastic/x-pack-elasticsearch@9cdc7b613c
2015-01-21 12:05:57 -05:00
Luca Cavanna 9fed91b795 Indices resolution: special treatment for get aliases request
GetAliasesRequest is the Java api request class behind the get alias and alias exists api. It allows for replacing its indices, as it implements IndicesRequest.Replaceable, but its authorization is only based on the indices specified on the request, the aliases are ignored.

This commit makes sure that the aliases are taken into account. Also get aliases is now part of the manage_aliases privilege and wildcards expression within aliases are replaced too with matching aliases that the current user is authorized for.

Closes elastic/elasticsearch#558
Closes elastic/elasticsearch#595

Original commit: elastic/x-pack-elasticsearch@b40b4cccc6
2015-01-21 16:14:58 +01:00
uboness 3bf687e726 [Cleanup] Removed redundant Inject annotations
Also removed unused constructors

Original commit: elastic/x-pack-elasticsearch@1d1d2dcbad
2015-01-21 13:44:54 +01:00
javanna fb7c731bd1 Tribe node: add support for tribe node in shield
Disabled license check on the tribe node to make sure that the tribe node can start, otherwise license plugin would try to generate a new trial license which is not possible since the node has no master. License check still happens for tribes though. This will be improved once es core supports merging cluster level custom metadata, then the tribe node will see some license coming from its tribes and won't require any additional license.

Added integration test to verify basic functionality against a tribe node, which also validates the settings needed on the tribes.

Made sure that shield is loaded and enabled on very tribe if loaded and enabled on the tribe node. We want to make sure that nobody manages to use shield on the tribe node only for free (since we disabled liccensing there), with no shield on the tribes. If we forcibly enable and make the shield plugin mandatory on the tribe clients, it means that they will not be able to join their corresponding clusters unless they have shield loaded and enabled too. As a result, shield is supported in the tribe node as long as all the tribes have shield loaded and enabled too.

Relates to elastic/elasticsearch#311
Closes elastic/elasticsearch#584

Original commit: elastic/x-pack-elasticsearch@317add553f
2015-01-21 12:27:30 +01:00
uboness 82fdf377a5 [Cleanup] removed FileRolesStore.Listener in favour of RefreshListener
Original commit: elastic/x-pack-elasticsearch@0c1a020dbb
2015-01-21 12:16:28 +01:00
jaymode ef979e4939 SSL/TLS: Only use TLS protocols by default
Only enables TLSv1, TLSv1.1, and TLSv1.2 protocols for transport, http, and ldaps. The supported
protocols are configurable in case one of these protocols is found to be insecure in the future.

Closes elastic/elasticsearch#594

Original commit: elastic/x-pack-elasticsearch@d4556091ef
2015-01-20 16:45:07 -05:00
c-a-m 1f8189fa12 LDAP: Adds SearchScope.Base tests and group.search settings for AD
Previously, AD group search used the user search dn so this adds configuration to separate the group search from the user search

 This adds tests for the newly introduced SearchScope.Base and includes general test cleanup.

Original commit: elastic/x-pack-elasticsearch@6ed1114b29
2015-01-20 13:53:25 -07:00
uboness da5299e4c5 LDAP refactoring
- Introduced a strategy for group search. This is applied on the `AbstractLdapConnection` level.
- The `LdapConnection` and `ActiveDirectoryConnection` are now clean of logic
- The `AbstractLdapConnection` holds a timeout
- Introduced `SearchScope` for better settings support.
- fixed a bug in `LdapConnectionFactory:74`... `settings.getAsArray` will never return `null`

Original commit: elastic/x-pack-elasticsearch@1f4a43d037
2015-01-20 13:53:25 -07:00
c-a-m 79d4b1e208 LDAP: Add configurable filters to LDAP group search and AD user search
This lets the user configure custom filters for group searches in LDAP, and user searches in AD

changed configuration in this commit:
group_search.group_search_dn -> group_search.base_dn
group_search.subtree_search -> group_search.subtree

added for LDAP:
group_search.filter
group_search.user_attribute

added for AD:
user_search.base_dn
user_search.filter
user_search.subtree

This also changes group_search.subtree to be true by default.
This fixes elastic/elasticsearch#567 and fixes elastic/elasticsearch#553

Original commit: elastic/x-pack-elasticsearch@8a1246aefd
2015-01-20 13:53:25 -07:00
Luca Cavanna f29cc62829 Authorization: split analyze api into cluster level action and original indices action
The analyze api allows to specify an index, to retrieve analyzers or token filters from a specific index. That is why it is categorized as indices level action. That said the index is optional and when not specified the action is executed at the cluster level. We have to remap the name of the action in that case, to make sure that it requires a different privilege under cluster: cluster:admin/analyze instead of indices:admin/analyze .

Closes elastic/elasticsearch#566
Closes elastic/elasticsearch#565
Closes elastic/elasticsearch#592

Original commit: elastic/x-pack-elasticsearch@9073b30d08
2015-01-20 18:33:49 +01:00
jaymode 166514651a SSL/TLS: Add option to disable reverse DNS resolution of hostname
This change adds the option to disable reverse DNS lookup of a hostname from
an IP address. This is needed if only an IP address is found in a SSL certificate
and hostname verification is enabled.

Closes elastic/elasticsearch#575

Original commit: elastic/x-pack-elasticsearch@07356bc885
2015-01-20 07:07:10 -05:00
Luca Cavanna 02682ff4ec Indices resolution: special treatment for IndicesAliasesRequest
IndicesAliasesRequest needs to be authorized against both indices and aliases. That means that the following request

curl -XPOST 'http://localhost:9200/_aliases' -d '
{
"actions" : [
{ "add" : { "index" : "test1", "alias" : "alias1" } }
]
}'

requires now indices:admin/aliases privileges for both test1 and alias1.

Added manage_aliases shortcut privilege that points to indices:admin/aliases.

Also, IndicesAliasesRequest used to not support replacing its indices (request doesn't implement IndicesRequest.Replaceable in es core). That can be worked around as well through the special treatment that we are introducing in shield for this specific request. Given that it is a composite action, every single operation has now its wildcards replaced with authorized indices (supported among aliases as well in case of remove operations). If any of the operation ends up relating to no indices after wildcards expansion, the whole request fails.

The DefaultIndicesResolver#explodeWildcards method, which used to expand wildcards as es core would do it, is not needed anymore, as all of the requests that support wildcards have now their indices properly replaced.

Added also special authorization pass for create index, if the request body contains aliases. The index can only be created if the current user has permission to create the index and to create the aliases that are part of the same request.

Closes elastic/elasticsearch#112
Closes elastic/elasticsearch#557
Closes elastic/elasticsearch#529

Original commit: elastic/x-pack-elasticsearch@d7201e8a8b
2015-01-20 12:23:52 +01:00
Luca Cavanna 16c7cfa50f review
Original commit: elastic/x-pack-elasticsearch@4f45b2de79
2015-01-20 11:01:03 +01:00
Luca Cavanna 58c5f95402 converted version check into a regular test class
Original commit: elastic/x-pack-elasticsearch@cefd362b70
2015-01-20 10:50:50 +01:00
javanna e04f4e2395 Track changes to be made once dependencies are upgraded
Added a class under test that is used to keep track of changes that we might have to make once we upgrade versions of dependencies, especially elasticsearch core.
 Every change is listed as a specific assert that trips with a future version of es core, with a meaningful description that explains what needs to be done.

 NOTE: changes suggested by asserts descriptions may break backwards compatibility. The same shield jar is supposed to work against multiple es core versions,
 thus if we make a change in shield that requires e.g. es core 1.4.1 it means that the next shield release won't support es core 1.4.0 anymore.
 In many cases we will just have to bump the version of the assert then, unless we want to break backwards compatibility, but the idea is that this class
 helps keeping track of this and eventually making changes when needed.

Closes elastic/elasticsearch#560

Original commit: elastic/x-pack-elasticsearch@fabe3858c1
2015-01-20 10:50:50 +01:00
javanna 1e2cea48d0 Authorization: enforced some assumptions through asserts
- made sure that clear_scroll all gets converted to the correspoinding shield cluster action in both action filter and transport filter (used to happen only on the action filter before): introduced the context of ShieldActionMapper that allows to convert action names based on an incoming request and its action name (will be useful for analyze api too)
- made sure that potential clear_scroll all errors contain the shield action name rather than the es core original one
- made it clearer that the only indices actions known not to be indices requests are scroll related ones, which we assert on and grant. Everything else gets denied.
- made it clearer that the only indices request whose indices might end up being resolved to an empty set is analyze request, as its index is optional
- simplified permissions check in Permission.Group by asserting on index argument not null

Original commit: elastic/x-pack-elasticsearch@7c01159b03
2015-01-20 10:45:21 +01:00
uboness 910d7c6372 Aligned users, users_roles and role_mapping file stores to behave like roles.yml
All three files are auto loaded by shield when modified. The behaviour that we agreed on is that when there's a parse failure in any of these files, we don't prevent the node from starting. Instead we skip the records that we failed to parse as if they don't exist. This is how `roles.yml` is handled today, and this commit makes sure that `users`,  `users_roles` and `role_mapping.yml` are aligned with this behaviour.

Also, the same behaviour is applied when the file is modified at runtime (so it's consistent with node start up).

This commit also adds a lot of missing tests for both `LdapGroupToRoleMapper` and `ActiveDirectoryGroupToRoleMapper` classes.

Original commit: elastic/x-pack-elasticsearch@7fdd6bb5cc
2015-01-20 02:20:25 +01:00
uboness d7d96d866e [Fix] removed null hosts in the audit logs
Some request are created locally by elasticsearch and therefore are not associated with a remote address (we only associate the remote address with a request that arrives remotely from via the transport layer). An example of such request is the periodic nodes info that is collected by elasticsearch. Also, requests that originate from the REST layer also create transport requests locally.

  This commit takes this behaviour into account and makes sure that we'll always log the host in the audit logs. We do that in the following way:

   - `host` is replaced by two attributes: `origin_type` and `origin_address`. `origin_type` can be either `rest`, `remote_node` or `local_node`. `origin_address` holds the host address of the origin
   - when no remote address is associated with the request, it's safe to assume it was created locally. We'll then output `origin_type=[local_node] origin_address=[<the localhost address>]`
   - when a rest request gets in, we'll copy and place its remote address in the context of the request (the context of the rest request is copied to the context of the transport request)
   - . in the audit logs, we'll inspect the transport request and look for a `rest_host` in its context. if we find it, we'll log the log entry under `origin_type=[rest], origin_address=[<the remote rest address>]` attributes. This way, the origin of the request won't get "lost" and we'll still differentiate between transport hosts and rest hosts.
   - if the request is holds a remote address, it can only come from the transport layer, so we'll output "origin_type=[transport] origin_address=[<remote address]"

 While at it, also changed the format of the log entries:

  - lowercased the whole message (e.g. `ANONYMOUS_ACCESS` to `[anonymous_access]` (for consistency sake)
  - introduced layer categorization for every entry to indicate whether its `[transport]`, `[rest]` or `[ip_filter]` related. I reckon this will make it easier to parse the logs if one wishes to do so.

Fixes elastic/elasticsearch#550

Original commit: elastic/x-pack-elasticsearch@b84f0c5548
2015-01-20 02:06:48 +01:00
javanna 98c3531bf9 move to es core 1.4.2
This commit moves the es core dependency to 1.4.2, which becomes the minimum version required from now on.

Changes made accordingly to this decision since we can break backwards compatibility and assume es core>=1.4.2

Closes elastic/elasticsearch#562

Original commit: elastic/x-pack-elasticsearch@484b4a2528
2015-01-19 08:24:18 +01:00
uboness 82c9b4fc61 [Cleanup] lowercased all log messages and exception messages
This to keep consistent with es core.

Also, where applicable, rephrased log messages to make it clearer.

Original commit: elastic/x-pack-elasticsearch@fae3188b17
2015-01-16 22:00:34 +01:00
Martijn van Groningen 2892154ace Merge pull request elastic/elasticsearch#64 from joehillen/fix-email-port
Fix email port setting

Original commit: elastic/x-pack-elasticsearch@ee669b55d8
2015-01-16 21:24:43 +01:00
uboness e3cb07132a [Fix] added the missing NoOpLogger
Original commit: elastic/x-pack-elasticsearch@2882baf8f9
2015-01-16 18:56:22 +01:00
uboness 66b4d5e6f8 Skip invalid roles in roles.yml
Today we require that the `roles.yml` file will be a valid yaml and all the role definitions there must be valid as well. If we can't fully parse this file, we simply throw an exception and ignore its content. After al long discussion, we decided that it would be much better to try and parse whatever we can out of this file and load the valid roles. Those invalid roles will be skipped and immediately removed from the system.

 This commit changes the way we parse the `roles.yml`. We first break it down to mini single-role yml constructs and then parse each separately from the others. This way, failing to parse one role, won't impact the others.

 Fixes elastic/elasticsearch#313

Original commit: elastic/x-pack-elasticsearch@31e3624594
2015-01-16 18:38:13 +01:00
uboness 9e078f4924 [Fix] a bug in ClearRealmCacheTests
Original commit: elastic/x-pack-elasticsearch@8d8fdf4c2a
2015-01-16 18:36:58 +01:00
uboness 56957f98bc [Fix] All loggers are not contextual
Meaning, all loggers are now settings aware, so all shield logs are now consistent with the rest of elasticsearch and will follow elasticsearch configuration and output format (printing out the node name by default).

Also:

- Changed the audit log to **not** be based on the elasticsearch settings as it needs to define its own format.
- Added the node name as a prefix to the audit logs by default (can be disabled but setting `shield.audit.logfile.prefix.node_name` to `false`
- As part of this change, the realms now changed and now created with a `RealmConfig`. This construct holds the realm settings, the environment and is served as a logger factory for all realm constructs.
- The only exceptions to the logs are the ssl socket factories.. the logs there are only used for tests by calling `clear`. This behaviour will change in the future such that `clear` will be removed and then there'll be no need for loggers in there.

 Fixes elastic/elasticsearch#446

Original commit: elastic/x-pack-elasticsearch@7a1058a54e
2015-01-16 17:55:25 +01:00
uboness 1d040c4c23 [Cleanup] - removed redundant overridden methods in esusers realm
Original commit: elastic/x-pack-elasticsearch@98ef6a9953
2015-01-16 02:53:23 +01:00
Brian Murphy c81a37a2ee Fix the email template test to be a lightweight unit test case and not start any unneeded services.
Original commit: elastic/x-pack-elasticsearch@93fc61d61a
2015-01-15 16:54:48 -05:00
Brian Murphy cef0976a6c Change the version to 1.0.0-beta2i
Original commit: elastic/x-pack-elasticsearch@aace18a423
2015-01-15 16:54:25 -05:00
Joe Hillenbrand 5c904dfafa Fix email port setting
The setting was mistyped as 'smpt' when it should have been 'smtp', but
it is better to change it to 'email' to be consistent with the other settings.

Original commit: elastic/x-pack-elasticsearch@0e610d89b5
2015-01-15 12:55:30 -08:00
uboness 2f373f692f Introduced an API to clear realms caches
Since both LDAP and AD realms are caching users. If the groups of the users change on the LDAP side, these changes will not be visible in shield until the relevant cached users will be evicted from cache. This poses a problem, specially when degrading users in terms of their permission (e.g. after degrading them on LDAP, they still have higher privileges until they're evicted from cache). The default cache timeout today is 1 hour. For this reason, a new API is introduced which will enable administrators to force cache evictions.

- Changed the default cache timeout to 20 minute
- `ClearRealmCacheAction was introduced (along with the relevant request and response constructs). This is a cluster action
- the corresponding rest action was introduced as well, under the `_shield/realm/{realm}/cache/clear` URI (where `{realm}` enables clearing specific realms, or all realms when passing `_all`.
- With the introduction of an action, the `ActionModule` now is no longer a node module - it's bound on both node and transport client.
- Added a new Cluster permission - `manage_shield`
- Also cleaned up the `Permission` and `AuthorizationService` class

Original commit: elastic/x-pack-elasticsearch@c59e244435
2015-01-15 19:13:02 +01:00
Brian Murphy a458e2df1d Register the ack handler again.
Original commit: elastic/x-pack-elasticsearch@b229c52f78
2015-01-15 10:31:59 -05:00