Instead of creating an automaton predicate on each request (very expensive) we now have a static create_index matcher (predicate) that is reused.
Original commit: elastic/x-pack-elasticsearch@f70dae13ac
- on license expiration, we only block cluster stats/health and indices stats.
- depend on the latest snapshot of the licensing plugin that supports registrations of expiration callbacks
- registering expiration callbacks to periodically log and warn about license expiration (pre and post expiration)
Original commit: elastic/x-pack-elasticsearch@5aee30fac4
This updates .esvmrc to get the latest license plugin, marvel, and reflects the latest configuration.
This sets the bind host and publish host to 127.0.0.1 so that hostname verification succeeds.
Original commit: elastic/x-pack-elasticsearch@a51046d130
Changes reflect the restructuring of elasticsearch maven repo
- changed the repository names (for consistency sake)
- elasticsearch repositories now point to `/releases` and `/snapshots`
- added `deploy-internal` and `deploy-public` profiles
Original commit: elastic/x-pack-elasticsearch@92709ce38a
- separated `get` privilege from `search`. This should make it simpler for users to only allow search (and not get) when working with filtered aliases
- added multi search under the `search` privilege
- added the multi get under the `get` privilege
Original commit: elastic/x-pack-elasticsearch@6fafb08a2c
This commit removes the requirement that a client using the SSLService must
have defined a keystore. Now for clients both the keystore and truststore are
optional; if neither are defined the system default trust managers will be used.
Closeselastic/elasticsearch#613
Original commit: elastic/x-pack-elasticsearch@1055a9666a
- The `anonymous_access_denied` clearly indicates that the requests were denied.
- In the future, if/when we add anonymous realm, we'll add another event type - `anonymous_access_granted` - plays nice with this change
Original commit: elastic/x-pack-elasticsearch@1fead24a0d
While IndicesAliasesRequest doesn't support empty aliases, thus only explicit _all needs to resolved to all existing authorized aliases, GetAliasesRequest does support empty aliases, thus we have to treat them the same as _all.
Closeselastic/elasticsearch#606
Original commit: elastic/x-pack-elasticsearch@3e993ea2bd
The default settings for the SSL session cache is unbounded with a timeout of
24 hours. This can lead to memory issues when clients do not resume connections.
This adds a default limit of 1000 sessions in the cache in addition to exposing
settings to control these values.
Closeselastic/elasticsearch#602
Original commit: elastic/x-pack-elasticsearch@9cdc7b613c
GetAliasesRequest is the Java api request class behind the get alias and alias exists api. It allows for replacing its indices, as it implements IndicesRequest.Replaceable, but its authorization is only based on the indices specified on the request, the aliases are ignored.
This commit makes sure that the aliases are taken into account. Also get aliases is now part of the manage_aliases privilege and wildcards expression within aliases are replaced too with matching aliases that the current user is authorized for.
Closeselastic/elasticsearch#558Closeselastic/elasticsearch#595
Original commit: elastic/x-pack-elasticsearch@b40b4cccc6
Disabled license check on the tribe node to make sure that the tribe node can start, otherwise license plugin would try to generate a new trial license which is not possible since the node has no master. License check still happens for tribes though. This will be improved once es core supports merging cluster level custom metadata, then the tribe node will see some license coming from its tribes and won't require any additional license.
Added integration test to verify basic functionality against a tribe node, which also validates the settings needed on the tribes.
Made sure that shield is loaded and enabled on very tribe if loaded and enabled on the tribe node. We want to make sure that nobody manages to use shield on the tribe node only for free (since we disabled liccensing there), with no shield on the tribes. If we forcibly enable and make the shield plugin mandatory on the tribe clients, it means that they will not be able to join their corresponding clusters unless they have shield loaded and enabled too. As a result, shield is supported in the tribe node as long as all the tribes have shield loaded and enabled too.
Relates to elastic/elasticsearch#311Closeselastic/elasticsearch#584
Original commit: elastic/x-pack-elasticsearch@317add553f
Only enables TLSv1, TLSv1.1, and TLSv1.2 protocols for transport, http, and ldaps. The supported
protocols are configurable in case one of these protocols is found to be insecure in the future.
Closeselastic/elasticsearch#594
Original commit: elastic/x-pack-elasticsearch@d4556091ef
Previously, AD group search used the user search dn so this adds configuration to separate the group search from the user search
This adds tests for the newly introduced SearchScope.Base and includes general test cleanup.
Original commit: elastic/x-pack-elasticsearch@6ed1114b29
- Introduced a strategy for group search. This is applied on the `AbstractLdapConnection` level.
- The `LdapConnection` and `ActiveDirectoryConnection` are now clean of logic
- The `AbstractLdapConnection` holds a timeout
- Introduced `SearchScope` for better settings support.
- fixed a bug in `LdapConnectionFactory:74`... `settings.getAsArray` will never return `null`
Original commit: elastic/x-pack-elasticsearch@1f4a43d037
This lets the user configure custom filters for group searches in LDAP, and user searches in AD
changed configuration in this commit:
group_search.group_search_dn -> group_search.base_dn
group_search.subtree_search -> group_search.subtree
added for LDAP:
group_search.filter
group_search.user_attribute
added for AD:
user_search.base_dn
user_search.filter
user_search.subtree
This also changes group_search.subtree to be true by default.
This fixeselastic/elasticsearch#567 and fixeselastic/elasticsearch#553
Original commit: elastic/x-pack-elasticsearch@8a1246aefd
The analyze api allows to specify an index, to retrieve analyzers or token filters from a specific index. That is why it is categorized as indices level action. That said the index is optional and when not specified the action is executed at the cluster level. We have to remap the name of the action in that case, to make sure that it requires a different privilege under cluster: cluster:admin/analyze instead of indices:admin/analyze .
Closeselastic/elasticsearch#566Closeselastic/elasticsearch#565Closeselastic/elasticsearch#592
Original commit: elastic/x-pack-elasticsearch@9073b30d08
This change adds the option to disable reverse DNS lookup of a hostname from
an IP address. This is needed if only an IP address is found in a SSL certificate
and hostname verification is enabled.
Closeselastic/elasticsearch#575
Original commit: elastic/x-pack-elasticsearch@07356bc885
IndicesAliasesRequest needs to be authorized against both indices and aliases. That means that the following request
curl -XPOST 'http://localhost:9200/_aliases' -d '
{
"actions" : [
{ "add" : { "index" : "test1", "alias" : "alias1" } }
]
}'
requires now indices:admin/aliases privileges for both test1 and alias1.
Added manage_aliases shortcut privilege that points to indices:admin/aliases.
Also, IndicesAliasesRequest used to not support replacing its indices (request doesn't implement IndicesRequest.Replaceable in es core). That can be worked around as well through the special treatment that we are introducing in shield for this specific request. Given that it is a composite action, every single operation has now its wildcards replaced with authorized indices (supported among aliases as well in case of remove operations). If any of the operation ends up relating to no indices after wildcards expansion, the whole request fails.
The DefaultIndicesResolver#explodeWildcards method, which used to expand wildcards as es core would do it, is not needed anymore, as all of the requests that support wildcards have now their indices properly replaced.
Added also special authorization pass for create index, if the request body contains aliases. The index can only be created if the current user has permission to create the index and to create the aliases that are part of the same request.
Closeselastic/elasticsearch#112Closeselastic/elasticsearch#557Closeselastic/elasticsearch#529
Original commit: elastic/x-pack-elasticsearch@d7201e8a8b
Added a class under test that is used to keep track of changes that we might have to make once we upgrade versions of dependencies, especially elasticsearch core.
Every change is listed as a specific assert that trips with a future version of es core, with a meaningful description that explains what needs to be done.
NOTE: changes suggested by asserts descriptions may break backwards compatibility. The same shield jar is supposed to work against multiple es core versions,
thus if we make a change in shield that requires e.g. es core 1.4.1 it means that the next shield release won't support es core 1.4.0 anymore.
In many cases we will just have to bump the version of the assert then, unless we want to break backwards compatibility, but the idea is that this class
helps keeping track of this and eventually making changes when needed.
Closeselastic/elasticsearch#560
Original commit: elastic/x-pack-elasticsearch@fabe3858c1
- made sure that clear_scroll all gets converted to the correspoinding shield cluster action in both action filter and transport filter (used to happen only on the action filter before): introduced the context of ShieldActionMapper that allows to convert action names based on an incoming request and its action name (will be useful for analyze api too)
- made sure that potential clear_scroll all errors contain the shield action name rather than the es core original one
- made it clearer that the only indices actions known not to be indices requests are scroll related ones, which we assert on and grant. Everything else gets denied.
- made it clearer that the only indices request whose indices might end up being resolved to an empty set is analyze request, as its index is optional
- simplified permissions check in Permission.Group by asserting on index argument not null
Original commit: elastic/x-pack-elasticsearch@7c01159b03
All three files are auto loaded by shield when modified. The behaviour that we agreed on is that when there's a parse failure in any of these files, we don't prevent the node from starting. Instead we skip the records that we failed to parse as if they don't exist. This is how `roles.yml` is handled today, and this commit makes sure that `users`, `users_roles` and `role_mapping.yml` are aligned with this behaviour.
Also, the same behaviour is applied when the file is modified at runtime (so it's consistent with node start up).
This commit also adds a lot of missing tests for both `LdapGroupToRoleMapper` and `ActiveDirectoryGroupToRoleMapper` classes.
Original commit: elastic/x-pack-elasticsearch@7fdd6bb5cc
Some request are created locally by elasticsearch and therefore are not associated with a remote address (we only associate the remote address with a request that arrives remotely from via the transport layer). An example of such request is the periodic nodes info that is collected by elasticsearch. Also, requests that originate from the REST layer also create transport requests locally.
This commit takes this behaviour into account and makes sure that we'll always log the host in the audit logs. We do that in the following way:
- `host` is replaced by two attributes: `origin_type` and `origin_address`. `origin_type` can be either `rest`, `remote_node` or `local_node`. `origin_address` holds the host address of the origin
- when no remote address is associated with the request, it's safe to assume it was created locally. We'll then output `origin_type=[local_node] origin_address=[<the localhost address>]`
- when a rest request gets in, we'll copy and place its remote address in the context of the request (the context of the rest request is copied to the context of the transport request)
- . in the audit logs, we'll inspect the transport request and look for a `rest_host` in its context. if we find it, we'll log the log entry under `origin_type=[rest], origin_address=[<the remote rest address>]` attributes. This way, the origin of the request won't get "lost" and we'll still differentiate between transport hosts and rest hosts.
- if the request is holds a remote address, it can only come from the transport layer, so we'll output "origin_type=[transport] origin_address=[<remote address]"
While at it, also changed the format of the log entries:
- lowercased the whole message (e.g. `ANONYMOUS_ACCESS` to `[anonymous_access]` (for consistency sake)
- introduced layer categorization for every entry to indicate whether its `[transport]`, `[rest]` or `[ip_filter]` related. I reckon this will make it easier to parse the logs if one wishes to do so.
Fixeselastic/elasticsearch#550
Original commit: elastic/x-pack-elasticsearch@b84f0c5548
This commit moves the es core dependency to 1.4.2, which becomes the minimum version required from now on.
Changes made accordingly to this decision since we can break backwards compatibility and assume es core>=1.4.2
Closeselastic/elasticsearch#562
Original commit: elastic/x-pack-elasticsearch@484b4a2528
This to keep consistent with es core.
Also, where applicable, rephrased log messages to make it clearer.
Original commit: elastic/x-pack-elasticsearch@fae3188b17
Today we require that the `roles.yml` file will be a valid yaml and all the role definitions there must be valid as well. If we can't fully parse this file, we simply throw an exception and ignore its content. After al long discussion, we decided that it would be much better to try and parse whatever we can out of this file and load the valid roles. Those invalid roles will be skipped and immediately removed from the system.
This commit changes the way we parse the `roles.yml`. We first break it down to mini single-role yml constructs and then parse each separately from the others. This way, failing to parse one role, won't impact the others.
Fixeselastic/elasticsearch#313
Original commit: elastic/x-pack-elasticsearch@31e3624594
Meaning, all loggers are now settings aware, so all shield logs are now consistent with the rest of elasticsearch and will follow elasticsearch configuration and output format (printing out the node name by default).
Also:
- Changed the audit log to **not** be based on the elasticsearch settings as it needs to define its own format.
- Added the node name as a prefix to the audit logs by default (can be disabled but setting `shield.audit.logfile.prefix.node_name` to `false`
- As part of this change, the realms now changed and now created with a `RealmConfig`. This construct holds the realm settings, the environment and is served as a logger factory for all realm constructs.
- The only exceptions to the logs are the ssl socket factories.. the logs there are only used for tests by calling `clear`. This behaviour will change in the future such that `clear` will be removed and then there'll be no need for loggers in there.
Fixeselastic/elasticsearch#446
Original commit: elastic/x-pack-elasticsearch@7a1058a54e
Since both LDAP and AD realms are caching users. If the groups of the users change on the LDAP side, these changes will not be visible in shield until the relevant cached users will be evicted from cache. This poses a problem, specially when degrading users in terms of their permission (e.g. after degrading them on LDAP, they still have higher privileges until they're evicted from cache). The default cache timeout today is 1 hour. For this reason, a new API is introduced which will enable administrators to force cache evictions.
- Changed the default cache timeout to 20 minute
- `ClearRealmCacheAction was introduced (along with the relevant request and response constructs). This is a cluster action
- the corresponding rest action was introduced as well, under the `_shield/realm/{realm}/cache/clear` URI (where `{realm}` enables clearing specific realms, or all realms when passing `_all`.
- With the introduction of an action, the `ActionModule` now is no longer a node module - it's bound on both node and transport client.
- Added a new Cluster permission - `manage_shield`
- Also cleaned up the `Permission` and `AuthorizationService` class
Original commit: elastic/x-pack-elasticsearch@c59e244435
This checkin ensures all objects allocated by jndi requests are freed up. It does this by wrapping NamingEnumerations with a ClosableNamingEnumeration that is placed in a try-with-resources block.
Original commit: elastic/x-pack-elasticsearch@8bed9585bd
A range is provided for the client profile and the test assumes that the first
port in the range is the port that the transport is bound to, which is not always
true. This change makes the test use the actual port that the client profile is
bound to.
Closeselastic/elasticsearch#531
Original commit: elastic/x-pack-elasticsearch@05962702ed
Hostname verification was previously randomized on a per node level, when it really
should have been a cluster level setting. This change makes hostname verification
randomization a cluster level settings.
Original commit: elastic/x-pack-elasticsearch@2a7da8aaf1
Since the system privilege also mapped to cluster/index monitoring actions, the access granted on those was only logged in `TRACE` level. This commit makes sure that these actions will be treated as any of the other actions, and only keep the *internal* system calls under `TRACE`
Fixeselastic/elasticsearch#554
Original commit: elastic/x-pack-elasticsearch@ffb719f547
uboness
c-a-m
jaymode
Alexander Reelsen
javanna