Commit Graph

388 Commits

Author SHA1 Message Date
Lee Hinman 05f72c0b91 Revert "Add a qa/rolling-upgrade test that does single-document index and deletes"
This reverts commit elastic/x-pack@ff05f28b7e.

Original commit: elastic/x-pack-elasticsearch@a05fc1f9e6
2017-01-10 11:19:15 -07:00
Lee Hinman 320ec6716d Merge branch 'master' into enhancement/use_shard_bulk_for_single_ops
Original commit: elastic/x-pack-elasticsearch@23761f3e16
2017-01-09 16:27:42 -07:00
Lee Hinman 45e84cde6e Add PreventFailingBuildIT in qa/smoke-test-watcher
The watcher tests were recently marked with `@Network`, which prevents them from
normally being run. Unfortunately, this means no tests run by default and the
entire suite fails.

Original commit: elastic/x-pack-elasticsearch@40cfc75b26
2017-01-09 15:59:32 -07:00
Lee Hinman 8326b6d83b Merge branch 'master' into enhancement/use_shard_bulk_for_single_ops
Original commit: elastic/x-pack-elasticsearch@98f4e74d2e
2017-01-09 14:22:18 -07:00
Jay Modi e0f0b4b7b8 rename the kibana role to kibana_system
This commit renames the kibana role to kibana_system and provides a backwards compatibility
layer so that kibana access still works properly during a rolling upgrade.

Closes elastic/elasticsearch#4525

Original commit: elastic/x-pack-elasticsearch@5c5796e53a
2017-01-09 16:06:50 -05:00
Lee Hinman 93720505b8 Merge branch 'master' into enhancement/use_shard_bulk_for_single_ops
Original commit: elastic/x-pack-elasticsearch@089fa9977d
2017-01-09 11:39:37 -07:00
Lee Hinman 7387d04139 Add a qa/rolling-upgrade test that does single-document index and deletes
Original commit: elastic/x-pack-elasticsearch@5850439b22
2017-01-09 11:16:33 -07:00
Tanguy Leroux a86b112f1e [Watcher] Delete JIRA issues after integration tests execution (elastic/elasticsearch#4552)
This commit delete the JIRA issues after the integration test execution. All issues from the testing project XWT are deleted, even if they have not been created during this specific test execution.

closes elastic/elasticsearch#4535

Original commit: elastic/x-pack-elasticsearch@0362463633
2017-01-09 10:41:48 +01:00
Tanguy Leroux 19cbab4ac3 [Watcher] Update triggering interval in Jira integration tests
In Jira integration tests, some watches are triggered every second whereas they are executed using the watch execute API. This commit increases the triggering interval to 1d so that the watches are not executed on slow machines.

Original commit: elastic/x-pack-elasticsearch@4d0462bc00
2017-01-05 21:00:53 +01:00
Tanguy Leroux 3d2d1d49b6 [Watcher] Move Jira integration tests to smoke-test-watcher (elastic/elasticsearch#4534)
This commit moves the Jira rest integration tests from the smoke-test-watcher-with-mustache project to the smoke-test-watcher project.

Original commit: elastic/x-pack-elasticsearch@c6b03d557f
2017-01-05 15:22:59 +01:00
Alexander Reelsen 63f4bbba98 Watcher: Use Apache HttpClient for internal Watcher HttpClient (elastic/elasticsearch#4434)
Watcher: Use Apache HttpClient for internal Watcher HttpClient

The current implementation based on URLConnection has several drawbacks.

* If server returned HTTP header but then got stuck, no timeout would help, the connection remained stuck
* GET requests with a body were not supported, the method was silently changed to POST
* More complex handling of input/error stream handling, the body could not be read from a single input stream

NOTE: This is a BWC breaker. From now on every part of the URL needs to be encoded properly before it is configured in the requeust builder. This requires an upgrade of all watches.

Closes elastic/elasticsearch#1141

Original commit: elastic/x-pack-elasticsearch@bbc8f85dd8
2017-01-05 14:25:58 +01:00
Tanguy Leroux 125a2c9c03 [Watcher] Enable Jira integration tests (elastic/elasticsearch#4331)
This commit enables the Jira integration tests with the Jira project and account provided by Edward Sy.

closes elastic/infraelastic/elasticsearch#1498

Original commit: elastic/x-pack-elasticsearch@78d1005064
2017-01-05 13:44:04 +01:00
Nik Everett 93b16f0615 Fix another qa project
Original commit: elastic/x-pack-elasticsearch@9609036dde
2017-01-05 10:56:09 +01:00
Nik Everett f2ae490b32 Switch from standalone-test to standalone-rest-test
standalone-rest-test doesn't configure unit tests and for these
integTest only projects that is what we want.

Original commit: elastic/x-pack-elasticsearch@f576dfdfbb
2017-01-05 10:56:09 +01:00
Nik Everett 5b6bfffa9a Require either BuildPlugin or StandaloneTestBasePlugin to use RestTestPlugin
It used to be that RestTestPlugin "came with" StandaloneTestBasePlugin
but we'd like to use it with BuildPlugin for the high level rest client.

Also fix some license headers.

Original commit: elastic/x-pack-elasticsearch@3d5549d170
2017-01-05 10:56:09 +01:00
Jay Modi 18a2cf23d4 Build a single role that represents a user's permissions (elastic/elasticsearch#4449)
This PR changes how we use roles and how we look at the roles of a user. Previously we looked up each role individually, parsed each into their own `Role` object, and had a wrapper that essentially served as an iterator over the roles. The same pattern was also followed for the permissions that composed a role (ClusterPermission, IndicesPermission, and RunAsPermission). This resulted in a lot of code that was hard to follow and could be inefficient. 

Now, we look up the roles for a user in bulk and only get the RoleDescriptor for each role. Once all role descriptors have been retrieved, we build a single Role that represents the user's permissions and we also cache this combination for better performance as authorization can happen many times for a single top level request as we authorize the top level request and any sub requests, which could be a large number in the case of shard requests.

This change also enabled a large cleanup of our permission and privilege classes, which should reduce the footprint of what needs to be followed. Some of the notable changes are:

* Consolidation of GeneralPrivilege and AbstractAutomatonPrivilege into the Privilege class
* The DefaultRole class has been removed and the permissions it provided were moved into the AuthorizationService
* The GlobalPermission class was removed as there is a single role that represents a user's permissions
* The Global inner classes for the various permissions were removed
* The Core inner class was removed and ClusterPermission, IndexPermission, RunAsPermission became final classes instead of interfaces
* The Permission interface has been removed. The isEmpty() method defined by this interface is not needed as we can simply evaluate the permission to get the same effect
* The ClusterPermission#check method only takes the action name again
* The AutomatonPredicate class was removed and replaced by Automatons#predicate
* IndicesAccessControl objects no longer need to be merged when evaluating permissions
* MergedFieldPermissions has been removed
* The Name class that was used to hold an array of strings has been removed and replaced with the use of a Set
* Privilege resolution is more efficient by only combining automata once

Other items:
* NativeRolesStore no longer does caching, so the RoleAndVersion class could be removed
* FileRolesStore doesn't need to be an AbstractLifecycleComponent

Relates elastic/elasticsearch#4327

Original commit: elastic/x-pack-elasticsearch@c1901bc82e
2016-12-30 09:27:49 -05:00
Nik Everett 4ff6279865 Remove exception from client suite
Original commit: elastic/x-pack-elasticsearch@51e94561d9
2016-12-22 09:10:42 -05:00
Areek Zillur 84db1b8731 x-pack changes for elasticsearchelastic/elasticsearch#21964
In https://github.com/elastic/elasticsearch/pull/21964, index
and delete operations are executed as single item bulk requests
internally. This means index and delete operations use the
bulk transport endpoints (indices:data/write/bulk[s][p] and
indices:data/write/bulk[s][r]).

This PR adds bulk transport endpoint to 'write' and 'delete'
index privilages and adds index and delete action as composite
actions to delay the authentication to the shard level.

Original commit: elastic/x-pack-elasticsearch@2305fc9ca0
2016-12-22 02:35:35 -05:00
jaymode 880808c428 test: disable security manager for custom realm integration tests
Original commit: elastic/x-pack-elasticsearch@4bcd7675b2
2016-12-21 16:05:43 -05:00
jaymode 9288401767 test: do not use security manager for client tests
Original commit: elastic/x-pack-elasticsearch@1200a1b757
2016-12-21 14:46:18 -05:00
Ryan Ernst 789df7d2fa Internal: Update signatures for EnvironmentAwareCommand (elastic/elasticsearch#4367)
This is the xplugins side of elastic/elasticsearchelastic/elasticsearch#22175

Original commit: elastic/x-pack-elasticsearch@4359cb1947
2016-12-19 15:23:53 -08:00
Ryan Ernst 6d4d599f91 Build: Convert xplugins to use new extra projects setup (elastic/elasticsearch#4175)
* Build: Convert xplugins to use new extra projects setup

This change makes the gradle initialization for xplugins look in the
correct location for elasticsearch, which is now as a sibling of an
elasticsearch-extra directory, with x-plugins as a child of the extra
directory.

The elasticsearch side of this change is
elastic/elasticsearchelastic/elasticsearch#21773. This change will enable renaming x-plugins
to x-pack, see elastic/elasticsearch#3643.

Original commit: elastic/x-pack-elasticsearch@09398aea5a
2016-12-14 15:02:28 -08:00
Alexander Reelsen b57c4f6ebe Watcher: Remove in memory watch store (elastic/elasticsearch#4201)
In order to prepare to the distributed watch execution, this commit
removes the in memory watch store.

Whenever a watch is needed now, a get request is executed and the parsing
is done. This happens when

* Put
* Get
* Ack
* Activate/Deactivate
* Execute

Note: This also means there are no usage stats currently regarding
the watch count, because we would need to execute a query. This would
require the usage stats to be async, see elastic/elasticsearch#3569

Another advantage is, that there is no dirty flag in the watch itself
needed anymore, because the watch is always the latest. Also write
operations store immediately and dont leave anything in memory.

Also ActionListener.wrap() was used a lot instead of more verbose anonmyous
inner classes.

Original commit: elastic/x-pack-elasticsearch@c47465b47c
2016-12-13 08:54:03 +01:00
Jay Modi ac34c3c37f Remove deprecated methods from the Realm class
This change removes the deprecated methods from the realm class. These methods include
blocking authentication and lookup and the lookup supported method.

Original commit: elastic/x-pack-elasticsearch@cff21e21ee
2016-12-09 07:28:11 -05:00
jaymode 2b7c03848f test: disable xpack plugins on tribe client nodes
The license test for the tribe node disables xpack features but these settings
are not passed to the tribe client nodes, so they need to manually be added.

Original commit: elastic/x-pack-elasticsearch@7273817c47
2016-12-03 14:02:55 -05:00
Simon Willnauer 7f17896927 Revert "Followup for elastic/elasticsearchelastic/elasticsearch#21915 - removal of legacy BWC test infrastructure (elastic/elasticsearch#4247)"
This reverts commit elastic/x-pack@c6c0ffd5d9.

Original commit: elastic/x-pack-elasticsearch@8b7386fb99
2016-12-02 10:55:46 +01:00
Alexander Reelsen 7c04897392 Watcher: Compile scripts on each invocation (elastic/elasticsearch#4239)
Transform and condition scripts were only compiled on its initial creation, so
when a new watch is created or when the master node loads all the watches. However
changing a script (like a stored one) did not lead to any changes in the in memory
watch store and thus the old script was executed again.

We do however have a mechanism in Elasticsearch's ScriptService that already does some
caching, and should reuse that one.

Closes elastic/elasticsearch#4237

Original commit: elastic/x-pack-elasticsearch@477548e237
2016-12-02 10:36:05 +01:00
Simon Willnauer ace1a7e6af Followup for elastic/elasticsearchelastic/elasticsearch#21915 - removal of legacy BWC test infrastructure (elastic/elasticsearch#4247)
Original commit: elastic/x-pack-elasticsearch@07cecdbf00
2016-12-02 08:06:46 +01:00
Jay Modi e8836f99b0 test: add tests that exercise the security index during rolling upgrades
This commit adds basic tests that store a user and a role using the native API. The test checks
that the user and role can be used prior to starting the upgrade. The realm and roles caches are
also cleared to ensure the next authentication will require a read from the security index; this
ensures we are actually testing reads from the index.

Original commit: elastic/x-pack-elasticsearch@396862da94
2016-12-01 11:15:15 -05:00
Jay Modi 139073e8f7 security: improve migrate tool output and remove trappy config option
This commit improves the output of the migrate tool in cases when there are errors parsing entries
in the roles or users files. This is done through the use of a logger that delegates its output to
the terminal.

Additionally, the `-c` option has been removed. This option was used to set the configuration directory
but this should be handled one way only and that is through the use of the `-Epath.conf` setting.

Closes elastic/elasticsearch#3757
Closes elastic/elasticsearch#3758

Original commit: elastic/x-pack-elasticsearch@811e367766
2016-12-01 10:17:28 -05:00
javanna f19be18b20 [TEST] add render search template qa test
Original commit: elastic/x-pack-elasticsearch@7febccbfb2
2016-11-29 23:45:52 +01:00
Luca Cavanna 34d6dc1db1 Categorize search template action as a composite indices request (elastic/elasticsearch#4209)
When we encounter a composite request, we authorize at first without looking at the indices, to see whether the action can be executed at all. We then rely on the action to delegate to an inner action per sub-request, which will be authorized based on the indices it refers to. The first step works great for the simulate mode of search template, as it doesn't involve any index. The second step will make sure that when search template involves a search, it will be authorized as a normal search request would, based on the indices it reads from.

Note that the wildcard expansion happens now on the search side, it doesn't have to happen when executing the first authorization step, hence SearchTemplateRequest doesn't have to implement IndicesRequest, only SearchRequest has to (which it does already).

Closes elastic/elasticsearch#4171

Original commit: elastic/x-pack-elasticsearch@d586bd90cb
2016-11-29 20:53:01 +01:00
Yannick Welsch c7d7a2bafc [TEST] Wait in rolling_upgrade rest test for monitoring indices to be fully allocated
Fixes previous commit elastic/x-pack@8bb4e6a so that it also accounts for monitoring indices that are automatically created by x-pack while the test is running.

Original commit: elastic/x-pack-elasticsearch@e50e1a2717
2016-11-25 12:54:41 +01:00
Yannick Welsch 16b624b1e4 [TEST] Wait in rolling_upgrade rest test for old cluster to have all indices fully allocated
When one of the 2 nodes in the old cluster is shut down, shards that were on that node will become unassigned and be marked to be
delay-allocated, i.e. either a node with shard data for that shard must be available or the allocation of the shards will be delayed for a minute.
In the mixed cluster the replica shard might not be allocated as the primary is already on the node with the newer version and replicas are not allowed
then to be allocated to a node of an older version of ES. Once both nodes are upgraded, the delay might still be in place, and can only be nullified if there
is shard data available on the node. If there never was a shard on that node though, it will take a minute and run into the timeout checking for green.

This commit ensures that all shards are fully-allocated before we do the rolling restart scenario

Original commit: elastic/x-pack-elasticsearch@a0d9b1b043
2016-11-24 16:43:08 +01:00
Yannick Welsch b503f05916 [Test] Fix rolling upgrade test to only wait for yellow in a mixed cluster
If the primary shard of an index with (number_of_replicas > 0) ends up on a new node in a mixed cluster, the replica cannot be allocated to the old node as
the new node might have written segments that use a new postings format or codec that is not available on the older node.
As x-pack automatically creates indices with number_of_replicas > 0, for example monitoring-data-*, the test can only wait for yellow in a mixed cluster.

Original commit: elastic/x-pack-elasticsearch@945d9e3811
2016-11-23 14:36:48 +01:00
Tanguy Leroux a32f2096a6 Add mappings for Jira action (elastic/elasticsearch#4155)
This commit updates the watch_history.json file so that it includes mappings for the new Jira action. It also update the JiraIssue format so that it now includes the name of the account used to create the Jira issue. It also update the REST tests to check that Jira action result are searchable and hide the user's password.

Original commit: elastic/x-pack-elasticsearch@75888f7748
2016-11-23 11:53:06 +01:00
Ryan Ernst 1dc839bd98 Remove groovy scripting language (elastic/elasticsearch#4162)
This is the xplugins side of elastic/elasticsearchelastic/elasticsearch#21607

Original commit: elastic/x-pack-elasticsearch@125843e814
2016-11-22 22:45:15 -08:00
jaymode 9126600fc3 test: add timeout to cluster health call that we are waiting on
This commit adds a timeout to the cluster health call that we wait on so that we can
see the status of the health request instead of getting timeouts failures with no
information to go on.

Original commit: elastic/x-pack-elasticsearch@2f34d01e00
2016-11-22 15:17:21 -05:00
jaymode 6170f3d22c test: use the right number of bwc nodes in old cluster
Original commit: elastic/x-pack-elasticsearch@0afc045bbc
2016-11-22 15:14:29 -05:00
Jay Modi 4239ba5415 allow reads of native users and roles when template version hasn't been updated
This change allows reads of our native users and roles when the template version has not been updated to
match the current version. This is useful for rolling upgrades where the nodes are also being actively
queried and/or indexed into. Without this, we can wreak havoc on a cluster by causing exceptions during
replication, which leads to shard failures. On nodes that match the version defined in the template,
write operations are allowed since we know that we are backwards compatible in terms of format but we
may have added new fields and shouldn't index them until the mappings and template have been updated.

As part of this, the rolling upgrade tests from core were used as the basis for a very basic set of tests
for doing a rolling upgrade with x-pack.

Closes elastic/elasticsearch#4126

Original commit: elastic/x-pack-elasticsearch@9be518ef00
2016-11-22 12:00:09 -05:00
Areek Zillur a9f3619b5a Enable merging license in tribe node (elastic/elasticsearch#4147)
Currently, a tribe node ignored underlying cluster licenses
due to inablity to select an appropriate license from  multiple
licenses. Now that tribe node supports merging custom metadata
(elasticsearch#elastic/elasticsearch#21552), we can enable license support in tribe
node.

Now, tribe node chooses license with the highest operation
mode from underlying cluster licenses. This commit also
adds integration tests for licensing to verify that:
 - autogenerated trial license propagates to tribe node
 - tribe node chooses the highest operation mode license
 - removing a license from underlying cluster license is
   removed from tribe

closes elastic/elasticsearch#3212

Original commit: elastic/x-pack-elasticsearch@b5c003decd
2016-11-22 11:42:51 -05:00
Tanguy Leroux 18478d63c2 Watcher: Add JIRA action (elastic/elasticsearch#4014)
closes elastic/elasticsearch#493

Original commit: elastic/x-pack-elasticsearch@6b7387d3e4
2016-11-21 10:52:55 +01:00
Simon Willnauer 92040ef72e Remove netty_3 support from xpack (elastic/elasticsearch#4097)
This is a followup from elastic/elasticsearchelastic/elasticsearch#21590 and needs to be
committed first or at the same time since netty_3 is removed

Original commit: elastic/x-pack-elasticsearch@131d74dd6b
2016-11-17 12:44:24 +01:00
Tanguy Leroux 4badf28a8d Add Vagrant Gradle plugin (elastic/elasticsearch#3993)
This commit adds a new Gradle sub project that makes use of the VagrantTestPlugin in order to test the installation of X-Pack.

Original commit: elastic/x-pack-elasticsearch@e09db6602c
2016-11-15 15:30:13 +01:00
Yannick Welsch 7b165504dc Use project-defined Java installation for keytool (elastic/elasticsearch#4066)
Companion commit for elastic/elasticsearchelastic/elasticsearch#21540

Original commit: elastic/x-pack-elasticsearch@a1c21ece25
2016-11-15 09:32:04 +01:00
Ryan Ernst 23e6cab7f1 Merge pull request elastic/elasticsearch#4044 from rjernst/realm_sig
Extensions: Make resource watcher available to custom realms

Original commit: elastic/x-pack-elasticsearch@3cb494e98d
2016-11-14 12:38:27 -08:00
Yannick Welsch 9d5ebe9e2a Use project-defined Java installation for keytool
Companion commit for elastic/elasticsearchelastic/elasticsearch#21540

Original commit: elastic/x-pack-elasticsearch@aa4e2df5bf
2016-11-14 15:43:11 +01:00
Nik Everett 700467c3a7 Use index_patterns in templates
And skip a REST test that won't pass.

Original commit: elastic/x-pack-elasticsearch@e297add6c1
2016-11-10 21:42:59 -05:00
Ryan Ernst bcd32ada4f Extensions: Make resource watcher available to custom realms
This simply adds ResourceWatcherService as an arg for getting custom
realms from xpack extensions.

closes elastic/elasticsearch#4038

Original commit: elastic/x-pack-elasticsearch@fe58d8a7ee
2016-11-10 12:43:28 -08:00
Ryan Ernst fa97a806ca Remove unneeded rest test params
This is the xplugins side of elastic/elasticsearchelastic/elasticsearch#21391

Original commit: elastic/x-pack-elasticsearch@30d36e340a
2016-11-07 14:46:39 -08:00
Jay Modi bd522191b2 test: use toJSON inside a role
This commit adds a test that uses toJSON templating inside of a role with
an array defined in the user's metadata.

Originates from user discussion at:
https://discuss.elastic.co/t/x-pack-security-role-definition-query-template-with-terms/62790

Original commit: elastic/x-pack-elasticsearch@196f7f597c
2016-11-07 07:21:41 -05:00
Jack Conradson 7dd4188299 Cleanup ScriptType (elastic/elasticsearch#3922)
Refactored ScriptType to clean up some of the variable and method names. Added more documentation. Deprecated the 'in' ParseField in favor of 'stored' to match the indexed scripts being replaced by stored scripts.

Original commit: elastic/x-pack-elasticsearch@d7c7bd7362
2016-10-31 13:49:10 -07:00
Nik Everett 0f54f9524d Handle reindex's change to whitelist
Reindex has grown support for simple whitelist patterns like
`localhost:*` and lost support for `myself`.

Original commit: elastic/x-pack-elasticsearch@31d9c4dc5c
2016-10-18 21:46:30 -04:00
javanna 4bb6e856f3 Authorize composite actions based on their action name only, subrequests and their indices will be later authorized individually
Eagerly authorizing CompositeIndicesRequests allowed the security plugin to fail fast up until now, but it makes it very hard to reason about each specific item in a multi items request. Either all items fail, or none do. We would rather want to adopt a similar behaviour to es core, where individual items fail without affecting other items that are part of the same request. We can rely on the fact that es core always authorizes both main action and every subaction too, and skip authorization for the main action. By subaction we mean either all sub search requests in msearch, as well as each shard level get in mget or shard level bulk request for bulk.

 BulkRequestInterceptor was converted to intercept BulkShardRequests rather than BulkRequest as that is where bulk is authorized after this change.

 Split IndicesAndAliasesResolverIntegrationTests into ReadActionsTests and WriteActionsTests as they require different set of permissions, lots of tests added.

Explicitly listing the composite actions makes sure that the actions that can bypass security are known, somebody adding a similar action must to add it to the list, so we know it doesn't happen by mistake. At this point the CompositeIndicesRequest can be used as a marker interface only (it is not really needed but can be used to verify that composite actions use a request that implements such interface).

Given that we don't authorize composite actions based on their indices anymore, but only their sub-requests which implement IndicesRequest, printing out the indices names in the audit log for requests like bulk and msearch is confusing. Removed support for that.

Authorize composite indices actions based on their name only, their indices will be authorized at the sub-request/shard level

Rather than simply granting bulk, mget, msearch etc. and relying on authorization at the sub-request/shard level, we check that the current user can at least execute the action. This justifies the grant line that gets written in the audit log, the action is potentially possible without looking at the indices. Each specific item will fail or succeed later and will yield its own specific audit log entry.

Original commit: elastic/x-pack-elasticsearch@4570caf019
2016-10-13 16:05:02 +02:00
javanna d27c4bee82 Support allowNoIndices option in security plugin
Supporting allowNoIndices means that the security plugin has a behaviour much more similar to vanilla es when dealing with wildcard expressions that match no indices, or empty clusters. The default for most request is to allow no indices, but security plugin could only disallow no indices all the time up until now.

The technical problem was that when anything gets resolved to an empty set of indices, we couldn't let that go through to es core, as that would become resolved to all indices by es core, which would be a security hole. We have now found a way though to replace an empty set of indices with something that es core will for sure resolve to no indices, so we can let the request through. We simply replace empty indices with '-*'.

Multi apis requests (e.g. _msearch) have yet to be fixed, as all their indices end up in the same bucket while they should each be authorized separately, so that every specific item can fail or be let through.

Original commit: elastic/x-pack-elasticsearch@0f67a0bfea
2016-10-13 16:05:02 +02:00
javanna 9b46b34bed Honour ignore_unavailable option when resolving indices
For all the requests that support multiple indices and wildcards, hence implementing IndicesRequest.Replaceable, we replace the wildcard expressions with the explicit names of the authorized indices they match. _all or empty indices is treated as a wildcard expression. We can also honour the ignore_unavailable option by going over all the explicit names and filter out the non authorized ones when ignore_unavailable is set to true. If ignore_unavailable is set to false, we leave everything as-is, which will cause an authorization exception to be thrown if only one of those explicit indices is not authorized for the current user.

This is the first step towards resolving elastic/elasticsearch#1250. The remaining issue is that in case we are left with no indices after stripping out the ones that the user is not authorized for, we throw an authorization exception rather than returning an empty response. That will require honouring the allow_no_indices option, which will also change the behaviour when a cluster is empty.

Relates to elastic/elasticsearch#1250

Original commit: elastic/x-pack-elasticsearch@e4ca940d05
2016-10-13 16:05:02 +02:00
Alexander Reelsen 8b83cf067c Watcher: Ensure awesome painless exceptions are propagated to the user (elastic/elasticsearch#3707)
When adding a watch which has a painless component, the scriptexception
was wrapped into a deprecated exception which means, that the awesome
painless descriptions were lost. This wrapping has been removed.

Closes elastic/elasticsearch#3161

Original commit: elastic/x-pack-elasticsearch@1703fe4eb6
2016-10-12 08:14:06 +02:00
Nik Everett 6e31ab8d99 Skip a new template rest test
It isn't compatible with security.

Relates to elastic/elasticsearch#20658

Original commit: elastic/x-pack-elasticsearch@33df690341
2016-10-11 12:30:45 -04:00
Tanguy Leroux 2e7b7be25c Watcher: Re enable array compare test (elastic/elasticsearch#3708)
This test has been blacklisted and deactivated months ago. This commit reenables this test and moves it at the right place. It also change the test to use the Execute Watch API instead of being sleep based.

Original commit: elastic/x-pack-elasticsearch@e7a9689375
2016-10-11 10:25:40 +02:00
Simon Willnauer 2f70ae92b6 Cut over to MockTcpTransport since LocalTransport is remove in core (elastic/elasticsearch#3684)
This is a followup commit to elastic/elasticsearchelastic/elasticsearch#20695

Original commit: elastic/x-pack-elasticsearch@27cd454ba6
2016-10-07 11:28:05 +02:00
Ryan Ernst 1fa0f835fe Build: Reorganize src roots
This change flattens the directory structure, both for the elasticsearch
specific directories, as well as within the elasticsearch x-pack plugin.

closes elastic/elasticsearch#2957

Original commit: elastic/x-pack-elasticsearch@45891a4632
2016-10-01 09:46:43 +02:00
Ryan Ernst 905237a56f Moved directories around
Original commit: elastic/x-pack-elasticsearch@2018bb5f9f
2016-09-29 12:03:14 +02:00
jaymode e5b0e7f5cb reorganize directory layout
See elastic/elasticsearch#1022

Original commit: elastic/x-pack-elasticsearch@3ee8761312
2015-12-03 16:22:37 +01:00
Robert Muir 9df905ff19 Simplify SSL test to not use openssl.
I think the intent here is to just test that our SSL layers work,
not invoke a long chain of keytool + openssl commands.

This simplifies the build and will work on windows.

Original commit: elastic/x-pack-elasticsearch@af07d0d4f7
2015-11-30 09:28:16 -05:00
Robert Muir 8a22ba0a08 smoke-test-plugins-ssl shoudl check if openssl is available
Today some jenkins servers dont have it (e.g. windows), and it constantly fails...

Original commit: elastic/x-pack-elasticsearch@6b561c73e0
2015-11-30 08:42:25 -05:00
javanna 171179d91f [TEST] non stored fields are not returned anymore via fields
Relates to https://github.com/elastic/elasticsearch/issues/14489

Original commit: elastic/x-pack-elasticsearch@2897dc5df7
2015-11-30 12:12:31 +01:00
Ryan Ernst a67aebc9fc Build: Remove hack to touch keystore before it is created
This will be fixed in ES with elastic/elasticsearchelastic/elasticsearch#15089

Original commit: elastic/x-pack-elasticsearch@55b42a7ad4
2015-11-28 18:07:37 -08:00
Daniel Mitterdorfer ad697c077e Reenable SSL smoke tests
Original commit: elastic/x-pack-elasticsearch@1969c6d020
2015-11-27 17:25:51 +01:00
Daniel Mitterdorfer b9b39efac8 Disable SSL integration tests temporarily
We disable SSL integration tests as  a workaround for
https://github.com/elastic/infra/issues/628 to ensure
other problems can still be caught.

Original commit: elastic/x-pack-elasticsearch@47bf56faec
2015-11-27 10:23:03 +01:00
Ryan Ernst 660ac633a6 Merge pull request elastic/elasticsearch#1043 from rjernst/remove_ant_contrib
Build: Simplify ssl test to not use ant

Original commit: elastic/x-pack-elasticsearch@14d41f6fc1
2015-11-25 11:28:46 -08:00
Ryan Ernst 59a10e6309 Build: Simplify ssl test to not use ant
This change ports the tasks from the ssl ant build file into gradle tasks.

Original commit: elastic/x-pack-elasticsearch@af88196050
2015-11-25 11:23:26 -08:00
Robert Muir ae24881484 Move disabled watcher+groovy "unit" tests to qa/messy-test-watcher-with-groovy
This is all the tests disabled from https://github.com/elastic/x-plugins/issues/724

At least, they will be running in the build in some way. If we can fix gradle to
add plugin metadata from lang-groovy to the test classpath, security manager
can be re-enabled for these as well.

But its also only 8 tests, maybe its easier to fix them?

Original commit: elastic/x-pack-elasticsearch@a5c407b80f
2015-11-25 13:29:00 -05:00
Robert Muir 7ed4ea56b2 re-enable smoke-test-plugins with ssl
Note, its a bit crazy/hackish, but it works.

Original commit: elastic/x-pack-elasticsearch@377113c1c2
2015-11-24 23:19:04 -05:00
Robert Muir 140a399dfb Merge pull request elastic/elasticsearch#1033 from rmuir/enable_shield_example_realm_qa
re-enable shield example realm QA test

Original commit: elastic/x-pack-elasticsearch@73177eacc8
2015-11-24 20:12:47 -05:00
Robert Muir ac898ef4f3 re-enable shield example realm QA test
Original commit: elastic/x-pack-elasticsearch@98fd46f3aa
2015-11-24 20:10:46 -05:00
Ryan Ernst 66f3d18af0 Build: Add back smoke test plugins for xplugins
This checks that all ES plugins and xplugins are installed. I also
changed the rest check to be a simple plugin count, so it does not fail
when new plugins are added.

Original commit: elastic/x-pack-elasticsearch@eaab182e43
2015-11-24 17:09:56 -08:00
Robert Muir f0c0f75dbd Merge pull request elastic/elasticsearch#1031 from rmuir/shield_audit_qa
add back shield audit qa tests

Original commit: elastic/x-pack-elasticsearch@ce1e637b5f
2015-11-24 19:44:05 -05:00
Robert Muir 71d50ec058 add back shield audit qa tests
Original commit: elastic/x-pack-elasticsearch@f34b2c99e9
2015-11-24 19:41:50 -05:00
Ryan Ernst 19b7cad39c Build: Add back shield client qa tests
Original commit: elastic/x-pack-elasticsearch@6cecea3992
2015-11-24 16:40:05 -08:00
Ryan Ernst f96a6700c4 Build: Remove hack in shield+watcher rest test for copying config file
Original commit: elastic/x-pack-elasticsearch@e31ef685d0
2015-11-24 16:13:22 -08:00
Robert Muir 4b35407510 re-enable smoke-test-watcher-with-shield qa test
Original commit: elastic/x-pack-elasticsearch@2710eb67ef
2015-11-24 18:49:15 -05:00
Robert Muir a4f596b204 get watcher+groovy QA test working again (without hack)
Original commit: elastic/x-pack-elasticsearch@843a5ea6e4
2015-11-24 17:41:21 -05:00
Ryan Ernst 4f44ccedb5 Build: Simplify plugin installs for integTests
This is the xplugins side of elastic/elasticsearchelastic/elasticsearch#14986, making use of
the simplification in configuration.

Original commit: elastic/x-pack-elasticsearch@a24ad7b08a
2015-11-24 12:54:11 -08:00
Ryan Ernst b011490320 Add more retries for wait condition, in case jenkins is slow
Original commit: elastic/x-pack-elasticsearch@04e5648cd9
2015-11-24 08:24:57 -08:00
Ryan Ernst 9da4b6160c Build: Get shield qa test with core rest tests working
This adds back the shield qa rest tests module with gradle. There is
also a small fix in ShieldPlugin for a bug that was discovered around
checking for a custom query cache (which was using the node settings
instead of index settings).

Original commit: elastic/x-pack-elasticsearch@28c6d58f37
2015-11-24 01:05:21 -08:00
Robert Muir cfb77bf572 Remove confusing pom.xml files
Original commit: elastic/x-pack-elasticsearch@772a1eb5d3
2015-11-05 10:39:28 -05:00
jaymode bb6fe2abcd test: re-sync the smoke test plugins lists with core
Original commit: elastic/x-pack-elasticsearch@ba45f84d97
2015-10-27 09:48:31 -04:00
jaymode 203e6f4c6e add ensureYellow
Original commit: elastic/x-pack-elasticsearch@698c00dd6b
2015-10-26 16:25:50 -04:00
jaymode f4991d862a remove the ensure green
The random index template can set a number of replicas that will prevent the index
from ever being green in a single node cluster...

Original commit: elastic/x-pack-elasticsearch@81ecfe0818
2015-10-26 16:22:08 -04:00
jaymode 6a46660d40 remove unnecessary wildcards
Original commit: elastic/x-pack-elasticsearch@65bad879f6
2015-10-26 15:36:33 -04:00
jaymode 09990dbeee add a basic cluster info IT to the found qa module
Original commit: elastic/x-pack-elasticsearch@8f6334ef2b
2015-10-26 15:01:19 -04:00
jaymode dd27d9afe0 add a LicensesManagerService for found and add marvel-agent to smoke test
Original commit: elastic/x-pack-elasticsearch@8793058058
2015-10-26 14:29:52 -04:00
uboness a4c505ceb1 Changed the Marvel module name to Marvel Agent
- this results in a `marvel-agent-2.0.0.zip` artifact, to better differentiate it from the kibana marvel plugin
- post 2.0 when we move the marvel kibana codebase to x-plugins we'll need to differentiate between these two modules anyway.

Closes elastic/elasticsearch#846

Original commit: elastic/x-pack-elasticsearch@6b6a76f7dd
2015-10-21 14:03:23 +02:00
Nik Everett 8231e856c2 Remove and ban @Test
Original commit: elastic/x-pack-elasticsearch@02425ca13d
2015-10-20 18:20:46 -04:00
Alexander Reelsen f523a476e1 Tests: Change queries to reflect latest master changes (removed filter query)
Original commit: elastic/x-pack-elasticsearch@6c57eb0d1f
2015-10-16 17:03:42 +02:00
Konrad Beiske 4a780637e9 Change found-license-plugin to use standard value for install name and initial smoke test for found-license-plugin
Original commit: elastic/x-pack-elasticsearch@526c0c6da4
2015-10-13 14:00:27 -04:00
Jason Tedor b468c74401 Remove Guava as a dependency
This commit removes Guava as a dependency. Note that Guava will remain
as a test-only dependency (transitively through Elasticsearch through
Jimfs).

Relates elastic/elasticsearchelastic/elasticsearch#13224

Original commit: elastic/x-pack-elasticsearch@fe23d5f25f
2015-10-09 14:56:03 -04:00
jaymode 7b0f2628cb updates to handle renamed RenderSearchTemplateAction
Original commit: elastic/x-pack-elasticsearch@03cb49ce52
2015-10-08 09:09:00 -04:00
Robert Muir 9e9b835213 Move watcher REST tests that require groovy to smoke-test-watcher-with-groovy
See https://github.com/elastic/x-plugins/issues/724

Original commit: elastic/x-pack-elasticsearch@ab95aef8e3
2015-09-29 07:49:44 -04:00
Robert Muir 15bb2581a0 Fix compile and unit test phase by disabling unit tests with direct groovy dependencies (temporary!)
See https://github.com/elastic/x-plugins/issues/724

Original commit: elastic/x-pack-elasticsearch@27862a76cb
2015-09-29 07:06:51 -04:00
jaymode b219e9c496 remove lucene expressions from qa pom
Original commit: elastic/x-pack-elasticsearch@389cf7b564
2015-09-23 10:13:51 -04:00
Martijn van Groningen 0a807f0b1d test: fix the rest spec test resource directory location
Original commit: elastic/x-pack-elasticsearch@59b2b6923f
2015-09-17 11:52:54 +02:00
jaymode a0557e23a6 sync smoke test plugins list with core lists
Closes elastic/elasticsearch#636

Original commit: elastic/x-pack-elasticsearch@418bceef64
2015-09-16 06:47:16 -04:00
Martijn van Groningen 920b92ffd3 test: re-enabled tribe qa test
Original commit: elastic/x-pack-elasticsearch@7cffe1b9f5
2015-09-15 14:50:20 +02:00
Martijn van Groningen 5e7bfcfa75 test: echo log file after the index is created to find out why sometimes on CI the indices of that index don't get into a started state.
Original commit: elastic/x-pack-elasticsearch@4213d162d6
2015-09-14 14:55:34 +02:00
Rashmi Kulkarni 71f630e07b Shield Audit IndexTrail Test
closes elastic/elasticsearch#630

Original commit: elastic/x-pack-elasticsearch@6deeb07412
2015-09-11 14:27:12 -07:00
Martijn van Groningen 7eda007c46 test: temporarily disable tribe smoke test
Original commit: elastic/x-pack-elasticsearch@23eccc9de9
2015-09-10 10:28:39 +02:00
Ryan Ernst 7aa612b62e Merge pull request elastic/elasticsearch#615 from rjernst/warnings_cleanup2
Add warning suppressions

Original commit: elastic/x-pack-elasticsearch@a787f9c2a4
2015-09-09 12:51:51 -07:00
Ryan Ernst fbbd3f6c2d Add warning suppressions
I fixed a couple more warnings and added suppressions, so that when
 elastic/elasticsearchelastic/elasticsearch#13410 lands, x-plugins will not break.

Original commit: elastic/x-pack-elasticsearch@8a19b2b71b
2015-09-09 12:45:20 -07:00
jaymode 714460c2f0 remove path.home from TransportClients in code and docs
After changes in core and elastic/elasticsearch#578, we do not need to set path.home in the settings for a
TransportClient anymore. This cleans up the usages of it in our tests and in our documentation.

Closes elastic/elasticsearch#605

Original commit: elastic/x-pack-elasticsearch@d70875fe2b
2015-09-09 15:16:30 -04:00
Martijn van Groningen a6dc1ad97d test: make sure to stop nodes after the tribe node integration test
Original commit: elastic/x-pack-elasticsearch@3aaaced6ec
2015-09-09 21:14:36 +02:00
jaymode 9e3bf47a87 update the transport client and add integration tests
Closes elastic/elasticsearch#477

Original commit: elastic/x-pack-elasticsearch@8926f6ca44
2015-09-09 12:30:41 -04:00
jaymode 154b10e901 add the ability to run as another user
This change adds a new permission that allows authorized users to execute a request as
another user. The flow is as follows:

1. The user making the request is authenticated
2. The user that is being impersonated is looked up
3. The requesting user is authorized for the privilege to run as the specified user
4. The impersonated user is then authorized for the given request

Additionally, the auditing has been updated to support this capability and indicates when a
user has been granted the ability to run as another user and then also indicates both the user
who is being impersonated and the requesting user when actions are granted/denied.

Closes elastic/elasticsearch#17

Original commit: elastic/x-pack-elasticsearch@00e5a6169b
2015-09-09 11:25:02 -04:00
Martijn van Groningen e7b338a077 test: added smoke test for the shield tribe node integration
Original commit: elastic/x-pack-elasticsearch@f7ab8b9044
2015-09-08 12:37:36 +02:00
uboness 533c14242f Bumped the version to 3.0.0-SNAPSHOT
Original commit: elastic/x-pack-elasticsearch@0771b3e589
2015-09-04 16:30:11 +02:00
Jason Tedor 77e74a9319 Add compare condition to handle arrays
This commit adds a new compare condition called “array_compare”. This
condition enables comparing a single resolved value to an array of
resolved values. The value can be compared for equality, non-equality,
and strict and non-strict ordering; the array compare condition will
evaluate to true if the value compares to true with respect to the
specified operator against all (“all”) or at least one (“some”) of the
values in the array specified by “array_path”. Each value in the array
can be resolved to a value using “path” (e.g., “array_path”:
“cx.payload.aggregations.some_field.buckets” and “path”: “doc_count”
would resolve each value in the buckets array to its “doc_count”).

Closes elastic/elasticsearch#345

Original commit: elastic/x-pack-elasticsearch@0d74b4dc11
2015-09-03 09:46:23 -04:00
Martijn van Groningen 067c2e0709 fix qa smoke tests
Original commit: elastic/x-pack-elasticsearch@b3a2e0bc38
2015-08-31 17:36:23 +02:00
jaymode 204bb2accb fix custom realm integration tests on windows
Original commit: elastic/x-pack-elasticsearch@d5a8722502
2015-08-21 14:08:43 -04:00
jaymode 8fd5fe7ed8 add the ability to register a custom authentication realms
This adds the extension points necessary to enable a user to write a elasticsearch plugin
that can integrate with Shield and add a custom authentication realm. For the most part,
the work here just exposes the existing interfaces we have been using for Realms and
factories to create realms. An additional interface was added to allow for a custom
authentication failure handler to be used. This was needed to support use cases like SSO
and Kerberos where additional headers may need to be sent to the user or a different
HTTP response code would need to be sent.

Relates to elastic/elasticsearch#24

Original commit: elastic/x-pack-elasticsearch@13442e5919
2015-08-21 10:39:05 -04:00
jaymode 7e552f393b fix all InetAddress forbidden apis and compile errors
This commit also fixes test shard routing compilation error and disables local address check in
the Shield IPFilter. This will be addressed in a followup, see elastic/elasticsearch#487

Original commit: elastic/x-pack-elasticsearch@984df0b131
2015-08-21 09:22:57 -04:00
Ryan Ernst 2b5cb6b9f2 Fix compile after removal of apache commons and refactoring of plugin api
Original commit: elastic/x-pack-elasticsearch@5171192d16
2015-08-18 15:35:01 -07:00
David Pilato 5899dc5f46 [maven] fix build issues with artifactId renaming
Related to elastic/elasticsearch#450

Original commit: elastic/x-pack-elasticsearch@d84fc8e85f
2015-08-18 17:29:09 +02:00
David Pilato 7b10f36775 [maven] rename artifactIds from `elasticsearch-something` to `something`
When https://github.com/elastic/elasticsearch/pull/12879 will be merged, this commit should be merged as well.

```
[INFO] Reactor Summary:
[INFO]
[INFO] Elasticsearch Commercial Plugin Build Resources .... SUCCESS [  0.228 s]
[INFO] Elasticsearch X-Plugins - Parent POM ............... SUCCESS [  0.282 s]
[INFO] X-Plugins: License: Parent POM ..................... SUCCESS [  0.089 s]
[INFO] X-Plugins: License: Core ........................... SUCCESS [  0.118 s]
[INFO] X-Plugins: License: Licensor ....................... SUCCESS [  0.150 s]
[INFO] X-Plugins: License: Plugin API ..................... SUCCESS [  0.106 s]
[INFO] X-Plugins: License: Plugin ......................... SUCCESS [  0.112 s]
[INFO] X-Plugins: Shield .................................. SUCCESS [  0.234 s]
[INFO] X-Plugins: Watcher ................................. SUCCESS [  0.264 s]
[INFO] X-Plugins: Marvel .................................. SUCCESS [  0.113 s]
[INFO] QA: Parent POM ..................................... SUCCESS [  0.097 s]
[INFO] QA: Smoke Test X-Plugins ........................... SUCCESS [  0.107 s]
[INFO] QA: Shield core REST tests ......................... SUCCESS [  0.093 s]
[INFO] QA: Smoke Test Watcher's Shield integration ........ SUCCESS [  0.109 s]
```

Original commit: elastic/x-pack-elasticsearch@e9871261cf
2015-08-18 13:55:11 +02:00
Martijn van Groningen d7665293cb Changed pom version to 2.1.0-SNAPSHOT
as ES core does in its master branch

Original commit: elastic/x-pack-elasticsearch@fc9b1a7327
2015-08-17 13:44:33 +02:00
jaymode 002c0282cc skip deployment of dev tools and qa modules when deploying publicly
Closes elastic/elasticsearch#433

Original commit: elastic/x-pack-elasticsearch@3f0f7fda4f
2015-08-14 13:53:16 -04:00
Adrien Grand 06d84f00e4 Tests: Move Shield "run core REST tests" to qa.
Close elastic/elasticsearch#404

Original commit: elastic/x-pack-elasticsearch@1250e1449e
2015-08-14 11:18:11 +02:00
uboness c4e213fc92 Updated version to 2.0.0-SNAPSHOT
Original commit: elastic/x-pack-elasticsearch@8fb8035596
2015-08-13 00:05:11 +02:00
Adrien Grand 268860be7b Disable Shield ssl tests until openssl is available on the Windows build machines.
Original commit: elastic/x-pack-elasticsearch@777375bb94
2015-08-12 17:00:20 +02:00
Martijn van Groningen 583799d3e7 applied feedback
Original commit: elastic/x-pack-elasticsearch@9042427219
2015-08-12 14:11:46 +02:00
Martijn van Groningen ca8a7bb262 added watcher+shield qa rest tests
only run watcher rest tests during verify phase
never run the rest tests with shield enabled, because that is now tested in the new qa module
removed the disabled license watcher rest tests, because the disabled license use case is already tested by the LicenseIntegrationTests
enabled the getting started rest test

Closes elastic/elasticsearch#403

Original commit: elastic/x-pack-elasticsearch@67f0f7f596
2015-08-12 14:11:45 +02:00
Adrien Grand cd5169745d Tests: Smoke tests with SSL enabled.
We should smoke test our plugins with ssl enabled to make sure that plugins
still work together and eg. Marvel still manages to export stats.

Close elastic/elasticsearch#402

Original commit: elastic/x-pack-elasticsearch@3bb7c2b96c
2015-08-12 14:04:49 +02:00
Adrien Grand 2842898c1c Build: cut over to the new startup-elasticsearch syntax.
Original commit: elastic/x-pack-elasticsearch@29582a18b9
2015-08-11 18:54:08 +02:00
jaymode 0b4512582f fix integration test runs due to changes in core around argument definition
Original commit: elastic/x-pack-elasticsearch@af2a85cf91
2015-08-11 09:01:14 -04:00
Adrien Grand 12e9dcc684 Add Shield testing to qa/smoke-test-plugins.
Original commit: elastic/x-pack-elasticsearch@fbbc6cefda
2015-08-11 10:54:50 +02:00
Adrien Grand d3789db974 Build: Remove definition of `convert-plugin-name`.
Follow-up of elastic/elasticsearchelastic/elasticsearch#12765.

Original commit: elastic/x-pack-elasticsearch@f75538e87b
2015-08-11 10:38:29 +02:00
Adrien Grand 214dc6e5be Build: Fix artifactId of qa/smoke-test-plugins.
Original commit: elastic/x-pack-elasticsearch@c84753e933
2015-08-10 14:01:26 +02:00
Adrien Grand f7cee828e3 Build: Make the qa module extend the x-plugins artifact.
This way it will inherit the repositories definitions.

Original commit: elastic/x-pack-elasticsearch@5d6ee26596
2015-08-10 11:56:46 +02:00
Adrien Grand 2cd124d263 Add a skeleton for QA tests.
For now this just tries to install license, marvel and watcher, and then checks
that these plugins are listed in the node infos. I can do shield once I figure
out how to set it up for REST tests.

Original commit: elastic/x-pack-elasticsearch@8549f4bc5a
2015-08-10 10:55:58 +02:00