Commit Graph

4044 Commits

Author SHA1 Message Date
Areek Zillur f6800f7b8b Fix license command line tools
Original commit: elastic/x-pack-elasticsearch@756c0f2384
2016-11-01 11:05:29 -04:00
Boaz Leskes 03c5d71c12 remove hard coded dates from testDateMathExpressionsCanBeAuthorized
Original commit: elastic/x-pack-elasticsearch@d7fac0b9a0
2016-11-01 09:02:39 +01:00
Jack Conradson 7dd4188299 Cleanup ScriptType (elastic/elasticsearch#3922)
Refactored ScriptType to clean up some of the variable and method names. Added more documentation. Deprecated the 'in' ParseField in favor of 'stored' to match the indexed scripts being replaced by stored scripts.

Original commit: elastic/x-pack-elasticsearch@d7c7bd7362
2016-10-31 13:49:10 -07:00
Yannick Welsch 8350a8b2d8 [TEST] Disconnect from newly added nodes if cluster state publishing fails
Companion commit for elastic/elasticsearchelastic/elasticsearch#21197

Original commit: elastic/x-pack-elasticsearch@248a6bfb7c
2016-10-31 15:36:54 +01:00
Simon Willnauer f696ad1d10 Skip authentication and warn if shards of the .security index are not available
Original commit: elastic/x-pack-elasticsearch@9970d80f2d
2016-10-28 15:04:06 +02:00
Alexander Reelsen e67847ca8c Tests: Increase logging to get more sync inside
My current assumption is, that creating the templates is not
yet finished (as this is async), so that we need to add
another check that the templates have been added before
continuing.

Relates elastic/elasticsearch#3892

Original commit: elastic/x-pack-elasticsearch@3880d200a1
2016-10-28 11:00:09 +02:00
Simon Willnauer f4da918b09 [TEST] Pass _analyze API params in the body rather than as params. Parameters are not supported anymore
Original commit: elastic/x-pack-elasticsearch@e04d425a89
2016-10-27 22:34:35 +02:00
Jason Tedor a15f565539 Mark BWC tests as awaits fix
These tests are awaiting the BWC indices script to be upgraded for 5.x.

Original commit: elastic/x-pack-elasticsearch@540fe73bd0
2016-10-26 21:40:00 -04:00
Jack Conradson 54a71289cc Merge pull request elastic/elasticsearch#3902 from jdconrad/stype
Refactor ScriptType to be a top-level class.

Original commit: elastic/x-pack-elasticsearch@22862a3727
2016-10-26 12:46:09 -07:00
Jack Conradson 4fd19aa00a Merge branch 'master' into stype
Original commit: elastic/x-pack-elasticsearch@37f27bef1a
2016-10-26 12:29:43 -07:00
Simon Willnauer 84b631643c Add utility method to fetch and collect results from a query (elastic/elasticsearch#3894)
Today we have the same madness in two places and no dedicated test. This
change moves the real madness into a single place and adds a test for it
to make sure it actually works and isn't just crazy.

Original commit: elastic/x-pack-elasticsearch@dabf5fdd63
2016-10-26 21:05:49 +02:00
Spencer 163e5feb6e Merge pull request elastic/elasticsearch#3864 from spalger/race-condition-xpack-info
Fix xpackInfo loading race condition

Original commit: elastic/x-pack-elasticsearch@3acd6bfe03
2016-10-26 11:06:18 -07:00
Jack Conradson 72a49015cc Refactor ScriptType to be a top-level class.
Original commit: elastic/x-pack-elasticsearch@39afcbfdf5
2016-10-26 10:21:47 -07:00
Simon Willnauer 9f57afbdf3 Return non-existing role if the .security index is not found (elastic/elasticsearch#3895)
We used to be very lenient with all kinds of exceptions related to the
`.security` index. Yet, sometimes in tests the index is not yet there but
transport clients already pinging the node this causes issues and transport
clients disconnect. Now if the index is not present we simply return no role.

Original commit: elastic/x-pack-elasticsearch@60948d0c2a
2016-10-26 17:25:20 +02:00
Simon Willnauer 6e1287bab9 Simplify TransportGetRolesAction (elastic/elasticsearch#3888)
TransportGetRolesAction optimizes for single role case while this
optimization can be simply inside the NativeRoleStore and being
way more contained.

Original commit: elastic/x-pack-elasticsearch@c43d8ba341
2016-10-26 14:55:39 +02:00
Jason Tedor 007e49c5d9 Reveal Content-Length on x-pack info HEAD requests
This commit permits x-pack info HEAD requests to reveal the
Content-Length of the response.

Relates elastic/elasticsearch#3887

Original commit: elastic/x-pack-elasticsearch@8696caa1f6
2016-10-25 23:12:54 -04:00
Simon Willnauer 0b24f022f7 Remove all blocking calls from TransportGetUsersAction (elastic/elasticsearch#3876)
`TransportGetUsersAction` does some funky blocking calls even though
it's specifying `SAME` as the thread-pool indicating that it's fast or
forking off quickly. Both might not be true today. This change adds
async support to the methods it calls without breaking the existing
Realm interface. Yet, we might need to do this down the road.

Original commit: elastic/x-pack-elasticsearch@d0959f87f3
2016-10-25 22:11:19 +02:00
Jay Modi 542a484031 security: cache negative lookups for native roles
This changes adds a special value for negative role lookups so that we can avoid scenarios
where we overload the cluster due to continually trying to load non-existing roles as is often
the case when `unmapped_groups_as_roles` is used with the active directory realm.

Relates elastic/elasticsearch#3530 

Original commit: elastic/x-pack-elasticsearch@62567b4c22
2016-10-25 16:00:27 -04:00
spalger a291fa77d3 Merge branch 'master' of github.com:elastic/x-plugins into race-condition-xpack-info
Original commit: elastic/x-pack-elasticsearch@ade5fae76b
2016-10-25 12:33:31 -07:00
Jay Modi 7d60f6b365 security: restore the correct user when switching to the system user
* security: restore the correct user when switching to the system user

For internal actions where we need to switch to the SystemUser, we should always restore the proper
context after execution. We were restoring an empty context for actions executed by the SystemUser
in the SecurityServerTransportInterceptor.

In order to accomplish this, a few changes have been made. Both the SecurityServerTransportInterceptor
and the SecurityActionFilter delegate to `SecurityContext#executeAsUser` when a user switch is necessary.
Tests were added for this method to ensure that the consumer is executed as the correct user and the proper
user is restored.

While working on this, a few other cleanups were made:

* SecurityContext can never have a null CryptoService, so a null check was removed
* We no longer replace the user with the system user when the system user is already associated with the request
* The security transport interceptor checks the license state and if auth is not allowed, delegate and return
* The security transport interceptor sendWithUser method now requires authentication to be present or a hard
exception is thrown.
* The TransportFilters integration test has been deleted. This was integration test that relied on the ability to
get instances from a node and trace the execution. This has been replaced by additional unit tests in
ServerTransportFilterTests

Closes elastic/elasticsearch#3845

Original commit: elastic/x-pack-elasticsearch@d8bcb59cb7
2016-10-25 13:48:28 -04:00
Simon Willnauer a50bc7946b Make request authorization non-blocking (elastic/elasticsearch#3837)
This change removes the blocking notion from fetching the roles
from a remote index. This also removes the blocking client calls
that can potentially deadlock a request if executed on the transport
thread.

Relates to elastic/elasticsearch#3790

Original commit: elastic/x-pack-elasticsearch@c2eda39043
2016-10-25 17:28:29 +02:00
Tim Sullivan c56eec6d26 Merge pull request elastic/elasticsearch#3862 from tsullivan/monitoring-ui-fix-kibana-stats-when-monitoring-disabled
monitoring ui: check monitoring is enabled before sending kibana stats

Original commit: elastic/x-pack-elasticsearch@f5cbf629c3
2016-10-24 14:55:07 -07:00
Tim Sullivan 1efddec0b0 Merge pull request elastic/elasticsearch#3854 from tsullivan/monitoring-ui-removed-unused-css-round-3
Monitoring ui removed unused css round 3

Original commit: elastic/x-pack-elasticsearch@1e0213d23a
2016-10-24 10:34:53 -07:00
Jay Modi f3d5d79a20 test: install a new signed license when running index BWC tests
This change now installs a signed license that has been generated at runtime so the
BWC tests can run without hitting licensing issues. The x-pack BWC tests pull in the
full cluster state, which contains the trial license from when the indices and state
was generated. After the trial license period and grace period issues arise with the
tests.

Closes elastic/elasticsearch#3858

Original commit: elastic/x-pack-elasticsearch@1c79e874e5
2016-10-24 09:18:59 -04:00
Simon Willnauer f8ba7f6fd8 Restore thread-context when executing with InternalClient (elastic/elasticsearch#3859)
Today when a request is executed with InternalClient the thread context might
be lost if another component like security exchanges it by executing an async call
or an internal action. This can be a serious security problem since if the async
call executes as the system user all subsequent calls made by the response
thread will also execute as the system user instead.

Original commit: elastic/x-pack-elasticsearch@80682f338d
2016-10-24 14:39:00 +02:00
Simon Willnauer 51b871f344 Followup API change for elastic/elasticsearchelastic/elasticsearch#21089
Original commit: elastic/x-pack-elasticsearch@5d9b2fe0c8
2016-10-24 14:06:13 +02:00
Adrien Grand 47079cf5d1 Disable bw testing due to license expiration.
Relates to elastic/elasticsearch#3858

Original commit: elastic/x-pack-elasticsearch@7d676b96d3
2016-10-24 11:47:23 +02:00
CJ Cenizal 546e364417 Merge pull request elastic/elasticsearch#3831 from cjcenizal/3650/improvement/license-page
Redesign license page in Monitoring.

Original commit: elastic/x-pack-elasticsearch@ca5c86096b
2016-10-21 14:30:04 -07:00
CJ Cenizal f7a4edc384 Merge pull request elastic/elasticsearch#3828 from cjcenizal/3666/improvement/side-nav-app-order
Reorder apps for more logical order.

Original commit: elastic/x-pack-elasticsearch@5b65fd6fcf
2016-10-21 09:58:31 -07:00
Tanguy Leroux fc88dfe1a6 CertificateTool must not generate world redeable files (elastic/elasticsearch#3810)
This commit changes the permissions of the files generated by the certgen tool to 600 (like syskeygen does)

Original commit: elastic/x-pack-elasticsearch@bca74e9c92
2016-10-20 16:36:35 +02:00
Jay Modi 05886cdf9f security: exclude the anonymous role from the xpack user
The calls made by the native users and roles store use the internal xpack user to make the request
and this user has a built-in role that has a single instance. A bug was introduced when fixing the logic
for applying the anonymous role to all users in elastic/elasticsearch#3716. The anonymous role was now being added to
the xpack user, even though the additional role would have no effect as this user is a superuser.

When the anonymous role is applied to the xpack user and exists as a native role or doesn't exist
at all, we run into a deadlock since we wind up querying for the role as a user that also has the
anonymous role.

This change special cases the XPackUser when getting the collection of roles so that the only role
applied to this user is the superuser role.

Closes elastic/elasticsearch#3822

Original commit: elastic/x-pack-elasticsearch@e3093904f1
2016-10-20 08:11:01 -04:00
jaymode 388bfd761d security: use lucene automatons and remove dependency on briks
This commit removes the dependency on the briks automatons library and instead uses the lucene
version. Shield was originally implemented using the lucene version, but issues arose with supporting
multiple versions of elasticsearch and API changes, so we moved to using the briks library.

x-pack and elasticsearch are always the same version so we can use the lucene version of the
automatons and remove the briks library. This also brings with it protection from huge automatons
that we did not have before.

Original commit: elastic/x-pack-elasticsearch@e3f34b6b55
2016-10-20 06:55:01 -04:00
Jay Modi ff3d685833 security: update unboundid-ldapsdk to the latest version
This changes updates the unboundid ldapsdk to the latest version to stay up to date
with their releases.

Original commit: elastic/x-pack-elasticsearch@b9e4f7f062
2016-10-20 06:37:30 -04:00
javanna b7a10239be [TEST] Verify that date math expressions work with security plugin
Original commit: elastic/x-pack-elasticsearch@d87c9fdb30
2016-10-20 12:07:26 +02:00
javanna 8d001237df fix compile error after https://github.com/elastic/elasticsearch/pull/21032
Original commit: elastic/x-pack-elasticsearch@8372cea977
2016-10-20 12:02:18 +02:00
javanna 508784554b fix compile error after https://github.com/elastic/elasticsearch/pull/21032
Original commit: elastic/x-pack-elasticsearch@c4f400c0f7
2016-10-20 11:58:13 +02:00
Simon Willnauer 8b6867b99b Deguice Watcher Actions and Transformations (elastic/elasticsearch#3818)
This change simplifies the creation of Actions and Transformations.
It moves all instantiation away from guice into straight forward
constructor based initialization.

Original commit: elastic/x-pack-elasticsearch@3c0bca2bea
2016-10-19 23:35:16 +02:00
javanna a4c0c49b43 [TEST] Only create indices if nodes exist
Some tests manually start nodes, hence in the before test phase there are no nodes around thus indices should not be created.
Relates to elastic/elasticsearch#3770

Closes elastic/elasticsearch#3812

Original commit: elastic/x-pack-elasticsearch@a21ad39903
2016-10-19 18:34:08 +02:00
Jason Tedor 3d658c3f1e Adjust ClusterStatsResponse constructor calls
A commit in core removed the UUID parameter from the
ClusterStatsResponse constructor. This commit adjusts x-plugins to this.

Original commit: elastic/x-pack-elasticsearch@6f2f26168e
2016-10-19 11:25:07 -04:00
Simon Willnauer 09a4882a4c Remove Notification Accounts abstraction (elastic/elasticsearch#3811)
This change is a first step towards a real abstraction on top of all the
notification services. There are a bunch of followup changes coming for this
that will remove most of the classes in here but this is a first small step
to actually have a notification service interface.

Original commit: elastic/x-pack-elasticsearch@e14abf8a8b
2016-10-19 16:58:17 +02:00
Alexander Reelsen b6e909658e Testing: Fix smoke tester
Original commit: elastic/x-pack-elasticsearch@929a45aa27
2016-10-19 15:21:38 +02:00
Jay Modi 9ea1786596 security: only log if we actually loaded the system key
This commit changes the logging to only log if we actually loaded the system key, otherwise
the message is misleading as the key file may not even exist but we output that it was
loaded.

Original commit: elastic/x-pack-elasticsearch@0af7953c64
2016-10-19 08:36:29 -04:00
Boaz Leskes 29e35267c3 SecurityTribeIT - only wait for number nodes if they are not already there.
Original commit: elastic/x-pack-elasticsearch@3fa5da519a
2016-10-19 14:30:09 +02:00
Alexander Reelsen baf1596418 Watcher: Introduce dedicated reporting attachment type (elastic/elasticsearch#3665)
Instead of using the long running and long blocking single polling HTTP attachment for our reporting,
we should use the async API provided by kibana. The new workflow (all blocking and in a single watch)
looks like this:

1. An initial request is sent to trigger the report generation, which returns a path
2. This path is used to continuously check if the report is done (then it is sent back) or kibana sends another HTTP error code, which will result in watcher to sleep for another interval until the report is finally returned.

Features include configurable interval time and retry count, so that the total amount of waiting can be tweaked into two directions.

This is what the reporting type looks like right now

```
{
   "my-attachment":{
      "reporting":{
         "url":"http://www.example.org/my-dashboard",
         "retries":6, // optional, default 40
         "interval":"1s", // optional, default 15s
         "auth":{
            "basic":{
               "username":"foo",
               "password":"secret"
            }
         }
      }
   }
}
```

The interval/retries can also be configured via settings.

Note, that this is just a temporal workaround until the watcher execution can execute in an asynchronous fashion.

Closes elastic/elasticsearch#3524

Original commit: elastic/x-pack-elasticsearch@d1eaa856b9
2016-10-19 12:21:25 +02:00
Alexander Reelsen 1c3baa61fe Security/Watcher: Increase index priority for indices (elastic/elasticsearch#3709)
The `.triggered-watches`, `.watches` and `.security` indices should load
as early as possible, and not wait for other indices (especially not
for time-based indices, that are old).

This commit adds an index.priority to the template for those indices.
The values 1000, 900 and 800 were chosen rather arbitrary, mainly we
did not want to go with 10, because it was used in the sample documentation.

Security should always be loaded first, because we might need this index for
other operations.

Any administrator can still change all the values in the indices, but this
cares for better defaults.

Original commit: elastic/x-pack-elasticsearch@6ed0fb7975
2016-10-19 11:14:47 +02:00
Alexander Reelsen 0228a94d80 Watcher: Add support for aliases for watches/triggered watches index (elastic/elasticsearch#3770)
As discussed in #elastic/elasticsearch-migration/79 supporting aliases for watcher allows
the migration plugin to work.

This adds the relevent checks in the WatchStore and the TriggeredWatchStore that aliases are
supported, as the current assumption was always to just load an index.

Also, this rarely sets those indices as aliases in all the integration tests, so that this
case gets tested.

Note: The new WatchStoreUtils.getConcreteIndex() method will be put into core, as this is a
useful helper for others.

Original commit: elastic/x-pack-elasticsearch@4a98af691d
2016-10-19 10:29:27 +02:00
Boaz Leskes aa1eedc062 SecurityTribeIT - wait for tribe node to full process incoming cluster states
Original commit: elastic/x-pack-elasticsearch@4da1303965
2016-10-19 10:00:54 +02:00
Nik Everett 0f54f9524d Handle reindex's change to whitelist
Reindex has grown support for simple whitelist patterns like
`localhost:*` and lost support for `myself`.

Original commit: elastic/x-pack-elasticsearch@31d9c4dc5c
2016-10-18 21:46:30 -04:00
Boaz Leskes 9bfd1721ca Fix SecurityTribeIT to properly add mock plugins in tribe node
Original commit: elastic/x-pack-elasticsearch@e82c39c5c4
2016-10-18 22:05:38 +02:00
Boaz Leskes 966600fc90 Adapt testing code to the removal of local discovery (elastic/elasticsearch#3767)
See https://github.com/elastic/elasticsearch/pull/20960

Original commit: elastic/x-pack-elasticsearch@f368fd4b1c
2016-10-18 21:12:36 +02:00