Commit Graph

7367 Commits

Author SHA1 Message Date
Lisa Cawley b6c9e96304 [DOCS] Added job ID requirements (elastic/x-pack-elasticsearch#3727)
Original commit: elastic/x-pack-elasticsearch@71f0073708
2018-01-25 09:23:56 -08:00
Michael Basnight 39b4587d02 Remove requiresKeystore when not actually required (elastic/x-pack-elasticsearch#3697)
Original commit: elastic/x-pack-elasticsearch@6bdd4ae2dc
2018-01-25 07:33:30 -06:00
Jim Ferenczi 3316c964c8 Move watcher's Cron into x-pack core (elastic/x-pack-elasticsearch#3716)
This change moves watcher's Cron class in the schedule package of xpack-core so that it can be used by other projects (rollup).

Original commit: elastic/x-pack-elasticsearch@f0aa32ccc2
2018-01-25 11:57:44 +01:00
Ryan Ernst 3e24ea3ed2 Test: Add internalClusterTest to xpack modules (elastic/x-pack-elasticsearch#3730)
This commit reenables running ITs in xpack by adding an internalClusterTest to xpack modules that contain ESIntegTestCase tests. The new task allows us to run these independently of rest integ tests, which are disabled for xpack modules because installing the bundled plugins directly is not quite the same as installing via the meta plugin. Some tests (ML) are moved to their own qa module to accommodate the need for a real cluster. A couple tests (monitoring and upgrade) have been marked as AwaitsFix.

Commits that have been folded into this commit:
* Move ML IT tests to qa/ml-native-tests
* Add internalClusterTest task and disable rest integ tests for xpack
modules. Also tweak ML tests and get upgrade tests working
* Adding the keystore and security back to the ml native tests
* Fixing native integ test
* Fix last ML test, add awaits fix to monitoring and upgrade tests
* cleanup PR
* fix checkstyle

Original commit: elastic/x-pack-elasticsearch@3c0ed6fd3b
2018-01-25 10:38:34 +01:00
Tanguy Leroux 7c90f874d7 Update x-pack packaging tests to work with meta plugins (elastic/x-pack-elasticsearch#3683)
relates elastic/x-pack-elasticsearch#3652

Original commit: elastic/x-pack-elasticsearch@e177b742f9
2018-01-25 10:20:52 +01:00
jaymode b456885b39 Test: increase time to wait for tribe tests with security
There have been some failures in CI due to the tribe tests with security taking too long for the
cluster to form and the expected number of node to join. In the thread dumps of the failure, it
can be seen that a node is still initializing and is in a method that could take a decent amount
of time on slow machines (TokenService#computeSecretKey). This commit increases the wait time to
allow for the node to startup and join.

Original commit: elastic/x-pack-elasticsearch@2bf4c96d8f
2018-01-24 16:30:02 -07:00
Jay Modi 3baf6f3a7a Remove production from the message about license installation without TLS (elastic/x-pack-elasticsearch#3666)
This change simply changes the wording of the message that is returned to the user when
installation of a license is attempted with security enabled and TLS disabled. The term
"production" has been removed as it means something different to users.

See elastic/x-pack-elasticsearch#2636

Original commit: elastic/x-pack-elasticsearch@9739c72d66
2018-01-24 12:44:25 -07:00
jaymode a547049303 Security: switch to IllegalArgumentException if request cannot be serialized
The commit switches the CreateTokenRequest and InvalidateTokenRequest to throw an
IllegalArgumentException when the version the request is being serialized to does not support the
request. This fixes test failures due to the use of the AssertingTransportInterceptor testing
serialization with versions prior to 6.2. The IAE is an indication to the
AssertingTranpsortInterceptor that the request does not support the version.

Original commit: elastic/x-pack-elasticsearch@c73abf1bc0
2018-01-24 12:19:09 -07:00
David Roberts ae1a90eba6 [ML] Move JobProvider and the Batched*Iterators out of core (elastic/x-pack-elasticsearch#3712)
By moving these into the ML module:

1. The classes are in the same module as their unit tests
2. We can extend the JobProvider in the future with functionality
   that is not in core

Original commit: elastic/x-pack-elasticsearch@610a89a3d9
2018-01-24 18:56:30 +00:00
Jay Modi 0baa45d9b3 [SAML] Find all tokens for a realm, not just the first 10 (elastic/x-pack-elasticsearch#3689)
This commit changes the token service to use a scroll based approach when finding all tokens by
the realm. Without this, we may only find a few tokens and leave some active that need to be
invalidated.

relates elastic/x-pack-elasticsearch#3688

Original commit: elastic/x-pack-elasticsearch@20e97b6aae
2018-01-24 11:07:51 -07:00
jaymode 2c46002c00 Test: get the rest-api-spec from the correct location
Original commit: elastic/x-pack-elasticsearch@a3f3c45213
2018-01-24 10:39:43 -07:00
jaymode 8059099697 Test: re-enable the rolling-upgrade tests after the refresh token backport
Original commit: elastic/x-pack-elasticsearch@a39736a4c0
2018-01-24 10:09:47 -07:00
Lisa Cawley 2428e98976 [DOCS] Clarify document level security (elastic/x-pack-elasticsearch#3701)
Original commit: elastic/x-pack-elasticsearch@b4bfe5706c
2018-01-24 09:07:21 -08:00
Lisa Cawley 9435ffe64b [DOCS] Clarify PKI realm support (elastic/x-pack-elasticsearch#3703)
Original commit: elastic/x-pack-elasticsearch@55da7a07d1
2018-01-24 08:32:23 -08:00
Lisa Cawley e7c78e05f8 [DOCS] Added ML add and delete calendar event APIs (elastic/x-pack-elasticsearch#3394)
Original commit: elastic/x-pack-elasticsearch@3283af2215
2018-01-24 08:14:23 -08:00
Alexander Reelsen cee33c0bc8 ML: Remove unneeded and empty lifecycle listener (elastic/x-pack-elasticsearch#3710)
Original commit: elastic/x-pack-elasticsearch@d8d77f4e74
2018-01-24 16:55:59 +01:00
David Kyle 29ece3c79f [ML] Refactor update calendar params (elastic/x-pack-elasticsearch#3714)
Original commit: elastic/x-pack-elasticsearch@a7ef33b58a
2018-01-24 15:16:21 +00:00
Jason Tedor e385b7dab4 Elevate privileges fetching metadata for SAML
We have to elevate privileges here as these invocations happen in a run
loop that will not have the correct privileges for socket connections.

Relates elastic/x-pack-elasticsearch#3671

Original commit: elastic/x-pack-elasticsearch@eab9f47583
2018-01-24 08:59:01 -05:00
Jason Tedor 3932635f98 Simplify security manager permissions
The security manager permissions were copied wholesale from pre-split
X-Pack. However, this grants unnecessary permissions to the child
plugins. This commit is a simple attempt at removing permissions that
are not needed in the child plugins.

Relates elastic/x-pack-elasticsearch#3651

Original commit: elastic/x-pack-elasticsearch@8325ed83d7
2018-01-24 08:57:54 -05:00
Tanguy Leroux ba7b84d4d0 Fix XPackExtension javadoc (elastic/x-pack-elasticsearch#3711)
Original commit: elastic/x-pack-elasticsearch@bbe950ce40
2018-01-24 14:24:39 +01:00
Simon Willnauer 3b7d1f4d98 Prevent watcher from starting after node has shutdown (elastic/x-pack-elasticsearch#3709)
Some tests seem to be pretty flaky due to concurrent watcher restarts.
This change makes sure we never restart watcher once the node is shutting down.

Original commit: elastic/x-pack-elasticsearch@f0bed7269b
2018-01-24 14:16:05 +01:00
Alexander Reelsen 1ebccfcf50 Settings: Cut over to affix settings where filters are needed (elastic/x-pack-elasticsearch#3685)
x-pack makes extensive use of group settings in its exporters as well as
its notification services. This does not cope well with settings being
marked as filtered.

This replaces a fair share of group settings (unfortunately not all of
them) with affix key settings, allowing for better filtering and
registering of settings.

Original commit: elastic/x-pack-elasticsearch@968bc8c2ee
2018-01-24 13:14:08 +01:00
Michael Basnight 4475b718a4 Remove invalid use of XPackPlugin class (elastic/x-pack-elasticsearch#3700)
Original commit: elastic/x-pack-elasticsearch@f4aabe1259
2018-01-23 21:11:24 -06:00
Jason Tedor c0790d6a49 Move x-pack-core to core package (elastic/x-pack-elasticsearch#3678)
This commit moves the source file in x-pack-core to a org.elasticsearch.xpack.core package. This is to prevent issues where we have compile-time success reaching through packages that will cross module boundaries at runtime (due to being in different classloaders). By moving these to a separate package, we have compile-time safety. Follow-ups can consider build time checking that only this package is defined in x-pack-core, or sealing x-pack-core until modules arrive for us.

Original commit: elastic/x-pack-elasticsearch@232e156e0e
2018-01-23 12:43:58 -06:00
David Kyle 97e018f1bd [ML] Return the updated calendar after removing a job (elastic/x-pack-elasticsearch#3690)
Original commit: elastic/x-pack-elasticsearch@1bf1f3228b
2018-01-23 17:40:41 +00:00
Alexander Reelsen d065b087ee Revert "Cleanup: Remove HaltedClock (elastic/x-pack-elasticsearch#3664)"
This reverts commit elastic/x-pack-elasticsearch@f91c401a60 due to
failing tests, like

./gradlew :x-pack-elasticsearch:plugin:watcher:test -Dtests.seed=AE30350FCE96D26D -Dtests.class=org.elasticsearch.xpack.watcher.watch.WatchTests -Dtests.method="testParserSelfGenerated" -Dtests.security.manager=true -Dtests.locale=ja-JP -Dtests.timezone=EET

Original commit: elastic/x-pack-elasticsearch@e45d79d643
2018-01-23 18:08:54 +01:00
David Roberts 697a08e742 [ML] Refactor so ML doesn't require PersistentTasksService at startup (elastic/x-pack-elasticsearch#3682)
At present the PersistentTasksService is created inside the ML plugin.
This is undesirable, as other plugins will use persistent tasks in the
near future.

This change refactors the startup code so that the PersistentTasksService
no longer needs to be passed to any constructors for ML components.

A future change will still be required to actually move the initialization
of the PersistentTasksClusterService, PersistentTasksService and
PersistentTasksExecutorRegistry out of the ML plugin, but following this
change it should be fairly simple.

Original commit: elastic/x-pack-elasticsearch@3c2a8e020e
2018-01-23 16:55:08 +00:00
Lisa Cawley 223d3c1f4c [DOCS] Added QA notes about scheduled events (elastic/x-pack-elasticsearch#3641)
Original commit: elastic/x-pack-elasticsearch@64b67aa0ad
2018-01-23 08:41:48 -08:00
Simon Willnauer 63c0e288af Ensure we protect Collections obtained from scripts from self-referencing (elastic/x-pack-elasticsearch#3681)
Self referencing maps can cause SOE if they are iterated ie. in their toString methods. This chance adds some protected to the usage of those collections.
see elastic/elasticsearch#28335

Original commit: elastic/x-pack-elasticsearch@c4f1089c74
2018-01-23 17:02:37 +01:00
Dimitris Athanasiou 215f9af1cc [ML] Add trace logging for when search response is obtained (elastic/x-pack-elasticsearch#3669)
This is useful for understanding performance characteristics
as it helps us understand whether the bottleneck is the search
part or the analytics part.

Relates elastic/x-pack-elasticsearch#3590

Original commit: elastic/x-pack-elasticsearch@dc8c095958
2018-01-23 14:16:34 +00:00
Alexander Reelsen e3da8fa4ae Cleanup: Remove HaltedClock (elastic/x-pack-elasticsearch#3664)
The HaltedClock was a leftover from moving over from our own Clock
implementation to a java.time one. java.time already has a fixed clock,
this one is not needed.

Original commit: elastic/x-pack-elasticsearch@f91c401a60
2018-01-23 14:13:22 +01:00
Simon Willnauer c70c1c7e98 Expose XPackExtensions via SPI (elastic/x-pack-elasticsearch#3530)
This change adds SPI loading for XPackExtensions that allows to extend
XPack via an ordinary plugin. This can co-exist with the existin
extension mechanism for the time being.

Original commit: elastic/x-pack-elasticsearch@bf02b56dee
2018-01-23 13:05:39 +01:00
David Kyle 8f393c3066 [ML] Improve error message when creating calendars (elastic/x-pack-elasticsearch#3668)
Original commit: elastic/x-pack-elasticsearch@996b0e2f65
2018-01-23 09:34:07 +00:00
Simon Willnauer c3efa4b6bc [TEST] disable ML when ML is not installed
Original commit: elastic/x-pack-elasticsearch@cd84acc3e0
2018-01-23 10:28:37 +01:00
Ryan Ernst 9d87b63ca4 Build: Fix third party audit task for xpack core (elastic/x-pack-elasticsearch#3656)
This commit re-enables thirdPartyAudit for x-pack core. Previously, when
xpack was a single plugin, it transitively picked up httpcore-nio
through the elasticsearch rest client. Now that xpack core does not
depend on the rest client, httpcore-nio must be added as a dependency.
Additionally, commons-logging was previously handled through the rest
client, but now xpack depends directly on this, thus excludes must be
added for the pesky missing classes there.

This commit also cleans up unnecessary parts of plugin/build.gradle no
longer necessary.

Original commit: elastic/x-pack-elasticsearch@70e936bdc3
2018-01-22 22:58:34 -08:00
Lisa Cawley c0edf2197b [DOCS] Replaced settings with links (elastic/x-pack-elasticsearch#3626)
Original commit: elastic/x-pack-elasticsearch@4ad018521e
2018-01-22 15:15:31 -08:00
Lisa Cawley 10827033c5 [DOCS] Added information about overall bucket scores (elastic/x-pack-elasticsearch#3333)
Original commit: elastic/x-pack-elasticsearch@68efc63f25
2018-01-22 13:52:47 -08:00
jaymode 698ad62150 Fix SAMLRealmTests#testReadIdpMetadataFromHttps failures
This commit fixes failures in SAMLRealmTests#testReadIdpMetadataFromHttps. The main test failure is
caused by the test relying too much on timing with a sleep; the sleep has been removed and replaced
with an assert busy. The second issue that was fixed in the test, is the test makes two requests to
the webserver but only one request is queued in the MockWebServer leading to a NPE.

relates elastic/x-pack-elasticsearch#3667

Original commit: elastic/x-pack-elasticsearch@6c7076c990
2018-01-22 12:20:52 -07:00
Jay Modi e577d2b776 Fix handling of IOException when computing key for tokens (elastic/x-pack-elasticsearch#3639)
The invalid token test has been failing due to the way the test expects an exception to be
returned. Recent changes allowed the exception to be returned via a listener but the test
was not adapted and the code in the token service was not properly handling these exceptions
when they were returned via a listener.

relates elastic/x-pack-elasticsearch#3630

Original commit: elastic/x-pack-elasticsearch@1cf2cc0427
2018-01-22 09:09:20 -07:00
Jason Tedor 2453da82e0 Force evaluation order between sub-plugins and core
We are relying on the fact that :x-pack-elasticsearch:plugin:core is
configured before these. This happens when these projects are
alphabetically after core, but is not the case when they are not
(ccr). To address this, we simplfy be explicit about evaluation order in
all sub-plugins.

Relates elastic/x-pack-elasticsearch#3663

Original commit: elastic/x-pack-elasticsearch@755f84258b
2018-01-22 10:55:03 -05:00
Jason Tedor 520b372549 Remove watcher as a security test dependency
This commit removes Watcher as a test dependency for Security.

Relates elastic/x-pack-elasticsearch#3655

Original commit: elastic/x-pack-elasticsearch@8d770ffb9d
2018-01-22 10:16:03 -05:00
Luca Cavanna d9eb4c049f Trim down usages of `ShardOperationFailedException` interface (elastic/x-pack-elasticsearch#3662)
In many cases we use the `ShardOperationFailedException` interface to abstract an exception that can only be of one type, namely `DefaultShardOperationException`. There is no need to use the interface in such cases, the concrete type should be used instead. That has the additional advantage of simplifying parsing such exceptions back from rest responses for the high-level REST client.

Original commit: elastic/x-pack-elasticsearch@b2259afcbf
2018-01-22 15:51:55 +01:00
Dimitrios Athanasiou 8ee5b608e0 [ML] Fix line width violation
Original commit: elastic/x-pack-elasticsearch@94d55c42b8
2018-01-22 14:17:42 +00:00
Dimitris Athanasiou cc432aee34 [ML] Check calendar exists before removing job (elastic/x-pack-elasticsearch#3661)
Also, removes check for whether a job-to-remove exists
and replaces it with a check of whether a job-to-remove
is already present in the calendar. This allows to
remove a job that may no longer exists and it improves
feedback for the case that an existing job is removed from
a calendar that doesn't contain it.

relates elastic/x-pack-elasticsearch#3620

Original commit: elastic/x-pack-elasticsearch@3ea39be1b6
2018-01-22 13:49:38 +00:00
Simon Willnauer 129f021843 [TEST] Fix SettingsFilterTests to also install the Security plugin
Original commit: elastic/x-pack-elasticsearch@b30997dc6a
2018-01-22 14:27:42 +01:00
Dimitris Athanasiou f3282b559f [ML] Check calendar exists before deleting event (elastic/x-pack-elasticsearch#3659)
Solves the event part of elastic/x-pack-elasticsearch#3620

Original commit: elastic/x-pack-elasticsearch@c79ca85c6e
2018-01-22 13:02:42 +00:00
Jason Tedor 928b6a6e04 Fix compilation in audit-tests
This commit fixes compilation in audit-tests which was broken after some
stray classes were moved out of plugin into specific plugin directories.

Original commit: elastic/x-pack-elasticsearch@8bfd8cfceb
2018-01-22 06:00:23 -05:00
Albert Zaharovits 0a1e352c5d [DOCS] for audit filtering (elastic/x-pack-elasticsearch#3594)
This documents the changes merged in elastic/x-pack-elasticsearch#3005 and elastic/x-pack-elasticsearch#3100 .

Original commit: elastic/x-pack-elasticsearch@d1702f0480
2018-01-22 11:45:12 +02:00
Ryan Ernst ac44ee8257 Build: Move test files from plugin/src to appropriate other projects (elastic/x-pack-elasticsearch#3653)
This commit moves mostly security tests that were left in plugin/src. It
also moves the CompositeTestingXPackPlugin into tribe license tests qa
project, which was the only remaining use. This class needs to be
removed, as it has unavoidable problems with dependency conflicts (ie
guava) between security and watcher, which it pulls in both into the
test classpath.

Original commit: elastic/x-pack-elasticsearch@756209e010
2018-01-21 14:08:47 -05:00
Jason Tedor 4d4f979d61 Remove debugging println from API JAR task
This commit removes a leftover println that was added while debugging
the execution of the API JAR task.

Original commit: elastic/x-pack-elasticsearch@37c2e8fe5b
2018-01-21 13:20:03 -05:00