Commit Graph

7341 Commits

Author SHA1 Message Date
David Roberts 697a08e742 [ML] Refactor so ML doesn't require PersistentTasksService at startup (elastic/x-pack-elasticsearch#3682)
At present the PersistentTasksService is created inside the ML plugin.
This is undesirable, as other plugins will use persistent tasks in the
near future.

This change refactors the startup code so that the PersistentTasksService
no longer needs to be passed to any constructors for ML components.

A future change will still be required to actually move the initialization
of the PersistentTasksClusterService, PersistentTasksService and
PersistentTasksExecutorRegistry out of the ML plugin, but following this
change it should be fairly simple.

Original commit: elastic/x-pack-elasticsearch@3c2a8e020e
2018-01-23 16:55:08 +00:00
Lisa Cawley 223d3c1f4c [DOCS] Added QA notes about scheduled events (elastic/x-pack-elasticsearch#3641)
Original commit: elastic/x-pack-elasticsearch@64b67aa0ad
2018-01-23 08:41:48 -08:00
Simon Willnauer 63c0e288af Ensure we protect Collections obtained from scripts from self-referencing (elastic/x-pack-elasticsearch#3681)
Self referencing maps can cause SOE if they are iterated ie. in their toString methods. This chance adds some protected to the usage of those collections.
see elastic/elasticsearch#28335

Original commit: elastic/x-pack-elasticsearch@c4f1089c74
2018-01-23 17:02:37 +01:00
Dimitris Athanasiou 215f9af1cc [ML] Add trace logging for when search response is obtained (elastic/x-pack-elasticsearch#3669)
This is useful for understanding performance characteristics
as it helps us understand whether the bottleneck is the search
part or the analytics part.

Relates elastic/x-pack-elasticsearch#3590

Original commit: elastic/x-pack-elasticsearch@dc8c095958
2018-01-23 14:16:34 +00:00
Alexander Reelsen e3da8fa4ae Cleanup: Remove HaltedClock (elastic/x-pack-elasticsearch#3664)
The HaltedClock was a leftover from moving over from our own Clock
implementation to a java.time one. java.time already has a fixed clock,
this one is not needed.

Original commit: elastic/x-pack-elasticsearch@f91c401a60
2018-01-23 14:13:22 +01:00
Simon Willnauer c70c1c7e98 Expose XPackExtensions via SPI (elastic/x-pack-elasticsearch#3530)
This change adds SPI loading for XPackExtensions that allows to extend
XPack via an ordinary plugin. This can co-exist with the existin
extension mechanism for the time being.

Original commit: elastic/x-pack-elasticsearch@bf02b56dee
2018-01-23 13:05:39 +01:00
David Kyle 8f393c3066 [ML] Improve error message when creating calendars (elastic/x-pack-elasticsearch#3668)
Original commit: elastic/x-pack-elasticsearch@996b0e2f65
2018-01-23 09:34:07 +00:00
Simon Willnauer c3efa4b6bc [TEST] disable ML when ML is not installed
Original commit: elastic/x-pack-elasticsearch@cd84acc3e0
2018-01-23 10:28:37 +01:00
Ryan Ernst 9d87b63ca4 Build: Fix third party audit task for xpack core (elastic/x-pack-elasticsearch#3656)
This commit re-enables thirdPartyAudit for x-pack core. Previously, when
xpack was a single plugin, it transitively picked up httpcore-nio
through the elasticsearch rest client. Now that xpack core does not
depend on the rest client, httpcore-nio must be added as a dependency.
Additionally, commons-logging was previously handled through the rest
client, but now xpack depends directly on this, thus excludes must be
added for the pesky missing classes there.

This commit also cleans up unnecessary parts of plugin/build.gradle no
longer necessary.

Original commit: elastic/x-pack-elasticsearch@70e936bdc3
2018-01-22 22:58:34 -08:00
Lisa Cawley c0edf2197b [DOCS] Replaced settings with links (elastic/x-pack-elasticsearch#3626)
Original commit: elastic/x-pack-elasticsearch@4ad018521e
2018-01-22 15:15:31 -08:00
Lisa Cawley 10827033c5 [DOCS] Added information about overall bucket scores (elastic/x-pack-elasticsearch#3333)
Original commit: elastic/x-pack-elasticsearch@68efc63f25
2018-01-22 13:52:47 -08:00
jaymode 698ad62150 Fix SAMLRealmTests#testReadIdpMetadataFromHttps failures
This commit fixes failures in SAMLRealmTests#testReadIdpMetadataFromHttps. The main test failure is
caused by the test relying too much on timing with a sleep; the sleep has been removed and replaced
with an assert busy. The second issue that was fixed in the test, is the test makes two requests to
the webserver but only one request is queued in the MockWebServer leading to a NPE.

relates elastic/x-pack-elasticsearch#3667

Original commit: elastic/x-pack-elasticsearch@6c7076c990
2018-01-22 12:20:52 -07:00
Jay Modi e577d2b776 Fix handling of IOException when computing key for tokens (elastic/x-pack-elasticsearch#3639)
The invalid token test has been failing due to the way the test expects an exception to be
returned. Recent changes allowed the exception to be returned via a listener but the test
was not adapted and the code in the token service was not properly handling these exceptions
when they were returned via a listener.

relates elastic/x-pack-elasticsearch#3630

Original commit: elastic/x-pack-elasticsearch@1cf2cc0427
2018-01-22 09:09:20 -07:00
Jason Tedor 2453da82e0 Force evaluation order between sub-plugins and core
We are relying on the fact that :x-pack-elasticsearch:plugin:core is
configured before these. This happens when these projects are
alphabetically after core, but is not the case when they are not
(ccr). To address this, we simplfy be explicit about evaluation order in
all sub-plugins.

Relates elastic/x-pack-elasticsearch#3663

Original commit: elastic/x-pack-elasticsearch@755f84258b
2018-01-22 10:55:03 -05:00
Jason Tedor 520b372549 Remove watcher as a security test dependency
This commit removes Watcher as a test dependency for Security.

Relates elastic/x-pack-elasticsearch#3655

Original commit: elastic/x-pack-elasticsearch@8d770ffb9d
2018-01-22 10:16:03 -05:00
Luca Cavanna d9eb4c049f Trim down usages of `ShardOperationFailedException` interface (elastic/x-pack-elasticsearch#3662)
In many cases we use the `ShardOperationFailedException` interface to abstract an exception that can only be of one type, namely `DefaultShardOperationException`. There is no need to use the interface in such cases, the concrete type should be used instead. That has the additional advantage of simplifying parsing such exceptions back from rest responses for the high-level REST client.

Original commit: elastic/x-pack-elasticsearch@b2259afcbf
2018-01-22 15:51:55 +01:00
Dimitrios Athanasiou 8ee5b608e0 [ML] Fix line width violation
Original commit: elastic/x-pack-elasticsearch@94d55c42b8
2018-01-22 14:17:42 +00:00
Dimitris Athanasiou cc432aee34 [ML] Check calendar exists before removing job (elastic/x-pack-elasticsearch#3661)
Also, removes check for whether a job-to-remove exists
and replaces it with a check of whether a job-to-remove
is already present in the calendar. This allows to
remove a job that may no longer exists and it improves
feedback for the case that an existing job is removed from
a calendar that doesn't contain it.

relates elastic/x-pack-elasticsearch#3620

Original commit: elastic/x-pack-elasticsearch@3ea39be1b6
2018-01-22 13:49:38 +00:00
Simon Willnauer 129f021843 [TEST] Fix SettingsFilterTests to also install the Security plugin
Original commit: elastic/x-pack-elasticsearch@b30997dc6a
2018-01-22 14:27:42 +01:00
Dimitris Athanasiou f3282b559f [ML] Check calendar exists before deleting event (elastic/x-pack-elasticsearch#3659)
Solves the event part of elastic/x-pack-elasticsearch#3620

Original commit: elastic/x-pack-elasticsearch@c79ca85c6e
2018-01-22 13:02:42 +00:00
Jason Tedor 928b6a6e04 Fix compilation in audit-tests
This commit fixes compilation in audit-tests which was broken after some
stray classes were moved out of plugin into specific plugin directories.

Original commit: elastic/x-pack-elasticsearch@8bfd8cfceb
2018-01-22 06:00:23 -05:00
Albert Zaharovits 0a1e352c5d [DOCS] for audit filtering (elastic/x-pack-elasticsearch#3594)
This documents the changes merged in elastic/x-pack-elasticsearch#3005 and elastic/x-pack-elasticsearch#3100 .

Original commit: elastic/x-pack-elasticsearch@d1702f0480
2018-01-22 11:45:12 +02:00
Ryan Ernst ac44ee8257 Build: Move test files from plugin/src to appropriate other projects (elastic/x-pack-elasticsearch#3653)
This commit moves mostly security tests that were left in plugin/src. It
also moves the CompositeTestingXPackPlugin into tribe license tests qa
project, which was the only remaining use. This class needs to be
removed, as it has unavoidable problems with dependency conflicts (ie
guava) between security and watcher, which it pulls in both into the
test classpath.

Original commit: elastic/x-pack-elasticsearch@756209e010
2018-01-21 14:08:47 -05:00
Jason Tedor 4d4f979d61 Remove debugging println from API JAR task
This commit removes a leftover println that was added while debugging
the execution of the API JAR task.

Original commit: elastic/x-pack-elasticsearch@37c2e8fe5b
2018-01-21 13:20:03 -05:00
Jason Tedor 04b48324aa Drop native controller from descriptors (except ML)
These were copied wholesale from the pre-split X-Pack
descriptor. However, only ML has a native controller. This removes the
plugin installation asking multiple times to approve the existence of a
native controlled for every bundled plugin.

Relates elastic/x-pack-elasticsearch#3650

Original commit: elastic/x-pack-elasticsearch@4fca606243
2018-01-21 11:00:51 -05:00
Jason Tedor a582acddf8 Add API JAR task to x-pack-core
This commit adds back the task for assembling the API JAR from the
x-pack-core JAR.

Original commit: elastic/x-pack-elasticsearch@10385ecf17
2018-01-21 10:00:47 -05:00
Jason Tedor 377bb53cca Fix format of third party audit in saml-idp-tests
The formatting here was off since it contains tabs and tabs are evil.

Original commit: elastic/x-pack-elasticsearch@fadbb02157
2018-01-20 19:53:26 -05:00
Tim Vernum 47213f5675 [Security] Add SAML authentication support (elastic/x-pack-elasticsearch#3646)
Introduces:

- SAML Realm
- REST & Transport actions to support SAML single signon / signout
- Tests for above
- More XML than you ever wanted to see.

Original commit: elastic/x-pack-elasticsearch@b0fe7bb652
2018-01-21 08:43:00 +10:00
Jason Tedor 50864eabce Introduce plugin-specific env scripts
With plugins of a meta-plugin now loaded in separate classloaders, we
should not be loading all classes in all plugins of a meta-plugin when
executing scripts. This is particularly problematic in the case of
security extensions where the install plugin extension command would be
running with the classpath of all plugins. However, if there is JAR hell
in this classpath, installation would fail. This is not realistic though
since the plugins are run in separate classloaders. To fix this, for the
scripts of a plugin, we only set the classpath to include the JARs for
that plugin and the JARs of core. This leads us to the introduction of
plugin-specific env scripts.

Relates elastic/x-pack-elasticsearch#3649

Original commit: elastic/x-pack-elasticsearch@543df37eed
2018-01-20 15:56:00 -05:00
Jason Tedor 51c53710d7 Keep imports consistent with 6.x
Some imports were changed in 6.x to address line-length issues
there. This commit pulls the same changes to master to keep the branches
consistent to simplify backports.

Original commit: elastic/x-pack-elasticsearch@190f9d41f5
2018-01-20 14:52:07 -05:00
David Roberts 64cfa017f0 [TEST] Update ML categorization expected result following C++ change
Original commit: elastic/x-pack-elasticsearch@cc78772aa2
2018-01-20 14:50:24 +00:00
Jason Tedor 60e577a9c4 Fix test base class that disables ML autodetect
This is no longer needed for general X-Pack since the tests will not be
depending on ML. We move this class to an ML specific directory and
remove the dependency from other tests.

Original commit: elastic/x-pack-elasticsearch@9b287f7460
2018-01-20 01:27:33 -05:00
Ryan Ernst 0bc496ae04 add pattern excludes for binary files
Original commit: elastic/x-pack-elasticsearch@f1077e35f8
2018-01-19 22:05:01 -08:00
Michael Basnight f3ec4a5208 Split up xpack plugins into their own modules (elastic/x-pack-elasticsearch#3643)
Thanks to some great work by a bunch of amazing people, the chuck norris xpack split is a go!

Original commit: elastic/x-pack-elasticsearch@dad98e28f4
2018-01-19 23:30:17 -06:00
Lisa Cawley 744d9325e9 [DOCS] Add forecasting to ML tutorial (elastic/x-pack-elasticsearch#3489)
Original commit: elastic/x-pack-elasticsearch@ffb681ae96
2018-01-19 12:43:58 -08:00
Jason Tedor 0602d38bd5 Split transport implementations into client/server
This commit splits the transport implementations into components that
can be used client-side (in the transport client) and server-side (in
the server). This enables removing security as a dependency for the
transport client.

Relates elastic/x-pack-elasticsearch#3635

Original commit: elastic/x-pack-elasticsearch@e480eb7eb2
2018-01-19 12:09:34 -05:00
Dimitrios Athanasiou 9e834cbf8a [ML] Fix javadoc in MlStrings#hasValidLengthForId
Original commit: elastic/x-pack-elasticsearch@ef5ecad5e2
2018-01-19 11:45:41 +00:00
Dimitris Athanasiou 21f692c02b [ML] Further validate calendar_id and add calendar description (elastic/x-pack-elasticsearch#3624)
relates elastic/x-pack-elasticsearch#3595

Original commit: elastic/x-pack-elasticsearch@fade977361
2018-01-19 10:44:39 +00:00
Ryan Ernst 19874e35ee More license file corrections
See elastic/x-pack-elasticsearch@0af9ffc

Original commit: elastic/x-pack-elasticsearch@360a88381f
2018-01-18 16:34:25 -08:00
Ryan Ernst 43486d6a70 Remove unnecessary license files
These are no longer necessary after https://github.com/elastic/elasticsearch/pull/28304

Original commit: elastic/x-pack-elasticsearch@0af9ffc4d1
2018-01-18 16:19:10 -08:00
Tim Brooks 685b75da3a Support changes in nio channel contexts (elastic/x-pack-elasticsearch#3609)
This is related to elastic/elasticsearch#elastic/x-pack-elasticsearch#28275. It modifies x-pack to
support the changes in channel contexts. Additionally, it simplifies
the SSLChannelContext by relying on some common work between it and
BytesChannelContext.

Original commit: elastic/x-pack-elasticsearch@8a8fcce050
2018-01-18 13:06:42 -07:00
jaymode e775e84a7e Test: add additional logging to SslMultiPortTests
This commit adds some additional logging to the SslMultiPortTests to help with debugging an
intermittent failure.

Relates elastic/x-pack-elasticsearch#1809

Original commit: elastic/x-pack-elasticsearch@ae9c967249
2018-01-18 12:46:01 -07:00
jaymode 40d315b244 Test: only use a single shard in RemoteIndexAuditTrailStartingTests
Relates elastic/x-pack-elasticsearch#435

Original commit: elastic/x-pack-elasticsearch@05db61297b
2018-01-18 12:07:30 -07:00
Tim Brooks fb12a0e383 Support TLS/SSL renegotiation (elastic/x-pack-elasticsearch#3600)
This commit is related to elastic/x-pack-elasticsearch#3246. It adds support for receiving TLS/SSL
renegotiation requests for peers.

Original commit: elastic/x-pack-elasticsearch@c22c16b3bc
2018-01-18 10:59:44 -07:00
Lisa Cawley 0ea43c1aa1 [DOCS] Move auditing settings to Elasticsearch Reference (elastic/x-pack-elasticsearch#3608)
Original commit: elastic/x-pack-elasticsearch@a108afd26b
2018-01-18 09:18:24 -08:00
Dimitris Athanasiou eefd8e7940 [ML] Allow categorical conditions to use all analysis fields (elastic/x-pack-elasticsearch#3615)
Categorical conditions should be allowed to use any of the analysis
fields as a field_name.

Original commit: elastic/x-pack-elasticsearch@a0d25e7445
2018-01-18 15:48:56 +00:00
Ryan Ernst b785f9c61b Build: Use md5 to determine whether ml snapshot needs downloading (elastic/x-pack-elasticsearch#3612)
This commit makes ML snapshot downloading happen less often. It does
that by first moving the download location to a directory outside the
destructive power of gradle clean, and then also uses the md5 of the zip
to compare to that found in s3. This allows us to do a cheap HEAD
request to find if the file has changed.

Original commit: elastic/x-pack-elasticsearch@cd8b00fd31
2018-01-17 22:12:23 -08:00
Lisa Cawley 9f6064f9ac [DOCS] Edited documentation for ML categorization_analyzer (elastic/x-pack-elasticsearch#3587)
Original commit: elastic/x-pack-elasticsearch@6dd179107a
2018-01-17 13:11:36 -08:00
Jay Modi 60d4b7e53e Add the ability to refresh tokens obtained via the API (elastic/x-pack-elasticsearch#3468)
This commit adds the ability to refresh tokens that have been obtained by the API using a refresh
token. Refresh tokens are one time use tokens that are valid for 24 hours. The tokens may be used
to get a new access and refresh token if the refresh token has not been invalidated or
already refreshed.

relates elastic/x-pack-elasticsearch#2595

Original commit: elastic/x-pack-elasticsearch@23435eb815
2018-01-17 12:18:44 -07:00
Dimitris Athanasiou f91631664a [ML] Rename Condition member valueFilter -> value (elastic/x-pack-elasticsearch#3599)
Original commit: elastic/x-pack-elasticsearch@5ef8415ca5
2018-01-17 18:39:47 +00:00