Commit Graph

7975 Commits

Author SHA1 Message Date
Alexander Reelsen d065b087ee Revert "Cleanup: Remove HaltedClock (elastic/x-pack-elasticsearch#3664)"
This reverts commit elastic/x-pack-elasticsearch@f91c401a60 due to
failing tests, like

./gradlew :x-pack-elasticsearch:plugin:watcher:test -Dtests.seed=AE30350FCE96D26D -Dtests.class=org.elasticsearch.xpack.watcher.watch.WatchTests -Dtests.method="testParserSelfGenerated" -Dtests.security.manager=true -Dtests.locale=ja-JP -Dtests.timezone=EET

Original commit: elastic/x-pack-elasticsearch@e45d79d643
2018-01-23 18:08:54 +01:00
David Roberts 697a08e742 [ML] Refactor so ML doesn't require PersistentTasksService at startup (elastic/x-pack-elasticsearch#3682)
At present the PersistentTasksService is created inside the ML plugin.
This is undesirable, as other plugins will use persistent tasks in the
near future.

This change refactors the startup code so that the PersistentTasksService
no longer needs to be passed to any constructors for ML components.

A future change will still be required to actually move the initialization
of the PersistentTasksClusterService, PersistentTasksService and
PersistentTasksExecutorRegistry out of the ML plugin, but following this
change it should be fairly simple.

Original commit: elastic/x-pack-elasticsearch@3c2a8e020e
2018-01-23 16:55:08 +00:00
Lisa Cawley 223d3c1f4c [DOCS] Added QA notes about scheduled events (elastic/x-pack-elasticsearch#3641)
Original commit: elastic/x-pack-elasticsearch@64b67aa0ad
2018-01-23 08:41:48 -08:00
Nik Everett 52f7ba8c5d SQL: Docs for basic aggregations
Adds documentation for basic aggregate functions supported by
Elasticsearch SQL.

Relates to elastic/x-pack-elasticsearch#2898

Original commit: elastic/x-pack-elasticsearch@ddc71165f2
2018-01-23 11:32:21 -05:00
Simon Willnauer 63c0e288af Ensure we protect Collections obtained from scripts from self-referencing (elastic/x-pack-elasticsearch#3681)
Self referencing maps can cause SOE if they are iterated ie. in their toString methods. This chance adds some protected to the usage of those collections.
see elastic/elasticsearch#28335

Original commit: elastic/x-pack-elasticsearch@c4f1089c74
2018-01-23 17:02:37 +01:00
Dimitris Athanasiou 215f9af1cc [ML] Add trace logging for when search response is obtained (elastic/x-pack-elasticsearch#3669)
This is useful for understanding performance characteristics
as it helps us understand whether the bottleneck is the search
part or the analytics part.

Relates elastic/x-pack-elasticsearch#3590

Original commit: elastic/x-pack-elasticsearch@dc8c095958
2018-01-23 14:16:34 +00:00
Nik Everett 7d429a74b7 SQL: Fix compilation after move
I moved something without running all of the tests. For shame!

Original commit: elastic/x-pack-elasticsearch@3aaeb320bc
2018-01-23 08:46:18 -05:00
Alexander Reelsen e3da8fa4ae Cleanup: Remove HaltedClock (elastic/x-pack-elasticsearch#3664)
The HaltedClock was a leftover from moving over from our own Clock
implementation to a java.time one. java.time already has a fixed clock,
this one is not needed.

Original commit: elastic/x-pack-elasticsearch@f91c401a60
2018-01-23 14:13:22 +01:00
Nik Everett d2ea36416b SQL: Expose WEEK function and document datetime functions (elastic/x-pack-elasticsearch#3680)
Adds documentation for all of the date time functions using the new
cli-like format extracted from the csv spec. In the process of doing
this I noticed that the `WEEK` function isn't exposed as a function.
This exposes it for consistency.

Relates to elastic/x-pack-elasticsearch#2898

Original commit: elastic/x-pack-elasticsearch@0459b24cb9
2018-01-23 07:11:43 -05:00
Simon Willnauer c70c1c7e98 Expose XPackExtensions via SPI (elastic/x-pack-elasticsearch#3530)
This change adds SPI loading for XPackExtensions that allows to extend
XPack via an ordinary plugin. This can co-exist with the existin
extension mechanism for the time being.

Original commit: elastic/x-pack-elasticsearch@bf02b56dee
2018-01-23 13:05:39 +01:00
David Kyle 8f393c3066 [ML] Improve error message when creating calendars (elastic/x-pack-elasticsearch#3668)
Original commit: elastic/x-pack-elasticsearch@996b0e2f65
2018-01-23 09:34:07 +00:00
Simon Willnauer c3efa4b6bc [TEST] disable ML when ML is not installed
Original commit: elastic/x-pack-elasticsearch@cd84acc3e0
2018-01-23 10:28:37 +01:00
Ryan Ernst 9d87b63ca4 Build: Fix third party audit task for xpack core (elastic/x-pack-elasticsearch#3656)
This commit re-enables thirdPartyAudit for x-pack core. Previously, when
xpack was a single plugin, it transitively picked up httpcore-nio
through the elasticsearch rest client. Now that xpack core does not
depend on the rest client, httpcore-nio must be added as a dependency.
Additionally, commons-logging was previously handled through the rest
client, but now xpack depends directly on this, thus excludes must be
added for the pesky missing classes there.

This commit also cleans up unnecessary parts of plugin/build.gradle no
longer necessary.

Original commit: elastic/x-pack-elasticsearch@70e936bdc3
2018-01-22 22:58:34 -08:00
Igor Motov ae55f506ca SQL: Move IT tests sql plugin project
Original commit: elastic/x-pack-elasticsearch@c7b59f05a5
2018-01-22 19:11:32 -05:00
Lisa Cawley c0edf2197b [DOCS] Replaced settings with links (elastic/x-pack-elasticsearch#3626)
Original commit: elastic/x-pack-elasticsearch@4ad018521e
2018-01-22 15:15:31 -08:00
Lisa Cawley 10827033c5 [DOCS] Added information about overall bucket scores (elastic/x-pack-elasticsearch#3333)
Original commit: elastic/x-pack-elasticsearch@68efc63f25
2018-01-22 13:52:47 -08:00
Igor Motov 6330209e7d Merge remote-tracking branch 'elastic/master' into feature/sql_2
Original commit: elastic/x-pack-elasticsearch@95cbd93803
2018-01-22 16:03:56 -05:00
jaymode 698ad62150 Fix SAMLRealmTests#testReadIdpMetadataFromHttps failures
This commit fixes failures in SAMLRealmTests#testReadIdpMetadataFromHttps. The main test failure is
caused by the test relying too much on timing with a sleep; the sleep has been removed and replaced
with an assert busy. The second issue that was fixed in the test, is the test makes two requests to
the webserver but only one request is queued in the MockWebServer leading to a NPE.

relates elastic/x-pack-elasticsearch#3667

Original commit: elastic/x-pack-elasticsearch@6c7076c990
2018-01-22 12:20:52 -07:00
Jay Modi e577d2b776 Fix handling of IOException when computing key for tokens (elastic/x-pack-elasticsearch#3639)
The invalid token test has been failing due to the way the test expects an exception to be
returned. Recent changes allowed the exception to be returned via a listener but the test
was not adapted and the code in the token service was not properly handling these exceptions
when they were returned via a listener.

relates elastic/x-pack-elasticsearch#3630

Original commit: elastic/x-pack-elasticsearch@1cf2cc0427
2018-01-22 09:09:20 -07:00
Jason Tedor 2453da82e0 Force evaluation order between sub-plugins and core
We are relying on the fact that :x-pack-elasticsearch:plugin:core is
configured before these. This happens when these projects are
alphabetically after core, but is not the case when they are not
(ccr). To address this, we simplfy be explicit about evaluation order in
all sub-plugins.

Relates elastic/x-pack-elasticsearch#3663

Original commit: elastic/x-pack-elasticsearch@755f84258b
2018-01-22 10:55:03 -05:00
Jason Tedor 520b372549 Remove watcher as a security test dependency
This commit removes Watcher as a test dependency for Security.

Relates elastic/x-pack-elasticsearch#3655

Original commit: elastic/x-pack-elasticsearch@8d770ffb9d
2018-01-22 10:16:03 -05:00
Luca Cavanna d9eb4c049f Trim down usages of `ShardOperationFailedException` interface (elastic/x-pack-elasticsearch#3662)
In many cases we use the `ShardOperationFailedException` interface to abstract an exception that can only be of one type, namely `DefaultShardOperationException`. There is no need to use the interface in such cases, the concrete type should be used instead. That has the additional advantage of simplifying parsing such exceptions back from rest responses for the high-level REST client.

Original commit: elastic/x-pack-elasticsearch@b2259afcbf
2018-01-22 15:51:55 +01:00
Dimitrios Athanasiou 8ee5b608e0 [ML] Fix line width violation
Original commit: elastic/x-pack-elasticsearch@94d55c42b8
2018-01-22 14:17:42 +00:00
Dimitris Athanasiou cc432aee34 [ML] Check calendar exists before removing job (elastic/x-pack-elasticsearch#3661)
Also, removes check for whether a job-to-remove exists
and replaces it with a check of whether a job-to-remove
is already present in the calendar. This allows to
remove a job that may no longer exists and it improves
feedback for the case that an existing job is removed from
a calendar that doesn't contain it.

relates elastic/x-pack-elasticsearch#3620

Original commit: elastic/x-pack-elasticsearch@3ea39be1b6
2018-01-22 13:49:38 +00:00
Simon Willnauer 129f021843 [TEST] Fix SettingsFilterTests to also install the Security plugin
Original commit: elastic/x-pack-elasticsearch@b30997dc6a
2018-01-22 14:27:42 +01:00
Dimitris Athanasiou f3282b559f [ML] Check calendar exists before deleting event (elastic/x-pack-elasticsearch#3659)
Solves the event part of elastic/x-pack-elasticsearch#3620

Original commit: elastic/x-pack-elasticsearch@c79ca85c6e
2018-01-22 13:02:42 +00:00
Jason Tedor 928b6a6e04 Fix compilation in audit-tests
This commit fixes compilation in audit-tests which was broken after some
stray classes were moved out of plugin into specific plugin directories.

Original commit: elastic/x-pack-elasticsearch@8bfd8cfceb
2018-01-22 06:00:23 -05:00
Albert Zaharovits 0a1e352c5d [DOCS] for audit filtering (elastic/x-pack-elasticsearch#3594)
This documents the changes merged in elastic/x-pack-elasticsearch#3005 and elastic/x-pack-elasticsearch#3100 .

Original commit: elastic/x-pack-elasticsearch@d1702f0480
2018-01-22 11:45:12 +02:00
Ryan Ernst ac44ee8257 Build: Move test files from plugin/src to appropriate other projects (elastic/x-pack-elasticsearch#3653)
This commit moves mostly security tests that were left in plugin/src. It
also moves the CompositeTestingXPackPlugin into tribe license tests qa
project, which was the only remaining use. This class needs to be
removed, as it has unavoidable problems with dependency conflicts (ie
guava) between security and watcher, which it pulls in both into the
test classpath.

Original commit: elastic/x-pack-elasticsearch@756209e010
2018-01-21 14:08:47 -05:00
Jason Tedor 4d4f979d61 Remove debugging println from API JAR task
This commit removes a leftover println that was added while debugging
the execution of the API JAR task.

Original commit: elastic/x-pack-elasticsearch@37c2e8fe5b
2018-01-21 13:20:03 -05:00
Jason Tedor 04b48324aa Drop native controller from descriptors (except ML)
These were copied wholesale from the pre-split X-Pack
descriptor. However, only ML has a native controller. This removes the
plugin installation asking multiple times to approve the existence of a
native controlled for every bundled plugin.

Relates elastic/x-pack-elasticsearch#3650

Original commit: elastic/x-pack-elasticsearch@4fca606243
2018-01-21 11:00:51 -05:00
Jason Tedor a582acddf8 Add API JAR task to x-pack-core
This commit adds back the task for assembling the API JAR from the
x-pack-core JAR.

Original commit: elastic/x-pack-elasticsearch@10385ecf17
2018-01-21 10:00:47 -05:00
Jason Tedor 377bb53cca Fix format of third party audit in saml-idp-tests
The formatting here was off since it contains tabs and tabs are evil.

Original commit: elastic/x-pack-elasticsearch@fadbb02157
2018-01-20 19:53:26 -05:00
Tim Vernum 47213f5675 [Security] Add SAML authentication support (elastic/x-pack-elasticsearch#3646)
Introduces:

- SAML Realm
- REST & Transport actions to support SAML single signon / signout
- Tests for above
- More XML than you ever wanted to see.

Original commit: elastic/x-pack-elasticsearch@b0fe7bb652
2018-01-21 08:43:00 +10:00
Jason Tedor 50864eabce Introduce plugin-specific env scripts
With plugins of a meta-plugin now loaded in separate classloaders, we
should not be loading all classes in all plugins of a meta-plugin when
executing scripts. This is particularly problematic in the case of
security extensions where the install plugin extension command would be
running with the classpath of all plugins. However, if there is JAR hell
in this classpath, installation would fail. This is not realistic though
since the plugins are run in separate classloaders. To fix this, for the
scripts of a plugin, we only set the classpath to include the JARs for
that plugin and the JARs of core. This leads us to the introduction of
plugin-specific env scripts.

Relates elastic/x-pack-elasticsearch#3649

Original commit: elastic/x-pack-elasticsearch@543df37eed
2018-01-20 15:56:00 -05:00
Jason Tedor 51c53710d7 Keep imports consistent with 6.x
Some imports were changed in 6.x to address line-length issues
there. This commit pulls the same changes to master to keep the branches
consistent to simplify backports.

Original commit: elastic/x-pack-elasticsearch@190f9d41f5
2018-01-20 14:52:07 -05:00
David Roberts 64cfa017f0 [TEST] Update ML categorization expected result following C++ change
Original commit: elastic/x-pack-elasticsearch@cc78772aa2
2018-01-20 14:50:24 +00:00
Jason Tedor 60e577a9c4 Fix test base class that disables ML autodetect
This is no longer needed for general X-Pack since the tests will not be
depending on ML. We move this class to an ML specific directory and
remove the dependency from other tests.

Original commit: elastic/x-pack-elasticsearch@9b287f7460
2018-01-20 01:27:33 -05:00
Ryan Ernst 0bc496ae04 add pattern excludes for binary files
Original commit: elastic/x-pack-elasticsearch@f1077e35f8
2018-01-19 22:05:01 -08:00
Michael Basnight f3ec4a5208 Split up xpack plugins into their own modules (elastic/x-pack-elasticsearch#3643)
Thanks to some great work by a bunch of amazing people, the chuck norris xpack split is a go!

Original commit: elastic/x-pack-elasticsearch@dad98e28f4
2018-01-19 23:30:17 -06:00
Nik Everett 51a6285ba1 SQL: Allow CSV tests to look more like the CLI (elastic/x-pack-elasticsearch#3640)
This allows CSV tests to include a line between the header and the
values that is ignored by the test framework. This optional line can be
added to the tests to make them a little easier to read which is useful
when they are included in the documentation. As a side effect they also
closely mimick the output of the CLI. To the point where you can copy
directly from the CLI and paste into the CSV tests.

Example:
```
constantYear
// tag::year
SELECT YEAR(CAST('2018-01-19T10:23:27Z' AS TIMESTAMP)) as year;
     year
---------------
2018
// end::year
;
```

This can be extracted with a construct like this in the docs:
```
["source","sql",subs="attributes,callouts,macros"]
--------------------------------------------------
include-tagged::{sql-specs}/datetime.csv-spec[year]
--------------------------------------------------
```

Which makes documentation that looks like this:
```
SELECT YEAR(CAST('2018-01-19T10:23:27Z' AS TIMESTAMP)) as year;
     year
---------------
2018
```

Which is fairly nice.

Original commit: elastic/x-pack-elasticsearch@8c10b5cb10
2018-01-19 16:24:42 -05:00
Lisa Cawley 744d9325e9 [DOCS] Add forecasting to ML tutorial (elastic/x-pack-elasticsearch#3489)
Original commit: elastic/x-pack-elasticsearch@ffb681ae96
2018-01-19 12:43:58 -08:00
Nik Everett b165f1c71e SQL: Fix constant folding of datetime functions (elastic/x-pack-elasticsearch#3637)
I went to write some docs for datetime functions that look like:
```
SELECT YEAR(CAST('2018-01-19T10:23:27Z' AS TIMESTAMP)) as year;

  year
2018

```
because I figured they'd be pretty easy to read because they didn't
require any knowledge of a data set. But it turns out that constant
folding doesn't work properly for date time functions because they don't
actually apply the extraction.

Original commit: elastic/x-pack-elasticsearch@aa9c66b2c7
2018-01-19 15:11:37 -05:00
Jason Tedor 0602d38bd5 Split transport implementations into client/server
This commit splits the transport implementations into components that
can be used client-side (in the transport client) and server-side (in
the server). This enables removing security as a dependency for the
transport client.

Relates elastic/x-pack-elasticsearch#3635

Original commit: elastic/x-pack-elasticsearch@e480eb7eb2
2018-01-19 12:09:34 -05:00
Dimitrios Athanasiou 9e834cbf8a [ML] Fix javadoc in MlStrings#hasValidLengthForId
Original commit: elastic/x-pack-elasticsearch@ef5ecad5e2
2018-01-19 11:45:41 +00:00
Dimitris Athanasiou 21f692c02b [ML] Further validate calendar_id and add calendar description (elastic/x-pack-elasticsearch#3624)
relates elastic/x-pack-elasticsearch#3595

Original commit: elastic/x-pack-elasticsearch@fade977361
2018-01-19 10:44:39 +00:00
Ryan Ernst 19874e35ee More license file corrections
See elastic/x-pack-elasticsearch@0af9ffc

Original commit: elastic/x-pack-elasticsearch@360a88381f
2018-01-18 16:34:25 -08:00
Ryan Ernst 43486d6a70 Remove unnecessary license files
These are no longer necessary after https://github.com/elastic/elasticsearch/pull/28304

Original commit: elastic/x-pack-elasticsearch@0af9ffc4d1
2018-01-18 16:19:10 -08:00
Lee Hinman f69a09ce83 SQL: Remove unneeded licenses, notices, and shas
Original commit: elastic/x-pack-elasticsearch@2407eeabd3
2018-01-18 17:16:25 -07:00
Lee Hinman 5818986775 Merge remote-tracking branch 'origin/master' into feature/sql_2
Original commit: elastic/x-pack-elasticsearch@8475419a00
2018-01-18 16:51:56 -07:00