Commit Graph

679 Commits

Author SHA1 Message Date
Nik Everett f66e01369a Merge branch 'master' into feature/sql_2
Original commit: elastic/x-pack-elasticsearch@6ba2651f46
2018-02-01 17:21:37 -05:00
Nik Everett c53e1f4b1c SQL: More tests and docs for permissions (elastic/x-pack-elasticsearch#3679)
Adds tests for what works and what doesn't when we're missing some of
SQL's required permissions.

Adds required permissions to the documentation of each SQL access
method.

relates elastic/x-pack-elasticsearch#3552

Original commit: elastic/x-pack-elasticsearch@971dabb3b4
2018-02-01 17:20:44 -05:00
Martijn van Groningen ee834026d1 Merge changes related to moving persistent tasks to elasticsearch repo into master branch.
Original commit: elastic/x-pack-elasticsearch@fb61e4853c
2018-02-01 15:31:57 +01:00
Lee Hinman 1b36133988 Merge remote-tracking branch 'origin/master' into feature/sql_2
Original commit: elastic/x-pack-elasticsearch@4176773659
2018-01-31 10:49:25 -07:00
Martijn van Groningen d2e7cee41d fixed compile error
Original commit: elastic/x-pack-elasticsearch@ff02a3244c
2018-01-31 15:38:14 +01:00
Dimitris Athanasiou ed11dad855 [ML] Expand detector function shortcuts (elastic/x-pack-elasticsearch#3789)
relates elastic/x-pack-elasticsearch#3162


Original commit: elastic/x-pack-elasticsearch@a3512c6693
2018-01-31 13:11:32 +00:00
Simon Willnauer 570411c2dc Remove all tribe related code, comments and documentation (elastic/x-pack-elasticsearch#3784)
Relates to elastic/elasticsearch#28443

Original commit: elastic/x-pack-elasticsearch@5c4e7fccc7
2018-01-30 20:40:46 +01:00
Nik Everett 6631fe0376 Merge branch 'master' into feature/sql_2
Original commit: elastic/x-pack-elasticsearch@ec3a82494e
2018-01-30 12:50:27 -05:00
Andy Bristol e45b738b4c [TEST] remove check for extensions executables (elastic/x-pack-elasticsearch#3778)
These executables have been removed because the old plugin
infrastructure has been replaced with SPI

Follow up from elastic/x-pack-elasticsearch#3734

Original commit: elastic/x-pack-elasticsearch@89dd1ab3b1
2018-01-30 07:42:44 -08:00
jaymode 0f6ca4716b Test: increase time to wait for the tribe node to start
See elastic/x-pack-elasticsearch#3754

Original commit: elastic/x-pack-elasticsearch@b1fb53ce8e
2018-01-30 08:13:19 -07:00
David Kyle 6c849534da [ML][TEST] Clean up jobs after SE test
Original commit: elastic/x-pack-elasticsearch@c71aafeb06
2018-01-30 14:34:36 +00:00
Tim Vernum 427da8de8b Switch SAML QA test to use a standalone cluster (elastic/x-pack-elasticsearch#3743)
Introduce a healthy dose of reality into the SAML integration tests.
Switches the test class from *Tests to *IT, and updates the build to run it during integTest

Original commit: elastic/x-pack-elasticsearch@edd2538e5f
2018-01-30 23:03:04 +11:00
David Kyle 10fae5e58f [ML][TEST] Scheduled events integration test (elastic/x-pack-elasticsearch#3555)
* Scheduled events integration test

Original commit: elastic/x-pack-elasticsearch@8181645fe5
2018-01-30 10:38:13 +00:00
Lee Hinman a789944b8d Merge remote-tracking branch 'origin/master' into feature/sql_2
Original commit: elastic/x-pack-elasticsearch@f96f80f0d0
2018-01-29 15:42:01 -07:00
Simon Willnauer 32d4e7f9d0 [TEST] Use `ensureYellowAndNoInitializingShards` in monitoring test (elastic/x-pack-elasticsearch#3768)
Monitoring creates indices concurrently to the test execution. In that
case ensureYellow might not be enough and will cause test-failures when shards
are not active etc. This change uses a new method added in core to ensure shards
are not initializing anymore.

relates elastic/x-pack-elasticsearch#2672
Relates to elastic/elasticsearch#28416

Original commit: elastic/x-pack-elasticsearch@661e87f2ee
2018-01-29 20:46:35 +01:00
Costin Leau 07658cc04f SQL: replace JDBC transport meta columns with SQL call (elastic/x-pack-elasticsearch#3740)
Replace meta column endpoint with SYS COLUMNS command

Original commit: elastic/x-pack-elasticsearch@819874bc5b
2018-01-29 19:18:24 +02:00
David Kyle b6bab14fd4 Revert "Mute ML rolling upgrade tests. Awaits fix elastic/x-pack-elasticsearch#1760"
This reverts commit elastic/x-pack-elasticsearch@deaf060818.

Original commit: elastic/x-pack-elasticsearch@2eece7d27e
2018-01-29 13:24:07 +00:00
Dimitris Athanasiou 664fcccb8b [ML][TEST] Add integ test for categorical rule (elastic/x-pack-elasticsearch#3739)
Original commit: elastic/x-pack-elasticsearch@975a10f810
2018-01-29 11:45:04 +00:00
Costin Leau e929d16d76 SQL: Replace ListTable transport with SQL command (elastic/x-pack-elasticsearch#3675)
Introduce system commands as alternative to meta HTTP endpoints
Pass in cluster name
Use 'BASE TABLE' instead of 'INDEX' when describing a table to stick
with the SQL terminology

Original commit: elastic/x-pack-elasticsearch@600312b8f7
2018-01-27 11:54:19 +02:00
Ryan Ernst 5a86450df7 Build: Replace references to x-pack-elasticsearch paths with helper methods (elastic/x-pack-elasticsearch#3748)
In order to more easily integrate xpack once it moves into the
elasticsearch repo, references to the existing x-pack-elasticsearch need
to be reduced. This commit introduces a few helper "methods" available
to any project within xpack (through gradle project extension
properties, as closures). All refeerences to project paths now use these
helper methods, except for those pertaining to bwc, which will be
handled in a followup.

Original commit: elastic/x-pack-elasticsearch@850668744c
2018-01-26 21:48:30 -08:00
Dimitrios Athanasiou b2f0745c7b [ML][TEST] Use condition_type instead of type in rule bwc test
Original commit: elastic/x-pack-elasticsearch@f49fe28b4e
2018-01-26 18:46:11 +00:00
Dimitrios Athanasiou 621f4f20b0 [ML][TEST] Add BWC tests for rules syntax change
Original commit: elastic/x-pack-elasticsearch@805d390417
2018-01-26 17:37:08 +00:00
Simon Willnauer 730e7075ab Remove XPackExtension in favor of SecurityExtensions (elastic/x-pack-elasticsearch#3734)
This change removes the XPackExtension mechanism in favor of
SecurityExtension that can be loaded via SPI and doesn't need
another (duplicate) plugin infrastructure

Original commit: elastic/x-pack-elasticsearch@f39e62a040
2018-01-26 16:14:11 +01:00
David Kyle 2356830782 [ML][TEST] Disable monitoring for ML QA tests (elastic/x-pack-elasticsearch#3751)
Original commit: elastic/x-pack-elasticsearch@4c3192e964
2018-01-26 10:10:52 +00:00
Ryan Ernst 3e24ea3ed2 Test: Add internalClusterTest to xpack modules (elastic/x-pack-elasticsearch#3730)
This commit reenables running ITs in xpack by adding an internalClusterTest to xpack modules that contain ESIntegTestCase tests. The new task allows us to run these independently of rest integ tests, which are disabled for xpack modules because installing the bundled plugins directly is not quite the same as installing via the meta plugin. Some tests (ML) are moved to their own qa module to accommodate the need for a real cluster. A couple tests (monitoring and upgrade) have been marked as AwaitsFix.

Commits that have been folded into this commit:
* Move ML IT tests to qa/ml-native-tests
* Add internalClusterTest task and disable rest integ tests for xpack
modules. Also tweak ML tests and get upgrade tests working
* Adding the keystore and security back to the ml native tests
* Fixing native integ test
* Fix last ML test, add awaits fix to monitoring and upgrade tests
* cleanup PR
* fix checkstyle

Original commit: elastic/x-pack-elasticsearch@3c0ed6fd3b
2018-01-25 10:38:34 +01:00
Tanguy Leroux 7c90f874d7 Update x-pack packaging tests to work with meta plugins (elastic/x-pack-elasticsearch#3683)
relates elastic/x-pack-elasticsearch#3652

Original commit: elastic/x-pack-elasticsearch@e177b742f9
2018-01-25 10:20:52 +01:00
jaymode b456885b39 Test: increase time to wait for tribe tests with security
There have been some failures in CI due to the tribe tests with security taking too long for the
cluster to form and the expected number of node to join. In the thread dumps of the failure, it
can be seen that a node is still initializing and is in a method that could take a decent amount
of time on slow machines (TokenService#computeSecretKey). This commit increases the wait time to
allow for the node to startup and join.

Original commit: elastic/x-pack-elasticsearch@2bf4c96d8f
2018-01-24 16:30:02 -07:00
jaymode 2c46002c00 Test: get the rest-api-spec from the correct location
Original commit: elastic/x-pack-elasticsearch@a3f3c45213
2018-01-24 10:39:43 -07:00
Costin Leau 3f4c789993 Merge branch 'master' into feature/sql_2
Original commit: elastic/x-pack-elasticsearch@348f0468b0
2018-01-24 19:19:51 +02:00
jaymode 8059099697 Test: re-enable the rolling-upgrade tests after the refresh token backport
Original commit: elastic/x-pack-elasticsearch@a39736a4c0
2018-01-24 10:09:47 -07:00
Igor Motov de15f7123b Merge remote-tracking branch 'elastic/master' into feature/sql_2
Original commit: elastic/x-pack-elasticsearch@bbabfb1328
2018-01-23 15:41:59 -05:00
Jason Tedor c0790d6a49 Move x-pack-core to core package (elastic/x-pack-elasticsearch#3678)
This commit moves the source file in x-pack-core to a org.elasticsearch.xpack.core package. This is to prevent issues where we have compile-time success reaching through packages that will cross module boundaries at runtime (due to being in different classloaders). By moving these to a separate package, we have compile-time safety. Follow-ups can consider build time checking that only this package is defined in x-pack-core, or sealing x-pack-core until modules arrive for us.

Original commit: elastic/x-pack-elasticsearch@232e156e0e
2018-01-23 12:43:58 -06:00
Igor Motov 2be9cabbe9 Merge remote-tracking branch 'elastic/master' into feature/sql_2
Original commit: elastic/x-pack-elasticsearch@d6451b0f6b
2018-01-23 12:53:13 -05:00
Nik Everett 52f7ba8c5d SQL: Docs for basic aggregations
Adds documentation for basic aggregate functions supported by
Elasticsearch SQL.

Relates to elastic/x-pack-elasticsearch#2898

Original commit: elastic/x-pack-elasticsearch@ddc71165f2
2018-01-23 11:32:21 -05:00
Simon Willnauer 63c0e288af Ensure we protect Collections obtained from scripts from self-referencing (elastic/x-pack-elasticsearch#3681)
Self referencing maps can cause SOE if they are iterated ie. in their toString methods. This chance adds some protected to the usage of those collections.
see elastic/elasticsearch#28335

Original commit: elastic/x-pack-elasticsearch@c4f1089c74
2018-01-23 17:02:37 +01:00
Nik Everett d2ea36416b SQL: Expose WEEK function and document datetime functions (elastic/x-pack-elasticsearch#3680)
Adds documentation for all of the date time functions using the new
cli-like format extracted from the csv spec. In the process of doing
this I noticed that the `WEEK` function isn't exposed as a function.
This exposes it for consistency.

Relates to elastic/x-pack-elasticsearch#2898

Original commit: elastic/x-pack-elasticsearch@0459b24cb9
2018-01-23 07:11:43 -05:00
Simon Willnauer c70c1c7e98 Expose XPackExtensions via SPI (elastic/x-pack-elasticsearch#3530)
This change adds SPI loading for XPackExtensions that allows to extend
XPack via an ordinary plugin. This can co-exist with the existin
extension mechanism for the time being.

Original commit: elastic/x-pack-elasticsearch@bf02b56dee
2018-01-23 13:05:39 +01:00
Igor Motov 6330209e7d Merge remote-tracking branch 'elastic/master' into feature/sql_2
Original commit: elastic/x-pack-elasticsearch@95cbd93803
2018-01-22 16:03:56 -05:00
Dimitris Athanasiou cc432aee34 [ML] Check calendar exists before removing job (elastic/x-pack-elasticsearch#3661)
Also, removes check for whether a job-to-remove exists
and replaces it with a check of whether a job-to-remove
is already present in the calendar. This allows to
remove a job that may no longer exists and it improves
feedback for the case that an existing job is removed from
a calendar that doesn't contain it.

relates elastic/x-pack-elasticsearch#3620

Original commit: elastic/x-pack-elasticsearch@3ea39be1b6
2018-01-22 13:49:38 +00:00
Dimitris Athanasiou f3282b559f [ML] Check calendar exists before deleting event (elastic/x-pack-elasticsearch#3659)
Solves the event part of elastic/x-pack-elasticsearch#3620

Original commit: elastic/x-pack-elasticsearch@c79ca85c6e
2018-01-22 13:02:42 +00:00
Jason Tedor 928b6a6e04 Fix compilation in audit-tests
This commit fixes compilation in audit-tests which was broken after some
stray classes were moved out of plugin into specific plugin directories.

Original commit: elastic/x-pack-elasticsearch@8bfd8cfceb
2018-01-22 06:00:23 -05:00
Ryan Ernst ac44ee8257 Build: Move test files from plugin/src to appropriate other projects (elastic/x-pack-elasticsearch#3653)
This commit moves mostly security tests that were left in plugin/src. It
also moves the CompositeTestingXPackPlugin into tribe license tests qa
project, which was the only remaining use. This class needs to be
removed, as it has unavoidable problems with dependency conflicts (ie
guava) between security and watcher, which it pulls in both into the
test classpath.

Original commit: elastic/x-pack-elasticsearch@756209e010
2018-01-21 14:08:47 -05:00
Jason Tedor 377bb53cca Fix format of third party audit in saml-idp-tests
The formatting here was off since it contains tabs and tabs are evil.

Original commit: elastic/x-pack-elasticsearch@fadbb02157
2018-01-20 19:53:26 -05:00
Tim Vernum 47213f5675 [Security] Add SAML authentication support (elastic/x-pack-elasticsearch#3646)
Introduces:

- SAML Realm
- REST & Transport actions to support SAML single signon / signout
- Tests for above
- More XML than you ever wanted to see.

Original commit: elastic/x-pack-elasticsearch@b0fe7bb652
2018-01-21 08:43:00 +10:00
Ryan Ernst 0bc496ae04 add pattern excludes for binary files
Original commit: elastic/x-pack-elasticsearch@f1077e35f8
2018-01-19 22:05:01 -08:00
Michael Basnight f3ec4a5208 Split up xpack plugins into their own modules (elastic/x-pack-elasticsearch#3643)
Thanks to some great work by a bunch of amazing people, the chuck norris xpack split is a go!

Original commit: elastic/x-pack-elasticsearch@dad98e28f4
2018-01-19 23:30:17 -06:00
Nik Everett 51a6285ba1 SQL: Allow CSV tests to look more like the CLI (elastic/x-pack-elasticsearch#3640)
This allows CSV tests to include a line between the header and the
values that is ignored by the test framework. This optional line can be
added to the tests to make them a little easier to read which is useful
when they are included in the documentation. As a side effect they also
closely mimick the output of the CLI. To the point where you can copy
directly from the CLI and paste into the CSV tests.

Example:
```
constantYear
// tag::year
SELECT YEAR(CAST('2018-01-19T10:23:27Z' AS TIMESTAMP)) as year;
     year
---------------
2018
// end::year
;
```

This can be extracted with a construct like this in the docs:
```
["source","sql",subs="attributes,callouts,macros"]
--------------------------------------------------
include-tagged::{sql-specs}/datetime.csv-spec[year]
--------------------------------------------------
```

Which makes documentation that looks like this:
```
SELECT YEAR(CAST('2018-01-19T10:23:27Z' AS TIMESTAMP)) as year;
     year
---------------
2018
```

Which is fairly nice.

Original commit: elastic/x-pack-elasticsearch@8c10b5cb10
2018-01-19 16:24:42 -05:00
Nik Everett b165f1c71e SQL: Fix constant folding of datetime functions (elastic/x-pack-elasticsearch#3637)
I went to write some docs for datetime functions that look like:
```
SELECT YEAR(CAST('2018-01-19T10:23:27Z' AS TIMESTAMP)) as year;

  year
2018

```
because I figured they'd be pretty easy to read because they didn't
require any knowledge of a data set. But it turns out that constant
folding doesn't work properly for date time functions because they don't
actually apply the extraction.

Original commit: elastic/x-pack-elasticsearch@aa9c66b2c7
2018-01-19 15:11:37 -05:00
Dimitris Athanasiou 21f692c02b [ML] Further validate calendar_id and add calendar description (elastic/x-pack-elasticsearch#3624)
relates elastic/x-pack-elasticsearch#3595

Original commit: elastic/x-pack-elasticsearch@fade977361
2018-01-19 10:44:39 +00:00
Lee Hinman 5818986775 Merge remote-tracking branch 'origin/master' into feature/sql_2
Original commit: elastic/x-pack-elasticsearch@8475419a00
2018-01-18 16:51:56 -07:00