Commit Graph

7841 Commits

Author SHA1 Message Date
Yannick Welsch 8ba08ccea9 [TEST] Create checkout directory on Windows before trimming path
The getShortPathName method can only be used on a directory that actually exists, otherwise it will fail with a cryptic message.

Original commit: elastic/x-pack-elasticsearch@44552dcfc8
2017-12-18 18:13:54 +01:00
Jason Tedor c92a216517 Revert "Fix elasticsearch-cli dependency"
This reverts commit elastic/x-pack-elasticsearch@68026168da.

Relates elastic/x-pack-elasticsearch#3349

Original commit: elastic/x-pack-elasticsearch@2c345ee5a4
2017-12-17 11:54:30 -05:00
Jason Tedor d97bfac8fc Fix elasticsearch-cli dependency
The API JAR POM picks up the wrong artifact name for the :core:cli
dependency, using the project name instead of the archive base
name. This commit fixes this issue by explicitly referring to the
artifact as a runtime dependency. With this change, the correct artifact
name is used in the API JAR POM.

Relates elastic/x-pack-elasticsearch#3336

Original commit: elastic/x-pack-elasticsearch@68026168da
2017-12-17 11:24:31 -05:00
Chris Earle 01f5318642 [Watcher] Use index.auto_expand_replicas: 0-1 (elastic/x-pack-elasticsearch#3284)
This changes the default behavior of .watch* indices to be green on one-node clusters, instead of constantly yellow.

Original commit: elastic/x-pack-elasticsearch@cdaee7cd72
2017-12-15 18:32:34 -05:00
Lisa Cawley 02129beec4 [DOCS] Reformatted machine learning overview (elastic/x-pack-elasticsearch#3346)
* [DOCS] Reformatted machine learning overview

* [DOCS] Added intro ML screenshot

Original commit: elastic/x-pack-elasticsearch@b6189000e0
2017-12-15 15:05:21 -08:00
Lee Hinman a8e9272994 SQL: Add remaining matrix aggregations (elastic/x-pack-elasticsearch#3330)
* Add remaining matrix aggregations

This adds the remaining [matrix aggregations](https://www.elastic.co/guide/en/elasticsearch/reference/6.1/search-aggregations-matrix-stats-aggregation.html).

These aggregations aren't currently implemented due to the inter-plugin
communication not being set up, so they return "innerkey/matrix stats not
handled (yet)".

For matrix aggs that share a name with an existing aggregation (like 'count'),
they have be prefixed with "matrix_", so, "matrix_count", "matrix_mean", and
"matrix_variance".

Relates to elastic/x-pack-elasticsearch#2885

* Return HTTP 400 for innerkey/matrix stats aggs

* Add integration test for unimplemented matrix aggs

Original commit: elastic/x-pack-elasticsearch@34459c59aa
2017-12-15 14:33:58 -07:00
Lisa Cawley cd245c8e86 [DOCS] Added xpack.ml.node_concurrent_job_allocations setting (elastic/x-pack-elasticsearch#3327)
* [DOCS] Added concurrent ML job setting

* [DOCS] Re-ordered ML settings

* [DOCS] Clarified concurrent job allocation setting

Original commit: elastic/x-pack-elasticsearch@cb2d501333
2017-12-15 11:19:11 -08:00
lcawley ab737982bb Merge remote-tracking branch 'upstream/master'
Original commit: elastic/x-pack-elasticsearch@5b94f9472a
2017-12-15 11:07:02 -08:00
lcawley d5e03f9bff [DOCS] Fixed troubleshooting titles
Original commit: elastic/x-pack-elasticsearch@4338580de6
2017-12-15 11:05:20 -08:00
Alexander Reelsen 5f8a0711f5 Watcher: Set index and type dynamically in index action (elastic/x-pack-elasticsearch#3264)
The index action allowed to set the id of a document dynamically,
however this was not allowed for the index or the type.

If a user wants to execute a search, modify the found documents and
index them back, then this would only work across a single index and a
single type. This change allows the watch writer to just take a search
result, read index and type out of that and configure this as part of
the index action.

On top of that the integration tests have been changed to become fast
running unit tests.

Original commit: elastic/x-pack-elasticsearch@640b085dd4
2017-12-15 16:59:29 +01:00
Lee Hinman 632c3e8238 SQL: Fix running `gradle run` from top-level directory (elastic/x-pack-elasticsearch#3329)
When running `gradle run` at the top level (at least with Gradle 4.4) it
attempts to run multiple instances of the server, causing the run to fail since
they can't both bind to 9200/9300.

This renames the tasks for the `qa` directories to be `runqa` and the task for
the `cli` directory to be `runcli`.

Original commit: elastic/x-pack-elasticsearch@734ab8e132
2017-12-15 08:42:18 -07:00
Alexander Reelsen f518501df4 Tests: Ensure that watcher is started in HipchatServiceTests
One of those tests requires watcher to be started, so a proper
assertBusy() block has been added to this tests.

relates elastic/x-pack-elasticsearch#3324

Original commit: elastic/x-pack-elasticsearch@324830316f
2017-12-15 16:05:43 +01:00
David Roberts b81c90d6fc [DOCS] Explain ML datafeed run-as integration/limitations (elastic/x-pack-elasticsearch#3311)
Docs for elastic/x-pack-elasticsearch#3254

Original commit: elastic/x-pack-elasticsearch@eec3c7ccce
2017-12-15 14:41:10 +00:00
Alexander Reelsen 758433a0fa Monitoring: Ensure all monitoring watches filter by timestamp (elastic/x-pack-elasticsearch#3238)
Only the Logstash and Kibana version mismatch watches contain a time
filter, the others are only sorting by timestamp. In combination with
searching in all `.monitoring-es-*` indices, this is IMO pretty resource
intensive, as we cannot exit early on any search request.

This commit adds time based filters to remaining three watches, using
the same range than the other two.

Original commit: elastic/x-pack-elasticsearch@3eb6bf0de2
2017-12-15 15:23:57 +01:00
Shaunak Kashyap 78f7c0e27a Fix license messaging for Logstash functionality (elastic/x-pack-elasticsearch#3268)
* Fix license messaging for Logstash functionality

With a Basic license, users are still able to perform CRUD operations on the `.logstash` index, therefore manage their Logstash pipelines. However, Logstash itself will not pick up any changes from this index and act on them. With an expired license Logstash functionality continues to operate as normal.

* Fixing messages after feedback

* Removing extraneous tabs at end of line

* Fixing typo

Original commit: elastic/x-pack-elasticsearch@bc069cf00f
2017-12-15 05:56:01 -08:00
Igor Motov 49a036cc5f SQL: Bring SQL Version in line with ES Version (elastic/x-pack-elasticsearch#3308)
It also makes it possible to use the Version class to parse the version that we get from Elasticsearch.

Original commit: elastic/x-pack-elasticsearch@73a3268b12
2017-12-14 22:01:19 -05:00
lcawley 4f1866db69 [DOCS] Updated titles of ML APIs
Original commit: elastic/x-pack-elasticsearch@3b3d856a89
2017-12-14 10:52:49 -08:00
Costin Leau f2792b8d93 Update cli to work on windows
Original commit: elastic/x-pack-elasticsearch@84f6ba3c1f
2017-12-14 18:53:21 +02:00
Nik Everett 28f1107dad SQL: Fix test for UnresolvedRelation
`UnresolvedRelation`'s equality test would sometimes not properly mutate
the object under test. This is because `ESTestCase#randomValueOtherThan`
will only run its provider one time if passed `null` for the "other
than" value.

Original commit: elastic/x-pack-elasticsearch@7b13e8dc98
2017-12-14 10:45:59 -05:00
Nik Everett 2268e592c6 Merge branch 'master' into feature/sql_2
Original commit: elastic/x-pack-elasticsearch@ded26a3834
2017-12-14 10:31:51 -05:00
Costin Leau 624a1530c0 SQL: Fix checkstyle / add missing hashCode (elastic/x-pack-elasticsearch#3323)
* Fix checkstyle / add missing hashCode
* Improve formatting

Original commit: elastic/x-pack-elasticsearch@a3ad2793bb
2017-12-14 17:17:26 +02:00
Nik Everett 59e6d34c29 SQL: Bundle the CLI into x-pack (elastic/x-pack-elasticsearch#3316)
This adds:

* The CLI jar itself into the `bin`. It is an executable jar.
* A shell and bat script to start the CLI. This isn't strictly required but folks will appreciate the consistency.
* Basic packaging tests for the CLI.

Relates to elastic/x-pack-elasticsearch#2979

Original commit: elastic/x-pack-elasticsearch@158f70a530
2017-12-14 09:57:03 -05:00
Nik Everett f5af60c7cf SQL: Fix error message on bad index (elastic/x-pack-elasticsearch#3312)
Fixes the error message that SQL produces when it sees unsupported
indexes. It was always returning all broken indexes as "unknown" even
though we have much better error messages. It was just throwing them
away.

I caught this originally when backporting to 6.x where we had a test
that we produced a useful error message when the user attempted to run
SQL on an index with more than one type. We couldn't run that in the
feature/sql branch because it is inside the full cluster restart tests
and the "old" version of Elasticsearch used in those tests in
feature/sql is 6.x which doesn't allow indexes with multiple types. When
I backported to 6.x the test failed because it hadn't been run before.

In addition to fixing that test and the problem, this adds another test
that will reveal the problem when run in the feature/sql and master
branch.

Original commit: elastic/x-pack-elasticsearch@c7b787baee
2017-12-14 09:56:36 -05:00
Colin Goodheart-Smithe 51b80aa998 Fixes test to support BytesSizeValue changes (elastic/x-pack-elasticsearch#3321)
Original commit: elastic/x-pack-elasticsearch@0bc8b0e847
2017-12-14 12:17:25 +00:00
Ioannis Kakavas d24921ea60 Check for existing x-pack directory when running the `users` CLI tool (elastic/x-pack-elasticsearch#3271)
Verify that the configuration directory `$ES_PATH_CONF/x-pack`
exists before attempting to run any of the `users` commands, and 
return a helpful error message to the user if it doesn't.

Original commit: elastic/x-pack-elasticsearch@6d663b6654
2017-12-14 13:45:53 +02:00
Chris Earle 876fc5612a [Monitoring] Use index.auto_expand_replicas: 0-1 (elastic/x-pack-elasticsearch#3282)
This changes the default behavior of .monitoring indices to be green on one-node clusters, instead of constantly yellow.

Note: This only impacts .monitoring* indices. Watcher indices currently still require a replica.

Original commit: elastic/x-pack-elasticsearch@6eb8a48a9f
2017-12-13 13:46:31 -05:00
Nik Everett e170021037 Revert "Revert "Fix version parsing in SQL""
This reverts commit elastic/x-pack-elasticsearch@a3ef6b43f9.

We'll merge this when we're good and ready.

Original commit: elastic/x-pack-elasticsearch@06d1e3311d
2017-12-13 10:19:54 -05:00
Costin Leau cdfe0f1f5a Revert "Revert "Merge branch 'feature/sql'""
This reverts commit elastic/x-pack-elasticsearch@cc79e19911.

We'll merge this when we're good and ready.

Original commit: elastic/x-pack-elasticsearch@b3ef4f2836
2017-12-13 10:19:31 -05:00
Costin Leau 2e60e831c0 Revert "Merge branch 'feature/sql'"
This reverts commit elastic/x-pack-elasticsearch@2b3f7697a5, reversing
changes made to elastic/x-pack-elasticsearch@b79f16673c.

We're backing out all of SQL.

Original commit: elastic/x-pack-elasticsearch@cc79e19911
2017-12-13 09:33:13 -05:00
Nik Everett 5a6cfafed2 Revert "Remove SQL ci script changes"
This reverts commit elastic/x-pack-elasticsearch@afb4347465.

We're backing out all of SQL.

Original commit: elastic/x-pack-elasticsearch@566729702f
2017-12-13 09:32:58 -05:00
Nik Everett d33c537711 Revert "Fix version parsing in SQL"
This reverts commit elastic/x-pack-elasticsearch@47020ed3aa.

We're backing out all of SQL.

Original commit: elastic/x-pack-elasticsearch@a3ef6b43f9
2017-12-13 09:32:44 -05:00
Jason Tedor 6ceee0be8f Fix version parsing in SQL
I have no idea why this logic was only looking at snapshot JARs (what is
suppose to happen in production?). This breaks the release tests though
which we really need to have stable. This commit fixes this by also
considering non-snapshot JARs.

Original commit: elastic/x-pack-elasticsearch@47020ed3aa
2017-12-12 22:15:59 -05:00
Lee Hinman ffeebc366d Add sql folder for upcoming migration (elastic/x-pack-elasticsearch#3307)
* Add sql folder for upcoming migration

This is the skeleton folder for SQL now that it's been merged.

Relates to elastic/x-pack-elasticsearch#3233

Original commit: elastic/x-pack-elasticsearch@c46548bd6c
2017-12-12 16:48:43 -07:00
Lee Hinman 86a04acb01 Rename folder x-pack-core -> core (elastic/x-pack-elasticsearch#3305)
* Rename folder x-pack-core -> core

The jar remains 'x-pack-core-*.jar'

* Put group in top-level build.gradle instead of plugin/core/build.gradle

Original commit: elastic/x-pack-elasticsearch@b23452fa55
2017-12-12 13:23:29 -07:00
Nik Everett c78328853f Remove SQL ci script changes
These were useful when SQL lived on its own branch but aren't now that
we've merged SQL to master.

Original commit: elastic/x-pack-elasticsearch@afb4347465
2017-12-12 14:49:46 -05:00
Nik Everett aef5ea6527 Merge branch 'feature/sql'
It is time.

Original commit: elastic/x-pack-elasticsearch@2b3f7697a5
2017-12-12 14:49:27 -05:00
Jay Modi d48ad6dbd6 Enhance security for alias and resize operations (elastic/x-pack-elasticsearch#3302)
This commit adds additional checks around resize operations and alias creation operations to
add an extra layer of security around these APIs.

Original commit: elastic/x-pack-elasticsearch@b79f16673c
2017-12-12 12:01:27 -07:00
Nik Everett eaa75cfdbb Merge branch 'master' into feature/sql
Original commit: elastic/x-pack-elasticsearch@b5fc2b484d
2017-12-12 11:29:23 -05:00
Nik Everett 2ca729afc2 Fix packaging tests after breaking up x-pack
The tests were failing after breaking up x-pack. This makes them pass
locally.

Original commit: elastic/x-pack-elasticsearch@af2a7866eb
2017-12-12 11:05:51 -05:00
Dimitris Athanasiou 92c40061c5 [ML] Skip index health checks for remote indices on datafeed node sel (elastic/x-pack-elasticsearch#3301)
Upon selecting a node to run a datafeed we normally check that the
data indices exists and their primaries are active. However, these
checks cannot be applied for CCS to a remote cluster. This commit
skips these checks for remote indices.

This removes the last obstacle for running CCS datafeeds.

Relates elastic/x-pack-elasticsearch#1071

Original commit: elastic/x-pack-elasticsearch@092f44feee
2017-12-12 15:12:33 +00:00
Tanguy Leroux 711254fd24 [Monitoring] Use the same Cluster State for all Collectors (elastic/x-pack-elasticsearch#3216)
This commit changes the Collectors so that they all use the 
same instance of ClusterState.

relates elastic/x-pack-elasticsearch#3156

Original commit: elastic/x-pack-elasticsearch@4f537b026c
2017-12-12 12:29:40 +01:00
David Kyle 0d46e9035c [ML] Get Filters should use executeAsyncWithOrigin (elastic/x-pack-elasticsearch#3295)
* Get Filters should use executeAsyncWithOrigin

Original commit: elastic/x-pack-elasticsearch@786c7bfd06
2017-12-12 11:15:54 +00:00
David Kyle 6113b86bdb [ML] Special events calendar CRUD endpoints (elastic/x-pack-elasticsearch#3267)
* Calendar CRUD endpoints

* Get calendars requires monitor permission

* Address review comments

* Add page params to get calendars

Original commit: elastic/x-pack-elasticsearch@badd1e6add
2017-12-12 09:21:44 +00:00
Nik Everett 6d56616d98 SQL: Ignore environment in CliFixture
If the currenct environment contains `JAVA_TOOLS` then the cli tests
will fail because they do not expect the notification about `JAVA_TOOLS`
that java prints to stdout. The clears the environment before starting
the CLI subprocess in an effort to remove the unexpected output.

Original commit: elastic/x-pack-elasticsearch@5729c1c09b
2017-12-11 16:42:52 -05:00
Nik Everett c27332be4a SQL: Fix docs build and cleanup docs index for merge
relates elastic/x-pack-elasticsearch#3084

Original commit: elastic/x-pack-elasticsearch@a64de41fbc
2017-12-11 16:39:18 -05:00
Nik Everett 2d9ce6132b SQL: Prepare docs for merge
Cleans up the index.asciidoc file so it merges cleanly with master and
builds SQL's docs.

Original commit: elastic/x-pack-elasticsearch@3c63c0bda0
2017-12-11 16:15:31 -05:00
Nik Everett 15f84840f2 SQL: Revert some security changes so NOOP merge
SQL used to have some changes to security. We've since reverted them but
we have some leftover stuff like import reordering and spacing changes.
We may as well remove them so merging SQL to master is smaller.

Original commit: elastic/x-pack-elasticsearch@c632256ddd
2017-12-11 16:06:48 -05:00
Nik Everett 236f64a70e SQL: Fix SSL for JDBC and CLI for real this time (elastic/x-pack-elasticsearch#3277)
Previously I'd added tests for JDBC and CLI that I *thought* used SSL but they didn't! I wasn't careful...

Testing changes:
* Actually enable SSL/HTTPS in the `qa:sql:security:ssl` subproject.
* Rework how `RemoteCli` handles security. This allows us to configure SSL, the keystore, and the username and password in a much less error prone way.
* Fix up JDBC tests to properly use SSL.
* Allow the `CliFixture` to specify the keystore location.
* Switch `CliFixture` and `RemoteCli` from sending the password in the connection string to filling out the prompt for it.
* Have `CliFixture` also send the keystore password when a keystore is configured.

This makes the following production code changes:
* Allow the CLI to configure the keystore location with the `-k`/`-keystore_location` parameters.
* If the keystore location is configured then the CLI will prompt for the password.
* Allow the configuration of urls starting with `https`.
* Improve the exception thrown when the URL doesn't parse by adding a suppressed exception with the original parse error, before we tried to add `http://` to the front of it.

Original commit: elastic/x-pack-elasticsearch@97fac4a3b4
2017-12-11 15:45:34 -05:00
David Roberts 249d06b256 [ML] Fix permissions for field caps in scroll data extractor
Follow up to elastic/x-pack-elasticsearch#3254

Original commit: elastic/x-pack-elasticsearch@d4df9446c0
2017-12-11 17:45:32 +00:00
Igor Motov 4bebc307c3 SQL: Add ability to close cursors (elastic/x-pack-elasticsearch#3249)
This commits adds a new end point for closing in-flight cursors, it also ensures that all cursors are properly closed by adding after test checks that ensures that we don't leave any search context open.

relates elastic/x-pack-elasticsearch#2878

Original commit: elastic/x-pack-elasticsearch@1052ea28dc
2017-12-11 11:36:02 -05:00