Commit Graph

7495 Commits

Author SHA1 Message Date
Nik Everett adae4019dd SQL: improve exception when error can't be parsed (elastic/x-pack-elasticsearch#3024)
When an error response wasn't parseable we would through fairly obscure
error messages about what is wrong with it. Great if you are fixing a
bug in Elasticsearch, no great if a proxy eats the response. This adds
the response to the error messages so you can see what was returned.

Original commit: elastic/x-pack-elasticsearch@00e542afc8
2017-11-15 15:33:53 -05:00
Nik Everett 56e4c66d1f SQL: Fix IDE build after shading (elastic/x-pack-elasticsearch#3026)
The IDEs don't participate in the shading but gradle does. So we have to
be a little more tricky about how we set up the IDE projects, sadly.

Original commit: elastic/x-pack-elasticsearch@5196756702
2017-11-15 15:33:25 -05:00
Tim Brooks d45c2d784c Update x-pack to support TcpChannel (elastic/x-pack-elasticsearch#2983)
This is related to elastic/elasticsearch#27132. This commit updates
x-pack to support signature changes introduced in that PR.

Original commit: elastic/x-pack-elasticsearch@461c96ffe6
2017-11-15 12:38:56 -07:00
lcawley 10a9334046 Merge remote-tracking branch 'upstream/master'
Original commit: elastic/x-pack-elasticsearch@7af3facb7d
2017-11-15 09:09:42 -08:00
Lisa Cawley f8497dc943 [DOCS] Added flow diagram for installation (elastic/x-pack-elasticsearch#3020)
Original commit: elastic/x-pack-elasticsearch@c7c39d9724
2017-11-15 09:08:48 -08:00
Lee Hinman 889b008298 Add "client-api-objects" dependency for xpack plugin and transport-client (elastic/x-pack-elasticsearch#2995)
* Add "client-api-objects" dependency for xpack plugin and transport-client

This adds another gradle project, "client-api-objects" which is intended to be a
common dependency so that the xpack plugin and transport-client can share the
same Request and Response objects.

Relates to elastic/x-pack-elasticsearch#2925

Original commit: elastic/x-pack-elasticsearch@a6d83299d0
2017-11-15 09:49:00 -07:00
Tim Brooks 5d47b96905 Modify x-pack to for `TransportChannel` changes (elastic/x-pack-elasticsearch#3013)
This is related to elastic/elasticsearch#elastic/x-pack-elasticsearch#27388. It modifies x-pack to
be compatible with the removal of the delegating transport channel.

Original commit: elastic/x-pack-elasticsearch@3bd7bf6773
2017-11-15 09:48:17 -07:00
lcawley 0eee69b6d3 [DOCS] Added flow diagram for installation
Original commit: elastic/x-pack-elasticsearch@048ac11a17
2017-11-15 08:27:12 -08:00
Alexander Reelsen 36105231c3 Watcher: Return useful error message when no accounts are found (elastic/x-pack-elasticsearch#2897)
When there were no accounts configured, watcher returned a cryptic
error message containing 'null' in the description. This fix returns
a more clear error message.

On top a dedicated NotificationServiceTests class was added to remove
redundant test cases in the hipchat/jira/slack unit tests, that all
basically tested NotificationService capabilties.

relates elastic/x-pack-elasticsearch#2666

Original commit: elastic/x-pack-elasticsearch@45d0d1df31
2017-11-15 11:51:23 +01:00
Alexander Reelsen f286f9b0f2 Watcher: Remove unused test helper class
Original commit: elastic/x-pack-elasticsearch@09f66a5f52
2017-11-15 11:05:59 +01:00
David Kyle afec0fd6c9 [ML] [DOCS] Clean up ML jobs after each docs test (elastic/x-pack-elasticsearch#2825)
Original commit: elastic/x-pack-elasticsearch@5ec4a4a752
2017-11-15 09:45:57 +00:00
Clinton Gormley d833af2046 Rest spec fixes (elastic/x-pack-elasticsearch#2965)
* Rename REST spec xpack.deprecation.info to xpack.migration.deprecations

* Fixed parameter-type naming in xpack.ml.get_model_snapshots

* Fixed QS multi-cluster search test to use cluster.remote_info

Original commit: elastic/x-pack-elasticsearch@ccd35b4a6c
2017-11-15 09:33:19 +01:00
Martijn van Groningen 56e7512500 Make the persistent task status available to PersistentTasksExecutor.nodeOperation(...) method
Original commit: elastic/x-pack-elasticsearch@ce6d788e77
2017-11-15 07:54:05 +01:00
Martijn van Groningen 439830890f Move the initialization of persistent tasks from the machine learning level to xpack level.
Today persistent tasks is only usable from machine learning, but others like ccr will need to use it too.
With this change ccr and other will be able to make use of it too.

Original commit: elastic/x-pack-elasticsearch@c90f01d5f6
2017-11-15 07:44:53 +01:00
Nik Everett 89e80e0cba Teach SQL to parse Elasticsearch's standard error responses (elastic/x-pack-elasticsearch#2764)
This teaches SQL to parse Elasticsearch's standard error responses
but doesn't change SQL to general Elasticsearch's standard error responses
in all cases. That can come in a followup. We do this parsing with
jackson-core, the same dependency Elasticsearch uses for parsing
json. We shade jackson-core in the JDBC driver so that users don't have to worry about
dependency clashes. We do not do so in the CLI because it is a standalone
application.

We get a few "bonus" changes along the way:
1. We save a copy operation. Before this change responses were spooled
into memory and then parsed. After this change they are parsed directly
from the response stream.
2. We had a few classes entirely to support the spooling operation that we
no longer need: `BytesArray`, `FastByteArrayInputStream`, and
`BasicByteArrayOutputStream`.
3. SQL's `Version` was incorrectly parsing the version from the jar manifest.
We didn't notice because the test was rigged to return `UNKNOWN` because
we *were* running the test from the compiled classes directory instead of the
jar. As part of shading jackson we moved running the tests to running against
the shaded jar. Now we can actually assert that we parse the version correctly.
It turns out we weren't. So I fixed it.

Original commit: elastic/x-pack-elasticsearch@2e8f397bf4
2017-11-14 21:31:35 -05:00
Lisa Cawley 245543d5cf [DOCS] Add upgrade details about stopping ML (elastic/x-pack-elasticsearch#2982)
* [DOCS] Add upgrade details about stopping ML

* [DOCS] Addressed feedback about stopping all ML

Original commit: elastic/x-pack-elasticsearch@1897390edc
2017-11-14 18:04:21 -08:00
Costin Leau 94d0a2d1ee Polishing for handling subtleties in the JDBC behavior: (elastic/x-pack-elasticsearch#2967)
1. decouple JdbcDriver from other classes to not trigger static
initialization (this happens through JDBC service discovery)
2. reduce visibility of JdbcDriver#close so only on jar unloading it
gets triggered
3. mark 3 methods introduced in Jdbc 4.1 as unsupported (their semantics
are somewhat weird)
4. Move versioning info in one class
5. Hook Version class in both JDBC entry points to perform cp sanity
checks
6. Remove JdbcDataSource#close (DebugLog are closed when the Driver gets
unloaded by the DriverManager) as there can be multiple instances of DS
but only one for Driver known by the DriverManager

Replace Strings with constants
Properly set TZ in security tests as well
JdbcDataSource is more defensive with its internal properties
JdbcConfiguration password parameter is aligned with JDBC DriverManager
Remove usage of JdbcConnection API
Removed JdbcConnection#setTimeZone - this encourages folks to use our
private API which would tie us down.
It is somewhat limiting for folks but it has less downsides overall and
does not trip debugging (which adds a proxy unaware of this method).
Update docs
Add JdbcDataSource into the Jdbc suite

Original commit: elastic/x-pack-elasticsearch@c713665d53
2017-11-15 00:29:41 +02:00
Nik Everett 1a434636fe Merge branch 'master' into feature/sql
Original commit: elastic/x-pack-elasticsearch@4c504025ce
2017-11-14 14:30:12 -05:00
Jay Modi 4eb87bba4b Remove create_bwc_indexes.py (elastic/x-pack-elasticsearch#2989)
This commit removes the create_bwc_indexes.py script as we no longer need this script to test for
BWC with indices. The last test relying on this script was removed in elastic/x-pack-elasticsearch#2832.

relates elastic/x-pack-elasticsearch#1629

Original commit: elastic/x-pack-elasticsearch@5dc3cbc395
2017-11-14 10:08:59 -07:00
jaymode 2f8cd77349 Test: TribeWithSecurityIT should wait for security index to be writeable
This commit adds checks to the TribeWithSecurityIT tests to ensure that the security index is
writeable before making modification operations. Otherwise, we hit errors in tests that are not
always reproducible.

relates elastic/x-pack-elasticsearch#2977

Original commit: elastic/x-pack-elasticsearch@c29bdff7ae
2017-11-14 08:18:55 -07:00
Jay Modi 5888174f24 Cleanup leftover tribe references in the plugin build.gradle file (elastic/x-pack-elasticsearch#2987)
In elastic/x-pack-elasticsearch#2901, the dependency on the tribe module was removed but a few leftover references were missed
in the build.gradle file of the x-pack-elasticsearch plugin. This commit removes these leftover
references.

Original commit: elastic/x-pack-elasticsearch@03f1cae1f5
2017-11-14 08:11:21 -07:00
Dimitrios Liappis a89bfe84ba [DOCS] Split long lines in Docker TLS getting-started snippet
and add warning for Windows users not using
PowerShell (e.g. `cmd.exe`) to remove the `\` character and join
lines.

Also fix trailing whitespace character in link back to `docker.asciidoc`.

Relates elastic/x-pack-elasticsearch#2999

Original commit: elastic/x-pack-elasticsearch@fe1c5dbc11
2017-11-14 14:25:52 +02:00
Alexander Reelsen dc42887396 Watcher: Move watcher-only packages into watcher hierarchy (elastic/x-pack-elasticsearch#2933)
In order to prepare for separate source directories, this commit moves
a few packages back into the watcher namespaces. A few of them have been
moved out previously as we thought that it might make sense to have a
dedicated notification API. This wont be the case for watcher on ES
anymore, so we can safely move those back into the watcher space.

Packages affected by this move:

* org.elasticsearch.xpack.common.http
* org.elasticsearch.xpack.common.text
* org.elasticsearch.xpack.common.secret
* org.elasticsearch.xpack.common.stats
* org.elasticsearch.xpack.support
* org.elasticsearch.xpack.notification

Tests have been moved accordingly.

The class `XContentUtils` has been split into one implementation for
watcher and one for security as different methods were used.

Relates elastic/x-pack-elasticsearch#2925

Original commit: elastic/x-pack-elasticsearch@0aec64a7e2
2017-11-14 11:35:10 +01:00
Dimitris Athanasiou e00b4bd197 [ML][TEST] Improve RevertModelSnapshotIT (elastic/x-pack-elasticsearch#2954)
X-Pack side testing of elastic/machine-learning-cpp#390



Original commit: elastic/x-pack-elasticsearch@64e8ca85eb
2017-11-14 10:32:42 +00:00
Costin Leau c8eb883c47 Fix incorrect assertions and javadoc
Original commit: elastic/x-pack-elasticsearch@0e6ebc0a69
2017-11-14 09:46:26 +02:00
Costin Leau 9a0b43cd17 Fix several NOCOMMITS (elastic/x-pack-elasticsearch#2968)
* Fix several NOCOMMITS

- renamed Assert to Check to make the intent clear
- clarify esMajor/Minor inside connection (thse are actually our own
methods, not part of JDBC API)
- wire pageTimeout into Cursor#nextPage


Original commit: elastic/x-pack-elasticsearch@7626c0a44a
2017-11-14 01:08:10 +02:00
Igor Motov ea0e58f971 SQL: introduce setting to disable SQL (elastic/x-pack-elasticsearch#2966)
Adds xpack.sql.enabled setting to provide ability to disable SQL on elasticsearch nodes.

relates elastic/x-pack-elasticsearch#2872

Original commit: elastic/x-pack-elasticsearch@d13b72e9ea
2017-11-13 15:10:47 -05:00
Igor Motov 774f423d9e Merge remote-tracking branch 'elastic/master' into feature/sql
Original commit: elastic/x-pack-elasticsearch@79f79ea1c2
2017-11-13 15:09:35 -05:00
jaymode 1e30b12f97 Test: increase security index wait times
Our tests currently rely on waiting for the security index to be available in some cases and in
CI, these checks have been timing out. This commit increases the amount of time that we will wait
for the index before failing to account for slow machines.

Original commit: elastic/x-pack-elasticsearch@639dccd5cb
2017-11-13 12:32:06 -07:00
Chris Earle def8c48dcb [Monitoring] Add "apm" to "stack_stats" for Phone Home (elastic/x-pack-elasticsearch#2848)
This checks if `apm-*` indices exist in the cluster to try to determine if APM is in use on the Elasticsearch cluster.

Original commit: elastic/x-pack-elasticsearch@7f9a9a4eee
2017-11-13 14:24:07 -05:00
Dimitrios Liappis 00ccac9203 [DOCS] Fix wrapped lines in code blocks of TLS getting started guide
Relates elastic/x-pack-elasticsearch#2970

Original commit: elastic/x-pack-elasticsearch@a279e57270
2017-11-13 20:00:35 +02:00
Simon Willnauer 92f87cab14 Xpack side of elastic/elasticsearch#27161 (elastic/x-pack-elasticsearch#2851)
Original commit: elastic/x-pack-elasticsearch@31fdcdf583
2017-11-13 12:07:04 +01:00
Hendrik Muhs fd392627a0 [ML-FC] Forecast status message rebase (elastic/x-pack-elasticsearch#2936)
Re-enables persistence of the forecast request stats document and adds more fields to it. Depends on https://github.com/elastic/machine-learning-cpp/pull/382.

Original commit: elastic/x-pack-elasticsearch@b6762005c0
2017-11-13 09:03:29 +01:00
Lisa Cawley 244cfa7181 [DOCS] Added link to TLS docker content (elastic/x-pack-elasticsearch#2959)
Original commit: elastic/x-pack-elasticsearch@686836c62c
2017-11-10 14:08:46 -08:00
Igor Motov 9b96c7eb33 SQL: clean unused classes and empty close() methods (elastic/x-pack-elasticsearch#2956)
relates elastic/x-pack-elasticsearch#2876

Original commit: elastic/x-pack-elasticsearch@c32c6a0058
2017-11-10 14:22:58 -05:00
Igor Motov 3c444c4719 SQL: Fix test when random JDK TZ doesn't exist in Joda (elastic/x-pack-elasticsearch#2903)
JodaTime timezone db can be out of date compared to that of the JDK which causes the JDBC Connection to fail when the randomized tests pick a timezone that's available in the JDK but not in Joda, like SystemV/PST8. This is happening because JdbcConnection configuration is using system default timezone and tries to pass it to Elasticsearch that is using joda. This commit, explicitly sets the time zone on JdbcConnection to a time zone randomly selected from a list of timezones that are known to both JDK and Joda.

relates elastic/x-pack-elasticsearch#2812

Original commit: elastic/x-pack-elasticsearch@b02e9794a8
2017-11-10 14:16:11 -05:00
Ryan Ernst 9a2ae4b7f2 Update security policy to use versionless codebase properties (elastic/x-pack-elasticsearch#2602)
This is the xpack side of
https://github.com/elastic/elasticsearch/pull/26756

Original commit: elastic/x-pack-elasticsearch@a219f5b6c0
2017-11-10 11:00:34 -08:00
Lisa Cawley d0f767a20a [DOCS] Move docker files to x-pack repo (elastic/x-pack-elasticsearch#2942)
Original commit: elastic/x-pack-elasticsearch@b4e518b178
2017-11-10 09:39:17 -08:00
Lisa Cawley fb769be92e [DOCS] Added TLS configuration info for Docker (elastic/x-pack-elasticsearch#2939)
* [DOCS] Add docker TLS configuration info

* [DOCS] Updated layout of TLS docker page

* [DOCS] Clean up docker TLS pages

* [DOCS] Changed nesting of TLS docker info

* [DOCS] More small updates to TLS docker page

Original commit: elastic/x-pack-elasticsearch@2b0504632a
2017-11-10 09:33:56 -08:00
David Roberts 742a052619 [ML] Account for the possibility of C++ log messages being UTF-16 (elastic/x-pack-elasticsearch#2952)
On Windows, log4cxx always writes to stderr in UTF-16, and we get the
logs from C++ to Java by redirecting stderr to our named pipe.  Hence
the log handler in Java needs to tolerate the log stream it's reading
being either UTF-16 (for Windows) or UTF-8 (for other platforms).

Fixes elastic/machine-learning-cpp#385

Original commit: elastic/x-pack-elasticsearch@89237d7125
2017-11-10 15:15:10 +00:00
Nik Everett b2285ae66e Merge branch 'master' into feature/sql
Original commit: elastic/x-pack-elasticsearch@b9d07ccd0f
2017-11-10 09:34:10 -05:00
Alexander Reelsen a90cd81f99 Docs: Fix broken watcher example using multi line strings
Original commit: elastic/x-pack-elasticsearch@7398be67c4
2017-11-10 10:06:47 +01:00
Tim Vernum 039cf42fdf Extend wait time to 20s in SecurityIntegTest
This kind of sucks, because we shouldn't have to wait that long for tests to run.
But they're failing CI with some regularity, and we rely on these integration tests.

Original commit: elastic/x-pack-elasticsearch@3f4acb2a32
2017-11-10 15:20:30 +11:00
lcawley b5cb814b32 [DOCS] Add security configuration section
Original commit: elastic/x-pack-elasticsearch@ccae9a84a9
2017-11-09 14:28:56 -08:00
Tal Levy 8c489b1a98 Prevent 7.x nodes from joining cluster with un-upgraded 6.x .security indices (elastic/x-pack-elasticsearch#2921) (elastic/x-pack-elasticsearch#2940)
This is a forward-port of elastic/x-pack-elasticsearch/pull/2921.

original commit message:

Before this commit, a cluster with security enabled and backed by
native-realm user permissions allowed rolled upgrades to clusters without
upgrading the `.security` index. This resulted in the newly established
6.0 cluster not able to register the native-realm users previously established
in the `.security` index. In order to fix this, one would have to rely on file-based
users to re-configure and upgrade the `.security` index. Since this state is easily
avoidable with an upgrade, this commit rejects the joining of upgraded nodes without
upgrading the security index beforehand.

modifications:

Test with 7.x vs 6.x nodes.

Original commit: elastic/x-pack-elasticsearch@56f81bfb20
2017-11-09 12:49:59 -08:00
Lisa Cawley 0c10d82f78 [DOCS] Added setup folder (elastic/x-pack-elasticsearch#2943)
Original commit: elastic/x-pack-elasticsearch@cc7027c7ec
2017-11-09 12:32:21 -08:00
jaymode 9d85f377c7 Test: update the branch logic for BWC tests
This commit updates the logic for determining which branch to use to make it consistent with the
logic in elasticsearch. This change means that testing BWC within the same major picks the correct
branch.

Original commit: elastic/x-pack-elasticsearch@2d75d15c41
2017-11-09 13:03:40 -07:00
Costin Leau 8f55a4192c Improve grammar so keywords have priority over identifiers
Original commit: elastic/x-pack-elasticsearch@8222e30060
2017-11-09 21:40:09 +02:00
Chris Earle efb5b8827b [Monitoring] Add Rolling Upgrade Tests (elastic/x-pack-elasticsearch#2832)
This adds a rolling upgrade test for X-Pack monitoring. It works by using the `_xpack/monitoring/_bulk` endpoint to send arbitrary data, then verify that it exists.

This forces a few things to happen, thereby testing the behavior: 

1. The templates must exist.
2. The elected master node must be "ready" to work (hence the first
point).
3. The same "system_api_version" is accepted by every version of ES.

Original commit: elastic/x-pack-elasticsearch@012e5738bb
2017-11-09 12:49:37 -05:00
Costin Leau af453a3fa4 Update docs structure
Original commit: elastic/x-pack-elasticsearch@78277df7fb
2017-11-09 18:45:49 +02:00