Commit Graph

7678 Commits

Author SHA1 Message Date
Lisa Cawley 42f9a990d1 [DOCS] Split TLS instructions for HTTP and transport layers (elastic/x-pack-elasticsearch#3895)
Original commit: elastic/x-pack-elasticsearch@77fe30f7d3
2018-02-15 11:41:01 -08:00
Andy Bristol b042afdfdc [TEST] AwaitsFix JdbcCsvSpecIT
For elastic/x-pack-elasticsearch#3960

Original commit: elastic/x-pack-elasticsearch@c0c1ff7d07
2018-02-15 11:25:57 -08:00
Andy Bristol 3eeb38762a [TEST] packaging: enable trial license
x-pack used to generate a trial platinum license on startup, but now it
generates a basic license. Configure security tests to use a trial
license at startup

Original commit: elastic/x-pack-elasticsearch@29382fe5ce
2018-02-15 10:39:05 -08:00
David Roberts 54b0fca88e [TEST] Update strings in ML categorization test (elastic/x-pack-elasticsearch#3959)
This makes the samples match the C++ code

Original commit: elastic/x-pack-elasticsearch@54b71f9299
2018-02-15 16:41:16 +00:00
Colin Goodheart-Smithe 2994dd15be [TEST] mute failing watcher with security yaml test
Original commit: elastic/x-pack-elasticsearch@034396693e
2018-02-15 16:02:22 +00:00
Costin Leau f04036a1bd SQL: update percentile tests to reflect the t-digest upgrade
The commit to t-digest 3.2 elastic/x-pack-elasticsearch#28305, changed the results of percentiles
(more information https://github.com/elastic/elasticsearch/pull/28305
#issuecomment-362542140)
This commit updates the tests accordingly.

Original commit: elastic/x-pack-elasticsearch@f2351ce747
2018-02-15 17:44:03 +02:00
Tim Vernum bad812d920 [TEST] Silence LocalExporterIntegTests.testExport
See: elastic/x-pack-elasticsearch#3954, elastic/x-pack-elasticsearch#3910

Original commit: elastic/x-pack-elasticsearch@a298c21579
2018-02-15 18:04:20 +11:00
Tim Brooks d45f28339b Support configuring license type for gradle run (elastic/x-pack-elasticsearch#3942)
This commit is related to elastic/x-pack-elasticsearch#3877. It allows the passing of a system
property to gradle run indicating whether a `basic` or `trial` license
should be generated. If the `trial` license is indicated, platinum
features will be enabled.

Original commit: elastic/x-pack-elasticsearch@b2b89a4606
2018-02-14 19:34:55 -07:00
Chris Earle 176411e55e [Monitoring] Ignore _bulk if Collection is Disabled (elastic/x-pack-elasticsearch#3910)
This blocks incoming requests from Kibana, Logstash, and Beats when X-Pack monitoring is effectively disabled by setting `xpack.monitoring.collection.interval: -1`.

Original commit: elastic/x-pack-elasticsearch@016a9472f1
2018-02-14 16:24:03 -05:00
Tanguy Leroux 00dec27d9f SQL: Relax RemoteFailure parsing so that it also parses error metadata (elastic/x-pack-elasticsearch#3938)
The current parsing of errors in the RemoteFailure class is strict and
fails on any field that is not one of [caused_by, reason, root_cause,
stack_trace, type]. Sadly some exceptions adds more headers or
metadata when they are printed as XContent and such fields can't be
easily ignored at parsing time.

This commit changes the RemoteFailure.parseFromResponse() method so that
it parses errors using the same behavior as the high level rest client:
it parses any unknown field as a metadata if it's string value or an
array of string and just ignores and skips everything else without
throwing an exception.

Original commit: elastic/x-pack-elasticsearch@1348706807
2018-02-14 21:05:23 +01:00
Igor Motov 89f15ed9d3 Tests: suppress testIndexUpgradeInfoLicense on release builds (elastic/x-pack-elasticsearch#3930)
The testIndexUpgradeInfoLicense test is generating a license, which
fails on release builds because they are using production public keys
for license verification.

relates elastic/x-pack-elasticsearch#3921

Original commit: elastic/x-pack-elasticsearch@e1c37bfd91
2018-02-14 13:56:40 -05:00
Lisa Cawley 22c3060fd6 [DOCS] Add ML limitation and troubleshooting (elastic/x-pack-elasticsearch#3929)
Original commit: elastic/x-pack-elasticsearch@bfabcbb456
2018-02-14 08:58:29 -08:00
Lee Hinman affc9e3563 Move more XContent.createParser calls to non-deprecated version (elastic/x-pack-elasticsearch#3928)
Part 2

This moves more of the callers to pass in the DeprecationHandler.

Relates to elastic/x-pack-elasticsearch#28504

Original commit: elastic/x-pack-elasticsearch@e0b52bf050
2018-02-14 09:41:26 -07:00
Lee Hinman e8ef20b219 Move more XContent.createParser calls to non-deprecated version (elastic/x-pack-elasticsearch#3927)
* Move more XContent.createParser calls to non-deprecated version

This moves more of the callers to pass in the DeprecationHandler.

Relates to https://github.com/elastic/elasticsearch/issues/28504

* Mockito ಠ_ಠ

Original commit: elastic/x-pack-elasticsearch@81cb99ba52
2018-02-14 09:24:47 -07:00
David Kyle 0023572acb [ML] Fix test logic in MlJobIT::testDelete_multipleRequest (elastic/x-pack-elasticsearch#3937)
Original commit: elastic/x-pack-elasticsearch@a9a457e286
2018-02-14 15:09:31 +00:00
Tim Vernum e7e3584560 Remove Issuer/Serial from AKI in certutil/certgen (elastic/x-pack-elasticsearch#3931)
The Issuer and Serial number are optional and can cause problems with
chain validation when using OpenSSL based tools

relates elastic/x-pack-elasticsearch#3915

Original commit: elastic/x-pack-elasticsearch@defcdd934f
2018-02-14 23:15:11 +11:00
Dimitris Athanasiou 245a063492 [ML][TEST] Rewrite RestoreModelSnapshotIT to make more robust (elastic/x-pack-elasticsearch#3925)
Now that the forecast API is available, we can use it to significantly
simplify this test. The test is rewritten to leverage the contract
that when a forecast is requested to a job without state it fails.

relates elastic/x-pack-elasticsearch#3909

Original commit: elastic/x-pack-elasticsearch@978ae352c5
2018-02-14 12:09:05 +00:00
Costin Leau a5b129c10c SQL: Introduce the catalog concept inside the grammar (elastic/x-pack-elasticsearch#3911)
Indices can now specify the catalog (the cluster) in which they belong.
This is a first step in supporting both CCS and BI tools which tend to
be explicit

Original commit: elastic/x-pack-elasticsearch@0b4633facc
2018-02-14 12:30:32 +02:00
Alexander Reelsen 742c7001c9 Watcher: Ensure watcher service calls are properly caught (elastic/x-pack-elasticsearch#3906)
In order to prevent exceptions to bubble up to the thread pool exception
handler, this properly wraps all the calls for the watcher service
within an executor into an AbstractRunnable to catch and log a possible
exception.

relates elastic/x-pack-elasticsearch#3854

Original commit: elastic/x-pack-elasticsearch@c0b39e6b5b
2018-02-14 10:45:29 +01:00
Adrien Grand 48847720e8 Disable IndexUpgradeIT.testIndexUpgradeInfoOnEmptyCluster too.
Relates elastic/x-pack-elasticsearch#3921

Original commit: elastic/x-pack-elasticsearch@88e66c5564
2018-02-14 09:05:51 +01:00
Adrien Grand 8dbd1fd2cd Disable IndexUpgradeIT.testIndexUpgradeInfo and testIndexUpgradeInfoLicense for now.
Relates elastic/x-pack-elasticsearch#3921

Original commit: elastic/x-pack-elasticsearch@a32e8fd557
2018-02-14 09:02:54 +01:00
Yogesh Gaikwad 20503a997f X-Pack-Security: Improve error messaging during setup password (elastic/x-pack-elasticsearch#3809)
* X-Pack-Security: Improve error messaging during setup password

Changes are done to improve logging and exception messages when
setup passwords. Now it checks whether
- X-Pack plugin is available
- X-Pack Security is available 
- X-Pack Security is enabled
by invoking /_xpack query and throws a specific error message.
1. Modifications to simple http client to return http response.
2. Changes in SetupPasswordTool for throwing appropriate exceptions.
2. Unit tests

relates elastic/x-pack-elasticsearch#3476

Original commit: elastic/x-pack-elasticsearch@eb0c5a19c5
2018-02-14 13:11:00 +11:00
Michael Basnight 495a0d83e5 Relax remote check for bwc project checkouts (elastic/x-pack-elasticsearch#3924)
Relates elastic/elasticsearch#28666

Original commit: elastic/x-pack-elasticsearch@7c67ee305a
2018-02-13 14:54:38 -06:00
Andy Bristol e520453551 Fix location of sql cli jar
The `elasticsearch` directory in the plugin folder structure was
removed, so make sure the CLI jar ends up in $ES_HOME/bin/x-pack as
intended

Follow up from elastic/x-pack-elasticsearch#28589

Original commit: elastic/x-pack-elasticsearch@6580b38fe0
2018-02-13 11:30:56 -08:00
Michael Basnight e10ffb4486 Remove snapshot conditional for bwc snapshots (elastic/x-pack-elasticsearch#3922)
Relates elastic/elasticsearch#28657

Original commit: elastic/x-pack-elasticsearch@0916a77179
2018-02-13 12:59:54 -06:00
Andy Bristol 4de793bdb0 [TEST] packaging: add debug logging (elastic/x-pack-elasticsearch#3892)
* [TEST] packaging: add debug logging

Some tests have recurring but intermittent failure on opensuse 42 (see
issues mentioned below). Add debug logging for the places where they
occur. Fail fast when checking to see if es-with-xpack has come up.

For elastic/x-pack-elasticsearch#3817, elastic/x-pack-elasticsearch#3827, and elastic/x-pack-elasticsearch#3862

Original commit: elastic/x-pack-elasticsearch@d69e988a44
2018-02-13 10:56:17 -08:00
Dimitris Athanasiou 2f4dcf36a9 [ML] Add notification for job updates coming from a user action (elastic/x-pack-elasticsearch#3890)
We were missing a notification for when a job is updated. This is
useful so users know that there's been changes which could justify
a change in the job behaviour.

In addition, having those notifications allows our integrations
tests to know when the update was processed which avoids having
to use `sleep()` with its instabilities.



Original commit: elastic/x-pack-elasticsearch@0b4eda2232
2018-02-13 18:46:00 +00:00
David Kyle 938cf239c9 [M] Clean noisy error message in Yaml tests (elastic/x-pack-elasticsearch#3923)
Original commit: elastic/x-pack-elasticsearch@5f0d4040dc
2018-02-13 17:05:27 +00:00
Tanguy Leroux 519807b50e [Tests] Wait for shards to be active in FullClusterRestartIT.testWatcher()
The test `testWatcher()` sometimes fails because the shards are not
active when searching in the `bwc_watch_index` or `.watcher-history`
index. This commit ensured that no shards are relocating or initializing
before executing the searches. It also swallows any error 503 Service
Unavailable when expecting a given number of search hits.

Original commit: elastic/x-pack-elasticsearch@b8e26e0e78
2018-02-13 15:50:57 +01:00
Martijn van Groningen ee0dcd3209 Tests: Decreased logging for watcher smoke tests
Original commit: elastic/x-pack-elasticsearch@e5f4830e7f
2018-02-13 14:35:37 +01:00
Martijn van Groningen 2afbad6c15 use generated license in qa test, see elastic/x-pack-elasticsearch#3878
Original commit: elastic/x-pack-elasticsearch@9a32eab685
2018-02-13 12:17:10 +01:00
Adrien Grand c633b919ab Watcher's ExecutionService is not thread-safe. (elastic/x-pack-elasticsearch#3861)
If start, stop and/or clearExecutions are called concurrently, you could end up
in an inconsistent state where `currentExecutions.sealAndAwaitEmpty` is called
twice on the same instance.

These `synchronized` keywords should make it a bit better, though I suspect
there are other issues and thread safety of this class should be reviewed.

This might have caused
https://internal-ci.elastic.co/job/elastic+x-pack-elasticsearch+5.6+multijob-unix-compatibility/932/consoleFull

Original commit: elastic/x-pack-elasticsearch@2f7e7a34d9
2018-02-13 11:27:11 +01:00
Ryan Ernst 19e9afdf10 Test: Re-enable upgrade integ tests (elastic/x-pack-elasticsearch#3917)
This commit removes awaitsfixes for upgrade tests. The fix was
previously made on a branch and did not make it to master until now.

relates elastic/x-pack-elasticsearch#3729

Original commit: elastic/x-pack-elasticsearch@3402bbe78e
2018-02-12 20:46:22 -08:00
Igor Motov c82fdad41d SQL: Consolidate more type information into DataType (elastic/x-pack-elasticsearch#3850)
Consolidates type handling into DataType, makes DataType available to 
JDBC by moving to sql-proto and removes support for all parameter types 
that cannot be handled by the server.

Original commit: elastic/x-pack-elasticsearch@b8024f5c46
2018-02-12 16:40:09 -05:00
Tim Brooks 5dbbe8fef8 Default to basic license at startup (elastic/x-pack-elasticsearch#3878)
This is related to elastic/x-pack-elasticsearch#3877. This commit modifies the license settings to
default to self generating a basic license.

Original commit: elastic/x-pack-elasticsearch@cd6ee8e06f
2018-02-12 12:57:04 -07:00
David Turner b95c971352 More logging in the rolling-upgrade `waitWithAuth` (elastic/x-pack-elasticsearch#3908)
Previously this could fail without logging anything, if there was no
exception thrown. Now it records the last status code as well as the last
exception, and logs something either way.

Original commit: elastic/x-pack-elasticsearch@753333e579
2018-02-12 18:45:46 +01:00
Martijn van Groningen fde2adb1b3 increase watcher logging when delegating to trigger service
Original commit: elastic/x-pack-elasticsearch@448ee71f37
2018-02-12 16:12:50 +01:00
Dimitrios Athanasiou 5d1a5f27b2 [ML][TEST] Readjust AutodetectMemoryLimitIT assertions
Model memory has decreased slightly due to
elastic/machine-learning-cpp#553. This commit adjusts
the test assertions accordingly.

Original commit: elastic/x-pack-elasticsearch@ea8d403f1a
2018-02-12 14:53:14 +00:00
Alexander Reelsen d5281e89b7 Watcher: Register HTML sanitizer settings (elastic/x-pack-elasticsearch#3904)
The HTML sanitizer settings were not registered correctly, when the
grouping functionality was removed in elastic/x-pack-elasticsearch#3685

This registers the settings properly in watcher.

Relates elastic/support-dev-help-elasticsearch#413

Original commit: elastic/x-pack-elasticsearch@787bb3d9d3
2018-02-12 15:34:04 +01:00
Alexander Reelsen 54cb890eb7 Watcher: Allow to defer startup for tests (elastic/x-pack-elasticsearch#3903)
This commit introduces a new watcher setting to defer starting watcher
until it has been called with the API for the first time. This is
primarily useful in testing environments, as this ensures that watcher
does not try to reload itself because of starting first and then
creating watcher indices.

In addition the undocumented and unused option
xpack.watcher.start_immediately has been removed.

Relates elastic/x-pack-elasticsearch#3854

Original commit: elastic/x-pack-elasticsearch@2b55aec4ad
2018-02-12 14:24:05 +01:00
Alexander Reelsen 48f6a752cb Watcher: Fix proxy scheme to default to HTTP (elastic/x-pack-elasticsearch#3844)
This fixes a regression introduced in Elasticsearch 6.0, when switching
from HttpURLConnection to the Apache HTTP Client.

In the old implementation there was no way to specify if you wanted to use HTTP
or HTTPS for your proxy, only HTTP. If people needed to use HTTPs, they
could just use the CONNECT feature of the proxy.

The new implementation used the scheme of the request that was about to
be sent out as the proxy scheme to be used. So if the request was HTTPS
but the proxy server was HTTP this created a problem.

This commit changes the default scheme to be just HTTP, so that then the
standard CONNECT procecure is taken care off.

Without a real proxy server this is super hard to test. I have verified
this with the following test against a tinyproxy running on port 8888,
but I do not have a great idea how to test this in a unit testable way using a real proxy.

Original commit: elastic/x-pack-elasticsearch@f68e72d8f1
2018-02-12 13:12:38 +01:00
Tanguy Leroux 8e501b6a7e [Tests] Add more time to start with X-Pack
This commit adds a bit more time to the packaging tests to start when
x-pack is installed.

Original commit: elastic/x-pack-elasticsearch@228a3d7c33
2018-02-12 11:35:46 +01:00
Alexander Reelsen f4fa274aa8 Tests: Do not set number of replicas for watcher index (elastic/x-pack-elasticsearch#3882)
The watcher index uses auto expanding of replicas nowadays, so there is
no need to set the number of replicas.

Also ensuring that all watcher indices are green on startup has been moved
to one call, in order to prevent indices being moved around after adding
the triggered watches or watch history index.

Original commit: elastic/x-pack-elasticsearch@dbe04adf64
2018-02-12 09:35:03 +01:00
Chris Earle 63c2d3e6db [Monitoring] Change MonitoringIT into ESIntegTestCase (elastic/x-pack-elasticsearch#3899)
Change MonitoringIT into ESIntegTestCase

Original commit: elastic/x-pack-elasticsearch@d06f6cc0b3
2018-02-10 12:37:17 -05:00
Michael Basnight f6b0de2536 Pull snapshots prior to 6.3 from elastic maven (elastic/x-pack-elasticsearch#3900)
Snapshot versions before 6.3 do not exist in the public, so they need to
be pulled from snapshots.elastic.co/maven.

Original commit: elastic/x-pack-elasticsearch@3cc94add91
2018-02-10 09:52:36 -06:00
Shaunak Kashyap 7ee5ef333c Add fields to `.logstash`'s mapping in template (elastic/x-pack-elasticsearch#3872)
* Add fields to `.logstash`'s mapping in template

This "makes room" in the index for pipeline settings and node groups. Due to this change, users will be able to specify settings and node groups for a pipeline via the Centralized Config Management UI in Kibana. Logstash will only retrieve pipelines associated with the node group specified via the `xpack.management.group.id` setting in `logstash.yml`. For the retrieved pipelines, Logstash will apply any (optionally) specified pipeline settings before (re)loading the pipelines.

* Making field name more explicit + adding multi field for better search

Original commit: elastic/x-pack-elasticsearch@2df101f0b1
2018-02-09 15:18:37 -08:00
Nik Everett 45ac81127a SQL: Drop requirement to have keystore (elastic/x-pack-elasticsearch#3896)
SQL doesn't need the keystore. We required it by mistake.

Original commit: elastic/x-pack-elasticsearch@6840637feb
2018-02-09 17:17:46 -05:00
Michael Basnight 24c6a21d8f BWC changes for x-pack (elastic/x-pack-elasticsearch#3841)
relates elastic/elasticsearch#28505

Original commit: elastic/x-pack-elasticsearch@eda97ded76
2018-02-09 15:03:08 -06:00
Chris Earle 9c3257e158 [Monitoring] Register affixed Header Setting (elastic/x-pack-elasticsearch#3893)
This adds the `HEADERS_SETTING` as setting reported by the Monitoring plugin, which was changed in 6.2.

It also adds an IT to ensure that it is not missed in the future (existing integration tests were passing in the Settings and not being validated by the cluster!).

Original commit: elastic/x-pack-elasticsearch@d6f0b4b503
2018-02-09 15:53:27 -05:00
Ryan Ernst 4e578fe7cc Build: Replace provided configuration with compileOnly (elastic/x-pack-elasticsearch#3868)
This is the xpack side of elastic/elasticsearch#28564

Original commit: elastic/x-pack-elasticsearch@60033824a4
2018-02-09 11:30:43 -08:00