Commit Graph

6755 Commits

Author SHA1 Message Date
David Roberts db8885a46e [ML] Do not download the ml-cpp zip when building it locally (elastic/x-pack-elasticsearch#2262)
When the machine-learning-cpp repo is built locally, the zip file it
creates is preferred over that downloaded from s3 when creating the
overall x-pack-elasticsearch zip.  However, prior to this change the
build would ALSO download an ml-cpp zip from s3, and just not use it.

Original commit: elastic/x-pack-elasticsearch@bd71637edd
2017-08-15 16:31:23 +01:00
Igor Motov 4242cc7370 SQL: Tests cleanup (elastic/x-pack-elasticsearch#2252)
Move Transport Request/Response tests to sql server project, remove unused ProtoTests.

Original commit: elastic/x-pack-elasticsearch@2ad0f3bde2
2017-08-14 17:23:57 -04:00
Jason Tedor f3a7d46698 Rename CONF_DIR to ES_PATH_CONF
This commit is following upstream Elasticsearch which has renamed the
environment variable used to specify a custom configuration directory
from CONF_DIR to ES_PATH_CONF.

Relates elastic/x-pack-elasticsearch#2261

Original commit: elastic/x-pack-elasticsearch@9ae29941e5
2017-08-15 06:19:39 +09:00
Tim Vernum a27dc257c9 Gracefully handle no content(-type) in Put License (elastic/x-pack-elasticsearch#2258)
PUT /_xpack/license with no content or content-type should fail with an appropriate error message rather than throwing NPE.

Original commit: elastic/x-pack-elasticsearch@f8c744d2a2
2017-08-14 20:39:39 +10:00
Alexander Reelsen 69b3ffa40a Tests: Remove useless wait time in watcher REST test
The rest test waited for the watch to run in the background, but there
were no guarantees that this really happened. Also it waited for five
seconds, instead of just executing the watch manually.

relates elastic/x-pack-elasticsearch#2255

Original commit: elastic/x-pack-elasticsearch@56765a649e
2017-08-14 11:52:26 +02:00
Alexander Reelsen 5416a6afd4 Tests: Fix timeout in watcher history template test
Due to an invalid timeout the test failed earlier than it should have.

relates elastic/x-pack-elasticsearch#2222

Original commit: elastic/x-pack-elasticsearch@2265c419e3
2017-08-14 09:53:52 +02:00
Lisa Cawley ea05ddd513 [DOCS] Fix principal access_granted attribute (elastic/x-pack-elasticsearch#2257)
Original commit: elastic/x-pack-elasticsearch@9c33afce9f
2017-08-11 16:53:21 -07:00
Nik Everett 2207c19afa Merge branch 'master' into feature/sql
Original commit: elastic/x-pack-elasticsearch@f9c0bc95d4
2017-08-11 15:41:22 -04:00
lcawley e2f7081693 [DOCS] Add abbreviated titles
Original commit: elastic/x-pack-elasticsearch@a4cf8a363f
2017-08-11 10:00:35 -07:00
lcawley 8d9dc6ff65 [DOCS] Clarify title for X-Pack install
Original commit: elastic/x-pack-elasticsearch@9712bbd1d5
2017-08-11 09:15:20 -07:00
David Roberts b5d159bc1c [ML] Handle simultaneous force delete datafeed and stop datafeed (elastic/x-pack-elasticsearch#2243)
This is an important case as the UI force stops datafeeds now.

Fixes elastic/x-pack-kibana#2083

Original commit: elastic/x-pack-elasticsearch@4d0f62ad2d
2017-08-11 14:43:24 +01:00
Alexander Reelsen 3ad2f5e9f5 Tests: Increase logging for watcher smoke tests
These tests have repeating but not reproducible failures,
where the stash is filled with a second PUT operation and the
watcher stats response does not match. Setting the log to trace
should shed some light on this.

As the smoke tests are only four tests this will not lead to a
log explosion.

Relates elastic/x-pack-elasticsearch#1513, elastic/x-pack-elasticsearch#1874

Original commit: elastic/x-pack-elasticsearch@5832dc7990
2017-08-11 15:29:52 +02:00
Alexander Reelsen 26c5766a0d Tests: Do not delete index templates on bwc tests
In order to run the bwc tests there is no need to delete the
index template at the end of a test run which results
in recreation of those due to all the cluster state listener.

Relates elastic/x-pack-elasticsearch#2228

Original commit: elastic/x-pack-elasticsearch@702d1c61ed
2017-08-11 14:11:50 +02:00
David Roberts cb3f3d2d04 [ML] Switch from max_running_jobs to xpack.ml.max_open_jobs (elastic/x-pack-elasticsearch#2232)
This change makes 2 improvements to the max_running_jobs setting:

1. Namespaces it by adding the xpack.ml. prefix
2. Renames "running" to "open", because the "running" terminology
   is not used elsewhere

The old max_running_jobs setting is used as a fallback if the new
xpack.ml.max_open_jobs setting is not specified.  max_running_jobs
is deprecated and (to ease backporting in the short term) will be
removed from 7.0 in a different PR closer to release of 7.0.

Relates elastic/x-pack-elasticsearch#2185

Original commit: elastic/x-pack-elasticsearch@18c539f9bb
2017-08-11 09:00:33 +01:00
Alexander Reelsen 5f30508efd Tests: Remove AwaitsFix annotation and increase logging
This test does not reproduce locally but fails regularly in CI.
Added more logging and proper comments.

Relates elastic/x-pack-elasticsearch#2222

Original commit: elastic/x-pack-elasticsearch@bf6b590629
2017-08-11 09:44:09 +02:00
Alexander Reelsen 11334b2df3 Tests: Fix TimeThrottleIntegrationTests to not rely on shard actions (elastic/x-pack-elasticsearch#2234)
These tests used to fail rarely, because during a watch execution
one of the watcher shards was relocated resulting in a second execution
of watch.

In order to prevent this, the tests do not need to actually create any
shards, which causes watcher potentially to be rebalanced.

This simplifies and speeds up the test as well.

relates elastic/x-pack-elasticsearch#1608

Original commit: elastic/x-pack-elasticsearch@1cfac1145d
2017-08-11 09:19:25 +02:00
Igor Motov a0de35d801 SQL: Add licensing support (elastic/x-pack-elasticsearch#2213)
Adds a granular licensing support to SQL. JDBC now requires a platinum license, everything else work with any non-expired license.

Original commit: elastic/x-pack-elasticsearch@a30470e2c9
2017-08-10 19:19:09 -04:00
Lisa Cawley cc7c9aeddb [DOCS] Remove redundant Logstash security page (elastic/x-pack-elasticsearch#2239)
Original commit: elastic/x-pack-elasticsearch@8f66e85fb0
2017-08-10 15:31:41 -07:00
Lisa Cawley e500fba354 [DOCS] Update links to Kibana security (elastic/x-pack-elasticsearch#2235)
Original commit: elastic/x-pack-elasticsearch@88f29b3321
2017-08-10 12:56:03 -07:00
Lisa Cawley 1066a6f877 [DOCS] Add loopback to X-Pack install info (elastic/x-pack-elasticsearch#2237)
Original commit: elastic/x-pack-elasticsearch@8c105818e2
2017-08-10 12:37:14 -07:00
Nik Everett 9d81805616 Merge branch 'master' into feature/sql
Original commit: elastic/x-pack-elasticsearch@75f438cd4a
2017-08-10 13:10:36 -04:00
lcawley 8c7828413d [DOCS] Fix typo
Original commit: elastic/x-pack-elasticsearch@03c2f708be
2017-08-10 09:20:57 -07:00
Lisa Cawley a03fa9dd45 [DOCS] Change indexes to indices (elastic/x-pack-elasticsearch#2231)
Original commit: elastic/x-pack-elasticsearch@1c2d24848e
2017-08-10 09:13:53 -07:00
David Kyle c4910098b0 [Docs] Change indexes -> indices in datafeed config example (elastic/x-pack-elasticsearch#2227)
Original commit: elastic/x-pack-elasticsearch@7ddd36360e
2017-08-10 16:22:59 +01:00
Alexander Reelsen 35ee552923 Tests: Fix logger initialization to include hostname in timewarped watcher
This cleans up logging, when starting several elasticsearch instances,
as otherwise you cannot see, which node emits this log message.

Original commit: elastic/x-pack-elasticsearch@c8c2819d86
2017-08-10 14:09:14 +02:00
Alexander Reelsen ceb13988e3 Watcher: Do not update watch status state during execution (elastic/x-pack-elasticsearch#2204)
When a watch is executed, it sends an update request to the watch to
udpate its status.

This update request also updates the status.state field, which contains
information, if the watch is active. If the watch gets executed, and
during execution a watch gets disabled, then the current execution will
set the watch back to active.

This commit fixes the current behaviour and never changes the state of
a watch when updating the status after executing, allowing
activate/deactivate calls to work as expected, regardless if a watch
is being executed.

This will fix not only the current behaviour but also some flaky tests.

Original commit: elastic/x-pack-elasticsearch@ca69109ecb
2017-08-10 13:16:39 +02:00
Alexander Reelsen 4012da662d Tests: Add @AwaitsFix to failing test
Original commit: elastic/x-pack-elasticsearch@545155ec13
2017-08-10 10:17:22 +02:00
Nik Everett 2b2f831116 Fix broke bwc test in sql
The test was enabled for the wrong versions....

Original commit: elastic/x-pack-elasticsearch@2662a11e0c
2017-08-09 15:58:47 -04:00
lcawley 1840e952af [DOCS] Remove outdated SSL info
Original commit: elastic/x-pack-elasticsearch@aeddc6dd5b
2017-08-09 10:11:57 -07:00
Lisa Cawley 0d79c80a8f [DOCS] Add user setup to X-Pack install info (elastic/x-pack-elasticsearch#2189)
* [DOCS] Add user setup to X-Pack install info

* [DOCS] Add TSL steps to X-Pack install

* [DOCS] Clarify SSL settings in Xpack install

Original commit: elastic/x-pack-elasticsearch@eee37729ff
2017-08-09 10:05:43 -07:00
Lisa Cawley 1bb3ad38c7 [DOCS] Update APIs for multiple jobs or datafeeds (elastic/x-pack-elasticsearch#2105)
* [DOCS] Update APIs for multiple jobs or datafeeds

* [DOCS] Fix syntax diagrams for ML stop/close APIs

* [DOCS] Removed TBD authorization for ML APIs

Original commit: elastic/x-pack-elasticsearch@1a9137a5a7
2017-08-09 08:30:21 -07:00
Colin Goodheart-Smithe a4dd177978 Migrates ToXContent classes (elastic/x-pack-elasticsearch#2024)
* Migrates ToXContent classes

* review comments

Original commit: elastic/x-pack-elasticsearch@74ce3755ab
2017-08-09 15:55:04 +01:00
Nik Everett f7d526036e Un-AwaitsFix SQL's jdbc integration tests (elastic/x-pack-elasticsearch#2217)
Most tests worked fine. The datetime tests are broken for some time
zones. The csv tests were broken because they accepted the default
fetch size which looks like it is broken.

Original commit: elastic/x-pack-elasticsearch@e034c2f102
2017-08-09 08:33:14 -04:00
Nik Everett 85937c6e78 Fix SQL's DatabaseMetaData (elastic/x-pack-elasticsearch#2214)
It was failing because it was missing config needed to properly
handle time zones.

Original commit: elastic/x-pack-elasticsearch@1166ea7475
2017-08-08 17:48:03 -04:00
Nik Everett e5973ec356 Un-@AwaitsFix ExplainIT
Original commit: elastic/x-pack-elasticsearch@dde8580afc
2017-08-08 16:09:28 -04:00
Lisa Cawley ccf0b6f2ed [DOCS] Fix typo (elastic/x-pack-elasticsearch#2211)
Original commit: elastic/x-pack-elasticsearch@6efb78c1b2
2017-08-08 12:52:52 -07:00
Nik Everett b31bc786d3 Remove isValidIndex from SQL's Catalog interface (elastic/x-pack-elasticsearch#2195)
This moves validating that each index contains only a single type
into EsCatalog which removes a race condition and removes a method
from the Catalog interface. Removing methods from the Catalog
interface is nice because the fewer methods that the interface has
the fewer have to be secured.

Original commit: elastic/x-pack-elasticsearch@85cd089e47
2017-08-08 15:42:47 -04:00
Nik Everett c47c66362e Fix content type in test
I'd left it off. Thanks Jenkins!

Original commit: elastic/x-pack-elasticsearch@57b708e414
2017-08-08 15:19:20 -04:00
Nik Everett 4c7e02191f Merge branch 'master' into feature/sql
Original commit: elastic/x-pack-elasticsearch@360ed34b70
2017-08-08 14:39:32 -04:00
Nik Everett 570b66638e Add test for index with two types (elastic/x-pack-elasticsearch#2194)
Adds a test that shows *how* SQL fails to address an index with two types
to the full cluster restart tests. Because we're writing this code
against 7.0 don't actually execute the test, but we will execute it when
we merge to 6.x and it *should* work.

Original commit: elastic/x-pack-elasticsearch@b536e9a142
2017-08-08 13:32:13 -04:00
Alexander Reelsen 0b5909fc65 Watcher: Stop swallowing exceptions, always return them instead of message (elastic/x-pack-elasticsearch#1933)
It is really hard to debug some issues with watcher, when only the
e.getMessage() is returned as failure reasons instead of the whole
stack trace.

This commit gets rid of ExceptionsHelper.detailedMessage(e) and always
returns the whole exception.

This commit also extends the watch history to have all fields named
error be treated like an object to be sure they do not get
indexed. No matter where it's placed in the hierarchy

In addition a few Field interface classes were removed, that only contained parse fields.

relates elastic/x-pack-elasticsearch#1816

Original commit: elastic/x-pack-elasticsearch@b2ce680139
2017-08-08 18:36:22 +02:00
David Roberts 22da5cf89e [ML] Add max open jobs per node as a node attribute (elastic/x-pack-elasticsearch#2203)
This commit adds the max_running_jobs setting from elasticsearch.yml
into a node attribute called ml.max_open_jobs.  Previously there was
an assumption that max_running_jobs would be the same for all nodes in
the cluster.  However, during a rolling cluster restart where the value
of the setting is being changed this clearly cannot be the case, and
would cause unexpected/unpredictable limits to be used during the period
when different nodes had different settings.

For backwards compatibility, if another node in the cluster has not added
its setting for max_running_jobs to the cluster state then the old
(flawed but better than nothing) approach is applied, i.e. assume the
remote node's setting for max_running_jobs is equal to that of the node
deciding the job allocation.

Relates elastic/x-pack-elasticsearch#2185

Original commit: elastic/x-pack-elasticsearch@1e62b89183
2017-08-08 16:16:27 +01:00
Dimitris Athanasiou 2864078da2 [ML] Move job group validation after parsing (elastic/x-pack-elasticsearch#2207)
Validating job groups during parsing results into
the validation error being wrapped into a parse
exception. The UI then does not display the cause of the
error. Finally, it is conceptually not a parse error, so
it belongs outside the parsing phase.

Original commit: elastic/x-pack-elasticsearch@a03f002bdc
2017-08-08 15:59:04 +01:00
Igor Motov e09eb41340 SQL: make JdbcAction transport friendly (elastic/x-pack-elasticsearch#2200)
Adds proper serialization to JdbcAction's requests and responses

Original commit: elastic/x-pack-elasticsearch@22d2c0582e
2017-08-08 10:26:13 -04:00
Alexander Reelsen cff3418e96 Tests: Only restart watcher on watcher tests
In order to not restart watcher on every test, this checks
if this is a watcher test and only restarts watcher in that case.

In addition this also checks if watcher is not marked as started.
as otherwise restarting does not make sense. Lastly, this waits until
watcher is marked as started before proceeding.

Original commit: elastic/x-pack-elasticsearch@a8d72f3ebb
2017-08-08 14:28:44 +02:00
David Roberts 6a159d2127 [ML] Fix fallout from bulk action requiring newlines (elastic/x-pack-elasticsearch#2205)
Only unit tests were broken.  Production ML code was always terminating
bulk requests with newlines.

Original commit: elastic/x-pack-elasticsearch@96ed06fed3
2017-08-08 11:07:13 +01:00
Alexander Reelsen 55e88d6857 Watcher: Ignore if template is missing when upgrade is running (elastic/x-pack-elasticsearch#2199)
If one of the old watcher templates does not exist when we try 
to delete it, the upgrade should just continue.

Original commit: elastic/x-pack-elasticsearch@6a52bad329
2017-08-08 10:17:58 +02:00
Igor Motov cc41a2daa0 SQL: make CliAction transport friendly (elastic/x-pack-elasticsearch#2198)
Adds proper serialization to CliAction's requests and responses

Original commit: elastic/x-pack-elasticsearch@2d2a15a5ba
2017-08-07 16:15:11 -04:00
Chris Earle 1d6f82dbe3 [Monitoring] Remove Dedicated IndicesStatsCollector (elastic/x-pack-elasticsearch#2192)
This removes the `IndicesStatsCollector` and, instead, it reuses the superset version of the call from the `IndexStatsCollector`.

On clusters with a large number of indices, this should actually help a good amount in reducing wasted calls and memory allocation without any difference in the output.

Original commit: elastic/x-pack-elasticsearch@93b09878e4
2017-08-07 13:00:41 -04:00
Tim Sullivan 7d86d75aa4 [Monitoring/Cluster Alerts] default admin email is snake_case (elastic/x-pack-elasticsearch#2177)
Original commit: elastic/x-pack-elasticsearch@d6129a0065
2017-08-07 09:15:14 -07:00