Commit Graph

6716 Commits

Author SHA1 Message Date
David Roberts 6741a62e80 [ML] Adjust memory limit test to account for greater accuracy (elastic/x-pack-elasticsearch#3047)
Due to elastic/machine-learning-cpp#409 the ML C++ code now instruments
memory more accurately.  This change modifies the expectations of the Java
integration test to account for the change.

Original commit: elastic/x-pack-elasticsearch@2ed7a75af4
2017-11-17 17:02:19 +00:00
Alexander Reelsen 0f97e28074 Watcher: Further preparations for source repo split (elastic/x-pack-elasticsearch#3006)
Changes to further prepare for feature split with regards to watcher:

- CryptoService has been moved into watcher
- CryptoService.generateKey() has been moved into SystemKeyTools, only
  used there
- The creation of the http client/notification classes have been moved
  into watcher, no further dependencies on watcher in XPackPlugin
- Each subproject now registers it's own named writeables

Relates elastic/x-pack-elasticsearch#2925

Original commit: elastic/x-pack-elasticsearch@a60c98ba7e
2017-11-17 17:05:07 +01:00
Alexander Reelsen 3d5fb54522 Watcher: Create templates on nodes newer than the master (elastic/x-pack-elasticsearch#2950)
This problem was introduced due to distributed watch execution.

When a node newer than the master node joins the cluster and gets a
.watches shard assigned it is supposed to start watcher. However
when a new version of the watch history template is part of that new
node (and we might increase that version anytime), this template does
not get installed, because only the master node is updating watcher
templates.

This commit checks if the local node version is higher than the master
node version and then also puts missing templates.

Currently this is done for all watcher templates, not only the watcher history.

relates elastic/x-pack-elasticsearch#2944

Original commit: elastic/x-pack-elasticsearch@4960231ea7
2017-11-17 16:56:29 +01:00
Alexander Reelsen 1933fc71f3 Tests: Mute more monitoring upgrade tests
Relates elastic/x-pack-elasticsearch#2948

Original commit: elastic/x-pack-elasticsearch@942ec95e6e
2017-11-17 13:54:50 +01:00
Hendrik Muhs a583cf270b [ML-FC] implement endpoint parameter that takes a duration (elastic/x-pack-elasticsearch#3027)
Adds a duration parameter to the forecast API. Also fixes issue if no parameter is given (forecast 1 day from time of last bucket), in which case it lets autodetect decide

depends on elastic/machine-learning-cpp#407

Original commit: elastic/x-pack-elasticsearch@3387478872
2017-11-17 06:37:03 +01:00
lcawley 7ae8f1cd9b [DOCS] Added sysgenkey command and watcher settings
Original commit: elastic/x-pack-elasticsearch@8e0b2027a6
2017-11-16 15:40:51 -08:00
lcawley 43a2572e7d [DOCS] Fixed typo in over_field_name
Original commit: elastic/x-pack-elasticsearch@c366d43448
2017-11-16 14:45:44 -08:00
Tim Brooks f94e6d427b Modify x-pack to for `TcpTransport` changes
This is related to elatic/elasticsearch#27407. That commit removed
parameterization from TcpTranport. This commit modifies the security
transport to be compatible with those changes.

Original commit: elastic/x-pack-elasticsearch@9878c26e14
2017-11-16 14:32:40 -07:00
Lisa Cawley 37c29a4006 [DOCS] Add categorization limitation (elastic/x-pack-elasticsearch#3022)
Original commit: elastic/x-pack-elasticsearch@387d7cf939
2017-11-16 10:05:11 -08:00
Hendrik Muhs 5efdb54eb8 add memoryUsage stats (elastic/x-pack-elasticsearch#3025)
adds memory usage statistics to forecast request stats

depends on: elastic/machine-learning-cpp#406

Original commit: elastic/x-pack-elasticsearch@abf90030fc
2017-11-16 18:38:13 +01:00
Dimitrios Athanasiou 80f4797f7a [ML][TEST] Add integ test for memory redution in population analysis
Original commit: elastic/x-pack-elasticsearch@ccd1f4835d
2017-11-16 17:10:59 +00:00
Zachary Tong 3c60c89504 [TEST] Skip "all" instead of "999"
The fake version was apparently making the yaml runner unhappy

Original commit: elastic/x-pack-elasticsearch@04193f9578
2017-11-16 01:29:00 +00:00
lcawley 9c0110678e [DOCS] Updated link to X-Pack breaking changes
Original commit: elastic/x-pack-elasticsearch@931bc10498
2017-11-15 16:39:05 -08:00
lcawley ff54c4190a [DOCS] Fixed links to X-Pack Kibana release notes
Original commit: elastic/x-pack-elasticsearch@57159e6bc6
2017-11-15 15:28:44 -08:00
lcawley e4518871ee Merge remote-tracking branch 'upstream/master'
Original commit: elastic/x-pack-elasticsearch@6a2df70985
2017-11-15 15:27:04 -08:00
Zachary Tong 34898f2717 [TEST] AwaitsFix/Skip REST test "qa/rolling_upgrade/60_monitoring"
Test appears to be broken and fails consistently.  See elastic/x-pack-elasticsearch#2948

Original commit: elastic/x-pack-elasticsearch@e3ba8277e3
2017-11-15 20:35:44 +00:00
Tim Brooks d45c2d784c Update x-pack to support TcpChannel (elastic/x-pack-elasticsearch#2983)
This is related to elastic/elasticsearch#27132. This commit updates
x-pack to support signature changes introduced in that PR.

Original commit: elastic/x-pack-elasticsearch@461c96ffe6
2017-11-15 12:38:56 -07:00
lcawley 10a9334046 Merge remote-tracking branch 'upstream/master'
Original commit: elastic/x-pack-elasticsearch@7af3facb7d
2017-11-15 09:09:42 -08:00
Lisa Cawley f8497dc943 [DOCS] Added flow diagram for installation (elastic/x-pack-elasticsearch#3020)
Original commit: elastic/x-pack-elasticsearch@c7c39d9724
2017-11-15 09:08:48 -08:00
Lee Hinman 889b008298 Add "client-api-objects" dependency for xpack plugin and transport-client (elastic/x-pack-elasticsearch#2995)
* Add "client-api-objects" dependency for xpack plugin and transport-client

This adds another gradle project, "client-api-objects" which is intended to be a
common dependency so that the xpack plugin and transport-client can share the
same Request and Response objects.

Relates to elastic/x-pack-elasticsearch#2925

Original commit: elastic/x-pack-elasticsearch@a6d83299d0
2017-11-15 09:49:00 -07:00
Tim Brooks 5d47b96905 Modify x-pack to for `TransportChannel` changes (elastic/x-pack-elasticsearch#3013)
This is related to elastic/elasticsearch#elastic/x-pack-elasticsearch#27388. It modifies x-pack to
be compatible with the removal of the delegating transport channel.

Original commit: elastic/x-pack-elasticsearch@3bd7bf6773
2017-11-15 09:48:17 -07:00
lcawley 0eee69b6d3 [DOCS] Added flow diagram for installation
Original commit: elastic/x-pack-elasticsearch@048ac11a17
2017-11-15 08:27:12 -08:00
Alexander Reelsen 36105231c3 Watcher: Return useful error message when no accounts are found (elastic/x-pack-elasticsearch#2897)
When there were no accounts configured, watcher returned a cryptic
error message containing 'null' in the description. This fix returns
a more clear error message.

On top a dedicated NotificationServiceTests class was added to remove
redundant test cases in the hipchat/jira/slack unit tests, that all
basically tested NotificationService capabilties.

relates elastic/x-pack-elasticsearch#2666

Original commit: elastic/x-pack-elasticsearch@45d0d1df31
2017-11-15 11:51:23 +01:00
Alexander Reelsen f286f9b0f2 Watcher: Remove unused test helper class
Original commit: elastic/x-pack-elasticsearch@09f66a5f52
2017-11-15 11:05:59 +01:00
David Kyle afec0fd6c9 [ML] [DOCS] Clean up ML jobs after each docs test (elastic/x-pack-elasticsearch#2825)
Original commit: elastic/x-pack-elasticsearch@5ec4a4a752
2017-11-15 09:45:57 +00:00
Clinton Gormley d833af2046 Rest spec fixes (elastic/x-pack-elasticsearch#2965)
* Rename REST spec xpack.deprecation.info to xpack.migration.deprecations

* Fixed parameter-type naming in xpack.ml.get_model_snapshots

* Fixed QS multi-cluster search test to use cluster.remote_info

Original commit: elastic/x-pack-elasticsearch@ccd35b4a6c
2017-11-15 09:33:19 +01:00
Martijn van Groningen 56e7512500 Make the persistent task status available to PersistentTasksExecutor.nodeOperation(...) method
Original commit: elastic/x-pack-elasticsearch@ce6d788e77
2017-11-15 07:54:05 +01:00
Martijn van Groningen 439830890f Move the initialization of persistent tasks from the machine learning level to xpack level.
Today persistent tasks is only usable from machine learning, but others like ccr will need to use it too.
With this change ccr and other will be able to make use of it too.

Original commit: elastic/x-pack-elasticsearch@c90f01d5f6
2017-11-15 07:44:53 +01:00
Lisa Cawley 245543d5cf [DOCS] Add upgrade details about stopping ML (elastic/x-pack-elasticsearch#2982)
* [DOCS] Add upgrade details about stopping ML

* [DOCS] Addressed feedback about stopping all ML

Original commit: elastic/x-pack-elasticsearch@1897390edc
2017-11-14 18:04:21 -08:00
Jay Modi 4eb87bba4b Remove create_bwc_indexes.py (elastic/x-pack-elasticsearch#2989)
This commit removes the create_bwc_indexes.py script as we no longer need this script to test for
BWC with indices. The last test relying on this script was removed in elastic/x-pack-elasticsearch#2832.

relates elastic/x-pack-elasticsearch#1629

Original commit: elastic/x-pack-elasticsearch@5dc3cbc395
2017-11-14 10:08:59 -07:00
jaymode 2f8cd77349 Test: TribeWithSecurityIT should wait for security index to be writeable
This commit adds checks to the TribeWithSecurityIT tests to ensure that the security index is
writeable before making modification operations. Otherwise, we hit errors in tests that are not
always reproducible.

relates elastic/x-pack-elasticsearch#2977

Original commit: elastic/x-pack-elasticsearch@c29bdff7ae
2017-11-14 08:18:55 -07:00
Jay Modi 5888174f24 Cleanup leftover tribe references in the plugin build.gradle file (elastic/x-pack-elasticsearch#2987)
In elastic/x-pack-elasticsearch#2901, the dependency on the tribe module was removed but a few leftover references were missed
in the build.gradle file of the x-pack-elasticsearch plugin. This commit removes these leftover
references.

Original commit: elastic/x-pack-elasticsearch@03f1cae1f5
2017-11-14 08:11:21 -07:00
Dimitrios Liappis a89bfe84ba [DOCS] Split long lines in Docker TLS getting-started snippet
and add warning for Windows users not using
PowerShell (e.g. `cmd.exe`) to remove the `\` character and join
lines.

Also fix trailing whitespace character in link back to `docker.asciidoc`.

Relates elastic/x-pack-elasticsearch#2999

Original commit: elastic/x-pack-elasticsearch@fe1c5dbc11
2017-11-14 14:25:52 +02:00
Alexander Reelsen dc42887396 Watcher: Move watcher-only packages into watcher hierarchy (elastic/x-pack-elasticsearch#2933)
In order to prepare for separate source directories, this commit moves
a few packages back into the watcher namespaces. A few of them have been
moved out previously as we thought that it might make sense to have a
dedicated notification API. This wont be the case for watcher on ES
anymore, so we can safely move those back into the watcher space.

Packages affected by this move:

* org.elasticsearch.xpack.common.http
* org.elasticsearch.xpack.common.text
* org.elasticsearch.xpack.common.secret
* org.elasticsearch.xpack.common.stats
* org.elasticsearch.xpack.support
* org.elasticsearch.xpack.notification

Tests have been moved accordingly.

The class `XContentUtils` has been split into one implementation for
watcher and one for security as different methods were used.

Relates elastic/x-pack-elasticsearch#2925

Original commit: elastic/x-pack-elasticsearch@0aec64a7e2
2017-11-14 11:35:10 +01:00
Dimitris Athanasiou e00b4bd197 [ML][TEST] Improve RevertModelSnapshotIT (elastic/x-pack-elasticsearch#2954)
X-Pack side testing of elastic/machine-learning-cpp#390



Original commit: elastic/x-pack-elasticsearch@64e8ca85eb
2017-11-14 10:32:42 +00:00
jaymode 1e30b12f97 Test: increase security index wait times
Our tests currently rely on waiting for the security index to be available in some cases and in
CI, these checks have been timing out. This commit increases the amount of time that we will wait
for the index before failing to account for slow machines.

Original commit: elastic/x-pack-elasticsearch@639dccd5cb
2017-11-13 12:32:06 -07:00
Chris Earle def8c48dcb [Monitoring] Add "apm" to "stack_stats" for Phone Home (elastic/x-pack-elasticsearch#2848)
This checks if `apm-*` indices exist in the cluster to try to determine if APM is in use on the Elasticsearch cluster.

Original commit: elastic/x-pack-elasticsearch@7f9a9a4eee
2017-11-13 14:24:07 -05:00
Dimitrios Liappis 00ccac9203 [DOCS] Fix wrapped lines in code blocks of TLS getting started guide
Relates elastic/x-pack-elasticsearch#2970

Original commit: elastic/x-pack-elasticsearch@a279e57270
2017-11-13 20:00:35 +02:00
Simon Willnauer 92f87cab14 Xpack side of elastic/elasticsearch#27161 (elastic/x-pack-elasticsearch#2851)
Original commit: elastic/x-pack-elasticsearch@31fdcdf583
2017-11-13 12:07:04 +01:00
Hendrik Muhs fd392627a0 [ML-FC] Forecast status message rebase (elastic/x-pack-elasticsearch#2936)
Re-enables persistence of the forecast request stats document and adds more fields to it. Depends on https://github.com/elastic/machine-learning-cpp/pull/382.

Original commit: elastic/x-pack-elasticsearch@b6762005c0
2017-11-13 09:03:29 +01:00
Lisa Cawley 244cfa7181 [DOCS] Added link to TLS docker content (elastic/x-pack-elasticsearch#2959)
Original commit: elastic/x-pack-elasticsearch@686836c62c
2017-11-10 14:08:46 -08:00
Ryan Ernst 9a2ae4b7f2 Update security policy to use versionless codebase properties (elastic/x-pack-elasticsearch#2602)
This is the xpack side of
https://github.com/elastic/elasticsearch/pull/26756

Original commit: elastic/x-pack-elasticsearch@a219f5b6c0
2017-11-10 11:00:34 -08:00
Lisa Cawley d0f767a20a [DOCS] Move docker files to x-pack repo (elastic/x-pack-elasticsearch#2942)
Original commit: elastic/x-pack-elasticsearch@b4e518b178
2017-11-10 09:39:17 -08:00
Lisa Cawley fb769be92e [DOCS] Added TLS configuration info for Docker (elastic/x-pack-elasticsearch#2939)
* [DOCS] Add docker TLS configuration info

* [DOCS] Updated layout of TLS docker page

* [DOCS] Clean up docker TLS pages

* [DOCS] Changed nesting of TLS docker info

* [DOCS] More small updates to TLS docker page

Original commit: elastic/x-pack-elasticsearch@2b0504632a
2017-11-10 09:33:56 -08:00
David Roberts 742a052619 [ML] Account for the possibility of C++ log messages being UTF-16 (elastic/x-pack-elasticsearch#2952)
On Windows, log4cxx always writes to stderr in UTF-16, and we get the
logs from C++ to Java by redirecting stderr to our named pipe.  Hence
the log handler in Java needs to tolerate the log stream it's reading
being either UTF-16 (for Windows) or UTF-8 (for other platforms).

Fixes elastic/machine-learning-cpp#385

Original commit: elastic/x-pack-elasticsearch@89237d7125
2017-11-10 15:15:10 +00:00
Alexander Reelsen a90cd81f99 Docs: Fix broken watcher example using multi line strings
Original commit: elastic/x-pack-elasticsearch@7398be67c4
2017-11-10 10:06:47 +01:00
Tim Vernum 039cf42fdf Extend wait time to 20s in SecurityIntegTest
This kind of sucks, because we shouldn't have to wait that long for tests to run.
But they're failing CI with some regularity, and we rely on these integration tests.

Original commit: elastic/x-pack-elasticsearch@3f4acb2a32
2017-11-10 15:20:30 +11:00
lcawley b5cb814b32 [DOCS] Add security configuration section
Original commit: elastic/x-pack-elasticsearch@ccae9a84a9
2017-11-09 14:28:56 -08:00
Tal Levy 8c489b1a98 Prevent 7.x nodes from joining cluster with un-upgraded 6.x .security indices (elastic/x-pack-elasticsearch#2921) (elastic/x-pack-elasticsearch#2940)
This is a forward-port of elastic/x-pack-elasticsearch/pull/2921.

original commit message:

Before this commit, a cluster with security enabled and backed by
native-realm user permissions allowed rolled upgrades to clusters without
upgrading the `.security` index. This resulted in the newly established
6.0 cluster not able to register the native-realm users previously established
in the `.security` index. In order to fix this, one would have to rely on file-based
users to re-configure and upgrade the `.security` index. Since this state is easily
avoidable with an upgrade, this commit rejects the joining of upgraded nodes without
upgrading the security index beforehand.

modifications:

Test with 7.x vs 6.x nodes.

Original commit: elastic/x-pack-elasticsearch@56f81bfb20
2017-11-09 12:49:59 -08:00
Lisa Cawley 0c10d82f78 [DOCS] Added setup folder (elastic/x-pack-elasticsearch#2943)
Original commit: elastic/x-pack-elasticsearch@cc7027c7ec
2017-11-09 12:32:21 -08:00