Commit Graph

8002 Commits

Author SHA1 Message Date
Alexander Reelsen ab99b82125 Tests: Replace YAML test waiting for watch execution (elastic/x-pack-elasticsearch#4008)
This is the last YAML test, that waits for a watch execution by
specifying some timeout value. This one also gets replaced with a java
test that uses `assertBusy()` and thus is much more likely to succeed.

relates elastic/x-pack-elasticsearch#1513

Original commit: elastic/x-pack-elasticsearch@c2ab8777f4
2018-03-15 14:20:58 -07:00
Alexander Reelsen 7712ab37d1 Documentation: Add username/password parameters for basic auth (elastic/x-pack-elasticsearch#4126)
relates elastic/x-pack-elasticsearch#4070

Original commit: elastic/x-pack-elasticsearch@19d7002d3c
2018-03-15 11:59:15 -07:00
Dimitris Athanasiou c10b2ea631 [ML] Ensure job is not assigned to node that cannot read model_snapshot (elastic/x-pack-elasticsearch#4091)
This adds a minimum compatible version to the model snapshot.
Nodes with a version earlier than that version cannot read
that model snapshot. Thus, such jobs are not assigned to
incompatible nodes.

relates elastic/x-pack-elasticsearch#4077

Original commit: elastic/x-pack-elasticsearch@2ffa6adce0
2018-03-15 17:38:52 +00:00
Alexander Reelsen 92379ca9af Watcher: Ensure usage stats work properly in distributed environment (elastic/x-pack-elasticsearch#4094)
This adds back usage stats by pickybacking on the watcher stats, which
are already running distributed in order to collect and merge watcher
statistics.

In order to be able to track statistics, we need to add information for
each watch in an in-memory data structure that is processed whenever a
usage request is coming in. This processing creates a number of counters
for each node, which then are merged together in the usage stats.

relates elastic/x-pack-elasticsearch#4071

Original commit: elastic/x-pack-elasticsearch@c8bfed288f
2018-03-15 10:28:03 -07:00
David Roberts 783cabbd2f [DOCS] Reflect recent improvements in notes on watch/datafeed privileges (elastic/x-pack-elasticsearch#4116)
Following elastic/x-pack-elasticsearch#3254 security for ML datafeeds has been improved.  The same goes
for watches since elastic/x-pack-elasticsearch#2808.

This change updates a section of the docs that was missed in those changes.
(The majority of the docs changes were made at the appropriate time.)

Original commit: elastic/x-pack-elasticsearch@b3b24ca483
2018-03-15 10:26:56 +00:00
Tim Brooks b121262b2d Fix expiration millis for start_basic (elastic/x-pack-elasticsearch#4124)
This is related to elastic/x-pack-elasticsearch#3877. There was a bug in the PR that introduced
start_basic route. The start basic had an expiration millis that rolled
over into negative numbers. This fixes that issue.

Original commit: elastic/x-pack-elasticsearch@aea9a13d2b
2018-03-14 17:28:36 -06:00
Alexander Reelsen 4109f6e5b7 Watcher: Fix compilation issue
Original commit: elastic/x-pack-elasticsearch@d94d18a12f
2018-03-14 16:20:25 -07:00
Alexander Reelsen c30256e9b5 Watcher: Fix TransformInput toXContent serialization (elastic/x-pack-elasticsearch#4061)
The toXContent serialization of the transform input was broken, which
could lead to the bad case that a watch could be stored with an invalid
toXContent serialization, that could not be read again, when the watch
should either be executed or even just returned by the Get watch API.

relates elastic/x-pack-elasticsearch#4049

Original commit: elastic/x-pack-elasticsearch@f06ad768b8
2018-03-14 15:53:39 -07:00
Lisa Cawley 76e141d390 [DOCS] Split out X-Pack installation substeps (elastic/x-pack-elasticsearch#4125)
Original commit: elastic/x-pack-elasticsearch@df3a4ff045
2018-03-14 14:52:48 -07:00
Boaz Leskes 3d6f4d3c1c remove FullClusterRestartIT.testMonitoring
Per @pickypg : The value of it is to prove that we can continue calling
the same API between versions, which every test we run does already and
does regardless of version.

relates elastic/x-pack-elasticsearch#3068

Original commit: elastic/x-pack-elasticsearch@ceb42d7f47
2018-03-14 21:57:22 +01:00
Nik Everett d1b0067e47 SQL: Ban PrintWriter#println in CLI (elastic/x-pack-elasticsearch#4118)
`PrintWriter#println` doesn't pay attention to the terminal
configuration that we specify so it breaks tests on Windows. If we
instead always use `PrintWriter#print('\n')` then the tests work
properly on windows *and* the CLI actually works properly on Windows.

relates elastic/x-pack-elasticsearch#4109

Original commit: elastic/x-pack-elasticsearch@ac17e691c8
2018-03-14 16:15:32 -04:00
Lee Hinman b14baf4a6f Decouple XContentBuilder from BytesReference (elastic/x-pack-elasticsearch#4119)
* Decouple XContentBuilder from BytesReference

This commit handles the removal of all mentions of BytesReference from
XContentBuilder. This is needed so that we can completely decouple the XContent
code and move it into its own dependency.

This is the x-pack side of https://github.com/elastic/elasticsearch/pull/28972

Original commit: elastic/x-pack-elasticsearch@8ba2e97b26
2018-03-14 13:48:05 -06:00
Tim Brooks a29972c498 Add start_basic action to x-pack client actions
This is related to elastic/x-pack-elasticsearch#3877. This commit adds the start_basic action as a
client action in x-pack.

Original commit: elastic/x-pack-elasticsearch@b89275b92d
2018-03-14 08:24:12 -06:00
Jason Tedor 5c48fc4eaa Adjust main response constructor arguments
This commit adjusts the invocation of the main response constructor as
these were changed upstream to remove a parameter.

Original commit: elastic/x-pack-elasticsearch@846b33c9e9
2018-03-14 07:44:40 -04:00
Tom Veasey 1d525f998c Model memory has increased so we need to update some integration test thresholds (elastic/x-pack-elasticsearch#4033)
Original commit: elastic/x-pack-elasticsearch@f9b77230ce
2018-03-14 10:59:25 +00:00
Tim Brooks 498c110073 Add type parameter to start_trial api (elastic/x-pack-elasticsearch#4102)
This is related to elastic/x-pack-elasticsearch#3877. This commit adds a paramer type to the
start_trial api. This parameter allows the user to pass a type (trial,
gold, or platinum) of license that will be generated. No matter what
type is choosen, you can only generate one per major version.

Original commit: elastic/x-pack-elasticsearch@b42234cbb5
2018-03-13 19:28:11 -06:00
Chris Earle 3c82f24637 [TEST][Monitoring] Remove check for requires_keystore (elastic/x-pack-elasticsearch#4108)
This removes the check for 'requires_keystore' from the plugin output,
which was removed unexpectedly to this test.

Original commit: elastic/x-pack-elasticsearch@80c5c03e20
2018-03-13 13:25:21 -07:00
Jason Tedor fbb752c273 Use Elasticsearch IOUtils (elastic/x-pack-elasticsearch#4105)
This commit replaces the usage of Lucene IOUtils with Elasticsearch
IOUtils, the former of which is now forbidden.

Original commit: elastic/x-pack-elasticsearch@8e0554001f
2018-03-13 12:49:52 -04:00
Nik Everett 5c3b69fe57 SQL: Clean up catch
Use a mutli-catch because it is a little cleaner.

Original commit: elastic/x-pack-elasticsearch@8555e82bca
2018-03-13 10:05:58 -04:00
Nik Everett 47a2f63d5e SQL: Be more careful with break and eof (elastic/x-pack-elasticsearch#4092)
The SQL CLI was being a bit cavalier about `null`, `ctrl-c`, and
`ctrl-d` while reading passwords to the point where it'd halt with
an exception if the user hit `ctrl-d` while typing a password. This
changes it so that the CLI will instead shut down if the user
`ctrl-c`s or `ctrl-d`s while on the password prompt with an
ENOPERM error code.

This also fixes a packaging test failure I caused by a copy and paste
error where the CLI was always enforcing things as though it was reading
a password all the time. This error was causing packaging test failures.

Original commit: elastic/x-pack-elasticsearch@a882c50fc7
2018-03-13 09:00:53 -04:00
Tim Brooks 7f7ac08447 Add api to start basic license (elastic/x-pack-elasticsearch#4083)
This is related to elastic/x-pack-elasticsearch#3877. This commit adds a route /start_basic that
will self generate a basic license. The only validation that is
performed is to check that you do not already have a basic license
installed. Additionally, if you lose features from switching to a basic
license, you must acknowledge the changes.

Original commit: elastic/x-pack-elasticsearch@7b8eeb50b1
2018-03-12 14:39:58 -06:00
Tim Brooks de10e61765 Disable ClusterStatsMonitoringDocTests test
ClusterStatsMonitoringDocTests.testToXContent is currently failing on
master. An issue (elastic/x-pack-elasticsearch#4100) has been created for this test. This commit
disables it in the meantime.

Original commit: elastic/x-pack-elasticsearch@76557313e6
2018-03-12 12:36:23 -06:00
Jason Tedor 839a776dad Adapt tests to package keystore creation (elastic/x-pack-elasticsearch#4068)
This commit adapts the X-Pack packaging tests to the change in
Elasticsearch to create the keystore on package installation.

Original commit: elastic/x-pack-elasticsearch@e86c98fa83
2018-03-12 12:49:50 -04:00
Yannick Welsch 4bce53a1ad Disallow logger methods with Object parameter
Relates to elastic/elasticsearch#28969

Original commit: elastic/x-pack-elasticsearch@1eff5eecd9
2018-03-12 11:03:23 +01:00
Tanguy Leroux 15ab4af157 [Monitoring] Align indices/index stats with local cluster state (elastic/x-pack-elasticsearch#4079)
A small bug in the `IndexStatsCollector` can potentially returns
statistics for newly created indices that does not exist yet in the
collector's `ClusterState` local instance.

It happens because an instance of the current `ClusterState` is
captured and passed to all the collectors before they are executed (so
that they all share the same view of the state of the cluster). On
some clusters, if an index is created after the `ClusterState` is
captured but before the `IndicesStatsRequest` is executed then it can
appears in the index stats but have no corresponding entry in the
local cluster state.

This commit changes the IndexStatsCollector so that it only return
statistics for indices that already exist in the cluster state. This
way a consistent view is possible between indices/index/shard stats.

Original commit: elastic/x-pack-elasticsearch@da173ae0b0
2018-03-12 10:32:54 +01:00
Tim Vernum 41af46688a Make PKI BootstrapCheck work with SecureSettings (elastic/x-pack-elasticsearch#3993)
SslConfiguration can depend on SecureSettings, so it must be
constructed during the correct lifecycle phase.
For PkiRealmBootstrapCheck, moved the construction of SslConfiguration
objets into the constructor rather than the check method

Original commit: elastic/x-pack-elasticsearch@1a4d147216
2018-03-12 10:48:35 +10:00
Albert Zaharovits d31d90d378 Auditing requests with null indices (elastic/x-pack-elasticsearch#4016)
Adds null check.

relates elastic/x-pack-elasticsearch#3988

Original commit: elastic/x-pack-elasticsearch@64bab62ca6
2018-03-11 13:13:14 +02:00
Tim Brooks 8d68b03cb6 Stop using basic license in put license test
This is related to elastic/x-pack-elasticsearch#4095. That test uses the a basic license in a test
of the route put license. Occasionally, that license is extended due to
recent work related to indefinite basic licenses before the test
assertions can be performed. This commit changes the test to use a gold
license.

Original commit: elastic/x-pack-elasticsearch@bf2550f044
2018-03-10 09:53:20 -07:00
David Roberts 32bc247789 [ML] Adjust the name of the ML C++ repo (elastic/x-pack-elasticsearch#4020)
Relates elastic/machine-learning-cpp#544

Companion to elastic/release-manager#296

Original commit: elastic/x-pack-elasticsearch@d65ea9add5
2018-03-09 22:53:38 +00:00
Lisa Cawley 58d6c79c00 [DOCS] Added ML limitation (elastic/x-pack-elasticsearch#4081)
Original commit: elastic/x-pack-elasticsearch@378bf49b1d
2018-03-09 09:22:12 -08:00
Tim Brooks ae383462a7 Modify self-generated basic licenses to not expire (elastic/x-pack-elasticsearch#3952)
This is related to elastic/x-pack-elasticsearch#3877. It modifies self-generated basic licenses to
(practically) never expire. Specifically, self-generated basic licenses
will be set with an expiration date 1 year before Long.MAX_VALUE 
Additionally, basic licenses with a different expiration date will be
replaced with a new self-generated basic licenses at startup.

Original commit: elastic/x-pack-elasticsearch@de8b343089
2018-03-09 09:54:30 -07:00
Ioannis Kakavas 1cc20c4c59 [DOCS] Explain possible values for IDP EntityID (elastic/x-pack-elasticsearch#3875)
Resolves elastic/x-pack-elasticsearch#3865

Original commit: elastic/x-pack-elasticsearch@9102bc1a61
2018-03-09 14:07:51 +02:00
Ioannis Kakavas 558679f997 Disregard comments in XML documents (elastic/x-pack-elasticsearch#4047)
* Disregard comments in XML documents
* Add tests to verify comments in XML are ignored

Original commit: elastic/x-pack-elasticsearch@8b2d8d32ef
2018-03-08 22:32:33 -08:00
Tim Vernum 3a4fa16f03 [SAML] Handle ACS URL with existing query params (elastic/x-pack-elasticsearch#4060)
If the Assertion Consumer Service URL already contained query
parameters, we would incorrectly append an addtional '?' rather than
adding the SAML parameters to the end with '&'

Original commit: elastic/x-pack-elasticsearch@60b6a977d8
2018-03-09 17:15:55 +11:00
Chris Earle c658238f33 [Logstash][Monitoring] Fix Registered Usage and Add Tests (elastic/x-pack-elasticsearch#4075)
This properly registers the `XPackFeatureSetUsage` for Logstash and
it tests it by invoking the Usage API in a Monitoring QA test.

Without those being properly registered, the test will consistently fail.

Original commit: elastic/x-pack-elasticsearch@2e8f2376fd
2018-03-08 14:53:05 -08:00
Lisa Cawley 277bd59e4f [DOCS] Remove ML CCS limitation (elastic/x-pack-elasticsearch#4082)
Original commit: elastic/x-pack-elasticsearch@8a44962435
2018-03-08 11:12:58 -08:00
Zachary Tong aa877161ff [Rollup] Register FeatureSetUsage with xpack, add tests (elastic/x-pack-elasticsearch#4040)
We had a Usage class before, but weren't registering it with XPack.
Would be nice to add more usage info in the future (like the running
jobs on each node), but unclear the best way to do it since we'd need
to filter through the list of allocated tasks.

Original commit: elastic/x-pack-elasticsearch@5207d2758b
2018-03-08 08:06:42 -08:00
Dimitris Athanasiou 1ed31af2c6 [ML] Allow model_memory_limit to be reduced (elastic/x-pack-elasticsearch#3998)
Up to now a job update that reduces the model memory limit
was not allowed. However, there could definitely be cases
where reducing the limit is necessary and reasonable.

This commit makes it possible to decrease the limit as long
as it does not go below the current memory usage. We obtain
the latter from the model size stats.

The conditions under which updating the model_memory_limit
is not allowed are now:

 - when the job is open
 - latest model_size_stats.model_bytes < new value

relates elastic/x-pack-elasticsearch#2461

Original commit: elastic/x-pack-elasticsearch@5b35923590
2018-03-08 06:14:18 -08:00
Shaunak Kashyap 11cd9097b6 Reverting change made for node group IDs (elastic/x-pack-elasticsearch#4069)
The node group feature merits further discussion so we're going to undo this change for now.

Original commit: elastic/x-pack-elasticsearch@48f82b4cc0
2018-03-07 14:40:19 -08:00
Lisa Cawley 095d6d466c [DOCS] Update types in datafeed resource (elastic/x-pack-elasticsearch#4011)
Original commit: elastic/x-pack-elasticsearch@6692b97c5e
2018-03-07 13:53:49 -08:00
Lee Hinman 2ecce78b13 Decouple XContentType from StreamInput/Output (elastic/x-pack-elasticsearch#4065)
This removes the readFrom and writeTo methods from XContentType, instead using
the more generic `readEnum` and `writeEnum` methods. Luckily they are both
encoded exactly the same way, so there is no compatibility layer needed for
backwards compatibility.

This is the x-pack side of https://github.com/elastic/elasticsearch/pull/28927

Original commit: elastic/x-pack-elasticsearch@f1c0928490
2018-03-07 14:50:33 -07:00
lcawley 39c1dd085a [DOCS] Added link to security commands
Original commit: elastic/x-pack-elasticsearch@168167517b
2018-03-07 13:11:32 -08:00
Lee Hinman 5f95e7ce87 Remove BytesRef usage from WatcherXContentParser (elastic/x-pack-elasticsearch#4024)
This conforms WatcherXContentParser for the changes in
https://github.com/elastic/elasticsearch/pull/28792

Original commit: elastic/x-pack-elasticsearch@d2d214ef89
2018-03-07 10:10:03 -07:00
Igor Motov a0800c0b09 SQL: Move parameter processing from the client to the server (elastic/x-pack-elasticsearch#3822)
Moves the `?` parameter substitution in prepared from the client to 
the server side, where it can be reused by the ODBC driver and can 
be implemented in a safer manner.

relates elastic/x-pack-elasticsearch#3699

Original commit: elastic/x-pack-elasticsearch@b876ccd8ae
2018-03-07 09:25:25 -05:00
Jason Tedor ead1c6c315 Fix Javadocs for o.e.x.r.j.RollupIndexer
This commit fixes the Javadocs for the class o.e.x.r.j.RollupIndexer as
these Javadocs were referring to instance methods on the class
incorrectly (using a this prefix).

Original commit: elastic/x-pack-elasticsearch@fdcc7338f9
2018-03-06 14:12:42 -08:00
Lisa Cawley 6e87d95f9b [DOCS] Added licensing APIs (elastic/x-pack-elasticsearch#4026)
Original commit: elastic/x-pack-elasticsearch@0e50cc0d64
2018-03-06 09:47:04 -08:00
Lee Hinman 2147d217df Wrap stream passed to createParser in try-with-resources (elastic/x-pack-elasticsearch#4055)
This wraps the stream (`.streamInput()`) that is passed to many of the
`createParser` instances in the enclosing (or a new) try-with-resources block.
This ensures the `BytesReference.streamInput()` is closed.

Relates to elastic/x-pack-elasticsearch#28504

Original commit: elastic/x-pack-elasticsearch@7546e3b4d4
2018-03-04 16:48:15 -07:00
Alexander Reelsen 792fcccf46 REST Tests: Remove human parameter (elastic/x-pack-elasticsearch#4032)
This parameter already exists in the core _common.json, no need to
repeat.

relates elastic/x-pack-elasticsearch#3575

Original commit: elastic/x-pack-elasticsearch@49cd3e0df5
2018-03-04 11:54:09 -08:00
Alexander Reelsen 918f62f97a Docs: Remove mentions of file based script in watcher
relates elastic/x-pack-elasticsearch#3755

Original commit: elastic/x-pack-elasticsearch@cfc023543d
2018-03-04 11:49:42 -08:00
Chris Earle af7ad4f366 [TEST][Monitoring] Move payload = null above call (elastic/x-pack-elasticsearch#4053)
This moves the `payload = null;` statement to above the asynchronous
HTTP call. This helps to avoid a race condition relative to `doClose`
asserting that it is `null`.

This is only a realistic situation during a shutdown situation because
the thread responds immediately before it can be nullified and freeable,
which is not a realistic scenario in any other situation.

Original commit: elastic/x-pack-elasticsearch@eb3a6ff118
2018-03-03 21:08:35 -08:00