Commit Graph

291 Commits

Author SHA1 Message Date
jaymode ff6fa6790e Test: fix kibana user role works in mixed cluster on older versions
The kibana user role in mixed cluster test expects a old cluster test to have run first that would
disable the kibana user, but that test is not executed on 5.1.1 and prior versions. This change
makes a update request an upsert to account for this.

relates elastic/x-pack-elasticsearch#1527

Original commit: elastic/x-pack-elasticsearch@df2945610e
2017-06-01 14:26:57 -06:00
David Kyle e16787ae7b Fix ML upgrade tests
Original commit: elastic/x-pack-elasticsearch@50e114e7eb
2017-06-01 20:26:57 +01:00
jaymode 68ad44c5c2 Test: enable default password migration test for versions > 5.1.1
Versions below 5.1.1 had a bug where the rest set enabled action would trip an assertion and cause
this test to fail so we skip those versions.

relates elastic/x-pack-elasticsearch#1523

Original commit: elastic/x-pack-elasticsearch@fba8ddd7e7
2017-06-01 08:37:39 -06:00
David Kyle 34f526b60b [ML] Yml rolling upgrade tests (elastic/x-pack-elasticsearch#1555)
* ML bwc jobs tests

* Test for starting old data feeds

Original commit: elastic/x-pack-elasticsearch@131c7b15eb
2017-06-01 14:29:10 +01:00
Nik Everett 592af606f8 Build: fix the rolling restarts for real
I had the numbers backwards. 🤦

Original commit: elastic/x-pack-elasticsearch@8f2d21db52
2017-05-26 17:31:50 -04:00
Nik Everett 928589bd89 Build: Use correct data node in test
We were getting lock errors because I hand ported something and made
a copy and paste error.

Original commit: elastic/x-pack-elasticsearch@9896ba4cc7
2017-05-26 15:20:44 -04:00
Nik Everett 43928708fa Build: handle core's change to data directory
Core changed how the data directory is configured so we have to
conform to the new way.

Original commit: elastic/x-pack-elasticsearch@360e2fea1a
2017-05-26 13:32:39 -04:00
Nik Everett b215c66778 Add tests for wire compatibility when system key is enabled (elastic/x-pack-elasticsearch#1557)
Reworks the rolling restart tests so they'd have caugh an
incompatibility in the wire protocol that we say between 5.4.0 and
5.4.1.

Original commit: elastic/x-pack-elasticsearch@f5e69cf58e
2017-05-26 12:17:00 -04:00
Alexander Reelsen e9fed20d96 Watcher: Use execute watch API in tests to prevent wrong watch history query
This test failed, because there could have been other results in the
watch history. However by switching to the execute watch API, there
is no need to query the watch history at all. This also removes a 5
second wait time in the test.

relates elastic/x-pack-elasticsearch#1549

Original commit: elastic/x-pack-elasticsearch@692779521e
2017-05-26 14:27:32 +02:00
Ryan Ernst 3b58334efb Refactor script context generic types (elastic/x-pack-elasticsearch#1547)
This is the xpack side of
https://github.com/elastic/elasticsearch/pull/24877

Original commit: elastic/x-pack-elasticsearch@4f3a1a27e6
2017-05-24 19:20:59 -07:00
Ryan Ernst 9a7c28786a Scripting: Use type aware script contexts (elastic/x-pack-elasticsearch#1538)
This creates two different script contexts for watcher, one which may be
used for SearchScript, and another for ExecutableScript.
This is the xpack side of elastic/elasticsearch#24868.

Original commit: elastic/x-pack-elasticsearch@9ae3d45fed
2017-05-24 14:29:25 -07:00
Alexander Reelsen 3d057991e0 Tests: Various watcher test improvements
* Reduced a longish timeout to a shorter one, as a watch should be
  executed in a HTTP test.
* Ensured that the TimeThrottleIntegration tests only query for own
  watches in the watch history, also use random names for watch ids
* HipChatServiceTests configured deprecated logging package, so it was
  not possible to follow the HTTP calls to the hipchat service endpoint.

relates elastic/x-pack-elasticsearch#1514
Relates elastic/x-pack-elasticsearch#1515

Original commit: elastic/x-pack-elasticsearch@adb492e4e9
2017-05-24 14:08:02 +02:00
Jason Tedor 284bf2512a Fix rolling upgrade test dependency
The rolling upgrade nodes need a keystore for SSL configuration but
there was no dependency on the task that copies the keystore into the
output directory for the nodes to pick up as an extra configuration
file. This commit addresses this by adding such a dependency. To do
this, we need to break the dependency of the keystore copy task on the
REST spec copy task; this is not an issue since the dependency was for
convenience of ordering the task and not actually needed.

Original commit: elastic/x-pack-elasticsearch@fddbc06e9f
2017-05-23 21:44:59 -04:00
Jason Tedor ed827970f1 Mark failing password migration test as skipped
A previous commit marked the wrong test as skipped, this commit marks
the correct issue.

Original commit: elastic/x-pack-elasticsearch@0723964206
2017-05-23 14:25:02 -04:00
Jason Tedor 3a151ac320 Mark failing password migration test as skipped
This test is failing reliably, so this commit marks it as skipped.

Original commit: elastic/x-pack-elasticsearch@15afd615d3
2017-05-23 14:13:57 -04:00
Jason Tedor dde2600181 Mark verify Kibana user works as skipped
This test fails reliably, so we mark it as skipped.

Original commit: elastic/x-pack-elasticsearch@4fc8a6883e
2017-05-23 13:35:44 -04:00
Jason Tedor 6ce8bd69ef Mark default password migration test as skipped
This test fails reliably, so this commit marks the test as awaits fix.

Original commit: elastic/x-pack-elasticsearch@8eb8b732ba
2017-05-23 13:35:44 -04:00
Jason Tedor 87ff7f0b52 Mark default password migration test as awaits fix
This commit marks the default password migration test as awaits fix as
the test is reliabily failling.

Original commit: elastic/x-pack-elasticsearch@4377480d98
2017-05-23 12:02:11 -04:00
David Kyle ea0f3fe4a0 [ML] Wait for a stopping datafeed (elastic/x-pack-elasticsearch#1461)
* [ML] Wait for a stopping datafeed

* Fix compilation after rebase

* Address review comments

Original commit: elastic/x-pack-elasticsearch@2baed641e9
2017-05-23 14:31:19 +01:00
Ryan Ernst 68e0bc419a Use simplified ScriptContext (elastic/x-pack-elasticsearch#1502)
This is the xpack side of elastic/elasticsearch#24818

Original commit: elastic/x-pack-elasticsearch@0edbbab431
2017-05-22 13:11:23 -07:00
Ryan Ernst 883f5d8a7a Remove overrides of isInlineScriptEnabled (elastic/x-pack-elasticsearch#1500)
This is the xpack side of elastic/elasticsearch#24815

Original commit: elastic/x-pack-elasticsearch@5aa314a49d
2017-05-20 12:01:34 -07:00
Jack Conradson b92a2f6582 Remove deprecated script settings. (elastic/x-pack-elasticsearch#1469)
Original commit: elastic/x-pack-elasticsearch@069e5a1087
2017-05-18 13:32:57 -07:00
Jay Modi 667f842f92 Fix authentication forward compatibility (elastic/x-pack-elasticsearch#1481)
The authentication object was changed in 5.4.0 in that it was conditionally signed depending on
the version and other factors. A bug was introduced however that causes the authentication to
actually get written with the version of the node it is being sent to even if that version is
greater than the version of the current node, which causes rolling upgrades to fail.

Original commit: elastic/x-pack-elasticsearch@a718ff8a52
2017-05-18 15:30:53 -04:00
Dimitris Athanasiou 1bb7651dba [ML] Refactor filters API to not use _type (elastic/x-pack-elasticsearch#1483)
- Removes dependence on _type for filters.
- Changes the put filter API to take the id in the URI
- Prepares .ml-meta index to be able to host more types in future

Relates elastic/x-pack-elasticsearch#668

Original commit: elastic/x-pack-elasticsearch@d4cffa9382
2017-05-18 18:09:20 +01:00
jaymode 1cc4ec95f3 Test: fix kibana write rolling upgrade test
This commit fixes the kibana write rolling upgrade test. The test needs to enable the kibana user
as another test explicitly disables it. After the test runs it disables the kibana user again.

relates elastic/x-pack-elasticsearch#1460

Original commit: elastic/x-pack-elasticsearch@437495432d
2017-05-18 10:12:34 -04:00
Ryan Ernst d6a9185857 Build: Convert rolling upgrade test to create tasks per bwc version (elastic/x-pack-elasticsearch#1477)
This is the xpack equivalent of elastic/elasticsearch#24758

Original commit: elastic/x-pack-elasticsearch@73e1a9c6b4
2017-05-18 02:25:54 -07:00
Ryan Ernst 58bd30e64f Fix put script calls in script tests with watcher
Original commit: elastic/x-pack-elasticsearch@2eac2e5192
2017-05-18 00:52:55 -07:00
Ryan Ernst d475010d1b Remove leftover file template references
Original commit: elastic/x-pack-elasticsearch@04e98c6261
2017-05-17 23:22:27 -07:00
Tim Vernum 7052194094 [TESTS] Fix filename pattern for templating in SSL smoke test
Relates to elastic/x-pack-elasticsearch@4c1d82c

Original commit: elastic/x-pack-elasticsearch@f5cbc1c6b3
2017-05-18 11:07:42 +10:00
Ryan Ernst f7705eac86 Remove file scripts (elastic/x-pack-elasticsearch#1399)
This is the xpack side of elastic/elasticsearch#24627


Original commit: elastic/x-pack-elasticsearch@4d1c745d74
2017-05-17 14:42:46 -07:00
Ryan Ernst 573da95e26 Use new wireCompatVersions property instead of bwcVersion (elastic/x-pack-elasticsearch#1466)
This is the xpack side of elastic/elasticsearch#24748

Original commit: elastic/x-pack-elasticsearch@8b7dd5cdbe
2017-05-17 12:58:51 -07:00
jaymode cc167467aa Test: mute kibana_write rest test that never ran or worked
See elastic/x-pack-elasticsearch#1460

Original commit: elastic/x-pack-elasticsearch@505c8b8832
2017-05-17 11:45:22 -04:00
Simon Willnauer bfc3779f51 Fix indentation in 30_kibana_write.yml
Original commit: elastic/x-pack-elasticsearch@6321493b03
2017-05-17 17:13:40 +02:00
Simon Willnauer 4f94624277 Rename test from `.yaml` to `yml`
We cut over to run tests only for `.yml` files in  elastic/elasticsearch#24659
The corresponding x-pack change was missing, this commit fixes it.

relates elastic/x-pack-elasticsearch#1457

Original commit: elastic/x-pack-elasticsearch@4c1d82c209
2017-05-17 15:39:57 +02:00
David Kyle abbdf232aa [ML] Test ML with the Transport Client (elastic/x-pack-elasticsearch#1440)
* Hide ML actions for tribe node client
* Remove unused parameters
* Enable ML actions and rest endpoints for the transport client
* Create the ML components for the transport client
* Add ml transport client tests

Original commit: elastic/x-pack-elasticsearch@509007ca29
2017-05-16 14:34:44 +01:00
Tim Vernum 463133b7de [Security] Cross cluster wildcard security (elastic/x-pack-elasticsearch#1290)
Support the resolution of remote index names, including those that contain wildcards in the cluster name or index part)

Specifically these work:
- `GET /remote*:foo/_search`
- `GET /*:foo/_search`
- `GET /*:foo,*/_search`
- `GET /remote:*/_search`
- `GET /*:*/_search`

This change assumes that every user is allowed to attempt a cross-cluster search against any remote index, and the actual authorisation of indices happens on the remote nodes. Thus ` GET /*:foo/_search` will expand to search the `foo` index on every registered remote without consideration of the roles and privileges that the user has on the source cluster.

Original commit: elastic/x-pack-elasticsearch@b45041aaa3
2017-05-15 15:02:13 +10:00
Dimitrios Athanasiou 7293bcdc1c [TEST] Set data_description in MlTransportClientIT
Original commit: elastic/x-pack-elasticsearch@0d392d26d8
2017-05-12 18:51:38 +01:00
Alexander Reelsen 50e9e413da Watcher: Make watch history use doc type instead of watch_record (elastic/x-pack-elasticsearch#1311)
As this does not require any reindexing this is easy to fix by just
changing the watch history template.

In addition the old templates are deleted on start up and the new ones
are instantiated.

Original commit: elastic/x-pack-elasticsearch@7e1ad495ad
2017-05-12 16:52:57 +02:00
Simon Willnauer 891c2a6c3f Replace XPacks delete_by_query impl with the core impl (elastic/x-pack-elasticsearch#1378)
This can now be shared with core once elastic/elasticsearch#24578

Original commit: elastic/x-pack-elasticsearch@42bbd75aee
2017-05-11 20:23:55 +02:00
David Roberts cd99024599 [ML] Validate job configs before transport (elastic/x-pack-elasticsearch#1375)
If invalid job configs are transported to the master node then the root
cause of the validation exception gets reported as a remote_transport_exception,
which is extremely confusing.

This commit moves the validation of job configurations to the first node that
handles the action.

Fixes elastic/x-pack-kibana#1172

Original commit: elastic/x-pack-elasticsearch@5ed59d2a6f
2017-05-10 10:55:16 +01:00
Ryan Ernst 85deb1f8f7 Rename ScriptEngineService to ScriptEngine (elastic/x-pack-elasticsearch#1374)
This is the xpack side of elastic/elasticsearch#24574

Original commit: elastic/x-pack-elasticsearch@4d37021f6d
2017-05-10 00:47:39 -07:00
Ryan Ernst 1c3d907748 Reverse runAs user setup to store authenticated user inside runAs user (elastic/x-pack-elasticsearch#1371)
Original commit: elastic/x-pack-elasticsearch@8276662298
2017-05-09 13:49:14 -07:00
David Kyle e5b11d0222 [ML] Not an error to close a job twice (elastic/x-pack-elasticsearch#1340)
* [ML] Not an error to close a job twice

* Error if job is opening

* Address review comments

* Test closed job isn’t resolved

Original commit: elastic/x-pack-elasticsearch@7da7b24c08
2017-05-08 16:34:46 +01:00
Simon Willnauer 5a4b7ee98b [TEST] Add REST test to search via unfiltered alias in CrossClusterSearch
Original commit: elastic/x-pack-elasticsearch@4e1b99824c
2017-05-05 10:25:06 +02:00
Simon Willnauer eddc7a2147 Add tests for cross cluster `_field_caps` API (elastic/x-pack-elasticsearch#1294)
This commit adds REST tests to ensure `_field_caps` works with cross cluster
search and security.

Relates to elastic/elasticsearch#24463

Original commit: elastic/x-pack-elasticsearch@2af0a8c179
2017-05-04 15:04:39 +02:00
Alexander Reelsen 4078b2f1b2 Watcher: Replace _status field with status (elastic/x-pack-elasticsearch#1285)
As fields with underscores will be disallowed in master, and we have to
prepare the upgrade, this commit renames the _status field to status.

When the 5.x upgrade logic is in place in the 5.x we can remove all the
old style _status handling from the master branch.

Note: All the BWC compatibility tests, that load 5.x indices are now
faking a finished upgrade by adding the `status` field to the mapping
of the watches index.

Original commit: elastic/x-pack-elasticsearch@9d5cc9aaec
2017-05-04 10:08:34 +02:00
Alexander Reelsen 50241717d3 Tests: Use execute watch API instead of persisting watch
To speed up the test and ensure no other failures occur,
simply use the execute watch API instead of persisting the watch
and executing then.

relates elastic/x-pack-elasticsearch#1350

Original commit: elastic/x-pack-elasticsearch@2888fb0e93
2017-05-02 14:04:04 +02:00
Alexander Reelsen c62f6f8177 Watcher: Distributed watch execution (elastic/x-pack-elasticsearch#544)
The distribution of watches now happens on the node which holds the
watches index, instead of on the master node. This requires several
changes to the current implementation.

1. Running on shards and replicas
   In order to run watches on the nodes with the watches index on its
   primaries and replicas. To ensure that watches do not run twice, there is
   a logic which checks the local shards, runs a murmurhash on the id and
   runs modulo against the number of shards and replicas, this is the way to
   find out, if a watch should run local. Reloading happens
2. Several master node actions moved to a HandledTransportAction, as they
   are basically just aliases for indexing actions, among them the
   put/delete/get watch actions, the acknowledgement action, the de/activate
   actions
3. Stats action moved to a broadcast node action, because we potentially
   have to query every node to get watcher statistics
4. Starting/Stopping watcher now is a master node action, which updates
   the cluster state and then listeners acts on those. Because of this watches
   can be running on two systems, if you those have different cluster state
   versions, until the new watcher state is propagated
5. Watcher is started on all nodes now. With the exception of the ticker
   schedule engine most classes do not need a lot of resources while running.
   However they have to run, because of the execute watch API, which can hit
   any node - it does not make sense to find the right shard for this watch
   and only then execute (as this also has to work with a watch, that has not
   been stored before)
6. By using a indexing operation listener, each storing of a watch now
   parses the watch first and only stores on successful parsing
7. Execute watch API now uses the watcher threadpool for execution
8. Getting the number of watches for the stats now simply queries the
   different execution engines, how many watches are scheduled, so this is
   not doing a search anymore

There will be follow up commits on this one, mainly to ensure BWC compatibility.

Original commit: elastic/x-pack-elasticsearch@0adb46e658
2017-05-02 10:12:46 +02:00
Tim Vernum ddf5fd68c2 Add ClusterSearchShardsAction to "read_cross_cluster" privilege (elastic/x-pack-elasticsearch#1231)
Cross cluster search uses ClusterSearchShardsAction under the covers.
Without this change, you would need both "read_cross_cluster" and "view_index_metadata" privilegs in order to have permission to execute searches from a remote cluster.

Original commit: elastic/x-pack-elasticsearch@65a6aff329
2017-04-27 22:39:13 +10:00
Yannick Welsch c6b8542758 [TEST] Adapt wait_for_nodes condition for ClusterFormationTasks with tribes
Companion commit for elastic/elasticsearch#24351

Original commit: elastic/x-pack-elasticsearch@0f57eecbdb
2017-04-27 09:57:54 +02:00