Commit Graph

319 Commits

Author SHA1 Message Date
Martijn van Groningen 8cc4f29f33 test: make sure analysis-common module is also available on the cluster 1 and cluster2 nodes.
Original commit: elastic/x-pack-elasticsearch@eef5d2b566
2017-06-15 23:25:45 +02:00
jaymode ee3d17adfd Test: fix security rolling upgrade tests that were failing
This commit fixes the default password migration tests that had been failing reproducibly. The
first fix skips tests using the set enabled api when running against a version prior to 5.1.2 as
this api would otherwise trip an assertion that the xcontent builder was not closed. The second
fix is to ensure we set the password field in the user object.

relates elastic/x-pack-elasticsearch#1529
relates elastic/x-pack-elasticsearch#1516

Original commit: elastic/x-pack-elasticsearch@2f9c804309
2017-06-15 14:50:26 -06:00
Martijn van Groningen 863755d2da Make sure that the module are installed too.
Original commit: elastic/x-pack-elasticsearch@f581d0902c
2017-06-15 22:03:18 +02:00
David Roberts b748da1880 [ML] Prevent time_field and control field name in analysis_config (elastic/x-pack-elasticsearch#1729)
In does not make sense for the time_field in the data_description to
be used as a by/over/partition field name, nor the summary_count_field,
categorization_field or as an influencer.  Therefore, configurations
where the time_field in the data_description is used in the
analysis_config are now rejected.

Additionally, it causes a problem communicating with the C++ code if
the control field name (which is '.') is used in the analysis_config,
so this is also rejected at the validation stage.

Relates elastic/x-pack-elasticsearch#1684

Original commit: elastic/x-pack-elasticsearch@e6750a2cda
2017-06-15 13:04:25 +01:00
Jason Tedor 8c5e7b589c Use master flag for disabling BWC tests
This commit skips the the BWC tests if the master BWC flag
bwc_tests_enabled in core is set to false.

Relates elastic/x-pack-elasticsearch#1725

Original commit: elastic/x-pack-elasticsearch@7b924066a9
2017-06-15 07:45:20 -04:00
Nik Everett 0970c509bc Remove the assemble task from projects not published (elastic/x-pack-elasticsearch#1721)
Removes the `assemble` task from projects that aren't published
to speed up `gradle assemble` so the unified release can call it.

Original commit: elastic/x-pack-elasticsearch@43dfcc15f3
2017-06-14 19:57:26 -04:00
Simon Willnauer 97693b9357 Add scroll support for cross cluster search (elastic/x-pack-elasticsearch#1706)
Original commit: elastic/x-pack-elasticsearch@eadffa396b
2017-06-14 20:38:58 +02:00
Jay Modi 9c8e12280b Test: increase the wait for green cluster health calls (elastic/x-pack-elasticsearch#1703)
This commit increases the amount of time to wait for green cluster health during a rolling upgrade
to account for the time that may be needed in the case of delayed shards. Additionally some old
timeout values were removed as they were used due to the default timeout of 30s.

Relates elastic/x-pack-elasticsearch#1683

Original commit: elastic/x-pack-elasticsearch@9996673db0
2017-06-14 10:25:40 -06:00
James Baiera 1d3921f581 Fixing vagrant build file issues with new vagrant support plugin
Original commit: elastic/x-pack-elasticsearch@dbb5d4a215
2017-06-12 10:03:30 -04:00
Dimitris Athanasiou 8eb62eac27 [ML] Automate detection of way to extract fields (elastic/x-pack-elasticsearch#1686)
In 5.4.x, the datafeed attempts to get all fields from
doc_values by default. It has a `_source` parameter which
when enabled changes the strategy to instead try to get
all fields from the source.

This has been the most common issue users have been
reporting as it means the datafeed will fail to fetch
any text fields by default.

This change uses the field capabilities API in order
to automatically detect whether a field is aggregatable.
It then extracts such fields from doc_values while the
rest are taken from source. The change also adds
validation to the start datafeed action so that if
fields are missing mappings or the time field is not
aggregatable we respond with an appropriate error.

relates elastic/x-pack-elasticsearch#1649

Original commit: elastic/x-pack-elasticsearch@76e2cc6cb2
2017-06-12 14:56:31 +01:00
Alexander Reelsen 27b5142de6 Watcher: Fix croneval tool for packaging (elastic/x-pack-elasticsearch#1689)
The croneval script used an old parameter to start up.
This commit removes this parameter, that is used, when a
package is used.

In addition a concrete vagrant test has been added.

relates elastic/x-pack-elasticsearch#1635

Original commit: elastic/x-pack-elasticsearch@ea7b8a08f4
2017-06-12 13:56:26 +02:00
Boaz Leskes b5ab68fac8 qa/upgrade_cluster/10_basic.yaml add shard level info on health failure
Original commit: elastic/x-pack-elasticsearch@46847ca262
2017-06-09 21:59:46 +02:00
Chris Earle c356074606 [Test] Rolling Upgrade should preserve templates (elastic/x-pack-elasticsearch#1687)
Real upgrades will have preserved their templates, so rolling upgrade tests should preserve them as well and internal services should be expected to replace them as needed.

Original commit: elastic/x-pack-elasticsearch@93a155951e
2017-06-09 13:41:22 -04:00
jaymode 9b3ee9f96c Remove security trace logging from rolling uprade tests
Original commit: elastic/x-pack-elasticsearch@b0ecb80816
2017-06-09 10:02:44 -06:00
Ryan Ernst c1a3f50e19 Convert script uses to use source/id keys (elastic/x-pack-elasticsearch#1670)
This is the xpack side of
https://github.com/elastic/elasticsearch/pull/25127

Original commit: elastic/x-pack-elasticsearch@e25bd90825
2017-06-09 08:29:36 -07:00
David Roberts 048ff24b79 [TEST] Wait for correct template version in rolling upgrade tests (elastic/x-pack-elasticsearch#1682)
We wait for index templates to be installed before running tests, but
these can get upgraded when the master node is upgraded.  If we don't
wait again in this case then tests can fail due to the cleanup code
being overwhelmed by pending cluster state updates that weren't waited
for before the test.

Original commit: elastic/x-pack-elasticsearch@eff0b24c11
2017-06-09 15:23:15 +01:00
Nik Everett b8d86682ef Rework rolling restart tests (elastic/x-pack-elasticsearch#1590)
Reworks the rolling restart tests so that all configuration
options share code. Now there is a project per configuration:
* `qa:rolling-upgrade:with-ssl-with-system-key`
* `qa:rolling-upgrade:with-ssl-without-system-key`

Original commit: elastic/x-pack-elasticsearch@5672b4a4f2
2017-06-08 12:48:41 -06:00
David Roberts 64330ff14d Move bracket
(Matches part of a change made to 5.5/5.x that doesn't need fully forward
porting.)

Original commit: elastic/x-pack-elasticsearch@01f0837e6c
2017-06-08 18:01:58 +01:00
David Roberts 7cb1c8bd35 [TEST] Fix security test blacklist for new ML test
Original commit: elastic/x-pack-elasticsearch@b6a054a2a7
2017-06-08 10:12:33 +01:00
David Roberts f865755259 [ML] Tolerate InternalAssumptionViolatedException in overridden test (elastic/x-pack-elasticsearch#1657)
In order for elastic/elasticsearch#25100 to work, overridden test()
methods that call ESClientYamlSuiteTestCase.test() must not consume
any InternalAssumptionViolatedException that
ESClientYamlSuiteTestCase.test() throws.

Relates elastic/x-pack-elasticsearch#1650

Original commit: elastic/x-pack-elasticsearch@081ccaa0a6
2017-06-07 14:22:05 +01:00
Alexander Reelsen 887538d6bc Watcher: Fix BWC tests, disable test requiring upgrade API
Original commit: elastic/x-pack-elasticsearch@e8de71cdf7
2017-06-07 11:09:31 +02:00
David Kyle ae299f633e [ML] Validate initial job settings (elastic/x-pack-elasticsearch#1646)
* [ML] Validate initial job settings

* Add same job creation checks to the validate endpoint

Original commit: elastic/x-pack-elasticsearch@ab76cf9ea2
2017-06-07 09:34:58 +01:00
Chris Earle b7c9fd7c93 Remove monitoring index checks until they exist without timing related issues elastic/x-pack-elasticsearch#1574
Original commit: elastic/x-pack-elasticsearch@97e44e4679
2017-06-07 00:32:10 -04:00
Chris Earle a357c97aaf Wait explicitly for .monitoring-es-* instead of the more generic wait
Original commit: elastic/x-pack-elasticsearch@4e1e4e379a
2017-06-06 18:24:17 -04:00
Chris Earle 3436ba6ecb With .monitoring-data-N index gone, we no longer want to wait for it
Original commit: elastic/x-pack-elasticsearch@312f1341ae
2017-06-06 18:22:00 -04:00
Jay Modi 2d893df7e9 Add better authorization for scroll requests and remove signing (elastic/x-pack-elasticsearch#1416)
This commit adds better security for scroll requests in that they are now tied to a single user as
we only authorize the request that creates the scroll. This is accomplished by adding a
SearchOperationListener that listens for new scroll contexts and stores the authentication on the
ScrollContext. Then upon
retrieval of the search context for a query or fetch, the current authentication is compared to the
authentication that was present when the scroll context was created. If the current authentication
belongs to a different user, then a SearchContextMissingException will be thrown to prevent leaking
a valid vs invalid scroll id.

Additionally, signing of a scroll id is only performed when there is a older node in the cluster
that would expect the scroll id to be signed. Once this is backported to 5.x, we can remove this
bwc layer for 6.0/master.

Original commit: elastic/x-pack-elasticsearch@0e5dcafd32
2017-06-06 10:23:18 -06:00
David Roberts b55d301a22 [TEST] Add more exclusions to ML security tests
Original commit: elastic/x-pack-elasticsearch@af9acc61ee
2017-06-05 11:37:12 +01:00
Ryan Ernst 7ee8eccf95 Script: Convert uses of CompiledTemplate to TemplateScript (elastic/x-pack-elasticsearch#1630)
This is the xpack side of elastic/elasticsearch#25032

Original commit: elastic/x-pack-elasticsearch@ba7df4f6ce
2017-06-02 13:41:33 -07:00
jaymode ff6fa6790e Test: fix kibana user role works in mixed cluster on older versions
The kibana user role in mixed cluster test expects a old cluster test to have run first that would
disable the kibana user, but that test is not executed on 5.1.1 and prior versions. This change
makes a update request an upsert to account for this.

relates elastic/x-pack-elasticsearch#1527

Original commit: elastic/x-pack-elasticsearch@df2945610e
2017-06-01 14:26:57 -06:00
David Kyle e16787ae7b Fix ML upgrade tests
Original commit: elastic/x-pack-elasticsearch@50e114e7eb
2017-06-01 20:26:57 +01:00
jaymode 68ad44c5c2 Test: enable default password migration test for versions > 5.1.1
Versions below 5.1.1 had a bug where the rest set enabled action would trip an assertion and cause
this test to fail so we skip those versions.

relates elastic/x-pack-elasticsearch#1523

Original commit: elastic/x-pack-elasticsearch@fba8ddd7e7
2017-06-01 08:37:39 -06:00
David Kyle 34f526b60b [ML] Yml rolling upgrade tests (elastic/x-pack-elasticsearch#1555)
* ML bwc jobs tests

* Test for starting old data feeds

Original commit: elastic/x-pack-elasticsearch@131c7b15eb
2017-06-01 14:29:10 +01:00
Nik Everett 592af606f8 Build: fix the rolling restarts for real
I had the numbers backwards. 🤦

Original commit: elastic/x-pack-elasticsearch@8f2d21db52
2017-05-26 17:31:50 -04:00
Nik Everett 928589bd89 Build: Use correct data node in test
We were getting lock errors because I hand ported something and made
a copy and paste error.

Original commit: elastic/x-pack-elasticsearch@9896ba4cc7
2017-05-26 15:20:44 -04:00
Nik Everett 43928708fa Build: handle core's change to data directory
Core changed how the data directory is configured so we have to
conform to the new way.

Original commit: elastic/x-pack-elasticsearch@360e2fea1a
2017-05-26 13:32:39 -04:00
Nik Everett b215c66778 Add tests for wire compatibility when system key is enabled (elastic/x-pack-elasticsearch#1557)
Reworks the rolling restart tests so they'd have caugh an
incompatibility in the wire protocol that we say between 5.4.0 and
5.4.1.

Original commit: elastic/x-pack-elasticsearch@f5e69cf58e
2017-05-26 12:17:00 -04:00
Alexander Reelsen e9fed20d96 Watcher: Use execute watch API in tests to prevent wrong watch history query
This test failed, because there could have been other results in the
watch history. However by switching to the execute watch API, there
is no need to query the watch history at all. This also removes a 5
second wait time in the test.

relates elastic/x-pack-elasticsearch#1549

Original commit: elastic/x-pack-elasticsearch@692779521e
2017-05-26 14:27:32 +02:00
Ryan Ernst 3b58334efb Refactor script context generic types (elastic/x-pack-elasticsearch#1547)
This is the xpack side of
https://github.com/elastic/elasticsearch/pull/24877

Original commit: elastic/x-pack-elasticsearch@4f3a1a27e6
2017-05-24 19:20:59 -07:00
Ryan Ernst 9a7c28786a Scripting: Use type aware script contexts (elastic/x-pack-elasticsearch#1538)
This creates two different script contexts for watcher, one which may be
used for SearchScript, and another for ExecutableScript.
This is the xpack side of elastic/elasticsearch#24868.

Original commit: elastic/x-pack-elasticsearch@9ae3d45fed
2017-05-24 14:29:25 -07:00
Alexander Reelsen 3d057991e0 Tests: Various watcher test improvements
* Reduced a longish timeout to a shorter one, as a watch should be
  executed in a HTTP test.
* Ensured that the TimeThrottleIntegration tests only query for own
  watches in the watch history, also use random names for watch ids
* HipChatServiceTests configured deprecated logging package, so it was
  not possible to follow the HTTP calls to the hipchat service endpoint.

relates elastic/x-pack-elasticsearch#1514
Relates elastic/x-pack-elasticsearch#1515

Original commit: elastic/x-pack-elasticsearch@adb492e4e9
2017-05-24 14:08:02 +02:00
Jason Tedor 284bf2512a Fix rolling upgrade test dependency
The rolling upgrade nodes need a keystore for SSL configuration but
there was no dependency on the task that copies the keystore into the
output directory for the nodes to pick up as an extra configuration
file. This commit addresses this by adding such a dependency. To do
this, we need to break the dependency of the keystore copy task on the
REST spec copy task; this is not an issue since the dependency was for
convenience of ordering the task and not actually needed.

Original commit: elastic/x-pack-elasticsearch@fddbc06e9f
2017-05-23 21:44:59 -04:00
Jason Tedor ed827970f1 Mark failing password migration test as skipped
A previous commit marked the wrong test as skipped, this commit marks
the correct issue.

Original commit: elastic/x-pack-elasticsearch@0723964206
2017-05-23 14:25:02 -04:00
Jason Tedor 3a151ac320 Mark failing password migration test as skipped
This test is failing reliably, so this commit marks it as skipped.

Original commit: elastic/x-pack-elasticsearch@15afd615d3
2017-05-23 14:13:57 -04:00
Jason Tedor dde2600181 Mark verify Kibana user works as skipped
This test fails reliably, so we mark it as skipped.

Original commit: elastic/x-pack-elasticsearch@4fc8a6883e
2017-05-23 13:35:44 -04:00
Jason Tedor 6ce8bd69ef Mark default password migration test as skipped
This test fails reliably, so this commit marks the test as awaits fix.

Original commit: elastic/x-pack-elasticsearch@8eb8b732ba
2017-05-23 13:35:44 -04:00
Jason Tedor 87ff7f0b52 Mark default password migration test as awaits fix
This commit marks the default password migration test as awaits fix as
the test is reliabily failling.

Original commit: elastic/x-pack-elasticsearch@4377480d98
2017-05-23 12:02:11 -04:00
David Kyle ea0f3fe4a0 [ML] Wait for a stopping datafeed (elastic/x-pack-elasticsearch#1461)
* [ML] Wait for a stopping datafeed

* Fix compilation after rebase

* Address review comments

Original commit: elastic/x-pack-elasticsearch@2baed641e9
2017-05-23 14:31:19 +01:00
Ryan Ernst 68e0bc419a Use simplified ScriptContext (elastic/x-pack-elasticsearch#1502)
This is the xpack side of elastic/elasticsearch#24818

Original commit: elastic/x-pack-elasticsearch@0edbbab431
2017-05-22 13:11:23 -07:00
Ryan Ernst 883f5d8a7a Remove overrides of isInlineScriptEnabled (elastic/x-pack-elasticsearch#1500)
This is the xpack side of elastic/elasticsearch#24815

Original commit: elastic/x-pack-elasticsearch@5aa314a49d
2017-05-20 12:01:34 -07:00
Jack Conradson b92a2f6582 Remove deprecated script settings. (elastic/x-pack-elasticsearch#1469)
Original commit: elastic/x-pack-elasticsearch@069e5a1087
2017-05-18 13:32:57 -07:00