Commit Graph

1945 Commits

Author SHA1 Message Date
Martijn van Groningen e685cfe8f9
[CCR] Fail with a better error if leader index is red (#35298)
as part of fetching history uuids from leader index.
2018-11-07 13:23:30 +01:00
Alpar Torok 8a85b2eada
Remove build qualifier from server's Version (#35172)
With this change, `Version` no longer carries information about the qualifier,
we still need a way to show the "display version" that does have both
qualifier and snapshot. This is now stored  by the build and red from `META-INF`.
2018-11-07 14:01:05 +02:00
Marios Trivyzas 42dcdd0aa8
SQL: Improve CircuitBreaker logic for SqlParser (#35300)
Grammar's identifiers can be completely skipped from counting depths
as they just add another level to the tree and they are always children
of some other expression which gets counted.

Increased maximum depth from 100 to 200. After testing on production
configuration with -Xss1m, depths of at least 250 can be used, so being
conservative we put the limit lower.

Fixes: #35299
2018-11-07 12:08:32 +01:00
Alexander Reelsen 9f3effd690
Watcher: Ignore system locale/timezone in croneval CLI tool (#33215)
The elasticsearch-croneval CLI tool uses local dates to display when
something gets triggered the next time. This is very confusing.

This commit ensures, that UTC and local timezone times will be written
out.

The output looks like this and contains localized dates for each trigger
date as well as for `now`.

Now is [Tue, 28 Aug 2018 17:23:51 +0000] in UTC, local time is [ᏔᎵᏁ, 28 ᎦᎶ 2018 12:23:51 -0500]
Here are the next 10 times this cron expression will trigger:
1.	Mon, 2 Jan 2040 11:00:00 +0000
	ᏉᏅᎯ, 2 ᎤᏃ 2040 06:00:00 -0500
2.      ...

This also removes an old outstanding TODO to use the jopt parsing to
cast the count to an integer instead of doing it ourselves.
2018-11-07 09:03:35 +01:00
Tim Vernum b4173c8393
Remove deprecated RealmConfig constructor (#35327)
This removes an obsolete constructor that was still being called from
some tests.

Relates: #30241
2018-11-07 18:21:30 +11:00
Martijn van Groningen 2395e16d84
[CCR] Change resume follow api to be a master node action (#35249)
In order to start shard follow tasks, the resume follow api already
needs execute N requests to the elected master node.

The pause follow API is also a master node action, which would make
how both APIs execute more consistent.
2018-11-07 07:38:44 +01:00
Tim Brooks f395b1eace
Open node connections asynchronously (#35144)
This is related to #29023. Additionally at other points we have
discussed a preference for removing the need to unnecessarily block
threads for opening new node connections. This commit lays the groudwork
for this by opening connections asynchronously at the transport level.
We still block, however, this work will make it possible to eventually
remove all blocking on new connections out of the TransportService
and Transport.
2018-11-06 17:58:20 -07:00
Andrei Stefan 432c518266
Upgrade jline to 3.8.2 (#35288) 2018-11-06 18:42:57 +02:00
Andrei Stefan c3e4575f61
SQL: new SQL CLI logo (#35261)
Added new SQL CLI logo representing the Elastic logo "painted" with
"Elastic" words, "SQL" under the logo and version on the last line
2018-11-06 17:51:52 +02:00
Tal Levy a85b4f42ca
[ILM] change remove-policy-from-index http method from DELETE to POST (#35268)
The remove-ilm-from-index API was using the DELETE http method
to signify that something is being removed. Although, metadata
about ILM for the index is being deleted, no entity/resource
is being deleted during this operation. POST is more in line with
what this API is actually doing, it is modifying the metadata for
an index. As part of this change, `remove` is also appended to the path 
to be more explicit about its actions.
2018-11-06 07:46:25 -08:00
Martijn van Groningen a937d7f5f3
[CCR] Forgot missing return statement,
Error was thrown if leader index had no soft deletes enabled, but it then continued creating the follower index.

The test caught this bug, but very rarely due to timing issue.

Build failure instance:

```
1> [2018-11-05T20:29:38,597][INFO ][o.e.x.c.LocalIndexFollowingIT] [testDoNotCreateFollowerIfLeaderDoesNotHaveSoftDeletes] before test
  1> [2018-11-05T20:29:38,599][INFO ][o.e.c.s.ClusterSettings  ] [node_s_0] updating [cluster.remote.local.seeds] from [[]] to [["127.0.0.1:9300"]]
  1> [2018-11-05T20:29:38,599][INFO ][o.e.c.s.ClusterSettings  ] [node_s_0] updating [cluster.remote.local.seeds] from [[]] to [["127.0.0.1:9300"]]
  1> [2018-11-05T20:29:38,609][INFO ][o.e.c.m.MetaDataCreateIndexService] [node_s_0] [leader-index] creating index, cause [api], templates [random-soft-deletes-templat
e, one_shard_index_template], shards [2]/[0], mappings []
  1> [2018-11-05T20:29:38,628][INFO ][o.e.c.r.a.AllocationService] [node_s_0] Cluster health status changed from [YELLOW] to [GREEN] (reason: [shards started [[leader-
index][0]] ...]).
  1> [2018-11-05T20:29:38,660][INFO ][o.e.x.c.a.TransportPutFollowAction] [node_s_0] [follower-index] creating index, cause [ccr_create_and_follow], shards [2]/[0]
  1> [2018-11-05T20:29:38,675][INFO ][o.e.c.s.ClusterSettings  ] [node_s_0] updating [cluster.remote.local.seeds] from [["127.0.0.1:9300"]] to [[]]
  1> [2018-11-05T20:29:38,676][INFO ][o.e.c.s.ClusterSettings  ] [node_s_0] updating [cluster.remote.local.seeds] from [["127.0.0.1:9300"]] to [[]]
  1> [2018-11-05T20:29:38,678][INFO ][o.e.x.c.LocalIndexFollowingIT] [testDoNotCreateFollowerIfLeaderDoesNotHaveSoftDeletes] after test
  1> [2018-11-05T20:29:38,678][INFO ][o.e.x.c.LocalIndexFollowingIT] [testDoNotCreateFollowerIfLeaderDoesNotHaveSoftDeletes] [LocalIndexFollowingIT#testDoNotCreateFoll
owerIfLeaderDoesNotHaveSoftDeletes]: cleaning up after test
  1> [2018-11-05T20:29:38,678][INFO ][o.e.c.m.MetaDataDeleteIndexService] [node_s_0] [follower-index/TlWlXp0JSVasju2Kr_hksQ] deleting index
  1> [2018-11-05T20:29:38,678][INFO ][o.e.c.m.MetaDataDeleteIndexService] [node_s_0] [leader-index/FQ6EwIWcRAKD8qvOg2eS8g] deleting index
FAILURE 0.23s J0 | LocalIndexFollowingIT.testDoNotCreateFollowerIfLeaderDoesNotHaveSoftDeletes <<< FAILURES!
   > Throwable #1: java.lang.AssertionError:
   > Expected: <false>
   >      but: was <true>
   >    at __randomizedtesting.SeedInfo.seed([7A3C89DA3BCA17DD:65C26CBF6FEF0B39]:0)
   >    at org.hamcrest.MatcherAssert.assertThat(MatcherAssert.java:20)
   >    at org.elasticsearch.xpack.ccr.LocalIndexFollowingIT.testDoNotCreateFollowerIfLeaderDoesNotHaveSoftDeletes(LocalIndexFollowingIT.java:83)
   >    at java.lang.Thread.run(Thread.java:748)
```

Build failure: https://elasticsearch-ci.elastic.co/job/elastic+elasticsearch+6.5+intake/46/console
2018-11-06 16:05:35 +01:00
Marios Trivyzas 1be64a7b8b
SQL: Fix null handling for AND and OR in SELECT (#35277)
Override `process()` in `BinaryLogicProcessor` which doesn't immediately
return null if left or right argument is null, which is the behaviour of
`process()` of the parent class `BinaryProcessor`.

Also, add more tests for `AND` and `OR` in SELECT clause with literal.

Fixes: #35240
2018-11-06 15:40:16 +01:00
Tal Levy 2bf843e768 [TEST] Mute ChangePolicyForIndexIT#testChangePolicyForIndex 2018-11-06 06:09:49 -08:00
Costin Leau 75e9a639ee
SQL: Introduce Coalesce function (#35253)
Add Coalesce conditional for replacing null values

Fix #35060
2018-11-06 13:12:24 +02:00
Nick Knize a5e1f4d3a2 Upgrade to lucene-8.0.0-snapshot-31d7dfe6b1 (#35224) 2018-11-06 11:55:23 +01:00
Simon Willnauer 833e0f8ecf
Prevent throttled indices to be searched through wildcards by default (#34354)
Today if a wildcard, date-math expression or alias expands/resolves
to an index that is search-throttled we still search it. This is likely
not the desired behavior since it can unexpectedly slow down searches
significantly.

This change adds a new indices option that allows `search`, `count`
and `msearch` to ignore throttled indices by default. Users can
force expansion to throttled indices by using `ignore_throttled=true`
on the rest request to expand also to throttled indices.

Relates to #34352
2018-11-06 09:45:30 +01:00
Martijn van Groningen 46c238d792
[CCR] Improve error when operations are missing (#35179)
Improve error when operations are missing
2018-11-06 08:42:47 +01:00
Martijn van Groningen cac67f8bcc
[CCR] Add extra validation to unfollow api (#35245)
Validate whether the follow index actually exists and
whether the follow index actually has custom ccr metadata.
2018-11-06 08:00:34 +01:00
Tim Vernum 574ec6686e
Include realm type in Security Realm setting keys (#30241)
This moves all Realm settings to an Affix definition.
However, because different realm types define different settings
(potentially conflicting settings) this requires that the realm type
become part of the setting key.

Thus, we now need to define realm settings as:

    xpack.security.authc.realms:
      file.file1:
        order: 0

      native.native1:
        order: 1

- This is a breaking change to realm config
- This is also a breaking change to custom security realms (SecurityExtension)
2018-11-06 14:56:50 +11:00
Shaunak Kashyap 00e66bab36
[Elasticsearch Monitoring] Collect only display_name (for now) (#35265)
This is a forward port of some of the changes made in #8445, specifically the change mentioned in https://github.com/elastic/elasticsearch/pull/34023#issuecomment-433212636.

Currently, in master, the `cluster_stats` collector collects _all_ cluster metadata and indexes it into `.monitoring-es-*`. However, per the discussion linked to above, we decided to collect _only_ the `display_name` cluster metadata setting for now. This PR makes this change.
2018-11-05 16:27:05 -08:00
Marios Trivyzas 9ac7af6b3e
SQL: Introduce NotEquals node to simplify expressions (#35234)
Add NotEquals node in parser to simplify expressions so that <value1> != <value2> is
no longer translated internally to NOT(<value1> = <value2>)

Closes: #35210
Fixes: #35233
2018-11-05 22:23:07 +01:00
Nik Everett f72ef9b5fd
Build: Pull "skip assemble on qa" to common build (#35214)
Pull all of the logic that we use to skip the `assemble` and
`dependenciesInfo` tasks on `qa` projects into one spot in our root
build file.
2018-11-05 16:16:00 -05:00
Lee Hinman 3ee004c7b9
Add dedicated step for checking shrink allocation status (#35161)
This adds a new step for checking whether an index is allocated correctly based
on the rules added prior to running the shrink step. It also fixes a bug where
for shrink we are not allowed to have the shards relocating for the shrink step.

This also allows us to simplify AllocationRoutedStep and provide better
feedback in the step info for why either the allocation or the shrink checks
have failed.

Resolves #34938
2018-11-05 12:02:59 -07:00
Marios Trivyzas fd82813660
SQL: [tests] Fix select csv spec and enable it (#35239)
Also, replace `||` and `&&` with `OR` and `AND` as the former are not
SQL standard operators.
2018-11-05 18:45:58 +01:00
Jake Landis c2766b65cf
ingest: raise visibility of ingest plugin documentation (#35048)
* move the set security user processor to the main documentation
* link to plugin processors

part of #33188
2018-11-05 11:44:10 -06:00
Gordon Brown 0fbb8a16bc
Skip Rollover step if next index already exists (#35168)
If the Rollover step would fail due to the next index in sequence
already existing, just skip to the next step instead of going to the
Error step.

This prevents spurious `ResourceAlreadyExistsException`s created by
simultaneous RolloverStep executions from causing ILM to error out
unnecessarily.
2018-11-05 09:20:43 -07:00
Lee Hinman 3473217563
Remove Joda usage from ILM (#35220)
This commit removes the Joda time usage from ILM and the HLRC components of ILM.
It also fixes an issue where using the `?human=true` flag could have caused the
parser not to work. These millisecond fields now follow the standard we use
elsewhere in the code, with additional fields added iff the `human` flag is
specified.

This is a breaking change for ILM, but since ILM has not yet been released, no
compatibility shim is needed.
2018-11-05 08:17:15 -07:00
Alexander Reelsen 409050e8de
Refactor: Remove settings from transport action CTOR (#35208)
As settings are not used in the transport action constructor, this
removes the passing of the settings in all the transport actions.
2018-11-05 13:08:18 +01:00
Martijn van Groningen ddda2d419c
[CCR] Change max_read_request_size default (#35247)
This changes the max_read_request_size default from unlimited to 32MB.
2018-11-05 12:51:42 +01:00
David Kyle 85f8458f06 [ML] Add comment describing test behaviour 2018-11-05 11:21:59 +00:00
Nhat Nguyen 54e1231ebd
CCR/TEST: Limit indexing docs in FollowerFailOverIT (#35228)
The suite FollowerFailOverIT is failing because some documents are not
replicated to the follower. Maybe the FollowTask is not working as
expected or the background indexers eat all resources while the follower
cluster is trying to reform after a failover; then CI is not fast enough
to replicate all the indexed docs within 60 seconds (sometimes I see 80k
docs on the leader).

This commit limits the number of documents to be indexed into the leader
index by the background threads so that we can eliminate the latter
case. This change also replaces a docCount assertion with a docIds
assertion so we can have more information if these tests fail again.

Relates #33337
2018-11-03 10:00:54 -04:00
Jay Modi ec1c2c4a76
Remove AbstractComponent usage in x-pack core (#35187)
This change removes the use of AbstractComponent in the security
module. The classes now declare their own loggers.

Relates #34488
2018-11-02 14:30:43 -06:00
Gordon Brown b3da3eae08
[ILM] Fix race condition in test (#35143)
Previously, testRunStateChangePolicyWithNextStep asserted that the
ClusterState before and after running the steps were equal. The test
only passed due to a race condition: The latch would be triggered by the
step execution, but the cluster state update thread would continue
running before committing the change to the cluster state. This allowed
the test to read the old cluster state and pass the equality check about
99.99% of the time.

The test now waits for the new cluster state to be committed before
checking that it is _not_ equal to the old cluster state.
2018-11-02 11:09:48 -06:00
Jason Tedor 1e241190eb
Disable assemble task from ILM qa projects
This commit disables the assemble tasks from all ILM qa projects. These
projects do not have an assemble task to execute.
2018-11-02 11:16:34 -04:00
David Kyle 43cff56aec Mute FollowerFailOverIT.testFollowIndexAndCloseNode
Issue #33337
2018-11-02 15:13:09 +00:00
Alan Woodward e2af849f70
Move ObjectPath and XContentUtils to libs/x-content (#34803)
These are generally useful utility classes that do not need to live in the Watcher code
2018-11-02 15:12:09 +00:00
Jay Modi 6b08d5fc89
Remove use of AbstractComponent in security (#35186)
This change removes the use of AbstractComponent in the security
module. The classes now declare their own loggers.

Relates #34488
2018-11-02 08:33:26 -06:00
Benjamin Trent 7fb95be8bd
Moving xpack.plugin.core ml response to use new base class (#35157)
* Moving xpack.plugin.core ml response to use new base class

* changing back to streamable
2018-11-02 07:46:53 -05:00
Colin Goodheart-Smithe 0aa66b077a
Merge branch 'master' into index-lifecycle 2018-11-02 11:20:54 +00:00
Nhat Nguyen 4875d6fb0b
CCR: Add NodeClosedException to retryable list (#35191)
This change adds NodeClosedException to the retry-able exception list.
2018-11-02 07:01:46 -04:00
Colin Goodheart-Smithe fc6e1f7f3f
Merge branch 'master' into index-lifecycle 2018-11-02 10:56:35 +00:00
Marios Trivyzas 340363b9fa
SQL: Fix null handling for IN => painless script (#35124)
Include `null` literals when generating the painless script for `IN` expressions.
Previously, they were skipped, because of an issue that had been fixed with #35108.

Fixes: #35122
2018-11-02 10:37:54 +01:00
Alpar Torok b432cbdafe Fix upgrade tests involving snapshot
Fixes
```
Could not determine the dependencies of task ':x-pack:qa:full-cluster-restart:with-system-key:v5.6.13#oldClusterTestCluster#node1.copyBwcPlugins'.
```
2018-11-02 09:51:41 +02:00
Martijn van Groningen 19d6cf1b9e
[CCR] Change response classes to not use from Streamable. (#35085)
Only the response classes of get auto follow pattern, the follow and stats APIs
were moved away from Streamable.  The other APIs use `AcknowledgedResponse`
or `BaseTasksResponse` as response class and
moving that class away from Streamable is a bigger change.
2018-11-02 08:02:17 +01:00
Nhat Nguyen dbc7c9259c CCR/TEST: Add debug log to testFailOverOnFollower 2018-11-01 22:28:06 -04:00
Tal Levy b260406ecf
ILM: use node _id attribute when allocating to one node (#35061)
ILM's Shrink Action was using a nodes "_name" attribute to
allocate to prepare for the shrink step. Since the name is
configurable by a user and may use the same name for
multiple nodes on one machine, _id is safer since it is guaranteed
to be unique.

closes #35043.
2018-11-01 14:38:00 -07:00
Tal Levy 6b312a500d uninherit from AbstractComponent in IndexLifecycleService 2018-11-01 10:22:55 -07:00
Tal Levy c3cf7dd305 Merge remote-tracking branch 'upstream/master' into index-lifecycle 2018-11-01 10:13:02 -07:00
Alpar Torok f8378d91a6
Implement VersionCollection in Java (#34050) 2018-11-01 17:43:57 +02:00
Benjamin Trent 2fadec5c3d
ML: Add support for rollup Indexes in Datafeeds (#34654)
* Adding rollup support for datafeeds

* Fixing tests and adjusting formatting

* minor formatting chagne

* fixing some syntax and removing redundancies

* Refactoring and fixing failing test

* Refactoring, adding paranoid null check

* Moving rollup into the aggregation package

* making AggregationToJsonProcessor package private again

* Addressing test failure

* Fixing validations, chunking

* Addressing failing test

* rolling back RollupJobCaps changes

* Adding comment and cleaning up test

* Addressing review comments and test failures

* Moving builder logic into separate methods

* Addressing PR comments, adding test for rollup permissions

* Fixing test failure

* Adding rollup priv check on datafeed put

* Handling missing index when getting caps

* Fixing unused import
2018-11-01 10:02:24 -05:00
Igor Motov 7b13d0592f
Watcher: fix metric stats names (#34951)
* Watcher: fix metric stats names

The current watcher stats metric names doesn't match the current
documentation. This commit fixes the behavior of `queued_watches`
metric, deprecates `pending_watches` metric and adds `current_watches`
to match the documented behavior. It also fixes the documentation, which
introduced `executing_watches` metric that was never added.

Fixes #34865
2018-11-01 10:12:05 -04:00
Andrei Stefan b6659847c2
SQL: handle wildcard expansion on incorrect fields (#35134) 2018-11-01 11:30:05 +02:00
Marios Trivyzas 4f78e2b576
SQL: Improve painless script generated from `IN` (#35055)
Replace standard `||` and `==` painless operators with
new `in` method introduced in `InternalSqlScriptUtils`.
This allows the list of values to become a script variable
which is replaced each time with the list of  values provided
by the user.

Move In to the same package as InPipe & InProcessor

Follow up to #34750

Co-authored-by: Costin Leau <costin.leau@gmail.com>
2018-11-01 10:23:49 +01:00
Nik Everett e28509fbfe
Core: Less settings to AbstractComponent (#35140)
Stop passing `Settings` to `AbstractComponent`'s ctor. This allows us to
stop passing around `Settings` in a *ton* of places. While this change
touches many files, it touches them all in fairly small, mechanical
ways, doing a few things per file:
1. Drop the `super(settings);` line on everything that extends
`AbstractComponent`.
2. Drop the `settings` argument to the ctor if it is no longer used.
3. If the file doesn't use `logger` then drop `extends
AbstractComponent` from it.
4. Clean up all compilation failure caused by the `settings` removal
and drop any now unused `settings` isntances and method arguments.

I've intentionally *not* removed the `settings` argument from a few
files:
1. TransportAction
2. AbstractLifecycleComponent
3. BaseRestHandler

These files don't *need* `settings` either, but this change is large
enough as is.

Relates to #34488
2018-10-31 21:23:20 -04:00
Tal Levy f8e23f6400
update ILM integ test cluster poll interval to 1s (#35113) 2018-10-31 17:09:35 -07:00
Costin Leau f87a53451b
SQL: Register missing processors (#35121)
Add registration (and tests) for missing processors in the serialization
chain.

Fix #35119
2018-10-31 23:07:36 +02:00
David Turner 0072c90e2a
Pre-populate unicast hosts files (#35136)
Today when ESIntegTestCase starts some nodes it writes out the unicast hosts
files each time a node starts its transport service. This does mean that a
number of nodes can start and perform their first pinging round without any
unicast hosts which, if the timing is unlucky and a lot of nodes are all
started at the same time, can lead to a split brain as in #35052.

Prior to #33554 this was unlikely to happen since the MockUncasedHostsProvider
would always have yielded the existing hosts, so the timing would have to have
been implausibly unlucky. Since #33554, however, it's more likely because the
race occurs between the start of the first round of pinging and the writing of
the unicast hosts file. It is realistic that new nodes will be configured with
the existing nodes from startup, so this change reinstates that behaviour.

Closes #35052.
2018-10-31 19:21:24 +00:00
Tal Levy 5f4b23f8c1
cleanup ILM qa structure (#35110)
This commit does a few things

- moves ILM-specifc rest yaml tests into plugin/ilm/qa, and creates special
  :plugin:ilm:qa:rest module to test them
- removes the with-security tests of the yaml tests since they are covered in
  the rest tests now
- moves ChangePolicyforIndexIT into the qa/multi-node project since that test is
  not currently running in main ilm since integTest is disabled
2018-10-31 11:49:29 -07:00
Tal Levy a294a7c6b5 fix IndexLifecycleService setting member
the settings variable was previously created by
the AbstractComponent class inherited by IndexLifecycleService.
this is no more.
2018-10-31 11:17:16 -07:00
Tal Levy d5d28420b6 Merge remote-tracking branch 'upstream/master' into index-lifecycle 2018-10-31 10:47:07 -07:00
Vladimir Dolzhenko 28bb1a3ac0
XPack/HLRC request/response compatibility tests (#34547)
Relates #34451 , #29827
2018-10-31 13:48:35 +01:00
Luca Cavanna 674225aaa1
[TEST] Enforce skip headers when needed (#34735)
The java yaml test runner supports sending request headers, yet not all clients support headers. This commit makes sure that we enforce adding a skip section with feature "headers" whenever headers are used in a do section as part of a test. That decreases the chance for new tests to break client builds due to the missing skip section.

Closes #34650
2018-10-31 13:07:02 +01:00
Dimitris Athanasiou 00dc2ba36f
[ML] Enable reusing field extraction logic when no time field is required (#35100) 2018-10-31 10:55:11 +00:00
Marios Trivyzas bf6f521827
SQL: Fix NPE thrown if HAVING filter evals to null (#35108)
Surround script with `nullSafeFilter` also for `AggFilter` translation.
Fixed query translation tests to properly set `onAggs`.

Fixes: #35107
2018-10-31 10:32:16 +01:00
Tal Levy fdb0db30c4
throw exception when ilm-rollover-step runs on index without rollover alias (#35065)
The ILM Rollover Step can execute on the incorrect index if the rollover alias
exists on another valid index, but not the one the step is executing against. This
is a problem and is now guarded against
2018-10-30 16:42:12 -07:00
Tal Levy 5141084048
rename CRUD api REST path prefix _ilm to _ilm/policy (#35056)
This PR renames the CRUD APIS for ILM

GET _ilm/<policy>, _ilm -> _ilm/policy/<policy>, _ilm/policy
PUT _ilm/<policy> -> _ilm/policy/<policy>
DELETE _ilm/<policy> -> _ilm/policy/<policy>

closes #34929.
2018-10-30 16:19:05 -07:00
Marios Trivyzas 455906d431
SQL: [tests] Remove AbstractBuilderTestCase dep (#35099)
Extend directly from `EsTestCase` to speed up execution.

Fixes: #35098
2018-10-30 22:31:29 +01:00
Nik Everett 086ada4c08
Core: Drop settings member from AbstractComponent (#35083)
Drops the `Settings` member from `AbstractComponent`, moving it from the
base class on to the classes that use it. For the most part this is a
mechanical change that doesn't drop `Settings` accesses. The one
exception to this is naming threads where it switches from an invocation
that passes `Settings` and extracts the node name to one that explicitly
passes the node name.

This change doesn't drop the `Settings` argument from
`AbstractComponent`'s ctor because this change is big enough as is.
We'll do that in a follow up change.
2018-10-30 16:10:38 -04:00
Tal Levy f5323cd86e
rename ILM Phase "minimum_age" to "min_age" (#35058)
closes #34927.
2018-10-30 12:54:02 -07:00
Jay Modi 09a124e8d6
Native roles store uses mget to retrieve roles (#33531)
The native roles store previously would issue a search if attempting to
retrieve more than a single role. This is fine when we are attempting
to retrieve all of the roles to list them in the api, but could cause
issues when attempting to find roles for a user. The search is not
prioritized over other search requests, so heavy aggregations/searches
or overloaded nodes could cause roles to be cached as missing if the
search is rejected.

When attempting to load specific roles, we know the document id for the
role that we are trying to load, which allows us to use the multi-get
api for loading these roles. This change makes use of the multi-get api
when attempting to load more than one role by name. This api is also
constrained by a threadpool but the tasks in the GET threadpool should
be quicker than searches.

See #33205
2018-10-30 13:35:07 -06:00
Gordon Brown 6ecb8ff344
Move to Error step if ClusterState* steps throw (#35069)
Previously, if ClusterStateActionSteps or ClusterStateWaitSteps threw an
exception executing, the exception would only be caught and logged by
the generic ClusterStateUpdateTask machinery and the index would become
stuck on that step.

Now, exceptions thrown in these steps will be caught and the index will
be moved to the Error step.
2018-10-30 13:33:32 -06:00
Nik Everett 7de9eda391
Graph: Move qa into plugin (#34517)
Move `x-pack/qa/smoke-test-graph-with-security` to
`x-pack/plugin/graph/qa/security` which should make it easier to run all
of the tests with a single command. It also lines up the directories
more closely with newer projects like cross cluster replication.
2018-10-30 15:14:33 -04:00
Nik Everett 6857d30527
SQL: Move qa directory into sql directory (#34496)
Moves `x-pack/qa/sql/*` into `x-pack/plugin/sql/qa` to make it simpler
to run all of the sql tests. This lines up with how newer projects like
cross cluster replication are testing themselves.
2018-10-30 14:54:15 -04:00
Costin Leau 6abddd8ca6
SQL: Optimizer rule for folding nullable expressions (#35080)
Add optimization for folding nullable expressions with a NULL 
argument. This is a variant of folding for the NULL case.

Fix #34826
2018-10-30 20:17:20 +02:00
Ryan Ernst 512319cef7
Test: Filter out deprecated joda tzs in tests (#34868)
This commit filters out usage of deprecated tzs by tests. These are
tested separately and should not require checking for warnings on any
test using random timezones.

closes #34188
2018-10-30 11:15:34 -07:00
Zachary Tong f9dd33a0b9
[Rollup] Proactively resolve index patterns in RollupSearch endoint (#34930)
This changes the RollupSearch endpoint to proactively resolve index
patterns.  If the index pattern(s) match more than one rollup index,
an exception is throw as before.  But if the pattern only matches one
rollup index, execution is allowed to continue (unlike before where
it would assume all patterns were for raw data).

This also allows the search endpoint to resolve aliases that point to
a rollup index.

Also tweaks the documentation to make this clear.

Closes #34828
2018-10-30 13:50:50 -04:00
Tal Levy 18c72e86c5 Merge remote-tracking branch 'upstream/master' into index-lifecycle 2018-10-30 08:09:57 -07:00
Tal Levy 6cf7f1205a Revert "add ILM rolling upgrade tests (#32828)"
This reverts commit 5c88356ad6.
2018-10-30 08:06:53 -07:00
Andrei Stefan 7bd113d284
Test for https://github.com/elastic/elasticsearch/issues/30054 bug (#35050)
Added a test for a bug in Elasticsearch (https://github.com/elastic/elasticsearch/issues/32941) that had implications in SQL and activated another test that didn't work in the past
2018-10-30 16:18:52 +02:00
Andrei Stefan 253152172c
SQL: Proper handling of nested fields at the beginning of the columns list (#35068) 2018-10-30 15:52:05 +02:00
Martijn van Groningen 6da2fb7d5b
Change CCR API request classes to use Writeable serialization instead of Streamable (#34911)
Only the follow stats request couldn't be changed to use Writeable serialization,
because that requires changes in `TransportTasksAction` and `BaseTasksRequest` base classes.
2018-10-30 11:03:30 +01:00
Marios Trivyzas e0fef52058
SQL: Fix failing test due to random timezone (#35067)
Use null values for `languages` instead of `birth_date` to avoid
differences with H2 because of timezone randomization.

Fixes: #35042
2018-10-30 10:12:33 +01:00
David Turner f7760ddacb Increase discovery log level in NetworkDisruptionIT
Sometimes the cluster forming here will split-brain when it grows up to 5
nodes. This could be a timing issue or could be something going wrong in
discovery, so this asks for more logs. Relates #35052
2018-10-30 08:45:23 +00:00
Dimitris Athanasiou d85a654ebb
[ML] Refactor doc value format into ExtractedField (#35053)
This commit moves the knowledge of which doc value format
to be used down to the `ExtractedField` instead of being
in the data extractor.
2018-10-29 22:56:53 +00:00
Tal Levy c9e4d26a53 Merge remote-tracking branch 'upstream/master' into index-lifecycle 2018-10-29 14:03:55 -07:00
Pratik Sanglikar f1135ef0ce Core: Replace deprecated Loggers calls with LogManager. (#34691)
Replace deprecated Loggers calls with LogManager.

Relates to #32174
2018-10-29 15:52:30 -04:00
Gordon Brown f6ac0e4bbc
[ILM] Fix Move To Step API causing ILM to hang (#34618)
The Move To Step API now checks to see if the target step is an
AsyncActionStep, and if so, runs it.

Previously, AsyncActionSteps would only be run when they are entered by
executing the previous step, so if an AsyncActionStep was entered via
the Move To Step API, ILM would never touch that index again.
2018-10-29 11:18:12 -06:00
Marios Trivyzas 42b4f23a76
SQL: Add `CAST` and `CONVERT` to `SHOW FUNCTIONS` (#34940)
Fixes: #34939
2018-10-29 16:27:07 +01:00
Nik Everett b093116a1e
Logging: Drop another deprecated Loggers method (#34520)
Drop a method from `Loggers` that we deprecated because it just
delegated to `LogManager`.
2018-10-29 10:05:24 -04:00
Jason Tedor cc9894d78a
Fix name of CCR stats transport action
This class name should include an indication that it is for CCR. This
commit adds that to the name of this class.
2018-10-29 09:50:13 -04:00
Jason Tedor 26f5c509af
Fix CCR API specification (#34963)
This commit fixes two issues with the CCR API specification:
 - remove the CCR stats endpoint, it is not currently implemented
 - fix the documentation links
2018-10-29 09:37:13 -04:00
Martijn van Groningen b2daaf15d1
[CCR] move tests that modify test cluster from the main test class to
a dedicated class.
2018-10-29 13:45:59 +01:00
Vladimir Dolzhenko 69fe9a1bf1
HLRC XPack Protocol clean up: Migration; Graph; Watcher (#34639)
Move parts related to XPack for packages Migration; Graph; Watcher from protocol to client

Drop protocol module

Relates #34451
2018-10-29 13:38:56 +01:00
Marios Trivyzas 8d7889d20c
SQL: Implement CAST between STRING and IP (#34949)
This allows comparison between IP fields and STRING input
from the user where automatic conversion is not yet supported (e.g.: for IN)

Closes: #34799
2018-10-29 13:06:30 +01:00
Marios Trivyzas 731e48beff
SQL: [Test] Fix and enable test with randomness (#34850)
Increase minimum number of elements for List<> ctor arguments
for specific classes that validate the size of the list.

Fixes: #34754
2018-10-29 12:02:28 +01:00
Marios Trivyzas c9ae1928e0
SQL: Fix function args verification and error msgs (#34926)
Extract data type verification for function arguments to a single place
so that NULL type can be treated as RESOLVED for all functions. Moreover
this enables to have consistent verification error messages for all functions.

Fixes: #34752
Fixes: #33469
2018-10-29 11:32:17 +01:00
David Roberts c455be7bc2
[ML] Rename the json file structure to ndjson (#34901)
The file structure finder endpoint can find the NDJSON
(newline-delimited JSON) file format, but called it
`json`.  This change renames the `format` for this file
structure to `ndjson`, which is more precise and will
hopefully avoid confusion.
2018-10-29 10:06:12 +01:00
Martijn van Groningen 1801518527
[CCR] Refactor stats APIs (#34912)
* Changed the auto follow stats to also include follow stats.
* Renamed the auto follow stats api to stats api and changed its url path
  from `/_ccr/auto_follow/stats` `/_ccr/stats`.
* Removed `/_ccr/stats` url path for the follow stats api, which makes
  the index parameter a required parameter.
* Fixed docs.
2018-10-29 07:45:27 +01:00
Martijn van Groningen bad5972f62
[CCR] Fix request serialization bug (#34917)
and some parameters that were not set in tests.
2018-10-29 07:38:55 +01:00
Tim Vernum bb5b59004e
[TEST] Manually trigger resource watching (#34893)
SSLTrustRestrictionsTests.testRestrictionsAreReloaded checks that the
SSL trust configuration is automatically updated reapplied if the
underlying "trust_restrictions.yml" file is modified.

Since the default resource watcher frequency is 5seconds, it could
take 10 second to run that test (as it waits for 2 reloaded).

Previously this test set that frequency to a very low value (3ms) so
that the elapsed time for the test would be reduced. However this
caused other problems, including that the resource watcher would
frequently run while the cluster was shutting down and files were
being cleaned up.

This change resets that watch frequency back to its default (5s) and
then manually calls the "notifyNow" method on the resource watcher
whenever the restrictions file is modified, so that the SSL trust
configuration is reloaded at exactly the right time.

Resolves: #34502
2018-10-29 11:23:36 +11:00
Costin Leau 14fb7edc1d
SQL: Fix incorrect AVG data type (#34948)
AVG now returns double (not the type of the target field)

Close #33773
2018-10-28 22:13:04 +02:00
Tal Levy 5c88356ad6
add ILM rolling upgrade tests (#32828) 2018-10-26 15:40:26 -07:00
Tal Levy d8322ca069 Merge remote-tracking branch 'upstream/master' into index-lifecycle 2018-10-26 12:46:21 -07:00
Nik Everett 9f87fdc7ab
Drop deprecationLogger from AbstractComponent (#34859)
Drops the `deprecationLogger` from `AbstractComponent`, moving it to
places where we need it. This saves us from building a bunch of
`DeprecationLogger`s that we don't need.

Relates to #34488
2018-10-26 15:40:16 -04:00
Nik Everett 10295b306d
Core: Drop nodeName from AbstractComponent (#34487)
`AbstractComponent` is trouble because its name implies that
*everything* should extend from it. It *is* useful, but maybe too
broadly useful. The things it offers access too, the `Settings` instance
for the entire server and a logger are nice to have around, but not
really needed *everywhere*. The `Settings` instance especially adds a
fair bit of ceremony to testing without any value.

This removes the `nodeName` method from `AbstractComponent` so it is
more clear where we actually need the node name.
2018-10-26 15:26:14 -04:00
Tal Levy e737ea7d4a
remove old doc placeholder and migrate ilm docs to top-level (#34615)
we are restructuring the docs, this migrates ILM docs outside of the x-pack doc structure.
2018-10-26 12:19:52 -07:00
Costin Leau 55d48e4cc1
SQL: Handle aggregation for null group (#34916)
When dealing with a null group, the associated metric aggs need to
return null as well

Fix #34896
2018-10-26 21:34:01 +03:00
Jay Modi a0279bc069
Responses can use Writeable.Reader interface (#34655)
In order to remove Streamable from the codebase, Response objects need
to be read using the Writeable.Reader interface which this change
enables. This change enables the use of Writeable.Reader by adding the
`Action#getResponseReader` method. The default implementation simply
uses the existing `newResponse` method and the readFrom method. As
responses are migrated to the Writeable.Reader interface, Action
classes can be updated to throw an UnsupportedOperationException when
`newResponse` is called and override the `getResponseReader` method.

Relates #34389
2018-10-26 09:21:54 -06:00
Costin Leau 5a7b8c0e69
SQL: Provide null-safe scripts for Not and Neg (#34877)
Introduce null-safe Painless scripts for Not and Neg
Simplify script generation for Unary functions

Close #34848
2018-10-26 18:21:14 +03:00
Lee Hinman 6880e2ad79 [TEST] Fix RolloverActionTests.mutateInstance
In some cases it wasn't mutating and caused test failures
2018-10-26 09:10:55 -06:00
Jason Tedor 43f6ba1c63
Fix put/resume follow request parsing (#34913)
This commit adds some fields that were missing from put follow, and
fixes a bug in resume follow.
2018-10-26 11:09:55 -04:00
Dimitris Athanasiou a39a67cd38
[ML] Extract common native process base class (#34856)
We currently have two different native processes:
autodetect & normalizer. There are plans for introducing
a new process. All these share many things in common.
This commit refactors the processes to extend an
`AbstractNativeProcess` class that encapsulates those
commonalities with the purpose of reusing the code
for new processes in the future.
2018-10-26 15:34:48 +01:00
Martijn van Groningen 306f1d78f8
[CCR] Retry when no index shard stats can be found (#34852)
Index shard stats for the follower shard are fetched, when a shard follow task is started.
This is needed in order to bootstap the shard follow task with the follower global checkpoint.

Sometimes index shard stats are not available (e.g. during a restart) and
we fail now, while it is very likely that these stats will be available some time later.
2018-10-26 15:14:24 +02:00
Albert Zaharovits 3f1fec1813
[Docs] audit logfile structured format (#34584)
Documents the new structured logfile format for auditing
that was introduced by #31931. Most changes herein
are for 6.x . In 7.0 the deprecated format is gone and a
follow-up PR is in order.
2018-10-26 15:19:35 +03:00
David Roberts 734088673e
[ML] Include message in field_stats for text log files (#34861)
This change ensures the `message` field is always included
in the `field_stats` for the semi-structured text log file
file structure.  Previously it was not, as it will almost
certainly contain all distinct values.  However, for
consistency in the UI it's useful to include it.
2018-10-26 07:45:02 +02:00
Tal Levy e1fdd00420
Lowercase static final DeprecationLogger instance names (#34887)
After discussing on the team's FixItFriday, we concluded that
static final instance variables that are mutable should be lowercased.

Historically, DeprecationLogger was uppercased more frequently than lowercased.
2018-10-25 21:12:19 -07:00
Tal Levy 4c4afef9cc fix PolicyStatsTests: mutate kept same indicesManaged 2018-10-25 16:13:09 -07:00
Tal Levy f6ce935444
fix `GET _ilm` response with uninitialized ILM metadata (#34881)
ILM would return a resource-not-found exception when requesting policies
while the IndexLifecycleMetaData is not initialized. The behavior here
should not be as extreme since it is not the user's fault.

This commit changes the behavior so that it succeeds and returns no policies
when no policy names are explicitely specified, otherwise keep the same behavior
of throwing an exception
2018-10-25 16:00:44 -07:00
Tal Levy f76ced4e14
update updating-policy docs (#34661)
This contains updates to the ILM Docs explaining how updates to policies work and
affect policy execution on indices.
2018-10-25 15:24:25 -07:00
Tal Levy 810cd46a30 Merge remote-tracking branch 'upstream/master' into index-lifecycle 2018-10-25 14:35:33 -07:00
Marios Trivyzas 06d7431950
SQL: Return error with ORDER BY on non-grouped. (#34855)
Previously, for some queries the validation for ORDER BY
fields didn't kick in since a HAVING close or an ORDER BY
with scalar function would add `Filter` and `Project` plans
between the `OrderBy` and the `Aggregate`.

Now the LogicalPlan tree beneath `OrderBy` is traversed and
the ORDER BY fields are properly verified.

Fixes: #34590
2018-10-25 22:15:28 +02:00
Tim Brooks cf9aff954e
Reduce channels in AbstractSimpleTransportTestCase (#34863)
This is related to #30876. The AbstractSimpleTransportTestCase initiates
many tcp connections. There are normally over 1,000 connections in
TIME_WAIT at the end of the test. This is because every test opens at
least two different transports that connect to each other with 13
channel connection profiles. This commit modifies the default
connection profile used by this test to 6. One connection for each
type, except for REG which gets 2 connections.
2018-10-25 13:37:49 -06:00
Lee Hinman a4b634e60e Remove unnecessary "acknowledge: true" statements in REST tests 2018-10-25 11:53:31 -06:00
Lee Hinman 3e7042832a Merge remote-tracking branch 'origin/master' into index-lifecycle 2018-10-25 11:00:36 -06:00
Tal Levy 17fe37e5dd
ILM Docs: update start/stop documentation (#34787)
adds documentation explaining the different operation modes in the
context of starting and stopping ILM.
2018-10-25 07:30:53 -07:00
Tal Levy 41eaa586e8
remove index.lifecycle.skip setting (#34823)
With the introduction of _ilm/stop and _ilm/start APIs, the
use cases where one would only target a select group
of indices to start/stop has been reduced. Since there is no
strong use-case for skipping specific indices, it is best to
remove this functionality and only adding if later desired, with the
hopes of keeping things more simple.
2018-10-25 07:27:04 -07:00
Tal Levy 21b9b024c7
fix PolicyStatsTests mutateInstance (#34835)
through randomization, there is a chance that the mutateInstance
for PolicyStatsTests does not actually mutate the original object.
This PR aims to fix this
2018-10-25 07:24:54 -07:00
Marios Trivyzas bd143334d3
SQL: Fix edge case: `<field> IN (null)` (#34802)
Handle the case when `null` is the only value in the list so that
it's translated to a `MatchNoDocsQuery`.

Followup to: #34750
2018-10-25 14:24:11 +02:00
Colin Goodheart-Smithe 0b26f8b14c
Fixes NPE in multi node qa testt 2018-10-25 10:45:30 +01:00
Costin Leau b97546a5a9
SQL: Introduce ODBC mode, similar to JDBC (#34825)
Close #34720
2018-10-25 12:40:00 +03:00
Andrei Stefan a7e08f462f
SQL: handle X-Pack or X-Pack SQL not being available in a more graceful way (#34736)
Throw a different error message for a http response code of 400, but also when the error itself is of a specific type.
2018-10-25 12:14:49 +03:00
Nhat Nguyen ff49e79d40 CCR: Rename follow-task parameters and stats (#34836)
* CCR: Rename follow parameters and stats

This commit renames the follow-task parameters and its stats.
Below are the changes:

## Params
- remote_cluster (unchanged)
- leader_index (unchanged)
- max_read_request_operation_count -> max_read_request_operation_count
- max_batch_size -> max_read_request_size
- max_write_request_operation_count (new)
- max_write_request_size (new)
- max_concurrent_read_batches -> max_outstanding_read_requests
- max_concurrent_write_batches -> max_outstanding_write_requests
- max_write_buffer_size (unchanged)
- max_write_buffer_count (unchanged)
- max_retry_delay (unchanged)
- poll_timeout -> read_poll_timeout

## Stats
- remote_cluster (unchanged)
- leader_index (unchanged)
- follower_index (unchanged)
- shard_id (unchanged)
- leader_global_checkpoint (unchanged)
- leader_max_seq_no (unchanged)
- follower_global_checkpoint (unchanged)
- follower_max_seq_no (unchanged)
- last_requested_seq_no (unchanged)
- number_of_concurrent_reads -> outstanding_read_requests
- number_of_concurrent_writes -> outstanding_write_requests
- buffer_size_in_bytes -> write_buffer_size_in_bytes (new)
- number_of_queued_writes -> write_buffer_operation_count
- mapping_version -> follower_mapping_version
- total_fetch_time_millis -> total_read_time_millis
- total_fetch_remote_time_millis -> total_read_remote_exec_time_millis
- number_of_successful_fetches -> successful_read_requests
- number_of_failed_fetches -> failed_read_requests
- operation_received -> operations_read
- total_transferred_bytes -> bytes_read
- total_index_time_millis -> total_write_time_millis [?]
- number_of_successful_bulk_operations -> successful_write_requests
- number_of_failed_bulk_operations -> failed_write_requests
- number_of_operations_indexed -> operations_written
- fetch_exception -> read_exceptions
- time_since_last_read_millis -> time_since_last_read_millis

* add test for max_write_request_(operation_count|size)
2018-10-25 10:36:15 +02:00
Armin Braun e7ced94a65
NETWORKING: Add SSL Handler before other Handlers (#34636)
* NETWORKING: Add SSL Handler before other Handlers

* The only way to run into the issue in #33998 is for `Netty4MessageChannelHandler`
to be in the pipeline while the SslHandler is not. Adding the SslHandler before any
other handlers should ensure correct ordering here even when we handle upstream events
in our own thread pool
* Ensure that channels that were closed concurrently don't trip the assertion
* Closes #33998
2018-10-25 07:51:14 +02:00
Alpar Torok 59536966c2
Add a new "contains" feature (#34738)
The contains syntax was added in #30874 but the skips were not properly
put in place.
The java runner has the feature so the tests will run as part of the
build, but language clients will be able to support it at their own
pace.
2018-10-25 08:50:50 +03:00
Ryan Ernst 687dc1eb11
Scripting: Remove SearchScript (#34730)
This commit removes the last non context based script class.
2018-10-24 15:03:38 -07:00
Martijn van Groningen 6fe0e62b7a
[CCR] Added write buffer size limit (#34797)
This limit is based on the size in bytes of the operations in the write buffer. If this limit is exceeded then no more read operations will be coordinated until the size in bytes of the write buffer has dropped below the configured write buffer size limit.

Renamed existing `max_write_buffer_size` to ``max_write_buffer_count` to indicate that limit is count based.

Closes #34705
2018-10-24 23:48:49 +02:00
Andrey Atapin 5f588180f9 Improve IndexNotFoundException's default error message (#34649)
This commit adds the index name to the error message when an index is not found.
2018-10-24 12:53:31 -07:00
Benjamin Trent cd27b0b996
Revert "Rollup add default metrics to histo groups (#34534)" (#34815)
This reverts commit 4236358f5d.
2018-10-24 14:25:10 -05:00
Costin Leau 7570d69254
SQL: Introduce support for IP fields (#34758)
IP fields are recognized and can be used through-out the query

Close #32499
2018-10-24 22:03:11 +03:00
Marios Trivyzas 98cd7ca861
SQL: Fix queries with filter resulting in NO_MATCH (#34812)
Previously, `Mapper` was returning an incorrect plan which resulted in an
```
SQLFeatureNotSupportedException: Found 1 problem(s)
line 1:8: Unexecutable item
```

Queries with a `WHERE` and/or `HAVING` clause which results in NO_MATCH
are now handled correctly and return 0 rows.

Fixes: #34613

Co-authored-by: Costin Leau <costin.leau@gmail.com>
2018-10-24 20:36:30 +02:00
Colin Goodheart-Smithe e7fddb5c93
Adds usage data for ILM (#33377)
* Adds usage data for ILM

* Adds tests for IndexLifecycleFeatureSetUsage and friends

* Adds tests for IndexLifecycleFeatureSet

* Fixes merge errors

* Add number of indices managed to usage stats

Also adds more tests

* Addresses Review comments
2018-10-24 18:28:46 +01:00
Costin Leau f19565c3e0
SQL: Verifier allows aliases aggregates for sorting (#34773)
Improve Verifier rule that prevented grouping with aliases inside aggregates
to not be accepted for ordering.

Close #34607
2018-10-24 19:31:26 +03:00
markharwood f7a6fb288f
Fix compilation error peculiar to eclipse2018-09 (#34798)
Fix compilation error peculiar to eclipse2018-09.
Added generics to make eclipse compiler happy
2018-10-24 16:58:31 +01:00
Nhat Nguyen d73768f812
CCR: Do not follow if leader does not have soft-deletes (#34767)
We should not create a follower index and abort a follow request if the
leader does not have soft-deletes. Moreover, we also should not
auto-follow an index if it does not have soft-deletes.
2018-10-24 11:19:39 -04:00
Colin Goodheart-Smithe c7fe87e43f
Removes Set Policy API in favour of setting index.lifecycle.name directly (#34304)
* Removes Set Policy API in favour of setting index.lifecycle.name
directly

* Reinstates matcher that will still be used

* Cleans up code after rebase

* Adds test to check changing policy with ndex settings works

* Fixes TimeseriesLifecycleActionsIT after API removal

* Fixes docs tests

* Fixes case on close where lifecycle service was never created
2018-10-24 16:14:59 +01:00
Shaunak Kashyap 0f1a5ecf61
Adding stack_monitoring_agent role (#34369)
* Adding stack_monitoring_agent role

* Fixing checkstyle issues

* Adding tests for new role

* Tighten up privileges around index templates

* s/stack_monitoring_user/remote_monitoring_collector/ + remote_monitoring_user

* Fixing checkstyle violation

* Fix test

* Removing unused field

* Adding missed code

* Fixing data type

* Update Integration Test for new builtin user
2018-10-24 07:20:46 -07:00
Alpar Torok 795d57b4f9
Auto configure all test tasks (#34666)
With this change, we apply the common test config automatically to all
newly created tasks instead of opting in specifically.

For plugin authors using the plugin externally this means that the
configuration will be applied to their RandomizedTestingTasks as well.

The purpose of the task is to simplify setup and make it easier to
change projects that use the `test` task but actually run integration
tests to use a task called `integTest` for clarity, but also because
we may want to configure and run them differently.
E.x. using different levels of concurrency.
2018-10-24 16:05:50 +03:00
Marios Trivyzas 4c73854da7
SQL: Implement null handling for `IN(v1, v2, ...)` (#34750)
Implemented null handling for both the value tested but also for
values inside the list of values tested against.

The null handling is implemented for local processors, painless scripts
and Lucene Terms queries making it available for `IN` expressions occuring
in `SELECT`, `WHERE` and `HAVING` clauses.

Closes: #34582
2018-10-24 14:42:40 +02:00
lipsill d5ad3de42e [test] Introduce strict deprecation mode for REST tests (#34338)
#33708 introduced a strict deprecation mode that makes a REST request
fail if there is a warning header in the response returned by
Elasticsearch (usually a deprecation message signaling that a feature
or a field has been deprecated).

This change adds the strict deprecation mode into the REST integration
tests, and makes the tests fail if a deprecated feature is used. Also
any test using a deprecated feature has been modified to pass the build.

The YAML integration tests already analyzed HTTP warnings so they do
not use this mode, keeping their "expected vs actual" behavior.
2018-10-24 08:21:24 -04:00
Colin Goodheart-Smithe 89404ced9b
Fixes SOTP docs test to restart ILM after 2018-10-24 13:05:41 +01:00