Commit Graph

42450 Commits

Author SHA1 Message Date
Andrei Stefan b6659847c2
SQL: handle wildcard expansion on incorrect fields (#35134) 2018-11-01 11:30:05 +02:00
Marios Trivyzas 4f78e2b576
SQL: Improve painless script generated from `IN` (#35055)
Replace standard `||` and `==` painless operators with
new `in` method introduced in `InternalSqlScriptUtils`.
This allows the list of values to become a script variable
which is replaced each time with the list of  values provided
by the user.

Move In to the same package as InPipe & InProcessor

Follow up to #34750

Co-authored-by: Costin Leau <costin.leau@gmail.com>
2018-11-01 10:23:49 +01:00
Alexandru Rusanescu f3e150b0ea [Docs] Update query_cache.asciidoc (#33340)
Add note about non-visibility of cache content.
2018-11-01 10:22:36 +01:00
Nik Everett e28509fbfe
Core: Less settings to AbstractComponent (#35140)
Stop passing `Settings` to `AbstractComponent`'s ctor. This allows us to
stop passing around `Settings` in a *ton* of places. While this change
touches many files, it touches them all in fairly small, mechanical
ways, doing a few things per file:
1. Drop the `super(settings);` line on everything that extends
`AbstractComponent`.
2. Drop the `settings` argument to the ctor if it is no longer used.
3. If the file doesn't use `logger` then drop `extends
AbstractComponent` from it.
4. Clean up all compilation failure caused by the `settings` removal
and drop any now unused `settings` isntances and method arguments.

I've intentionally *not* removed the `settings` argument from a few
files:
1. TransportAction
2. AbstractLifecycleComponent
3. BaseRestHandler

These files don't *need* `settings` either, but this change is large
enough as is.

Relates to #34488
2018-10-31 21:23:20 -04:00
Igor Motov b5e5e93c46
Fixes randomDateTimeZone method (#35145)
The randomDateTimeZone method shouldn't return deprecated timezones
this causes some tests to fail with deprecation warning.
2018-10-31 20:32:18 -04:00
Tal Levy f8e23f6400
update ILM integ test cluster poll interval to 1s (#35113) 2018-10-31 17:09:35 -07:00
Ryan Ernst bfaa94f53e
Packaging: Update procrun executables to version 1.1.0 (#35147)
This commit updates the procrun manager and service exes to 1.1.0. There
are a few bug fixes, including for a bug which can cause lingering
processes when removing the service.
2018-10-31 16:31:14 -07:00
Seong-hyun, Oh 9ef4788c13 Make XContentBuilder in AliasActions build `is_write_index` field (#35071)
Make XContentBuilder in AliasesActions build `is_write_index` field
2018-10-31 14:15:46 -07:00
lipsill d181d1bab1 Remove deprecated url parameters `_source_include` and `_source_exclude` (#35097)
Removes `_source_include` and `_source_exclude` url parameters. 
These parameters have been deprecated in #33475.

Closes #22792
2018-10-31 17:11:59 -04:00
Costin Leau f87a53451b
SQL: Register missing processors (#35121)
Add registration (and tests) for missing processors in the serialization
chain.

Fix #35119
2018-10-31 23:07:36 +02:00
Armin Braun e6f9f0666e
NETWORKING: MockTransportService Wait for Close (#35038)
* NETWORKING: MockTransportService Wait for Close

* Make `MockTransportService` wait `30s` for close listeners to run before failing the assertion
* Closes #34990
2018-10-31 21:33:49 +01:00
David Turner 0072c90e2a
Pre-populate unicast hosts files (#35136)
Today when ESIntegTestCase starts some nodes it writes out the unicast hosts
files each time a node starts its transport service. This does mean that a
number of nodes can start and perform their first pinging round without any
unicast hosts which, if the timing is unlucky and a lot of nodes are all
started at the same time, can lead to a split brain as in #35052.

Prior to #33554 this was unlikely to happen since the MockUncasedHostsProvider
would always have yielded the existing hosts, so the timing would have to have
been implausibly unlucky. Since #33554, however, it's more likely because the
race occurs between the start of the first round of pinging and the writing of
the unicast hosts file. It is realistic that new nodes will be configured with
the existing nodes from startup, so this change reinstates that behaviour.

Closes #35052.
2018-10-31 19:21:24 +00:00
Tal Levy 5f4b23f8c1
cleanup ILM qa structure (#35110)
This commit does a few things

- moves ILM-specifc rest yaml tests into plugin/ilm/qa, and creates special
  :plugin:ilm:qa:rest module to test them
- removes the with-security tests of the yaml tests since they are covered in
  the rest tests now
- moves ChangePolicyforIndexIT into the qa/multi-node project since that test is
  not currently running in main ilm since integTest is disabled
2018-10-31 11:49:29 -07:00
Andy Bristol 6492eaa84d [test] mad tests more lenient approximation 2018-10-31 11:48:58 -07:00
Nik Everett ca620ff4ce
Loggers: Drop last deprecated logger function (#35082)
Drop the last function from `Loggers` that just wraps Log4j2.

Relates to #32174
2018-10-31 14:38:29 -04:00
Andy Bristol 010845b233 [test] dont skip last version for mad 2018-10-31 11:36:25 -07:00
Tal Levy a294a7c6b5 fix IndexLifecycleService setting member
the settings variable was previously created by
the AbstractComponent class inherited by IndexLifecycleService.
this is no more.
2018-10-31 11:17:16 -07:00
Qian Hong f27251dfcb [Docs] Improve Redindex/Update/Delete By Query. (#35127)
Fix a copy paste error and improve formatting in
Redindex/Update/Delete By Query and fix formula in Update By Query.
2018-10-31 14:04:48 -04:00
Tal Levy d5d28420b6 Merge remote-tracking branch 'upstream/master' into index-lifecycle 2018-10-31 10:47:07 -07:00
Luca Cavanna ef5181c678
Allow to enable pings for specific remote clusters (#34753)
When we connect to remote clusters, there may be a few more routers/firewalls in-between compared to when we connect to nodes in the same cluster. We've experienced cases where firewalls drop connections completely and keep-alives seem not to be enough, or they are not properly configured. With this commit we allow to enable application-level pings specifically from CCS nodes to the selected remote nodes through the new setting `cluster.remote.${clusterAlias}.transport.ping_schedule`.  The new setting is similar `transport.ping_schedule` but it does not affect intra-cluster communication, pings are only sent to specific remote cluster when specifically enabled, as they are disabled by default.

Relates to #34405
2018-10-31 17:32:53 +01:00
Lisa Cawley 72cb885abc
[DOCS] Updates monitoring diagram (#35096) 2018-10-31 09:18:14 -07:00
Mayya Sharipova 0b8465f2c0
Update type to "seat" (#35086)
otherwise this throws an exception: java.lang.IllegalArgumentException: Rejecting mapping update to [seats] as the final mapping would have more than 1 type: [seat, _doc]

May be, later for the types removal, we can modify `seats.json` to have a type `_doc` instead of `seat`
2018-10-31 12:06:48 -04:00
Sue Gallagher 1ce3c92a2d [DOCS] Add info on calendar vs fixed interval. (#31638)
Extensive edit to add additional information on the difference between calendar intervals and fixed-length intervals.
2018-10-31 10:16:36 -04:00
Albert Zaharovits bdd8460906
HLRest: add security authenticate API (#33552)
This adds the security `_authenticate` API to the HLREST client.
It is unlike some of the other APIs because the request does not
have a body.
The commit also creates the `User` entity. It is important
to note that the `User` entity does not have the `enabled`
flag. The `enabled` flag is part of the response, alongside
the `User` entity.
Moreover this adds the `SecurityIT` test class 
(extending `ESRestHighLevelClientTestCase`).

Relates #29827
2018-10-31 14:49:33 +02:00
Vladimir Dolzhenko 28bb1a3ac0
XPack/HLRC request/response compatibility tests (#34547)
Relates #34451 , #29827
2018-10-31 13:48:35 +01:00
Luca Cavanna 674225aaa1
[TEST] Enforce skip headers when needed (#34735)
The java yaml test runner supports sending request headers, yet not all clients support headers. This commit makes sure that we enforce adding a skip section with feature "headers" whenever headers are used in a do section as part of a test. That decreases the chance for new tests to break client builds due to the missing skip section.

Closes #34650
2018-10-31 13:07:02 +01:00
Dimitris Athanasiou 00dc2ba36f
[ML] Enable reusing field extraction logic when no time field is required (#35100) 2018-10-31 10:55:11 +00:00
Marios Trivyzas bf6f521827
SQL: Fix NPE thrown if HAVING filter evals to null (#35108)
Surround script with `nullSafeFilter` also for `AggFilter` translation.
Fixed query translation tests to properly set `onAggs`.

Fixes: #35107
2018-10-31 10:32:16 +01:00
Armin Braun 3fa67c5d8a
DISCOVERY: Cleanup AbstractDisruptionTestCase (#34808)
* DISCOVERY: Cleanup AbstractDisruptionTestCase

* Make the internal test cluster manage minimum master nodes where we used the default of (nodes / 2 + 1) before
* Remove use of the `NodeConfigurationSource` indirection
* Relates #33675
2018-10-31 07:52:37 +01:00
debadair e540abd551
[DOCS] Fixed edit links for ILM APIs and added the APIs to the REST ref (#35070)
* [DOCS] Fixed edit links for ILM APIs and added the APIs to the REST API section.

* [DOCS] Fixed link to ILM APIs.
2018-10-30 18:27:43 -07:00
Tal Levy fdb0db30c4
throw exception when ilm-rollover-step runs on index without rollover alias (#35065)
The ILM Rollover Step can execute on the incorrect index if the rollover alias
exists on another valid index, but not the one the step is executing against. This
is a problem and is now guarded against
2018-10-30 16:42:12 -07:00
Tal Levy 5141084048
rename CRUD api REST path prefix _ilm to _ilm/policy (#35056)
This PR renames the CRUD APIS for ILM

GET _ilm/<policy>, _ilm -> _ilm/policy/<policy>, _ilm/policy
PUT _ilm/<policy> -> _ilm/policy/<policy>
DELETE _ilm/<policy> -> _ilm/policy/<policy>

closes #34929.
2018-10-30 16:19:05 -07:00
Marios Trivyzas 455906d431
SQL: [tests] Remove AbstractBuilderTestCase dep (#35099)
Extend directly from `EsTestCase` to speed up execution.

Fixes: #35098
2018-10-30 22:31:29 +01:00
Nik Everett 086ada4c08
Core: Drop settings member from AbstractComponent (#35083)
Drops the `Settings` member from `AbstractComponent`, moving it from the
base class on to the classes that use it. For the most part this is a
mechanical change that doesn't drop `Settings` accesses. The one
exception to this is naming threads where it switches from an invocation
that passes `Settings` and extracts the node name to one that explicitly
passes the node name.

This change doesn't drop the `Settings` argument from
`AbstractComponent`'s ctor because this change is big enough as is.
We'll do that in a follow up change.
2018-10-30 16:10:38 -04:00
Tal Levy f5323cd86e
rename ILM Phase "minimum_age" to "min_age" (#35058)
closes #34927.
2018-10-30 12:54:02 -07:00
Nik Everett 0f9a7b3344 Docs: Fix build after moving SQL QA
I moved SQL's QA in #34496 to make it easier to run all of SQL tests in
one go but forgot to update the docs build. This fixes that.
2018-10-30 15:47:25 -04:00
Jay Modi 09a124e8d6
Native roles store uses mget to retrieve roles (#33531)
The native roles store previously would issue a search if attempting to
retrieve more than a single role. This is fine when we are attempting
to retrieve all of the roles to list them in the api, but could cause
issues when attempting to find roles for a user. The search is not
prioritized over other search requests, so heavy aggregations/searches
or overloaded nodes could cause roles to be cached as missing if the
search is rejected.

When attempting to load specific roles, we know the document id for the
role that we are trying to load, which allows us to use the multi-get
api for loading these roles. This change makes use of the multi-get api
when attempting to load more than one role by name. This api is also
constrained by a threadpool but the tasks in the GET threadpool should
be quicker than searches.

See #33205
2018-10-30 13:35:07 -06:00
Gordon Brown 6ecb8ff344
Move to Error step if ClusterState* steps throw (#35069)
Previously, if ClusterStateActionSteps or ClusterStateWaitSteps threw an
exception executing, the exception would only be caught and logged by
the generic ClusterStateUpdateTask machinery and the index would become
stuck on that step.

Now, exceptions thrown in these steps will be caught and the index will
be moved to the Error step.
2018-10-30 13:33:32 -06:00
Nik Everett 7de9eda391
Graph: Move qa into plugin (#34517)
Move `x-pack/qa/smoke-test-graph-with-security` to
`x-pack/plugin/graph/qa/security` which should make it easier to run all
of the tests with a single command. It also lines up the directories
more closely with newer projects like cross cluster replication.
2018-10-30 15:14:33 -04:00
Nik Everett 6857d30527
SQL: Move qa directory into sql directory (#34496)
Moves `x-pack/qa/sql/*` into `x-pack/plugin/sql/qa` to make it simpler
to run all of the sql tests. This lines up with how newer projects like
cross cluster replication are testing themselves.
2018-10-30 14:54:15 -04:00
Costin Leau 6abddd8ca6
SQL: Optimizer rule for folding nullable expressions (#35080)
Add optimization for folding nullable expressions with a NULL 
argument. This is a variant of folding for the NULL case.

Fix #34826
2018-10-30 20:17:20 +02:00
Ryan Ernst 512319cef7
Test: Filter out deprecated joda tzs in tests (#34868)
This commit filters out usage of deprecated tzs by tests. These are
tested separately and should not require checking for warnings on any
test using random timezones.

closes #34188
2018-10-30 11:15:34 -07:00
Nik Everett 2b2e208c4c
Docs: Remove range notation from random score docs (#35093)
The `random_score` function produces values between 0 (inclusive) and 1
(exclusive) and documented it with fancy methematical range notation. It
is so fancy I thought it was a typo. This changes the documentation to
use words.

Relates to #35084
2018-10-30 14:12:59 -04:00
Zachary Tong f9dd33a0b9
[Rollup] Proactively resolve index patterns in RollupSearch endoint (#34930)
This changes the RollupSearch endpoint to proactively resolve index
patterns.  If the index pattern(s) match more than one rollup index,
an exception is throw as before.  But if the pattern only matches one
rollup index, execution is allowed to continue (unlike before where
it would assume all patterns were for raw data).

This also allows the search endpoint to resolve aliases that point to
a rollup index.

Also tweaks the documentation to make this clear.

Closes #34828
2018-10-30 13:50:50 -04:00
Julie Tibshirani 70da490f34
Remove some documentation that only makes sense with multiple types. (#35066)
* Remove a tip about ignore_above that only makes sense with multiple types.
* Remove a line from the percolator documentation that refers to multiple types.
2018-10-30 10:19:12 -07:00
Vladimir Dolzhenko be75b40a29
Fix LineLength Check Suppressions: index.mapper (#35087)
Relates #34884
2018-10-30 18:00:14 +01:00
Nik Everett 70e939ee38 Search: Test deprecated field in mixed cluster
Update the test for the deprecated `_source_exclude` parameter to run
against 6.6.0+ now that we've backported the deprecation.

Relates to #33475
2018-10-30 11:59:09 -04:00
Tal Levy 18c72e86c5 Merge remote-tracking branch 'upstream/master' into index-lifecycle 2018-10-30 08:09:57 -07:00
Tal Levy 6cf7f1205a Revert "add ILM rolling upgrade tests (#32828)"
This reverts commit 5c88356ad6.
2018-10-30 08:06:53 -07:00
Luca Cavanna 7ef65dedc3
[TEST] improve validation of yaml suites (#34957)
Validation of test sections and suites consists of checking that the proper skip features sections are in place depending on the features used in tests.

The validation logic was previously only performed on do sections included in each test section, and the skip needed to be present in the same test section. What happens often though is that the skip is added to the setup section, or the teardown section.

This commit improves the validation of test suites by validating setup and teardown section first, then looking at each test section while still eventually reading the skip section from setup or teardown.

We are also making SkipSection, SetupSection, TearDownSection, ClientYamlTestSection and ClientYamlTestSuite immutable. Previously it was possible to utilize constants like SetupSection.EMPTY, which were modifiable and affect every other future users by modifiying them. This has been corrected.

Also, validation has been improved to cumulate errors so that all the errors from a suite will be listed at once.

Relates to #34735
2018-10-30 16:06:31 +01:00