Commit Graph

44755 Commits

Author SHA1 Message Date
Yuri Astrakhan 6a13a252e9
Abstract GeoHashGridAggregatorFactory creation, renamed geohash -> hash (#37836)
* Delegate `new GeoHashGridAggregatorFactory(...)` inside the `GeoGridAggregationBuilder` to the child classes.
* Rename all `geohash...` to `hash...`
2019-01-24 23:45:18 -05:00
Nhat Nguyen 3ccd488755 Remove testMappingsPropagatedToMasterNodeImmediately
This test is obsolete since #31140 where an index request with dynamic
mapping update no longer requires acking.

Closes #37816
2019-01-24 21:48:50 -05:00
Mayya Sharipova a30ce6a00a
Rename feature, feature_vector and feature_query (#37794)
Ranaming as follows:
feature -> rank_feature
feature_vector -> rank_features
feature query -> rank_feature query

Ranaming is done to distinguish from other vector types.

Closes #36723
2019-01-24 19:18:48 -05:00
Boaz Leskes 9d87ca567a 300_sequence_numbers should not rely on 7.0 total hits structure 2019-01-24 23:41:10 +01:00
Peter Dyson 4313c2d4a7
[DOCS] More info on disabling swap (#37248)
* [DOCS] More info on disabling swap, inform that no Elasticsearch restart is required.
2019-01-25 08:39:58 +10:00
Lee Hinman 0f3c542850
Deprecate xpack.watcher.history.cleaner_service.enabled (#37782)
This deprecates the `xpack.watcher.history.cleaner_service.enabled` setting,
since all newly created `.watch-history` indices in 7.0 will use ILM to manage
their retention.

In 8.0 the setting itself and cleanup actions will be removed.

Resolves #32041
2019-01-24 15:31:31 -07:00
Julie Tibshirani d473bcda8d Remove outdated callouts from the 'create index' HLRC docs 2019-01-24 14:08:58 -08:00
Christoph Büscher 967de04257
Uppercasing some docs section title (#37781)
Section titles are mostly uppercase, only a few cases where query DSL parameters
or Java method names are used as the title they should be lowercased.
2019-01-24 22:54:55 +01:00
Julie Tibshirani e1d8df4ffa
Deprecate types in create index requests. (#37134)
From #29453 and #37285, the include_type_name parameter was already present and defaulted to false. This PR makes the following updates:
* Add deprecation warnings to RestCreateIndexAction, plus tests in RestCreateIndexActionTests.
* Add a typeless 'create index' method to the Java HLRC, and deprecate the old typed version. To do this cleanly, I created new CreateIndexRequest and CreateIndexResponse objects that differ from the existing server ones.
2019-01-24 13:17:47 -08:00
Boaz Leskes af2f4c8f73 enable bwc tests and bump versions after backporting https://github.com/elastic/elasticsearch/pull/37639 2019-01-24 20:55:55 +01:00
Nhat Nguyen 864e465515 Adjust minRetainedSeqNo asssertion in CombinedDeletionPolicyTests
In these tests, we initialize the retained_seq_no with NO_OPS_PERFORMED,
thus we should verify that the min of the retained_seq_no is at least
NO_OPS_PERFORMED not 0.

Closes #35994
2019-01-24 13:43:51 -05:00
Andrey Ershov 4974684003
Add tool elasticsearch-node unsafe-bootstrap (#37696)
elasticsearch-node tool helps to restore cluster if half or more of
master eligible nodes are lost. Of course, all bets are off, regarding
data consistency.

There are two parts of the tool: unsafe-bootstrap to be used when there
is still at least one master-eligible node alive and detach-cluster,
when there are no master-eligible nodes left.
This commit implements the first part.

Docs for the tool will be added separately as a part of #37812.
2019-01-24 19:25:55 +01:00
Tal Levy 289106a578
Refactor GeoHashGrid to be abstract and re-usable (#37742)
This change split out all the specific GeoHash
classes for the geohash_grid aggregation into
abstract GeoGrid classes that can be re-used for
specific hashing types, like `geohash`
2019-01-24 10:12:14 -08:00
Lisa Cawley 33cac52b2c
[DOCS] Remove beta tag from metricbeat monitoring (#37791) 2019-01-24 09:14:08 -08:00
Nhat Nguyen 76fb573569
Do not allow put mapping on follower (#37675)
Today, the mapping on the follower is managed and replicated from its
leader index by the ShardFollowTask. Thus, we should prevent users
from modifying the mapping on the follower indices.

Relates #30086
2019-01-24 12:13:00 -05:00
David Turner 187b233571 Read m_m_n from cluster states from 6.7
This completes the BWC serialisation changes required for a 6.7 master to
inform other nodes of the node-level value of the `minimum_master_nodes`
setting.

Relates #37701, #37811
2019-01-24 17:05:49 +00:00
David Roberts 0e36adc35f Mute SimpleClusterStateIT testMetadataVersion
Due to https://github.com/elastic/elasticsearch/issues/37820
2019-01-24 16:50:55 +00:00
Marios Trivyzas 74b6f308e9
SQL: Fix issue with complex expression as args of PERCENTILE/_RANK (#37102)
When the arguements of PERCENTILE and PERCENTILE_RANK can be folded,
the `ConstantFolding` rule kicks in and calls the `replaceChildren()`
method on `InnerAggregate` which is created from the aggregation rules
of the `Optimizerz. `InnerAggregate` in turn, cannot implement the method
as the logic of creating a new `InnerAggregate` instance from a list of
`Expression`s resides in the Optimizer. So, instead, `ConstantFolding`
should be applied before any of the aggregations related rules.

Fixes: #37099
2019-01-24 18:40:20 +02:00
Alpar Torok f6b6f927ec
Fail randomized testing tasks if no tests ran (#37764)
Reverts #36259 in part to make randomized test fail if no tests are ran.
This is useful when filtering tests as it's easy to make a typo and
think the test ran trough successfully.
2019-01-24 17:33:08 +02:00
Alpar Torok 37768b7eac
Testing conventions now checks for tests in main (#37321)
* Testing conventions now checks for tests in main

This is the last outstanding feature of the old NamingConventionsTask,
so time to remove it.

* PR review
2019-01-24 17:30:50 +02:00
Alpar Torok e7aa7e909a
Switch testclusters to use nio (#37365) 2019-01-24 17:30:24 +02:00
niloct ddee1926f9 Update update-by-query.asciidoc (#37555)
Similar fix as #37370.
2019-01-24 16:27:42 +01:00
Alpar Torok 4e08cca6bc
Ground work to start up the docker image in the build (#37754)
This change adds a docker compose configuration that's used with
the `elasticsearch.test.fixtures` plugin to start up the image
and check that the TCP ports are up.

We can build on this to add other checks for culster health,
run REST tests, etc.

We can add multiple containers and configurations to the compose
file (e.x. test different env vars) and form clusters.
2019-01-24 17:26:42 +02:00
David Roberts bd02ca4b7b Mute NoMasterNodeIT testNoMasterActionsWriteMasterBlock
Due to https://github.com/elastic/elasticsearch/issues/37823
2019-01-24 15:17:13 +00:00
Ioannis Kakavas 265710e658
Better msg on unmapped principal attribute (#37805)
When we can't map the principal attribute from the configured SAML
attribute in the realm settings, we can't complete the
authentication. We return an error to the user indicating this and
we present them with a list of attributes we did get from the SAML
response to point out that the expected one was not part of that
list. This list will never contain the NameIDs though as they are
not part of the SAMLAttribute list. So we might have a NameID but
just with a different format.
2019-01-24 17:05:01 +02:00
Nhat Nguyen a6abb28abf
Fix InternalEngineTests#assertOpsOnPrimary (#37746)
The assertion `assertOpsOnPrimary` does not store seq_no and primary
term of successful deletes to the `lastOpSeqNo` and `lastOpTerm`. This
leads to failures of the subsequence CAS deletes or indexes with seq_no
and term. Moreover, this assertion trips a translog assertion because it
bumps the primary term of some operations but not the primary term of
the engine.

Relates #36467
Closes #37684
2019-01-24 10:02:48 -05:00
David Roberts a81931bb2a Mute DynamicMappingIT testMappingsPropagatedToMasterNodeImmediately
Due to https://github.com/elastic/elasticsearch/issues/37816
2019-01-24 14:32:44 +00:00
Andrei Stefan 163a27b93c
SQL: Fix BasicFormatter NPE (#37804) 2019-01-24 15:40:51 +02:00
Jason Tedor 7517e3a7bd
Optimize warning header de-duplication (#37725)
Now that warning headers no longer contain a timestamp of when the
warning was generated, we no longer need to extract the warning value
from the warning to determine whether or not the warning value is
duplicated. Instead, we can compare strings directly.

Further, when de-duplicating warning headers, are constantly rebuilding
sets. Instead of doing that, we can carry about the set with us and
rebuild it if we find a new warning value.

This commit applies both of these optimizations.
2019-01-24 08:39:24 -05:00
Yannick Welsch feab59df03
Bubble exceptions up in ClusterApplierService (#37729)
Exceptions thrown by the cluster applier service's settings and cluster appliers are bubbled up, and
block the state from being applied instead of silently being ignored. In combination with the cluster
state publishing lag detector, this will throw a node out of the cluster that can't properly apply
cluster state updates.
2019-01-24 14:09:03 +01:00
Marios Trivyzas 9357929309
SQL: Improve handling of invalid args for PERCENTILE/PERCENTILE_RANK (#37803)
Improve the Exception and the error message returned when 2nd argument
of PERCENTILE and PERCENTILE_RANK is not a constant.
2019-01-24 15:03:49 +02:00
Simon Willnauer c7b16162ae
Remove unused ThreadBarrier class (#37666)
This class is pretty complex and only used in a test where we can simply
fail the test with an assertion error.
2019-01-24 13:52:22 +01:00
Yulong 20533c5990
Add built-in user and role for code plugin (#37030)
* Add built-in roles for code plugin

* Fix rest-client get-roles test count

* Fix broken test
2019-01-24 20:12:32 +08:00
Alpar Torok bc20142791
Consolidate testclusters tests into a single project (#37362) 2019-01-24 13:47:21 +02:00
Yannick Welsch 2bf269e628 Fix docs for MappingUpdatedAction
Follow-up to #31140
2019-01-24 12:44:36 +01:00
Marios Trivyzas f707fa9e0a
SQL: Introduce SQL DATE data type (#37693)
* SQL: Introduce SQL DATE data type

Support ANSI SQL's DATE type by introducing a runtime-only
ES SQL date type.

Closes: #37340
2019-01-24 13:41:58 +02:00
Boaz Leskes b6317ed70b disabling bwc test while backporting https://github.com/elastic/elasticsearch/pull/37639 2019-01-24 12:01:32 +01:00
David Roberts bcf5a4ca47 Mute ClusterDisruptionIT testAckedIndexing
Due to https://github.com/elastic/elasticsearch/issues/37810
2019-01-24 10:58:02 +00:00
Yannick Welsch 64adb5ad5b
Set acking timeout to 0 on dynamic mapping update (#31140)
As acking can fail for any reason (unrelated node being too slow, node disconnecting), it should not
be required for acking to succeed in order for index requests with dynamic mapping updates to
successfully complete.

Relates to #30672 and Closes #30844
2019-01-24 11:39:46 +01:00
Albert Zaharovits b6936e3c1e
Remove index audit output type (#37707)
This commit removes the Index Audit Output type, following its deprecation
in 6.7 by 8765a31d4e6770. It also adds the migration notice (settings notice).

In general, the problem with the index audit output is that event indexing
can be slower than the rate with which audit events are generated,
especially during the daily rollovers or the rolling cluster upgrades.
In this situation audit events will be lost which is a terrible failure situation
for an audit system.
Besides of the settings under the `xpack.security.audit.index` namespace, the
`xpack.security.audit.outputs` setting has also been deprecated and will be
removed in 7. Although explicitly configuring the logfile output does not touch
any deprecation bits, this setting is made redundant in 7 so this PR deprecates
it as well.

Relates #29881
2019-01-24 12:36:10 +02:00
David Roberts f12bfb4684 Mute FollowerFailOverIT testReadRequestsReturnsLatestMappingVersion
Due to https://github.com/elastic/elasticsearch/issues/37807
2019-01-24 09:58:50 +00:00
David Kyle e1226f69b7
[ML] Increase close job timeout and lower the max number (#37770) 2019-01-24 09:18:48 +00:00
Armin Braun 36889e8a2f
Remove Custom Listeners from SnapshotsService (#37629)
* Remove Custom Listeners from SnapshotsService

Motivations:
    * Shorten the code some more
    * Use ActionListener#wrap to get easy to reason about behavior in failure scenarios
    * Remove duplication in the logic of handling snapshot completion listeners (listeners removing themselves and comparing snapshots to their targets)
        * Also here, move all listener handling into `SnapshotsService` and remove custom listener class by putting listeners in a map
2019-01-24 10:11:18 +01:00
David Turner bdef2ab8c0
Use m_m_nodes from Zen1 master for Zen2 bootstrap (#37701)
Today we support a smooth rolling upgrade from Zen1 to Zen2 by automatically
bootstrapping the cluster once all the Zen1 nodes have left, as long as the
`minimum_master_nodes` count is satisfied. However this means that Zen2 nodes
also require the `minimum_master_nodes` setting for this one specific and
transient situation.

Since nodes only perform this automatic bootstrapping if they previously
belonged to a Zen1 cluster, they can keep track of the `minimum_master_nodes`
setting from the previous master instead of requiring it to be set on the Zen2
node.
2019-01-24 08:57:40 +00:00
Martijn van Groningen 2908ca1b35
Fix index filtering in follow info api. (#37752)
The filtering by follower index was completely broken.
Also the wrong persistent tasks were selected, causing the
wrong status to be reported.

Closes #37738
2019-01-24 08:50:23 +01:00
Ryan Ernst d9d13f3414
Use project dependency instead of substitutions for distributions (#37730)
Currently integration tests which use either bwc snapshot versions or
the current version of elasticsearch depend on project substitutions to
link to the build of those artifacts. Likewise, vagrant tests use
dependency substitutions to get to bwc snapshots of rpm and debs.
This commit changes those to depend on the relevant project/configuration
and removes the dependency substitutions for distributions we do not
publish.
2019-01-23 23:41:21 -08:00
Michael Basnight 04c64147bd
Update authenticate to allow unknown fields (#37713)
AuthenticateResponse did not allow unknown fields. This commit fixes the
test and ConstructingObjectParser such that it does now allow unknown
fields.

Relates #36938
2019-01-23 22:14:01 -06:00
Michael Basnight 944972a249
Deprecate HLRC EmptyResponse used by security (#37540)
The EmptyResponse is essentially the same as returning a boolean, which
is done in other places. This commit deprecates all the existing
EmptyResponse methods and creates new boolean methods that have method
params reordered so they can exist with the deprecated methods. A
followup PR in master will remove the existing deprecated methods, fix
the parameter ordering and deprecate the incorrectly ordered parameter
methods.

Relates #36938
2019-01-23 22:13:16 -06:00
Ryan Ernst 49b8b07758
Use explicit version for build-tools in example plugin integ tests (#37792)
The example plugins are currently built within the build-tools integ
tests as a means to ensure the gradle plugin works for external plugin
builds. These tests generate a dummy build.gradle, and a dummy local
maven repository to find the local builds dependencies in. Currently
that build-tools dependency uses "+" as the version. However, this
allows gradle to find the "latest" version, and unfortunately gradle has
its own plugin repository which is apparently connected to jcenter. This
recently triggered a flood of CI failures when jcenter suddenly pulled
alpha2, and all builds started trying to use that instead of the locally
built build-tools. This commit uses the explicit version of build-tools
that was build locally, which will cause resolution to stop when the
local repo is first checked.
2019-01-23 17:39:13 -08:00
Mayya Sharipova fdb66039d4
Change `rational` to `saturation` in script_score (#37766)
This change of the function name is necessary for conformity
with feature queries.

Closes #37714
2019-01-23 14:28:20 -05:00