Commit Graph

44419 Commits

Author SHA1 Message Date
Jim Ferenczi 787acb14b9
Track total hits up to 10,000 by default (#37466)
This commit changes the default for the `track_total_hits` option of the search request
to `10,000`. This means that by default search requests will accurately track the total hit count
up to `10,000` documents, requests that match more than this value will set the `"total.relation"`
to `"gte"` (e.g. greater than or equals) and the `"total.value"` to `10,000` in the search response.
Scroll queries are not impacted, they will continue to count the total hits accurately.
The default is set back to `true` (accurate hit count) if `rest_total_hits_as_int` is set in the search request.
I choose `10,000` as the default because that's also the number we use to limit pagination. This means that users will be able to know how far they can jump (up to 10,000) even if the total number of hits is not accurate.

Closes #33028
2019-01-25 13:45:39 +01:00
Alpar Torok e7f0adb0c5
Add path sensitivity annotations (#37762)
The plugin builder plugin generates warnings for these.
There's no immediate impact as we don't have a shared build cache.
2019-01-25 14:15:09 +02:00
Alpar Torok 7b516f99b9
Resolve BWC dependencies too when resolving deps (#37412)
This will allow those to be cached too leading to faster builds and
fewer opportunity for network failures
2019-01-25 14:14:12 +02:00
Mayya Sharipova 70af3c7983
Correct deprec log in RestGetFieldMappingAction (#37843)
* Correct deprec log in RestGetFieldMappingAction

Correct a class used for deprecation logging in
RestGetFieldMappingAction

* Correct deprec log in RestCreateIndexAction

Correct a class used for deprecation logging in
RestCreateIndexAction
2019-01-25 07:13:46 -05:00
Alpar Torok 97e89ad0bc
Testclusters logging improovements (#37354)
- Cluster logs are only indented as  node name is already in the logs
- silence logging on shutdown
- have fully qualified name as node and cluster name
2019-01-25 14:13:34 +02:00
Armin Braun be6bdab346
Use TestFixturesPlugin to Run Minio in Tests (#37852)
* Use TestFixturesPlugin to Run Minio in Tests

* Closes #37680
* Closes #37783
2019-01-25 12:56:40 +01:00
Tanguy Leroux a3baa8f5ef
Freezing an index should increase its index settings version (#37813)
When an index is frozen, two index settings are updated (index.frozen and 
index.search.throttled) but the settings version is left unchanged and does 
not reflect the settings update. This commit change the 
TransportFreezeIndexAction so that it also increases the settings version 
when an index is frozen/unfrozen.

This issue has been caught while working on the replication of closed 
indices (#3388) in which index metadata for a closed index are updated 
to frozen metadata and this specific assertion tripped.
2019-01-25 11:27:27 +01:00
Andrey Ershov 9e7fd8caed
Migrate ZenDiscoveryIT to Zen2 (#37465)
ZenDiscoveryIT contained 5 tests. 3 run without changes, testNodeRejectsClusterStateWithWrongMasterNode removed, testHandleNodeJoin_incompatibleClusterState changed.
2019-01-25 11:17:09 +01:00
David Roberts 170d7413d0
[ML] Fix gaps in reserved roles tests (#37772)
Some of our newer endpoints and indices were missing from
the tests.
2019-01-25 09:29:53 +00:00
Armin Braun 7692b607b9
Fix ClusterDisruptionIT#testAckedIndexing (#37853)
* Stop threads before logging the list of exceptions
* For the broken case of concurrent iteration in the finally block and the threads not having shut down,
use `CopyOnWriteArrayList` to have concurrency safe iteration
* Closes #37810
2019-01-25 09:38:29 +01:00
Martijn van Groningen 5a9dadb3ff
changed versionAdded now that #37767 is backedported 2019-01-25 09:18:42 +01:00
Martijn van Groningen 1151f3b3ff
Fail with a dedicated exception if remote connection is missing or (#37767)
or connectivity to the remote connection is failing.

Relates to #37681
2019-01-25 08:53:18 +01:00
Ricardo Ferreira df8fa9781e Remove Abstract Component (#35898)
TransportAction and BaseRestHandler now no longer extends AbstractComponent. The AbstractComponent no longer has usages so it was deleted.

Closes #34488
2019-01-25 08:35:19 +01:00
Tim Vernum 03690d12b2
Remove TLS 1.0 as a default SSL protocol (#37512)
The default value for ssl.supported_protocols no longer includes TLSv1
as this is an old protocol with known security issues.
Administrators can enable TLSv1.0 support by configuring the
appropriate `ssl.supported_protocols` setting, for example:

xpack.security.http.ssl.supported_protocols: ["TLSv1.2","TLSv1.1","TLSv1"]

Relates: #36021
2019-01-25 15:46:39 +11:00
Yuri Astrakhan 6a13a252e9
Abstract GeoHashGridAggregatorFactory creation, renamed geohash -> hash (#37836)
* Delegate `new GeoHashGridAggregatorFactory(...)` inside the `GeoGridAggregationBuilder` to the child classes.
* Rename all `geohash...` to `hash...`
2019-01-24 23:45:18 -05:00
Nhat Nguyen 3ccd488755 Remove testMappingsPropagatedToMasterNodeImmediately
This test is obsolete since #31140 where an index request with dynamic
mapping update no longer requires acking.

Closes #37816
2019-01-24 21:48:50 -05:00
Mayya Sharipova a30ce6a00a
Rename feature, feature_vector and feature_query (#37794)
Ranaming as follows:
feature -> rank_feature
feature_vector -> rank_features
feature query -> rank_feature query

Ranaming is done to distinguish from other vector types.

Closes #36723
2019-01-24 19:18:48 -05:00
Boaz Leskes 9d87ca567a 300_sequence_numbers should not rely on 7.0 total hits structure 2019-01-24 23:41:10 +01:00
Peter Dyson 4313c2d4a7
[DOCS] More info on disabling swap (#37248)
* [DOCS] More info on disabling swap, inform that no Elasticsearch restart is required.
2019-01-25 08:39:58 +10:00
Lee Hinman 0f3c542850
Deprecate xpack.watcher.history.cleaner_service.enabled (#37782)
This deprecates the `xpack.watcher.history.cleaner_service.enabled` setting,
since all newly created `.watch-history` indices in 7.0 will use ILM to manage
their retention.

In 8.0 the setting itself and cleanup actions will be removed.

Resolves #32041
2019-01-24 15:31:31 -07:00
Julie Tibshirani d473bcda8d Remove outdated callouts from the 'create index' HLRC docs 2019-01-24 14:08:58 -08:00
Christoph Büscher 967de04257
Uppercasing some docs section title (#37781)
Section titles are mostly uppercase, only a few cases where query DSL parameters
or Java method names are used as the title they should be lowercased.
2019-01-24 22:54:55 +01:00
Julie Tibshirani e1d8df4ffa
Deprecate types in create index requests. (#37134)
From #29453 and #37285, the include_type_name parameter was already present and defaulted to false. This PR makes the following updates:
* Add deprecation warnings to RestCreateIndexAction, plus tests in RestCreateIndexActionTests.
* Add a typeless 'create index' method to the Java HLRC, and deprecate the old typed version. To do this cleanly, I created new CreateIndexRequest and CreateIndexResponse objects that differ from the existing server ones.
2019-01-24 13:17:47 -08:00
Boaz Leskes af2f4c8f73 enable bwc tests and bump versions after backporting https://github.com/elastic/elasticsearch/pull/37639 2019-01-24 20:55:55 +01:00
Nhat Nguyen 864e465515 Adjust minRetainedSeqNo asssertion in CombinedDeletionPolicyTests
In these tests, we initialize the retained_seq_no with NO_OPS_PERFORMED,
thus we should verify that the min of the retained_seq_no is at least
NO_OPS_PERFORMED not 0.

Closes #35994
2019-01-24 13:43:51 -05:00
Andrey Ershov 4974684003
Add tool elasticsearch-node unsafe-bootstrap (#37696)
elasticsearch-node tool helps to restore cluster if half or more of
master eligible nodes are lost. Of course, all bets are off, regarding
data consistency.

There are two parts of the tool: unsafe-bootstrap to be used when there
is still at least one master-eligible node alive and detach-cluster,
when there are no master-eligible nodes left.
This commit implements the first part.

Docs for the tool will be added separately as a part of #37812.
2019-01-24 19:25:55 +01:00
Tal Levy 289106a578
Refactor GeoHashGrid to be abstract and re-usable (#37742)
This change split out all the specific GeoHash
classes for the geohash_grid aggregation into
abstract GeoGrid classes that can be re-used for
specific hashing types, like `geohash`
2019-01-24 10:12:14 -08:00
Lisa Cawley 33cac52b2c
[DOCS] Remove beta tag from metricbeat monitoring (#37791) 2019-01-24 09:14:08 -08:00
Nhat Nguyen 76fb573569
Do not allow put mapping on follower (#37675)
Today, the mapping on the follower is managed and replicated from its
leader index by the ShardFollowTask. Thus, we should prevent users
from modifying the mapping on the follower indices.

Relates #30086
2019-01-24 12:13:00 -05:00
David Turner 187b233571 Read m_m_n from cluster states from 6.7
This completes the BWC serialisation changes required for a 6.7 master to
inform other nodes of the node-level value of the `minimum_master_nodes`
setting.

Relates #37701, #37811
2019-01-24 17:05:49 +00:00
David Roberts 0e36adc35f Mute SimpleClusterStateIT testMetadataVersion
Due to https://github.com/elastic/elasticsearch/issues/37820
2019-01-24 16:50:55 +00:00
Marios Trivyzas 74b6f308e9
SQL: Fix issue with complex expression as args of PERCENTILE/_RANK (#37102)
When the arguements of PERCENTILE and PERCENTILE_RANK can be folded,
the `ConstantFolding` rule kicks in and calls the `replaceChildren()`
method on `InnerAggregate` which is created from the aggregation rules
of the `Optimizerz. `InnerAggregate` in turn, cannot implement the method
as the logic of creating a new `InnerAggregate` instance from a list of
`Expression`s resides in the Optimizer. So, instead, `ConstantFolding`
should be applied before any of the aggregations related rules.

Fixes: #37099
2019-01-24 18:40:20 +02:00
Alpar Torok f6b6f927ec
Fail randomized testing tasks if no tests ran (#37764)
Reverts #36259 in part to make randomized test fail if no tests are ran.
This is useful when filtering tests as it's easy to make a typo and
think the test ran trough successfully.
2019-01-24 17:33:08 +02:00
Alpar Torok 37768b7eac
Testing conventions now checks for tests in main (#37321)
* Testing conventions now checks for tests in main

This is the last outstanding feature of the old NamingConventionsTask,
so time to remove it.

* PR review
2019-01-24 17:30:50 +02:00
Alpar Torok e7aa7e909a
Switch testclusters to use nio (#37365) 2019-01-24 17:30:24 +02:00
niloct ddee1926f9 Update update-by-query.asciidoc (#37555)
Similar fix as #37370.
2019-01-24 16:27:42 +01:00
Alpar Torok 4e08cca6bc
Ground work to start up the docker image in the build (#37754)
This change adds a docker compose configuration that's used with
the `elasticsearch.test.fixtures` plugin to start up the image
and check that the TCP ports are up.

We can build on this to add other checks for culster health,
run REST tests, etc.

We can add multiple containers and configurations to the compose
file (e.x. test different env vars) and form clusters.
2019-01-24 17:26:42 +02:00
David Roberts bd02ca4b7b Mute NoMasterNodeIT testNoMasterActionsWriteMasterBlock
Due to https://github.com/elastic/elasticsearch/issues/37823
2019-01-24 15:17:13 +00:00
Ioannis Kakavas 265710e658
Better msg on unmapped principal attribute (#37805)
When we can't map the principal attribute from the configured SAML
attribute in the realm settings, we can't complete the
authentication. We return an error to the user indicating this and
we present them with a list of attributes we did get from the SAML
response to point out that the expected one was not part of that
list. This list will never contain the NameIDs though as they are
not part of the SAMLAttribute list. So we might have a NameID but
just with a different format.
2019-01-24 17:05:01 +02:00
Nhat Nguyen a6abb28abf
Fix InternalEngineTests#assertOpsOnPrimary (#37746)
The assertion `assertOpsOnPrimary` does not store seq_no and primary
term of successful deletes to the `lastOpSeqNo` and `lastOpTerm`. This
leads to failures of the subsequence CAS deletes or indexes with seq_no
and term. Moreover, this assertion trips a translog assertion because it
bumps the primary term of some operations but not the primary term of
the engine.

Relates #36467
Closes #37684
2019-01-24 10:02:48 -05:00
David Roberts a81931bb2a Mute DynamicMappingIT testMappingsPropagatedToMasterNodeImmediately
Due to https://github.com/elastic/elasticsearch/issues/37816
2019-01-24 14:32:44 +00:00
Andrei Stefan 163a27b93c
SQL: Fix BasicFormatter NPE (#37804) 2019-01-24 15:40:51 +02:00
Jason Tedor 7517e3a7bd
Optimize warning header de-duplication (#37725)
Now that warning headers no longer contain a timestamp of when the
warning was generated, we no longer need to extract the warning value
from the warning to determine whether or not the warning value is
duplicated. Instead, we can compare strings directly.

Further, when de-duplicating warning headers, are constantly rebuilding
sets. Instead of doing that, we can carry about the set with us and
rebuild it if we find a new warning value.

This commit applies both of these optimizations.
2019-01-24 08:39:24 -05:00
Yannick Welsch feab59df03
Bubble exceptions up in ClusterApplierService (#37729)
Exceptions thrown by the cluster applier service's settings and cluster appliers are bubbled up, and
block the state from being applied instead of silently being ignored. In combination with the cluster
state publishing lag detector, this will throw a node out of the cluster that can't properly apply
cluster state updates.
2019-01-24 14:09:03 +01:00
Marios Trivyzas 9357929309
SQL: Improve handling of invalid args for PERCENTILE/PERCENTILE_RANK (#37803)
Improve the Exception and the error message returned when 2nd argument
of PERCENTILE and PERCENTILE_RANK is not a constant.
2019-01-24 15:03:49 +02:00
Simon Willnauer c7b16162ae
Remove unused ThreadBarrier class (#37666)
This class is pretty complex and only used in a test where we can simply
fail the test with an assertion error.
2019-01-24 13:52:22 +01:00
Yulong 20533c5990
Add built-in user and role for code plugin (#37030)
* Add built-in roles for code plugin

* Fix rest-client get-roles test count

* Fix broken test
2019-01-24 20:12:32 +08:00
Alpar Torok bc20142791
Consolidate testclusters tests into a single project (#37362) 2019-01-24 13:47:21 +02:00
Yannick Welsch 2bf269e628 Fix docs for MappingUpdatedAction
Follow-up to #31140
2019-01-24 12:44:36 +01:00
Marios Trivyzas f707fa9e0a
SQL: Introduce SQL DATE data type (#37693)
* SQL: Introduce SQL DATE data type

Support ANSI SQL's DATE type by introducing a runtime-only
ES SQL date type.

Closes: #37340
2019-01-24 13:41:58 +02:00