The maximum number of fields per index is limited to 1000 by default by the
`index.mapping.total_fields.limit` setting to prevent accidental mapping
explosions due to too many fields. Currently all metadata fields also count
towards this limit, which can lead to some confusion when using lower limits.
It is not obvious for users that they cannot actually add as many fields as
are specified by the limit in this case.
This change takes the number of metadata fields out of the field count that we
check against the field limit. It also adds tests that check that we can add
fields up to the specified limit, but throw an exception for any additional field added.
Closes#24096
Deprecating a some setting specializations (e.g., list settings) does
not cause deprecation warning headers and deprecation log messages to
appear. This is due to a missed check for deprecation. This commit fixes
this for all setting specializations, and ensures that this can not be
missed again.
This allows tokenfilters to be applied selectively, depending on the status of the current token in the tokenstream. The filter takes a scripted predicate, and only applies its subfilter when the predicate returns true.
Solves all of the xpack line length suppressions and then merges the
remainder of the xpack checkstyle_suppressions.xml file into the core
checkstyle_suppressions.xml file. At this point that just means the
antlr generated files for sql.
It also adds an exclusion to the line length tests for javadocs that
are just a URL. We have one such javadoc and breaking up the line would
make the link difficult to use.
The log structure endpoint will return these in addition to
pure structure information so that it can be used to drive
pre-import data visualizer functionality.
The statistics for every field are count, cardinality
(distinct count) and top hits (most common values). Extra
statistics are calculated if the field is numeric: min, max,
mean and median.
Gradle triggers the build of artifacts even if assemble is disabled.
Most users will not need bwc distributions after running `./gradlew
assemble` so instead of forcing them to add `-x buildBwcVersion`, we
detect this and skip the configuration of the artifacts.
Adds a place for users to store cluster-wide data they wish to associate
with the cluster via the Cluster Settings API. This is strictly for
user-defined data, Elasticsearch makes no other other use of these
settings.
With the introduction of the default distribution, it means that by
default the query cache is wrapped in the security implementation of the
query cache. This cache does not allow caching if the request does not
carry indices permissions. Yet, this will not happen if authorization is
not allowed, which it is not by default. This means that with the
introduction of the default distribution, query caching was disabled by
default! This commit addresses this by checking if authorization is
allowed and if not, delegating to the default indices query
cache. Otherwise, we proceed as before with security. Additionally, we
clear the cache on license state changes.
Now that types are unique per mapping we can retrieve the document mapper
without referencing the type. This fixes an NPE when stored fields are disabled.
For 6x we'll need a different fix since mappings can still have multiple types.
Relates #32941
Extend SHOW TABLES, DESCRIBE and SHOW COLUMNS to support table
identifiers not just SQL LIKE pattern.
This allows both Elasticsearch-style multi-index patterns and SQL LIKE.
To disambiguate between the two (as the " vs ' can be easy to miss),
the grammar now requires LIKE keyword as a prefix for all LIKE-like
patterns.
Also added some docs comparing the two types of patterns.
Fix#33294
This is not changing the behaviour as when the sort field was set
to `influencer_score` the secondary sort would be used and that
was using the `record_score` at the highest priority.
1. The TOMCAT_DATESTAMP format needs to be checked before
TIMESTAMP_ISO8601, otherwise TIMESTAMP_ISO8601 will
match the start of the Tomcat datestamp.
2. Exclude more characters before and after numbers. For
example, in 1.2.3 we don't want to match 1.2 as a float.
This commit introduces the formal notion of a private setting. This
enables us to register some settings that we had previously not
registered as fully-fledged settings to avoid them being exposed via
APIs such as the create index API. For example, we had hacks in the
codebase to allow index.version.created to be passed around inside of
settings objects, but was not registered as a setting so that if a user
tried to use the setting on any API then they would get an
exception. This prevented users from setting index.version.created on
index creation, or updating it via the index settings API. By
introducing private settings, we can continue to reject these attempts,
yet now we can represent these settings as actual settings. In this
change, we register index.version.created as an actual setting. We do
not cutover all settings that we had been treating as private in this
pull request, it is already quite large due to moving some tests around
to account for the fact that some tests need to be able to set the
index.version.created. This can be done in a follow-up change.
Global search timeouts and timeouts specified in the search request body use the
same internal mechanism as search cancellation. Therefore the same caveats
apply, mostly around the responsiveness of the timeout which gets only checked
by a running search on segment boundaries by default.
Closes#31263
* TESTS: Fix Race Condition in Temp Path Creation
* Calling `createTempDir` concurrently here in
the `Follower`s causes collisions at times
which lead to `createEngine` throwing because
of unexpected files in the newly created temp
dir
* Fixed by creating all temp dirs in the main test thread
* closes#33344
We can have multiple documents in Lucene with the same seq_no for
parent-child documents (or without rollback). In this case, the usage
"lastSeenSeqNo + 1" is an off-by-one error as it may miss some
documents. This error merely affects the `skippedOperations` contract.
See: https://github.com/elastic/elasticsearch/pull/33222#discussion_r213842257Closes#33318
The existing implemention could not deal with negative numbers as well
as +- 999 milliseconds around the epoch.
This commit uses Instant.ofEpochMilli() and parses the input to
a number instead of using a date formatter.
This change merges two sections in the "Tune for search speed" documentation
that recommend mapping numeric identifiers as keywords. Both sections contain
mostly the same advice, so they can be merged.
Closes#32733
The comparator used TimeValue parsing, which meant it couldn't handle
calendar time. This fixes the comparator to handle either (and potentially
mixed). The mixing shouldn't be an issue since the validation code
upstream will prevent it, but was simplest to allow the comparator
to handle both.
In Lucene 8 the statistics for a field (doc_count, sum_doc_count, ...) are
checked and invalid values (v < 0) are rejected. Though for the _field_names
field we hide the statistics of the field if security is enabled since
some terms (field names) may be filtered. However this statistics are never
used, this field is not used for ranking and cannot be used to generate
term vectors. For these reasons this commit restores the original statistics
for the field in order to be compliant with Lucene 8.
This commit adds the support to early terminate the collection of a leaf
in the aggregation framework. This change introduces a MultiBucketCollector which
handles CollectionTerminatedException exactly like the Lucene MultiCollector.
Any aggregator can now throw a CollectionTerminatedException without stopping
the collection of a sibling aggregator. This is useful for aggregators that
can infer their result without visiting all documents (e.g.: a min/max aggregation on a match_all query).
Adrien Grand
Christoph Büscher
Jason Tedor
Michael Basnight
Dimitris Athanasiou
Alan Woodward
Nik Everett
Armin Braun
David Roberts
Paul Sanwald
Costin Leau
Alpar Torok
Gordon Brown
Lisa Cawley
Costin Leau
Jim Ferenczi
Nhat Nguyen
Sohaib Iftikhar
Nikolay Vasiliev
Julie Tibshirani
Benjamin Trent
tony-dillon
Colin Goodheart-Smithe
David Kyle
Alexander Reelsen
Zachary Tong
Jim Ferenczi