FetchSubPhase#getProcessor currently takes a SearchLookup parameter. This
however is only needed by a couple of subphases, and will almost certainly change in
future as we want to simplify how fetch phases retrieve values for individual hits.
To future-proof against further signature changes, this commit moves the SearchLookup
reference into FetchContext instead.
Currently we throw an error if stored fields are disabled, but hit version metadata is
requested on a search. This doesn't make much sense, as the version information
is stored in docvalues and so has no connection with stored fields.
This commit removes the link between the two, allowing version metadata to be loaded
even when stored fields are disabled in a request.
Fixes#62456
The data frame structure in c++ has a limit on 2^32 documents. This commit
adds a check that the number of documents involved in the analysis are
less than that and fails to start otherwise. That saves the cost of
reindexing when it is unnecessary.
Backport of #62547
In #57666 we changed when null_value was parsed for ip and date fields. Previously,
the null value was stored as a string, and parsed into a date or InetAddress whenever
a document containing a null value was encountered. Now, the values are parsed when
the mappings are built, which means that bad values are detected up front; if you try and
add a mapping with a badly-parsed ip or date for a null_value, the mapping will be
rejected.
This causes problems for upgrades in the case when you have a badly-formed null_value
in a pre-7.9 cluster. This commit fixes the upgrade case by changing the logic to only
logging a warning on the badly formed value, replicating the earlier behaviour.
Fixes#62363
The AssertingInputStream in S3BlobContainerRetriesTests verifies
that InputStream are either fully consumed or aborted, but the
eof flag is only set when the underlying stream returns it.
When buffered read are executed and when the exact number
of remaining bytes are read, the eof flag is not set to true. Instead
the test should rely on the total number of bytes read to know if
the stream has been fully consumed.
Close#62390
This adds ILM support for automatically migrating the managed
indices between data tiers.
This proposal makes use of a MigrateAction that is injected
(similar to how the Unfollow action is injected) in phases that
don't define index allocation rules using the AllocateAction or
don't explicitly define the MigrateAction itself (regardless if it's
enabled or disabled).
(cherry picked from commit c1746afffd61048d0c12d3a77e6d8191a804ed49)
Signed-off-by: Andrei Dan <andrei.dan@elastic.co>
ReplaceDataStreamBackingIndexStep#performAction seems to perform an equality
check on an original Index and the write indexes names, but because this
compares an Index instance to a String, the condition can never be met. This PR
changes this comparison.
ShardClusterSnapshotRestoreIT is confusing as we already have a
very complete SharedClusterSnapshotRestoreIT test suite. This
commit removes ShardClusterSnapshotRestoreIT and folds its
unique test in DataStreamsSnapshotsIT.
Since #61857 we test using BCJSSE (Bouncy Castle SSL) when running on
Zulu8 because Azul have backported SSL changes from Java11 into their
Java8 JRE which prevents us from using Sun JSSE in FIPS mode.
BCJSSE uses different exception messages than Sun JSSE, so we needed
to update
RestrictedTrustManagerTests.testThatDelegateTrustManagerIsRespected
to reflect the fact that sometimes we might be receive BCJSSE error
messages on a Java8 JVM
Resolves: #62281
The usage of single quotes to wrap a string literal is forbidden
and an error encouraging the user to user double quotes is returned.
Tests are properly adjusted.
Relates to #61659
(cherry picked from commit 8be400b77370bf4cf68c89f492c2d235f3cce43c)
Backport of #62484 to 7.x branch.
It is possible in mixed version clusters (nodes prior to 7.10)
that a 404 is returned when wiping all data streams.
This is because there are no data streams and
the coordinator node is on a version that doesn't
mark the delete request for wildcard usage.
We currently pass a SearchContext around to share configuration among
FetchSubPhases. With the introduction of runtime fields, it would be useful
to start storing some state on this context to be shared between different
subphases (for example, stored fields or search lookups can be loaded lazily
but referred to by many different subphases). However, SearchContext is a
very large and unwieldy class, and adding more methods or state here feels
like a bridge too far.
This commit introduces a new FetchContext class that exposes only those
methods on SearchContext that are required for fetch phases. This reduces
the API surface area for fetch phases considerably, and should give us some
leeway to add further state.
Current implementations of the indexer are using aggregations.
Thus each search step executes a search action. However,
we can generalize that to allow for any action that returns a `SearchResponse`.
This commit abstracts the search phase from the search action.
Backport of #61739
The CodecReader wrapper we use to remove the `_recovery_source` field
doesn't override `StoredFieldsreader#getMergeInstance`, which has the
undesired side-effect of preventing the wrapped stored fields reader
from optimizing merging.
`VersionConflictEngineException` is thrown on the hot path for updates,
but stack traces are expensive to compute and transport and rarely
useful for this kind of exception. This commit avoids computing the
stack trace for these exceptions.
This new snapshot contains the following JIRAs that we're interested in:
- [LUCENE-9525](https://issues.apache.org/jira/browse/LUCENE-9525)
Better handling of small documents. This should improve retrieval times
when documents are less than ~1kB.
- [LUCENE-9510](https://issues.apache.org/jira/browse/LUCENE-9510)
Faster flushes when index sorting is enabled by not compressing the
temporary files that store stored fields and term vectors.
Today we only emit `DEBUG` logs if the source disconnects from the
target during a recovery. This deserves to be noisier by default since
it should be rare and may help users identify other problems with their
network or with their shard movements.
This commit promotes this message to `INFO`. There's no need for `WARN`
since these days we will normally resume the recovery where it left off.
With this commit we rename all of the fielddata, doc_values and mapped field type classes for runtime fields to not start with the Script prefix but rather their runtime type (e.g. Boolean) and only then Script
To preserve the PIT semantics, the retrieval of results has moved from
using multi-get to using an idsQuery.
(cherry picked from commit 1c2362fcf2be62ce568b3772924abce7331ef23c)
This commit makes the LocalNodeMasterListener interface extend the
ClusterStateListener interface and use a default implementation for
detecting whether the local node master status changed.
Backport of #62422
This backport incorporates all the changes to improve compiler extensibility. The reason for this
backport is the changes are now required to support runtime fields.
With this commit we rename the script classes used for each mapped field type used for runtime fields. The new naming is a shorter version of the previous one: from e.g. BooleanScriptFieldScrip to BooleanScript . We also move such classes to the existing mapper package.
Constructing the timout checker FIRST and THEN registering the watcher allows the test to have a race condition.
The timeout value could be reached BEFORE the matcher is added. To prevent the matcher never being interrupted, a new timedOut value is added to the watcher thread entry. Then when a new matcher is registered, if the thread was previously timedout, we interrupt the matcher immediately.
closes#48861
Alan Woodward
Adam Locke
Dimitris Athanasiou
Mark Vieira
Nik Everett
Tanguy Leroux
Ignacio Vera
Lee Hinman
Andrei Dan
Luca Cavanna
James Rodewig
David Kyle
Fernando Briano
Christoph Büscher
Tim Vernum
Marios Trivyzas
Martijn van Groningen
Adrien Grand
David Turner
Armin Braun
Julie Tibshirani
Costin Leau
Jay Modi
Jack Conradson
Benjamin Trent