In 7.x we can't just by default generate this setting as it might not be
supported by data nodes that are assigned shards for an older version in mixed version
clusters.
Closes#64152
The new fields option allows to fetch the value of all fields in the mapping.
However, internal fields that are used by some field mappers are also shown when
concrete fields retrieved through a pattern (`*` or `foo*`).
We have a [long term plan](https://github.com/elastic/elasticsearch/issues/63446) to hide these fields in field_caps and from pattern resolution
so this change is just a hot fix to ensure that they don't break the retrieval in the meantime.
The `flattened._keyed field will show up as an empty field when using a pattern that match the
flattened field.
Relates #63446
This commit fixes an issue with the detection on macOS for whether or
not the bundled JDK is being used. The logic between macOS and non-macOS
is different because the JDK has a different directory structure on
macOS versus non-macOS. However, due to notarization issues, we changed
the top-level directory from jdk to jdk.app, yet never updated this
detection logic to account for that.
Ideally, we would have a packaging test that asserts that we have the
behavior here correct, and it maintains over time. Alas, we do not
currently have packaging tests on macOS.
This commit adjusts the defaults for the tiered data roles so that they
are enabled by default, or if the node has the legacy data role. This
ensures that the default experience is that the tiered data roles are
enabled.
To fully specifiy the behavior for the tiered data roles then:
- starting a new node with the defaults: enabled
- starting a new node with node.roles configured: enabled if and only
if the tiered data roles are explicitly configured, independently
of the node having the data role
- starting a new node with node.data enabled: enabled unless the
tiered data roles are explicitly disabled
- starting a new node with node.data disabled: disabled unless the
tiered data roles are explicitly enabled
XPack usage starts out on management threads, but depending on the
implementation of the usage plugin, they could end up running on
transport threads instead. Fixed to always reschedule on a management
thread.
With this change, we will always return the same point in time in a
search response as its input until we implement the retry mechanism
for the point in times.
The formatting of the global bottom value does not take the resolution of the provided
numeric_type into account. This change fixes this bug by providing the resolution
directly in the doc value format if the numeric_type is provided as `date_nanos`.
Closes#63719
We must not remove the snapshot from the initializing set
in the `timeout` getter. This was a plain oversight/mistake
and went unnoticed. It can lead to the removal of a valid
snapshot clone from the cluster state in rare circumstances
(e.g. when a node concurrently joins the cluster or a routing
change happens as it did in the linked test failure).
Closes#64115
The `NodeNotConnectedException` exception can be nested as well in the
fairly unlikley case of the disconnect occuring between the connected check
and actually sending the request in the transport service.
Closes#63233
There is a small chance that the file deletion will run
on the searchable snapshot thread pool and not on the test
thread now that the cache is non-blocking in which case
we fail the assertion unless we wait for that thread.
Adds support for the unsigned_long type to data frame analytics.
This type is handled in the same way as the long type. Values
sent to the ML native processes are converted to floats and
hence will lose accuracy when outside the range where a float
can uniquely represent long values.
Backport of #64066
If we run into a background merge between creating the snapshot and closing the index
then with compound files we could be in a situation where we get zero file reuse
on restore.
Force merging before the snapshot gives us a single segment that won't change down the line
so the restore always sees file reuse from the close index.
Closes#63476
Replacing the mechanism for eviction and listener references via a read-write lock by
a reference counting implementation.
This fixes a bug that caused test failure #63586 in which concurrently trying to acquire or release
an eviction listener while doing a file operation would sometimes lead to throwing an exception
since the `tryLock` call on the read lock would fail in this case.
Also this removes the possibility of blocking cluster state updates as a result of them waiting
on the write-lock which might take a long time if a slow read operation executes concurrently.
Closes#63586
Assuming the clone failed when the request failed is not sufficient.
There are failure modes where the request fails but the clone still works out
because the data node resent the requeest after the first clone had already been
failed and removed from the cluster state when master was restarted.
Closes#63473
The setting name is script.context.$CONTEXT.cache_max_size rather than
script.context.$CONTEXT.context_max_size.
Co-authored-by: Andrew Kroh <andrew.kroh@elastic.co>
This commit updates the list of system index names to be complete and
correct for Kibana and APM. The pattern `.kibana*` is too inclusive for
system indices and actually includes the
`.kibana-event-log-${version}-${int}` pattern for the Kibana event log,
which should only be hidden and not a system index. Additionally, the
`.apm-custom-link` index was not included in the list of system
indices. Finally, the reporting pattern has been updated to match that
of the permissions given to the kibana_system role.
Backport of #63950
* Add APM index to Kibana system indices, making it
accessible through the _kibana endpoint and giving it the
same access privileges as the other Kibana system indices.
* Parameterize kibana system index tests by index name
Backport of #63756
Co-authored-by: William Brafford <williamrandolphbrafford@gmail.com>
The deprecation indexing code was writing to a regular data stream,
and it is not yet possible to hide a data stream or prefix it with
a period. This functionality we be re-added once it is possible to
mark a data stream as hidden, and also to not rely on the standard
logs template since that can be disabled.
We have to wait for no more operations here not for `1`. This mostly worked
because the test thread would add the listener quickly enough so that it sees the
state where either the snapshot or clone but not both have already finished
but randomly the test thread would be slow and time out on a state without snaphots in it.
Armin Braun
jimczi
Jason Tedor
Jake Landis
Costin Leau
Adam Locke
Henning Andersen
James Rodewig
Nhat Nguyen
David Roberts
Andrei Dan
David Kyle
debadair
Marios Trivyzas
Przemyslaw Gomulka
Jay Modi
Lisa Cawley
Rory Hunter
Yang Wang
markharwood