We have a packaging test that tries to install all plugins, and then
asserts that all expected plugins are installed. The expected plugins
are dervied from the list of plugins in the plugins sub-project. The
plugin transport-nio was recently added here, but explicit commands to
install and remove this plugin were never added. This commit addresses
this.
If a newly created index from a rollover request matches with an index
template whose aliases contains the rollover request alias, the alias
will point to multiple indices. This will cause indexing requests to be
rejected. To avoid such situation, we make sure that there is no
duplicated alias before creating a new index; otherwise abort and report
an error to the caller.
Closes#26976
This commit removes the finalization of a snapshot by the snapshot deletion request. This way, the deletion marks the snapshot as ABORTED in cluster state and waits for the snapshot completion. It is the responsability of the snapshot execution to detect the abortion and terminates itself correctly. This
avoids concurrent snapshot finalizations and also ordinates the operations: the deletion
aborts the snapshot and waits for the snapshot completion, the creation detects the abortion
and stops by itself and finalizes the snapshot, then the deletion resumes and continues
the deletion process.
Several responses include the shards_acknowledged flag (indicating whether the
requisite number of shard copies started before the completion of the operation)
and there are two different getters used : isShardsAcknowledged() and isShardsAcked().
This PR deprecates the isShardsAcked() in favour of isShardsAcknowledged() in
CreateIndexResponse, RolloverResponse and CreateIndexClusterStateUpdateResponse.
Closes#27784
The full-cluster-restart tests are run with two nodes. This can lead to situations where the shrink tests fail because
the replicas are not allocated yet and the shrink operation needs the source shards to be available on the same node.
Previously this would default to the version of the remote Node.
However, if the remote cluster was mixed-version (e.g. it was part way through a
rolling upgrade), then the TransportService may have negotiated a connection
version that is not identical to connected Node's version.
This mismatch would cause the Stream and the (Remote)Connection to report
different version numbers, which could cause data to be sent over the wire
using an incorrect serialization version.
* master:
Remove out-of-date projectile file
Remove Gradle cheatsheet
Fix reproduction info to point to Gradle wrapper
Update platforms tests to use Gradle wrapper
Update testing docs to reflect Gradle wrapper
Painless: Modify Loader to Load Classes Directly from Definition (#28088)
Update contributing docs to use the Gradle wrapper
Create nio-transport plugin for NioTransport (#27949)
test: replaced try-catch statements with expectThrows(...)
Add getWarmer and getTranslog method to NodeIndicesStats (#28092)
fix doc mistake
Added ASN support for Ingest GeoIP plugin.
Fix global aggregation that requires breadth first and scores (#27942)
We are long past the Maven to Gradle transition, and Gradle is
commonplace enough these days that we do not need to devote a top-level
file on a Gradle cheatsheet.
Relates #28113
Gradle will no longer be needed in the test VMs as the Gradle wrapper
can be used to run the platform tests. This commit updates the platform
tests to use the Gradle wrapper, and removes installing Gradle in the VM
which will speed up VM provisioning.
Relates #28105
This is related to #27260. This commit moves the NioTransport from
:test:framework to a new nio-transport plugin. Additionally, supporting
tcp decoding classes are moved to this plugin. Generic byte reading and
writing contexts are moved to the nio library.
Additionally, this commit adds a basic MockNioTransport to
:test:framework that is a TcpTransport implementation for testing that
is driven by nio.
When calling the node stats action via the Java API it was not able to return
information about warmers and translog even though that is available through
the REST API. This change adds methods to make the responses more
consistent.
* Fix global aggregation that requires breadth first and scores
This change fixes the deferring collector when it is executed in a global context
with a sub collector thats requires to access scores (e.g. top_hits aggregation).
The deferring collector replays the best buckets for each document and re-executes the original query
if scores are needed. When executed in a global context, the query to replay is a simple match_all
query and not the original query.
Closes#22321Closes#27928
* es/master:
Introduce Gradle wrapper
Ignore GIT_COMMIT when calculating commit hash
Re-enable bwc tests after #27881 was backported
Set the elasticsearch-nio codebase for tests (#28067)
We have agreed to introduce the Gradle wrapper to simplify workflows for
developers, and managing infrastructure (e.g., CI, release builds, etc.)
as well as consideration for the fact that other projects in our stack
use Gradle and do not necessarily want to be tied to our Gradle version.
Relates #28065
When finding the commit hash for the build to place in the JAR manifest
(which is used to identity the build), the scm-info plugin assumes that
GIT_COMMIT is the commit for this build. That assumption is wrong, this
build could be a sub-build of another build that GIT_COMMIT belongs
to. If GIT_COMMIT is set, we ignore the commit hash calculated by
scm-info and calculate the hash ourselves.
Relates #28082
This commit sets the elasticsearch-nio code base in the
BootstrapForTesting class. This is necessary as that codebase needs
socket permissions. Setting the codebase manually is necessary as
intellij does not package our internal libraries when running tests.
* es/master: (53 commits)
Bump compat version for local depdendent test to 6.2.0
Pass `java.locale.providers=COMPAT` to Java 9 onwards (#28080)
Allow shrinking of indices from a previous major (#28076)
Remove deprecated exceptions (#28059)
Add Writeable.Reader support to TransportResponseHandler (#28010)
Plugins: Add plugin extension capabilities (#27881)
Fix cluster.routing.allocation.enable and cluster.routing.rebalance.enable casing (#28037)
[Test] Fix scores for dcg in RankEvalRequestIT and RankEvalYamlIT
[Docs] Add note on limitation for significant_text with nested objects (#28052)
Enable convert processor to support Long and Double. (#27957)
Enable Wildfly tests on JDK 9 and JDK 10
[Test] Fix allowed delta for calculated scores in DiscountedCumulativeGainTests
[Test] Mute DiscountedCumulativeGainTests on ARM
Only bind loopback addresses when binding to local
Fix assertion in Wildfly build
Fix typo in comment in Wildfly build
Use ephemeral ports in Wildfly tests
Update fuzzy-query.asciidoc (#28032)
Just another elasticsearch library (#27996)
Disable nio test transport (#28028)
...
Java 9 added some enhancements to the internationalization support that
impact our date parsing support. To ensure flawless BWC and consistent
behavior going forward Java 9 runtimes requrie the system property
`java.locale.providers=COMPAT` to be set.
Closes#10984
Lucene does not allow adding Lucene 6 files to a Lucene 7 index. This commit ensures that we carry over the Lucene version to the newly created Lucene index.
Closes#28061
Allows TransportResponse objects not to implement Streamable anymore. As an example, I've adapted the response handler for ShardActiveResponse, allowing the fields in that class to become final.
This commit adds the infrastructure to plugin building and loading to
allow one plugin to extend another. That is, one plugin may extend
another by the "parent" plugin allowing itself to be extended through
java SPI. When all plugins extending a plugin are finished loading, the
"parent" plugin has a callback (through the ExtensiblePlugin interface)
allowing it to reload SPI.
This commit also adds an example plugin which uses as-yet implemented
extensibility (adding to the painless whitelist).
Previously we disabled these tests on JDK 9 and JDK 10 because Wildfly
10 did not support JDK 9 and JDK 10. With the release of Wildfly 11
supporting JDK 9 and JDK 10 and now that we depend on Wildfly 11 in
these tests, we can enable these tests on JDK 9 and JDK 10.
Relates #28042
* Only bind loopback addresses when binding to local
Today when binding to local (the default) we bind to any address that is
a loopback address, or any address on an interface that declares itself
as a loopback interface. Yet, not all addresses on loopback interfaces
are loopback addresses. This arises on macOS where there is a link-local
address assigned to the loopback interface (fe80::1%lo0) and in Docker
services where virtual IPs of the service are assigned to the loopback
interface (docker/libnetwork#1877). These situations cause problems:
- because we do not handle the scope ID of a link-local address, we end
up bound to an address for which publishing of that address does not
allow that address to be reached (since we drop the scope)
- the virtual IPs in the Docker situation are not loopback addresses,
they are not link-local addresses, so we end up bound to interfaces
that cause the bootstrap checks to be enforced even though the
instance is only bound to local
We address this by only binding to actual loopback addresses, and skip
binding to any address on a loopback interface that is not a loopback
address. This lets us simplify some code where in the bootstrap checks
we were skipping link-local addresses, and in writing the ports file
where we had to skip link-local addresses because again the formatting
of them does not allow them to be connected to by another node (to be
clear, they could be connected to via the scope-qualified address, but
that information is not written out).
Relates #28029
This commit fixes a flubbed assertion in the Wildfly build file; the
intention was to assert that we have only set the value of the
management port once except the assertion that the management port was
equal to zero when finding a matching log line was left off.
The Wildfly tests previously needed hardcoded ports which would
sometimes already be in use leading to Wildfly test failures because the
server could not bind during startup. We had to wait until some upstream
changes (wildfly/wildfly#9943) and (wildfly/wildfly-core#2390) were
released before we could use ephemeral ports in these tests. These
changes are released now so this commit changes the Wildfly tests to use
ephemeral ports when starting Wildfly.
Relates #28040
This commit disables the nio transport as an option for the test
transport in integration tests. This is because it does not currently
run properly in intellij due to socket permissions. It should be
reenabled once #27881 is merged (and the proper permissions are added).
Jason Tedor
Andrew Banchich
Nhat Nguyen
Tanguy Leroux
olcbean
Yannick Welsch
Tim Vernum
Martijn van Groningen
Ryan Ernst
Jack Conradson
hyeonyoung na
Tim Brooks
Scott Somerville
Sian Lerk Lau
Jim Ferenczi
Simon Willnauer
David Turner
kel
Christoph Büscher
Gytis Šk
Timoshenko