Gradle will no longer be needed in the test VMs as the Gradle wrapper
can be used to run the platform tests. This commit updates the platform
tests to use the Gradle wrapper, and removes installing Gradle in the VM
which will speed up VM provisioning.
Relates #28105
When finding the commit hash for the build to place in the JAR manifest
(which is used to identity the build), the scm-info plugin assumes that
GIT_COMMIT is the commit for this build. That assumption is wrong, this
build could be a sub-build of another build that GIT_COMMIT belongs
to. If GIT_COMMIT is set, we ignore the commit hash calculated by
scm-info and calculate the hash ourselves.
Relates #28082
This commit adds the infrastructure to plugin building and loading to
allow one plugin to extend another. That is, one plugin may extend
another by the "parent" plugin allowing itself to be extended through
java SPI. When all plugins extending a plugin are finished loading, the
"parent" plugin has a callback (through the ExtensiblePlugin interface)
allowing it to reload SPI.
This commit also adds an example plugin which uses as-yet implemented
extensibility (adding to the painless whitelist).
* Adds task dependenciesInfo to BuildPlugin to generate a CSV file with dependencies information (name,version,url,license)
* Adds `ConcatFilesTask.groovy` to concatenates multiple files into one
* Adds task `:distribution:generateDependenciesReport` to concatenate `dependencies.csv` files into a single file (`es-dependencies.csv` by default)
# Examples:
$ gradle dependenciesInfo :distribution:generateDependenciesReport
## Use `csv` system property to customize the output file path
$ gradle dependenciesInfo :distribution:generateDependenciesReport -Dcsv=/tmp/elasticsearch-dependencies.csv
## When branch is not master, use `build.branch` system property to generate correct licenses URLs
$ gradle dependenciesInfo :distribution:generateDependenciesReport -Dbuild.branch=6.x -Dcsv=/tmp/elasticsearch-dependencies.csv
We disabled the Javadoc task on JDK 10 due to an apparent bug in Javadoc
generation on JDK 10. However, the client JAR task sets up its own
Javadoc task for client JARs (because it merely copies the non-client
Javadoc JAR). This commit diables that task too, since the Javadocs for
the non-client JAR will not exist.
Relates #27962
There appears to be a bug in JDK 10 for generating Javadocs with some
nested anonymous classes. This commit disables these on JDK 10 until the
upstream issue is resolved.
Relates #27952
JDK 10 has gone fully-modular. This means that:
- when targeting JDK 8 with a JDK 10 compiler, we have to use the full
profile
- when targeting JDK 10 with a JDK 10 compiler, we have to use
-add-modules java.base
Today we only target JDK 8 (our minimum compatibility version) so we
need to change the compiler flags conditional on using a JDK 10
compiler. This commit does that.
Relates #27884
When running the release tests, we set build.snapshot to false and this
causes all version numbers to not have "-SNAPSHOT". This is true even
for the tips of the branches (e.g., currently 5.6.6 on the 5.6
branch). Yet, if we do not set snapshot to false, then we would still be
trying to find artifacts with "-SNAPSHOT" appended which would not have
been build since build.snapshot is false. To fix this, we have to push
build.snapshot into the version logic.
Relates #27778
We have tests that manually unpackage the RPM and Debian package
distributions and start a cluster manually (not from the service) and
run a basic suite of integration tests against them. This is problematic
because it is not how the packages are intended to be used (instead,
they are intended to be installed using the package installation tools,
and started as services) and so violates assumptions that we make about
directory paths. This commit removes these integration tests, instead
relying on the packaging tests to ensure the packages are not
broken. Additionally, we add a sanity check that the package
distributions can be unpackaged. Finally, with this change we can remove
some leniency from elasticsearch-env about checking for the existence of
the environment file which the leniency was there solely for these
integration tests.
Relates #27725
The problem here is that splitting was using a method that intentionally
trims whitespace (the method is really meant to be used for splitting
parameters where whitespace should be trimmed like list
settings). However, for routing values whitespace should not be trimmed
because we allow routing with leading and trailing spaces. This commit
switches the parsing of these routing values to a method that does not
trim whitespace.
Relates #27712
This new snapshot mostly brings a change to TopFieldCollector which can now
early terminate collection when trackTotalHits is `false`.
As a follow-up, we should replace our usage of
`EarlyTerminatingSortingCollector` with this new option.
Gradle 4.2 introduced a feature for safer handling of stale output files. Unfortunately, due to the way some of our tasks are written, this broke execution of our REST tests tasks. The reason for this is that the extract task (which extracts the ES distribution) would clean its output directory, and thereby also remove the empty cwd subdirectory which was created by the clean task. The reason why Gradle would remove the directory is that the earlier running clean task would programmatically create the empty cwd directory, but not make Gradle aware of this fact, which would result in Gradle believing that it could just safely clean the directory.
This commit explicitly defines a task to create the cwd subdirectory, and marks the directory as output of the task, so that the subsequent extract task won't eagerly clean it. It thereby restores full compatibility of the build with Gradle 4.2 and above.
This commit also removes the @Input annotation on the waitCondition closure of AntFixture, which conflicted with the extended input/output validation of Gradle 4.3.
The main highlight of this new snapshot is that it introduces the opportunity
for queries to opt out of caching. In case a query opts out of caching, not only
will it never be cached, but also no compound query that wraps it will be
cached.
The build currently does not work with Gradle 4.3.1 as the Gradle team stopped publishing the gradle-logging dependency to jcenter, starting with 4.3.1 (not sure why). There are two options:
- Add the repository managed by Gradle team (https://repo.gradle.org/gradle/libs-releases-local) to our build
- Use an older version (4.3) of the dependency when running with Gradle 4.3.1.
Not to depend on another external repo, I've chosen solution 2. Note that this solution could break on future versions, but as this is a compileOnly dependency, and the interface we use has been super stable since forever, I don't envision this to be an issue (and easily detected by a breaking build).
PR #26911 set minimum_master_nodes from number_of_nodes to (number_of_nodes / 2) + 1 in our REST tests. This has led to test failures (see #27233) as the REST tests only configure the first node in its unicast.hosts pinging list (see explanation here: #27233 (comment)). Until we have a proper fix for this, I'm reverting the change in #26911.
Gradle 5.0 will remove support for colons in configuration and task
names. This commit fixes this for our build by removing all current uses
of colons in configuration and task names.
Relates #27305
While it's not possible to upgrade the Jackson dependencies
to their latest versions yet (see #27032 (comment) for more)
it's still possible to upgrade to the latest 2.8.x version.
An upstream Gradle change has broken us starting on version 4.2. This
commit blacklists these versions until we can either find a workaround,
or the upstream issue is addressed.
Relates #27087
Upgrade to Jackson 2.9.2 and also use a boolean `closed` flag to
indicate that a FastStringReader instance is closed, so that length
is still correctly reported after the reader is closed.
The rolling-upgrade test was only writing the "minimum_master_nodes" setting to the configuration file of the old nodes, but not the upgraded ones.
Also changes the value of "minimum_master_nodes" from "number_of_nodes" to "(number_of_nodes / 2) + 1".
Update provides:
* Adds support for Gradle 4.0 (without deprecation warning)
* Full Java 9 support through ASM 6.0
Version 2.4 was buggy (it broke Gradle dependencies on SourceSets), but this version fixes it.
Today we have all non-plugin mappers in core. I'd like to start moving those
that neither map to json datatypes nor are very frequently used like `date` or
`ip` to a module.
This commit creates a new module called `mappers-extra` and moves the
`scaled_float` and `token_count` mappers to it. I'd like to eventually move
`range` fields there but it's more complicated due to their intimate
relationship with range queries.
Relates #10368
This commit adds a dependency to the install module task on the task
that builds the module. This is needed for standalone integration
tests that require other modules to be installed. Without this, we do
not have a guarantee that the module is bundled.
Javadoc linking between projects currently relies on
projectSubstitutions. However, that is an extension variable that is not
part of BuildPlugin. This commit moves the javadoc linking into the root
build.gradle, alongside where projectSubstitutions are defined.
There is a bug in Log4j on JDK 9 for walking the stack to find where a
log line is coming from. This bug is impacting some of our testing, so
this commit marks these tests as skippable only on JDK 9 until the bug
is fixed upstream.
Relates #26467
The current script service has a script compilation limit for a one
minute window. This is set to a small default value of 15. Instead of
increasing that default value, this commit introduces a new setting
that allows to configure a rate per time unit, so that the script service can deal with bursts better.
The new setting is named `script.max_compilations_rate`,
requires a nonnegative number and a positive time value.
The default is `75/5m`, which is equivalent to the existing 15 per minute.
When changing how the config path is configured (from a command-line
flag to an environment variable) we had to add BWC code in the build so
that we could form clusters with 5.x nodes in them. Now that this branch
has moved to 7.x, we no longer need to be BWC with 5.x for starting
nodes. This commit removes this dead BWC code.
Relates #26446
Jason Tedor
Ryan Ernst
Daniel Mitterdorfer
Maxime Gréau
Adrien Grand
Yannick Welsch
David Turner
Martijn van Groningen
Dimitrios Liappis
Tanguy Leroux
David Pilato
Colin Goodheart-Smithe
Uwe Schindler
Kevin Risden
Alexander Reelsen