This change adds a Fixture class for use by gradle. A Fixture is an
external process that integration tests will use. It can be added as a
dependsOn for integTest, and will automatically be shutdown upon success
or failure, as well as relevant information dumped on failure. There is
also an example fixture in this change.
This new task allows setting code, similar to a doLast or doFirst,
except it is specifically geared at running ant (and thus called doAnt).
It adjusts the ant logging while running the ant so that the log
level/behavior can be tweaked, and automatically buffers based on gradle
logging level, and dumps the ant output upon failure.
This fixes the `lenient` parameter to be `missingClasses`. I will remove this boolean and we can handle them via the normal whitelist.
It also adds a check for sheisty classes (jar hell with the jdk).
This is inspired by the lucene "sheisty" classes check, but it has false positives. This check is more evil, it validates every class file against the extension classloader as a resource, to see if it exists there. If so: jar hell.
This jar hell is a problem for several reasons:
1. causes insanely-hard-to-debug problems (like bugs in forbidden-apis)
2. hides problems (like internal api access)
3. the code you think is executing, is not really executing
4. security permissions are not what you think they are
5. brings in unnecessary dependencies
6. its jar hell
The more difficult problems are stuff like jython, where these classes are simply 'uberjared' directly in, so you cant just fix them by removing a bogus dependency. And there is a legit reason for them to do that, they want to support java 1.4.
This change removes hardcoded ports from cluster formation. It passes
port 0 for http and transport, and then uses a special property to have
the node log the ports used for http and transport (just for tests).
This does not yet work for multi node tests. This brings us one step
closer to working with --parallel.
This commit removes and now forbids all uses of
Collections#shuffle(List) and Random#<init>() across the codebase. The
rationale for removing and forbidding these methods is to increase test
reproducibility. As these methods use non-reproducible seeds, production
code and tests that rely on these methods contribute to
non-reproducbility of tests.
Instead of Collections#shuffle(List) the method
Collections#shuffle(List, Random) can be used. All that is required then
is a reproducible source of randomness. Consequently, the utility class
Randomness has been added to assist in creating reproducible sources of
randomness.
Instead of Random#<init>(), Random#<init>(long) with a reproducible seed
or the aforementioned Randomess class can be used.
Closes#15287
Wildcard imports are terrible, they cause ambiguity in the code,
make it not compile with the future versions of java in many cases.
We should simply fail the build on this, it is messiness, caused by
messy Intellij configuration
We have some tests which have crazy dependencies, like on other plugins.
This change adds a "messy-test" gradle plugin which can be used for qa
projects that these types of tests can run in. What this adds over
regular standalone tests is the plugin properties and metadata on the
classpath, so that the plugins are properly initialized.
This is a relic from shading where it was trickier to implement.
Third party signatures are already in e.g. the test list, there
is no reason to separate them out.
Instead, we could have a third party signatures that does
something different... like keep tabs on third party libraries.
This commit removes and now forbids all uses of the type-unsafe empty
Collections fields Collections#EMPTY_LIST, Collections#EMPTY_MAP, and
Collections#EMPTY_SET. The type-safe methods Collections#emptyList,
Collections#emptyMap, and Collections#emptySet should be used instead.
Typical failure:
```
:test-framework:dependencyLicenses (Thread[main,5,main]) started.
:test-framework:dependencyLicenses
Executing task ':test-framework:dependencyLicenses' (up-to-date check took 0.0 secs) due to:
Task has not declared any outputs.
:test-framework:dependencyLicenses FAILED
:test-framework:dependencyLicenses (Thread[main,5,main]) completed. Took 0.023 secs.
FAILURE: Build failed with an exception.
* What went wrong:
Execution failed for task ':test-framework:dependencyLicenses'.
> Licences dir /mnt/jenkins/workspace/es_core_master_strong/test-framework/licenses does not exist, but there are dependencies
```
Related to #15168
This change attempts to simplify the gradle tasks for precommit. One
major part of that is using a "less groovy style", as well as being more
consistent about how tasks are created and where they are configured. It
also allows the things creating the tasks to set up inter task
dependencies, instead of assuming them (ie decoupling from tasks
eleswhere in the build).
We had increased this in maven, but it was lost in the transition to
gradle. This change adds it as a configurable setting the the logger for
randomized testing and bumps it to 25.
The current delay waits until later after normal configuration, but
still just after resolution (ie when paths would be known for
dependencies but not actual execution). This delays the checks further
to be done right before we actually execute the copy task.
closes#15068
Currently if running with --info, the command line for ES, along with
env vars, are logged before they may be ammended to add debug options.
This moves the adding JAVA_OPTS to before we print the command.
This adds the standalone tests so they will compile (and thus can be
modified with import completion) within IntelliJ. It also explicitly
sets up buildSrc as a module.
Note that this does *not* mean eg evil-tests can be run from intellij.
These are special tests that require special settings (eg disabling
security manager). They need to be run from the command line.
closes#15075
The Exec task outputs stdout/stderr to the standard streams by default.
However, to keep output short, we currently capture this, and only
output if the task failed. This change makes a small wrapper around Exec
to facilitate this behavior anywhere we use Exec.
This change delays the lookup for whatever is passed to extra config as
the source file to happen at execution time. This allows using eg a task
which generates a file, but maintains the checks that the file is not a
dir and that it exists at runtime.
This change allows copy extra files into the integ test cluster before
it runs. However, it explicitly forbids overwriting elasticsearch.yml,
since that is generated.
The current mechanism for adding plugins to the integTest cluster is to
have a FileCollection. This works well for the integTests for a single
plugin, which automatically adds itself to be installed. However, for qa
tests where many plugins may be installed, and from other projects, it
is cumbersome to add configurations, dependencies and dependsOn
statements over and over. This simplifies installing a plugin from
another project by moving this common setup into the cluster
configuration code.
The current wait condition for an integ test cluster being up is a
simple http get on the root path for elasticsearch. However, it is
useful to allow having arbitrary wait conditions. This change reworks
the wait task to first check that each node process started successfully
and has a socket up, followed by an arbitrary wait condition which
defaults to the current http get.
Also, cluster settings are allowed to be added, and overriden. Finally,
custom setup commands are made relative to the elasticsearch home dir
for each node.
This change adds back the last of the missing test options to the juni4
wrapper. leaveTemporary is important in that setting it to true (which
is how we had it set in maven) removes the warnings we currently get
about a leftover file that cannot be deleted (from jna).
This change adds back the multi node smoke test, as well as making the
cluster formation for any test allow multiple nodes. The main changes in
cluster formation are abstracting out the node specific configuration to
a helper struct, as well as making a single wait task that waits for all
nodes after their start tasks have run. The output on failure was also
improved to log which node's info is being printed.
* Forbid System.setProperties & co in forbidden APIs.
* Ban property write access at runtime with security manager.
Plugins that need to modify system properties will need to request permission in their plugin-security.policy
As part of the refactoring to allow --debug-jvm with gradle run, the way
java options are passed for integ tests was changed. However, we need to
make sure the jvm argline passed goes to ES_GC_OPTS because this
allows overriding things like which garbage collector we run, which we
do for testing from jenkins. This change adds back ES_GC_OPTS.
Because jar hell checks run during static initialization of tests, a
failure will result in all tests failing. However, only the first test
within each jvm shows the jarhell failure, and later tests get a class
not found exception for the class that failed to load when static init
failed.
This change adds a task to run as part of precommit, which checks the
test runtime classpath for jarhell.
closes#14721
If there is a failure in the elasticsearch start script, we currently
completely lose the failure. This is due to how spawning works with ant.
This change avoids the issue by introducing an intermediate script,
built dynamically before running ant exec, which runs elasticsearch and
redirects the output to a log file. This essentially causes us to run
elasticsearch in the foreground and capture the output, but at the same
time keep a running script which ant can pump streams from (which will
always be empty).
There were a number of subtle issues with the existing logging that
wraps events from Junit4 and ant. This change:
* Tweaks at what level certain events are logged
* Fixes -Dtests.output=always to force everything to be logged
* Makes -Dtests.class imply -Dtests.output=always
* Captures ant logging from junit4, so that direct jvm output will be
logged on failure when not using gradle info logging
When the build tries to start an elasticsearch instance but the start fails
if it fails to find the log file then we log a line about how we can't find
the file.
Sometimes when running elasticsearch, it is useful to attach a remote
debugger. This change adds a --debug-jvm option (the same name gradle
uses for its tests debug option), which adds java agent config for a
remote debugger. The configuration is set to hava java suspend until the
remove debugger is attached.
closes#14772
If you build elasticsearch without a git repository it was creating a null
shortHash which was causing Elasticsearch not to be able to form transport
connections.
Closes#14748
This makes the rest tests **tons** more responsive.
Also stop test progress output from jumping by using formating. The progess
now looks like:
Suites [004/549], Tests [0019|0|0], in 1.58s J2 completed UpdateNumberOfReplicasTests
The changes included are:
1. The suites, total tests, and JVM id are now padded based on their maximum
size. The maximum number of tests is just a guess because that data isn't
easily available when the suite starts. JVM id rarely matters because only
the most crazy individuals use more than 10 JVMs.
2. The suite information is reordered. Now its runtime, jvm id, suite name,
and, optionally, method name. This reordering is useful because the thing
that varies in length, the suite and method name, are on the right hand
side. This means that nothing jumps around during the test run.
We recently got a run command with gradle, but it is sometimes useful to
run ES with a specific plugin. This is a start, by making each esplugin
have a run command which installs the plugin and runs elasticsearch in
the foreground.
This makes forbidden patterns a little smarter, so it does not need to
run on every build. It works because the marker file timestamp will be
compared against the source files (which are the inputs to forbidden
patterns).
closes#14788
We currently enforce JAVA_HOME is set, but use gradle's java version to
compile, and JAVA_HOME version to test. This change makes the compile
and test both use JAVA_HOME, and clarifies which java version is used by
gradle and that used for the compile/test.
With gradle, deploying to maven means first generating poms. These are
filled in based on dependencies of the project. Recently, we started
disallowing transitive dependencies. However, this configuration does
not translate to maven poms because maven has no concept of excluding
all transitive dependencies.
This change adds exclusions for each of the transitive deps of each
dependency being added to the maven pom. It does so by creating dummy
configurations for each direct dependency (which does not have
transitive deps excluded), so that we can iterate the transitive deps
when building the pom.
Note, this should be simpler (just modifying maven's pom model), but
gradle tries to hide that from their api, causing us to need to
manipulate the xml directly.
https://discuss.gradle.org/t/modifying-maven-pom-generation-to-add-excludes/12744
Currently elasticsearch in integ tests is started using an ant task on
windows, or gradle exec on everything else. However, gradle exec has
some flaws, one being Ctrl-C does not run finalizedBy tasks, which means
interrupting integ tests will leak a jvm. This change makes all systems
use ant exec. One caveat is, if there is any output by the jvm, we lose
it in ant bit heaven. But this is no different than what we had with
gradle. In the future, we should look at using a separate thread to
pump streams from the elasticsearch process.
closes#14701
closes#14726
Squashed commit of the following:
commit 5b591e98570e3fa481b2816a44063b98bff36ddf
Author: Robert Muir <rmuir@apache.org>
Date: Fri Nov 13 00:54:08 2015 -0500
add assumption for self-signing in PluginManagerTests
commit ed11e5371b6f71591dc41c6f60d033502cfcf029
Author: Robert Muir <rmuir@apache.org>
Date: Fri Nov 13 00:20:59 2015 -0500
show error output from integ test startup
commit d8b187a10e95d89a0e775333dcbe1aaa903fb376
Author: Robert Muir <rmuir@apache.org>
Date: Thu Nov 12 22:14:11 2015 -0500
fix gradle check under jigsaw
If we use JAVA_HOME consistently for tests, we can run tests with a
different version of java than gradle runs with. For example, this
enables running tests with jigsaw, but building with java 8. The only
caveat is intellij does not set JAVA_HOME. This change enforces
JAVA_HOME is set, but ignores for intellij.
This moves the min java version used by elasticsearch to one place, a
constant in BuildPlugin. For me on java 9, this fixed my jar to have the
correct target/source versions.
closes#14702
Transitive dependencies can be confusing and hard to deal with when
conflicts arise between them. This change removes transitive
dependencies from elasticsearch, and forces any dependency conflicts to
be resolved manually, instead of automatically by gradle.
closes#14627
In gradle 2.7 (or groovy 2.3.10, not sure which), there appears to be a
bug on linux where using a fully qualified class (without an import
statement) does not work. This change forces gradle 2.8 or above. It
also moves the logic around a little for the version check so the build
info is printed before checks against that info.
Some dependencies must be specified in a couple places in the build.
e.g. randomized runner is specified both in buildSrc (for the gradle
wrapper plugin), as well as in the test-framework.
This change creates buildSrc/versions.properties which acts similar to
the set of shared version properties we used to have in the maven parent
pom.
The esplugin gradle plugin automatically adds the pluging being built to
the integTest cluster. However, there were two issues with this. First
was a bug in the name, which should have been the configured
esplugin.name instead of the project name. Second, the files
configuration was overcomplicated (trying to use the groovy spreader
operator after delaying calls to singleFile). Instead, we can just pass
the file collections (which will just be a single file at execution
time).
This commit addresses an issue in getting a path to the jps bin. The
solution is to get the path to the JDK relative to the JVM running
Gradle.
Closes#14614
Latest version of lucene deprecated Query#setBoost and Query#getBoost which made queries effectively immutable. Those methods need to be replaced with `BoostQuery` that wraps any query that needs boosting.
This commit replaces usages of setBoost with BoostQuery and adds it to forbidden-apis for prod code.
Usages of `getBoost` are only partially removed, as some will have to stay for backwards compatibility.
Closes#14264
run.sh and run.bat were calling out to the old maven build system.
This is no longer in place, so we've created new gradle tasks to
start an elasticsearch node from the current codebase.
fixed#14423
The plugin name currently defaults to the gradle project name. But the
gradle project name for standalone repo (like an external plugin would
be) defaults to the directory name of the repo. This is trappy, since it
depends on how the repo was checked out.
This change enforces the plugin name is always set.
closes#14603
This makes it a groovy project that works in eclipse.
You will have to install a plugin for groovy language support
(I used a snapshot build from https://github.com/groovy/groovy-eclipse/wiki)
Many other improvements:
* Use spaces in ES path
* Use space in path for plugin file installation
* Use a different cwd than ES home
* Use jps to ensure process being stopped is actually elasticsearch
* Stop ES if pid file already exists
* Delete pid file when successfully killed
Also, refactored the cluster formation code to be a little more organized.
closes#14464
This gets the tar and tar_plugins tests working in gradle. It does so by
adding a subproject, qa/vagrant, which adds the following tasks:
Verification
------------
checkPackages - Check the packages against a representative sample of the
linux distributions we have in our Vagrantfile
checkPackagesAllDistros - Check the packages against all the linux
distributions we have in our Vagrantfile
Package Verification
--------------------
checkCentos6 - Run packaging tests against centos-6
checkCentos7 - Run packaging tests against centos-7
checkDebian8 - Run packaging tests against debian-8
checkFedora22 - Run packaging tests against fedora-22
checkOel7 - Run packaging tests against oel-7
checkOpensuse13 - Run packaging tests against opensuse-13
checkSles12 - Run packaging tests against sles-12
checkUbuntu1204 - Run packaging tests against ubuntu-1204
checkUbuntu1404 - Run packaging tests against ubuntu-1404
checkUbuntu1504 - Run packaging tests against ubuntu-1504
Vagrant
-------
smokeTestCentos6 - Smoke test the centos-6 VM
smokeTestCentos7 - Smoke test the centos-7 VM
smokeTestDebian8 - Smoke test the debian-8 VM
smokeTestFedora22 - Smoke test the fedora-22 VM
smokeTestOel7 - Smoke test the oel-7 VM
smokeTestOpensuse13 - Smoke test the opensuse-13 VM
smokeTestSles12 - Smoke test the sles-12 VM
smokeTestUbuntu1204 - Smoke test the ubuntu-1204 VM
smokeTestUbuntu1404 - Smoke test the ubuntu-1404 VM
smokeTestUbuntu1504 - Smoke test the ubuntu-1504 VM
vagrantHaltCentos6 - Shutdown the vagrant VM running centos-6
vagrantHaltCentos7 - Shutdown the vagrant VM running centos-7
vagrantHaltDebian8 - Shutdown the vagrant VM running debian-8
vagrantHaltFedora22 - Shutdown the vagrant VM running fedora-22
vagrantHaltOel7 - Shutdown the vagrant VM running oel-7
vagrantHaltOpensuse13 - Shutdown the vagrant VM running opensuse-13
vagrantHaltSles12 - Shutdown the vagrant VM running sles-12
vagrantHaltUbuntu1204 - Shutdown the vagrant VM running ubuntu-1204
vagrantHaltUbuntu1404 - Shutdown the vagrant VM running ubuntu-1404
vagrantHaltUbuntu1504 - Shutdown the vagrant VM running ubuntu-1504
vagrantSmokeTest - Smoke test some representative distros from the Vagrantfile
vagrantSmokeTestAllDistros - Smoke test all distros from the Vagrantfile
vagrantUpCentos6 - Startup a vagrant VM running centos-6
vagrantUpCentos7 - Startup a vagrant VM running centos-7
vagrantUpDebian8 - Startup a vagrant VM running debian-8
vagrantUpFedora22 - Startup a vagrant VM running fedora-22
vagrantUpOel7 - Startup a vagrant VM running oel-7
vagrantUpOpensuse13 - Startup a vagrant VM running opensuse-13
vagrantUpSles12 - Startup a vagrant VM running sles-12
vagrantUpUbuntu1204 - Startup a vagrant VM running ubuntu-1204
vagrantUpUbuntu1404 - Startup a vagrant VM running ubuntu-1404
vagrantUpUbuntu1504 - Startup a vagrant VM running ubuntu-1504
It does not make the "check" task depend on "checkPackages" so running the
vagrant tests is still optional. They are slow and depend on vagrant and
virtualbox.
The Package Verification tasks are useful for testing individual distros.
The Vagrant tasks are listed in `gradle tasks` primarily for discoverability.
Gradle defaults to tgz extension when tar is compressed. This changes
the tar distribution back to tar.gz. Note that this also means the maven
packaging type is now tar.gz.
Ryan Ernst
Robert Muir
Jason Tedor
Robert Muir
David Pilato
Nik Everett
javanna
andrejserafim
Daniel Mitterdorfer