Commit Graph

135 Commits

Author SHA1 Message Date
Ryan Ernst f5b6b40a92 Fix distros to put modules dir int he right place and qa rest tests to
use the real zip
2015-12-03 23:03:14 -08:00
Ryan Ernst a8e9403204 added gradle checks for modules configuration, and ability to add
modules to integ test cluster
2015-12-03 20:53:06 -08:00
Ryan Ernst f2cebd9b3b Merge branch 'master' into jigsaw 2015-12-03 20:02:44 -08:00
Ryan Ernst 0be141e021 Build: Add .settings for buildSrc on gradle eclipse
We have eclipse settings added to all projects when running gradle
eclipse, but buildSrc is its own special project that is not
encapsulated by allprojects blocks. This adds eclipse settings to
buildSrc.
2015-12-03 19:56:11 -08:00
Ryan Ernst caf77f7eea Add integ-test-zip distribution 2015-12-03 14:52:51 -08:00
Robert Muir 1f626d3458 forbidden third-party-signatures -> core-signatures
This is a relic from shading where it was trickier to implement.
Third party signatures are already in e.g. the test list, there
is no reason to separate them out.

Instead, we could have a third party signatures that does
something different... like keep tabs on third party libraries.
2015-12-03 01:45:06 -05:00
Ryan Ernst f8026ed8fd Build: Fix dependency licenses check to correctly skip projects without dependencies 2015-12-02 11:03:56 -08:00
Jason Tedor 05430a788a Remove and forbid use of the type-unsafe empty Collections fields
This commit removes and now forbids all uses of the type-unsafe empty
Collections fields Collections#EMPTY_LIST, Collections#EMPTY_MAP, and
Collections#EMPTY_SET. The type-safe methods Collections#emptyList,
Collections#emptyMap, and Collections#emptySet should be used instead.
2015-12-02 10:41:59 -05:00
Robert Muir b7f497627f Merge pull request #15172 from rmuir/nuke_unused_deps
remove unused core dependencies
2015-12-02 07:20:11 -05:00
David Pilato 9c77cdc201 Fix build failure caused by Licenses Check in test-framework module
Typical failure:

```
:test-framework:dependencyLicenses (Thread[main,5,main]) started.
:test-framework:dependencyLicenses
Executing task ':test-framework:dependencyLicenses' (up-to-date check took 0.0 secs) due to:
  Task has not declared any outputs.
:test-framework:dependencyLicenses FAILED
:test-framework:dependencyLicenses (Thread[main,5,main]) completed. Took 0.023 secs.

FAILURE: Build failed with an exception.

* What went wrong:
Execution failed for task ':test-framework:dependencyLicenses'.
> Licences dir /mnt/jenkins/workspace/es_core_master_strong/test-framework/licenses does not exist, but there are dependencies
```

Related to #15168
2015-12-02 10:08:35 +01:00
Ryan Ernst d68c6673a2 Build: Cleanup precommit task gradle code
This change attempts to simplify the gradle tasks for precommit. One
major part of that is using a "less groovy style", as well as being more
consistent about how tasks are created and where they are configured. It
also allows the things creating the tasks to set up inter task
dependencies, instead of assuming them (ie decoupling from tasks
eleswhere in the build).
2015-12-01 22:36:54 -08:00
Robert Muir 6e5c301739 remove unused dependencies 2015-12-01 22:22:11 -05:00
Ryan Ernst fcb6f44acf Build: Add ability to specify the compact profile for javac 2015-12-01 11:04:46 -08:00
Robert Muir f7e7a6bfad compile against compact3 profile 2015-12-01 09:40:54 -05:00
Ryan Ernst 4bb1eed766 Build: Increase the number of failed tests shown in test summary
We had increased this in maven, but it was lost in the transition to
gradle. This change adds it as a configurable setting the the logger for
randomized testing and bumps it to 25.
2015-11-30 13:40:39 -08:00
Ryan Ernst 0fa05edf03 Build: Improve output when integ test fails
This outputs a lot more info when integ tests fail to start, as well as
(should) fix windows (at least in my VM run) integ tests.
2015-11-29 19:44:43 -08:00
Robert Muir 93758874fb Merge pull request #15093 from rmuir/gradle_windoze_integ_tests
Fix integration tests and 'gradle run' to work on windoze
2015-11-29 12:15:30 -05:00
Robert Muir a01f86aaac Fix integration tests and gradle run to work on windoze 2015-11-29 07:39:18 -08:00
Ryan Ernst 8e25dc4f80 Merge pull request #15089 from rjernst/extra_config_file_delay
Fix delayed extra config file checks to be right before copy
2015-11-28 19:44:16 -08:00
Ryan Ernst 018c766bf7 Build: Fix delayed extra config file checks to be right before copy
The current delay waits until later after normal configuration, but
still just after resolution (ie when paths would be known for
dependencies but not actual execution). This delays the checks further
to be done right before we actually execute the copy task.

closes #15068
2015-11-28 18:06:36 -08:00
Ryan Ernst 1ee59e4977 Merge pull request #15088 from rjernst/integ_test_command_line
Move debug options setting to before command line is logged
2015-11-28 17:19:39 -08:00
Ryan Ernst f15ee9fa54 Build: Move debug options setting to before command line is logged
Currently if running with --info, the command line for ES, along with
env vars, are logged before they may be ammended to add debug options.
This moves the adding JAVA_OPTS to before we print the command.
2015-11-28 17:05:11 -08:00
Nik Everett eba594fc08 Remove strange checks and clean dirs 2015-11-28 19:44:53 -05:00
Nik Everett 45e71329ee [build] Fix deb and rpm tests
Gradle is causing so much trouble here! Its too cute for its own good.
2015-11-28 19:07:06 -05:00
Robert Muir 66ae46ef8b Give enough ram so javac won't OOM
Today its based on the automatic -Xmx, but this
will fail in some environments (e.g. vagrant).
2015-11-26 02:16:46 -07:00
Ryan Ernst 139d688e24 Merge pull request #14996 from rjernst/logged_exec
Add wrapper around gradle Exec task for easier logging
2015-11-25 11:26:30 -08:00
Ryan Ernst bcb4be322f Build: Add wrapper around gradle Exec task for easier logging
The Exec task outputs stdout/stderr to the standard streams by default.
However, to keep output short, we currently capture this, and only
output if the task failed. This change makes a small wrapper around Exec
to facilitate this behavior anywhere we use Exec.
2015-11-25 11:26:02 -08:00
Ryan Ernst 52f31ee14a Build: Allow extra config for integ test to be anything project.file() accepts
This change delays the lookup for whatever is passed to extra config as
the source file to happen at execution time. This allows using eg a task
which generates a file, but maintains the checks that the file is not a
dir and that it exists at runtime.
2015-11-25 00:34:56 -08:00
Ryan Ernst d6969fcf3a Build: Split extra config files for integ test into separate task
This doesn't work when there are no extra config files, since gradle
sees the copy task and says "there is nothing to do".
2015-11-24 16:37:32 -08:00
Ryan Ernst 84d6cbd32a Build: Fix extra config to create a file, not a directory for the destination file
Also added some checks to catch misconfiguration (dir or non existing file).
2015-11-24 16:09:47 -08:00
Ryan Ernst 7b0a0ce0d9 Build: Add ability to specify extra configuration files for integ test
This change allows copy extra files into the integ test cluster before
it runs. However, it explicitly forbids overwriting elasticsearch.yml,
since that is generated.
2015-11-24 15:19:51 -08:00
Ryan Ernst 3a77d993fd Build: Copy rest specs for plugins that are added to integ test cluster
This change allows having rest tests using the api spec of plugins that
the rest test is testing against.
2015-11-24 14:34:21 -08:00
Ryan Ernst 1e0f929281 Build: Simplify adding plugins that are another project in the build
The current mechanism for adding plugins to the integTest cluster is to
have a FileCollection. This works well for the integTests for a single
plugin, which automatically adds itself to be installed. However, for qa
tests where many plugins may be installed, and from other projects, it
is cumbersome to add configurations, dependencies and dependsOn
statements over and over. This simplifies installing a plugin from
another project by moving this common setup into the cluster
configuration code.
2015-11-24 12:53:11 -08:00
Ryan Ernst f0e780bf86 Merge pull request #14959 from rjernst/cluster_waiting
Allow customizing wait condition and cluster settings
2015-11-24 08:24:00 -08:00
Ryan Ernst 647f324c46 Change waitfor max time back to 30 seconds, in case jenkins is slow 2015-11-24 08:23:16 -08:00
Ryan Ernst af496f5e7f Merge pull request #14925 from rjernst/maven_test_parity
Add back leaveTemporary and ifNoTests
2015-11-24 00:53:25 -08:00
Ryan Ernst 31b7e91e5a Build: Allow customizing wait condition and cluster settings
The current wait condition for an integ test cluster being up is a
simple http get on the root path for elasticsearch. However, it is
useful to allow having arbitrary wait conditions. This change reworks
the wait task to first check that each node process started successfully
and has a socket up, followed by an arbitrary wait condition which
defaults to the current http get.

Also, cluster settings are allowed to be added, and overriden. Finally,
custom setup commands are made relative to the elasticsearch home dir
for each node.
2015-11-24 00:48:58 -08:00
Ryan Ernst 529e442e2a Merge branch 'master' into attach_without_subdir 2015-11-23 15:01:55 -08:00
Ryan Ernst 5a62282150 Build: Rework extra plugins support to be through sibling directories
This change removes the subdirectory support for extra-plugins, and
replaces it with an iteration of sibling directories with the prefix
"extra-plugin-".
2015-11-23 14:15:45 -08:00
Michael McCandless e13b0d4bde upgrade lucene to 5.4.0-snapshot-1715952 2015-11-23 17:13:49 -05:00
Ryan Ernst 503d836ee4 Build: Add back leaveTemporary and ifNoTests
This change adds back the last of the missing test options to the juni4
wrapper. leaveTemporary is important in that setting it to true (which
is how we had it set in maven) removes the warnings we currently get
about a leftover file that cannot be deleted (from jna).
2015-11-22 22:18:42 -08:00
Ryan Ernst 7d3da7e6ee Build: Get multi node smoke tests working
This change adds back the multi node smoke test, as well as making the
cluster formation for any test allow multiple nodes. The main changes in
cluster formation are abstracting out the node specific configuration to
a helper struct, as well as making a single wait task that waits for all
nodes after their start tasks have run. The output on failure was also
improved to log which node's info is being printed.
2015-11-22 14:49:05 -08:00
Robert Muir 30529c008d Ban write access to system properties
* Forbid System.setProperties & co in forbidden APIs.
* Ban property write access at runtime with security manager.

Plugins that need to modify system properties will need to request permission in their plugin-security.policy
2015-11-21 22:33:06 -05:00
Ryan Ernst a2ef228d12 Build: Add precommit checks to standalone tests 2015-11-21 11:50:43 -08:00
Nik Everett 9b6e59bd57 [build] Survive missing directories 2015-11-20 17:38:50 -05:00
Ryan Ernst 61d62078f5 Merge pull request #14779 from rjernst/fork_compile
Use JDK at JAVA_HOME for compiling/testing, and improve build info output
2015-11-20 12:12:49 -08:00
Ryan Ernst df9cd96e9f Merge pull request #14858 from rjernst/check_jarhell
Add jar hell check before tests run
2015-11-20 11:35:49 -08:00
Ryan Ernst 6b734dc009 Remove debugging output from running jjs scripts 2015-11-20 11:31:46 -08:00
Ryan Ernst 63a7b155d8 Merge branch 'master' into fork_compile 2015-11-20 11:30:36 -08:00
Ryan Ernst c650d31ef1 Build: Improve integ test startup behavior
As part of the refactoring to allow --debug-jvm with gradle run, the way
java options are passed for integ tests was changed. However, we need to
make sure the jvm argline passed goes to ES_GC_OPTS because this
allows overriding things like which garbage collector we run, which we
do for testing from jenkins. This change adds back ES_GC_OPTS.
2015-11-19 13:06:47 -08:00