Commit Graph

183 Commits

Author SHA1 Message Date
Yannick Welsch d5191518ba Fix thirdPartyAudit check on Windows in case of jar hell with JDK 2015-12-31 12:10:48 +01:00
Robert Muir 180ab2493e Improve thirdPartyAudit check, round 3 2015-12-28 22:38:55 -05:00
Robert Muir 010d1a89c5 Merge branch 'master' into hdfs2-only 2015-12-22 00:40:54 -05:00
Adrien Grand cf52e96c42 Upgrade to lucene-5.5.0-snapshot-1721183.
Some files that implement or use the Scorer API had to be changed because of
https://issues.apache.org/jira/browse/LUCENE-6919.
2015-12-21 17:02:08 +01:00
Robert Muir f4f8b6e3fe Merge branch 'master' of github.com:elastic/elasticsearch into hdfs2-only 2015-12-20 21:59:02 -05:00
Ryan Ernst a518599e27 Allow plugins to upgrade slf4j-api to a compile dep
This was originally intended to be general purpose in #15555, but
that still had problems. Instead, this change fixes the issue explicitly
for slf4j-api, since that is the problematic dep that is not actually
included in the distributions.
2015-12-20 17:59:00 -08:00
Robert Muir 12a8428dfb Add MiniHDFS test fixture, started before integTest and shut down after.
Currently uses a hardcoded port (9999), need to apply MavenFilteringHack after it starts.
2015-12-20 16:00:37 -05:00
Ryan Ernst 9cb4c82c58 Build: Add fixture capabilities to integ tests
This change adds a Fixture class for use by gradle. A Fixture is an
external process that integration tests will use. It can be added as a
dependsOn for integTest, and will automatically be shutdown upon success
or failure, as well as relevant information dumped on failure. There is
also an example fixture in this change.
2015-12-19 15:46:21 -08:00
Ryan Ernst ad46a09cf5 Revert "Build: Allow plugins to "upgrade" provided deps"
This reverts commit 583ba76d0b.
2015-12-19 01:22:12 -08:00
Ryan Ernst 583ba76d0b Build: Allow plugins to "upgrade" provided deps
This only really applies to slf4j, since the others are in the
distribution, but it is necessary if a plugin wants to depend on slf4j.
2015-12-19 00:06:29 -08:00
Ryan Ernst af11707da0 Remove unused method impl in AntTask and make abstract 2015-12-18 14:37:15 -08:00
Ryan Ernst beec7ca9db Merge branch 'master' into wildcard_imports 2015-12-18 13:18:08 -08:00
Ryan Ernst d0a10b337e Fix silly typo in variable name... 2015-12-18 13:17:05 -08:00
Ryan Ernst a639b3d3e4 Remove wildcard imports for groovy too 2015-12-18 12:47:35 -08:00
Ryan Ernst 4ea19995cf Remove wildcard imports 2015-12-18 12:43:47 -08:00
Ryan Ernst 853e9c0fd1 Merge branch 'master' into wildcard_imports 2015-12-18 12:24:30 -08:00
Ryan Ernst 44edac0081 use better variable name for build listener, and change access back to
private for internal method of thirdPartyAudit
2015-12-18 12:07:30 -08:00
Ryan Ernst 5b9bf8e738 Make a new ant builder per AntTask invocation 2015-12-18 12:01:54 -08:00
Ryan Ernst 9f1dfdbaea Build: Add AntTask to simplify controlling logging when running ant from gradle
This new task allows setting code, similar to a doLast or doFirst,
except it is specifically geared at running ant (and thus called doAnt).
It adjusts the ant logging while running the ant so that the log
level/behavior can be tweaked, and automatically buffers based on gradle
logging level, and dumps the ant output upon failure.
2015-12-18 11:45:52 -08:00
Robert Muir 2ce54640f5 Remove unnecessary license categories/matchers 2015-12-18 13:46:59 -05:00
Robert Muir c4f823903a fix indent 2015-12-18 12:00:52 -05:00
Robert Muir e82808917c don't fail on missing source dirs 2015-12-18 11:58:18 -05:00
Robert Muir 94d6b22137 add gradle licenseHeaders to precommit
This is a port of the logic from apache lucene that uses Rat
2015-12-18 11:43:58 -05:00
Ryan Ernst fbff877ec3 Fix thirdpartyaudit to ignore all elasticsearch packages 2015-12-17 22:31:40 -08:00
Ryan Ernst 6a99796b02 Build: Move test framework under a "test" top level dir
This allows adding more test projects, eg integ test fixtures that will
be coming soon.
2015-12-17 21:25:06 -08:00
Robert Muir 6692e42d9a thirdPartyAudit round 2
This fixes the `lenient` parameter to be `missingClasses`. I will remove this boolean and we can handle them via the normal whitelist.
It also adds a check for sheisty classes (jar hell with the jdk).
This is inspired by the lucene "sheisty" classes check, but it has false positives. This check is more evil, it validates every class file against the extension classloader as a resource, to see if it exists there. If so: jar hell.

This jar hell is a problem for several reasons:

1. causes insanely-hard-to-debug problems (like bugs in forbidden-apis)
2. hides problems (like internal api access)
3. the code you think is executing, is not really executing
4. security permissions are not what you think they are
5. brings in unnecessary dependencies
6. its jar hell

The more difficult problems are stuff like jython, where these classes are simply 'uberjared' directly in, so you cant just fix them by removing a bogus dependency. And there is a legit reason for them to do that, they want to support java 1.4.
2015-12-17 02:35:00 -05:00
Robert Muir 42138007db add some more comments about internal api usage 2015-12-16 18:56:02 -05:00
Robert Muir 49f37a526c fix bad indent 2015-12-16 16:50:52 -05:00
Robert Muir ee79d46583 Add gradle thirdPartyAudit to precommit tasks 2015-12-16 16:38:16 -05:00
Ryan Ernst 5c8a0da1fd Build: Change rest integ tests to not have hardcoded ports
This change removes hardcoded ports from cluster formation. It passes
port 0 for http and transport, and then uses a special property to have
the node log the ports used for http and transport (just for tests).
This does not yet work for multi node tests. This brings us one step
closer to working with --parallel.
2015-12-11 17:36:08 -08:00
Ryan Ernst 47e922ba85 Add excludes and patterns as explicit inputs to forbidden patterns task 2015-12-11 08:26:38 -08:00
Jason Tedor 3383c24be0 Remove and forbid use of Collections#shuffle(List) and Random#<init>()
This commit removes and now forbids all uses of
Collections#shuffle(List) and Random#<init>() across the codebase. The
rationale for removing and forbidding these methods is to increase test
reproducibility. As these methods use non-reproducible seeds, production
code and tests that rely on these methods contribute to
non-reproducbility of tests.

Instead of Collections#shuffle(List) the method
Collections#shuffle(List, Random) can be used. All that is required then
is a reproducible source of randomness. Consequently, the utility class
Randomness has been added to assist in creating reproducible sources of
randomness.

Instead of Random#<init>(), Random#<init>(long) with a reproducible seed
or the aforementioned Randomess class can be used.

Closes #15287
2015-12-11 11:16:38 -05:00
Robert Muir bc140659a0 fail build on wildcard imports
Wildcard imports are terrible, they cause ambiguity in the code,
make it not compile with the future versions of java in many cases.

We should simply fail the build on this, it is messiness, caused by
messy Intellij configuration
2015-12-11 11:03:44 -05:00
Robert Muir 2741888498 Remove RuntimePermission("accessDeclaredMembers")
Upgrades lucene to 5.5.0-1719088, randomizedtesting to 2.3.2, and securemock to 1.2
2015-12-10 14:26:55 -05:00
Robert Muir e454fadc22 Merge branch 'master' into shave_mustache 2015-12-10 07:58:24 -05:00
Robert Muir b6fbd18e09 Merge pull request #15322 from rmuir/awesome
make slow tests more obvious
2015-12-09 15:05:53 -05:00
Ryan Ernst 46d73a13ff Merge branch 'master' into shave_mustache 2015-12-08 21:11:48 -08:00
Ryan Ernst 18698b9308 Build: Add build plugin for "messy" tests
We have some tests which have crazy dependencies, like on other plugins.
This change adds a "messy-test" gradle plugin which can be used for qa
projects that these types of tests can run in. What this adds over
regular standalone tests is the plugin properties and metadata on the
classpath, so that the plugins are properly initialized.
2015-12-08 19:48:39 -08:00
Robert Muir 2e299860e5 refactor mustache to lang-mustache plugin. No rest tests yet. 2015-12-08 22:45:21 -05:00
Robert Muir b0febc1b9f make slow tests more obvious 2015-12-08 13:05:15 -05:00
Ryan Ernst fa1c708ccd Merge branch 'master' into standalone_tests_intellij 2015-12-07 16:01:32 -08:00
Ryan Ernst 2dbd93d3b0 Tweaked module error message against non integ-test-zip and made checks
for limiting configuration to certain distributions prettier
2015-12-04 11:40:08 -08:00
Ryan Ernst 01d48e2062 Merge branch 'master' into jigsaw 2015-12-04 11:29:49 -08:00
Ryan Ernst a6752c4da9 Merge pull request #15210 from rjernst/buildsrc_test_cleanup
Cleanup integ test classes in buildSrc to be less groovyish
2015-12-04 11:24:21 -08:00
Ryan Ernst 4b1fc8ff22 Merge pull request #15226 from rjernst/fix_update_shas
Fix updateShas to not barf on disabled license checks and even compile correctly
2015-12-03 23:34:33 -08:00
Ryan Ernst f5b6b40a92 Fix distros to put modules dir int he right place and qa rest tests to
use the real zip
2015-12-03 23:03:14 -08:00
Ryan Ernst a8e9403204 added gradle checks for modules configuration, and ability to add
modules to integ test cluster
2015-12-03 20:53:06 -08:00
Ryan Ernst f2cebd9b3b Merge branch 'master' into jigsaw 2015-12-03 20:02:44 -08:00
Ryan Ernst 0be141e021 Build: Add .settings for buildSrc on gradle eclipse
We have eclipse settings added to all projects when running gradle
eclipse, but buildSrc is its own special project that is not
encapsulated by allprojects blocks. This adds eclipse settings to
buildSrc.
2015-12-03 19:56:11 -08:00
Ryan Ernst 17e6195fd6 Build: Fix updateShas to not barf on disabled license checks and even compile correctly
These were just results of not testing properly after refactoring.

closes #15224
2015-12-03 15:21:14 -08:00