Commit Graph

524 Commits

Author SHA1 Message Date
Robert Muir b01c17959e fix REPRODUCE WITH for integration tests 2015-07-16 21:45:20 -04:00
Robert Muir 9e7882e604 protect logic under skipTests for stupid pom packaging 2015-07-16 20:49:16 -04:00
Robert Muir 6b7561ac9a Use junit4 for running integration tests, too
failsafe uses surefire, which sucks. It also mean integ tests act alien right now.
I would rather have the consistency, e.g. things formatted the same way, running integ tests under security manager, etc.
2015-07-16 19:43:33 -04:00
Martijn van Groningen 9687670e8d upgraded groovy from 2.4.0 to 2.4.4 2015-07-16 13:56:32 +02:00
jaymode a7d4a97325 integration tests should use the baseVersion when copying artifacts 2015-07-15 15:25:43 -04:00
Simon Willnauer 6faa19c3de Deploy artifacts to S3 as well as sonatype when running a release
This change allows when running a release build with:

```
 mvn -Prelease deploy
```

To deploy all artifacts to S3 as well as to sonatype at the same time.
Both sources will be consistent in terms of content and no further action
is required to publish our artifacts including .rpm, .deb, .zip, .tar.gz, .jar
etc. on to out S3 download service. Albeit this changes the structure of our
downloads to pretty much matching the maven repository layout, this makes
releaseing core as well as the plugins extremely simple. This will allow to
remove most of our python script used for release and it will automatically
allow to release and integrate new modules without further interaction.

This also allows us to bascially streamline our release process on CI such that
CI builds can simply run maven deploy which is all we do during a release.

With this commit only the git related modifications like tagging, version bumping
on our pom files and publishing RPM and .deb in their dedicated repository is left
for the python script.

With this change our artifact are available as follows:

```
http://download.elasticsearch.org/elasticsearch/release/org/elasticsearch/elasticsearch/2.0.0-beta1/elasticsearch-2.0.0-beta1.deb
http://download.elasticsearch.org/elasticsearch/release/org/elasticsearch/elasticsearch/2.0.0-beta1/elasticsearch-2.0.0-beta1.zip
http://download.elasticsearch.org/elasticsearch/release/org/elasticsearch/elasticsearch/2.0.0-beta1/elasticsearch-2.0.0-beta1.rpm
```

Plugins are deployed to URLs like this:

```
http://download.elasticsearch.org/elasticsearch/release/org/elasticsearch/plugin/elasticsearch-analysis-icu/2.0.0-beta1/elasticsearch-analysis-icu-2.0.0-beta1.zip
```

All artifacts like .jar as well as their checksums and gpg signatures are also available next to it.
2015-07-15 17:14:34 +02:00
Robert Muir 68143dedf2 Refactor integration tests
1. tests don't have a bogus test dependency on zips anymore,
   instead we handle this in pre-integration-test. This reduces
   lots of confusion for e.g. mvn clean test.
2. refactor integ logic so that core/ and plugin/ share it.
   previously they were duplicates but the above change simplifies life.
   it also makes it easier for doing more interesting stuff
2015-07-14 23:37:04 -04:00
Robert Muir ed2f8ace51 enable system assertions 2015-07-14 21:25:32 -04:00
Robert Muir 882b46d65f remove shadow m2e build 2015-07-14 20:31:12 -04:00
Robert Muir 38054ff4ee Merge pull request #12243 from rmuir/enforcer
Use enforcer:display-info to print version info
2015-07-14 15:01:14 -04:00
Simon Willnauer b159a36559 Merge pull request #12227 from s1monw/release_profile
Add a release profile to the parent pom.xml
2015-07-14 21:00:40 +02:00
Robert Muir 15f233f672 Use enforcer:display-info to print version info
Currently this is done with ant tasks, but this is simpler and
designed for this purpose.
2015-07-14 14:48:20 -04:00
Robert Muir 0808c98eeb enable enforcer plugin 2015-07-14 13:42:12 -04:00
Robert Muir 01601e9a3d Add permissions to all classpath elements in BootstrapForTesting
The previous strategy (target/xxx + .m2/repository) is obviously broken for multi-module
builds.

Includes hack for crazy jython, which "finds its own jar" then looks for a Lib/ beside it
2015-07-14 12:01:03 -04:00
Simon Willnauer 295f1971d2 Add a release profile to the parent pom.xml
This commit adds a release profile that runs additional checks like:
  * check for `norelease` in the source
  * check for `AwaitsFix` annotations
  * deploys only if all reactor builds where successful
  * signs all artifacts with the GPG key provided (required for release)
  * builds the RPM together with the zip, tar.gz and .deb during package phase

Note this is WIP of replacing major functionality of the release pythong script but
won't work by itself.
2015-07-14 12:04:16 +02:00
Ryan Ernst fd6bc9fc57 tests: explicitly disable security manager for maven integration tests 2015-07-13 17:09:06 -07:00
Ryan Ernst adfd55bf89 Tests: Explicitly use newer junit runner for maven integ tests
The older runner (junit 4.0?) hides exceptions in static initialization
(and runs the tests anyways!). With this change, the actual cause
of the recent windows integ test failures with security manager
are shown.
2015-07-13 15:33:02 -07:00
uboness 686413771c Removed duplication in parent pom
Found 2 definition for  `maven-install-plugin`
2015-07-13 22:40:18 +02:00
uboness b40186652c updated the elasticsearch versioning format
Moving to from `X.Y.Z.beta1`/`X.Y.Z.RC1` to `X.Y.Z-beta1`/`X.Y.Z-rc1`
2015-07-13 20:26:37 +02:00
Simon Willnauer 0eb9f07f0e Calculate artifact checksums in maven
Today we calculate the checksums in a python scritp to do the release.
We can also do this way simpler in maven using an ant task.
2015-07-10 13:30:31 +02:00
Ryan Ernst 5d059313dc appease maven warnings about using ${version} in pom 2015-07-09 13:38:15 -07:00
Simon Willnauer e0708813a9 Make 2.0.0.beta1-SNAPSHOT the current version.
Today everything is tight to having the next version as the latest.
In order to work towards 2.0.0.beta1 we need to fix all the usage of
2.0.0-SNAPSHOT to reflect the version we will release soon.
Usually we do this on the release branch but to simplify things I wanna
keep this on master for now and move to 2.1.0-SNAPSHOT on master once
we created a 2.0 branch.

Closes #12148
2015-07-09 21:24:32 +02:00
Robert Muir 8ef73b7dba Get integration tests working on windows.
We also run our license checker in `mvn verify`, but there
are problems with checksum calculation on windows there, so I've
disabled the license checker on windows to prevent those false fails.
2015-07-08 02:40:00 -04:00
Robert Muir 8f0b7164b7 Fix tophints noise.
previously this was done wrong, junit4 ant tasks were brought into
the test classpath. This created jar hell for tests, and also encouraged
people to use _internal_ stuff like its bundled guava in tests.

also the task was set to be lenient and ignore errors. And we were
passing in a messload of unnecessary classpaths to run this. It
only needs the module classpath: the explicit ant dependencies we declare.
2015-07-08 00:45:02 -04:00
Robert Muir fb62d945aa isolate integ tests a little better.
They are mostly just ES clients, but we can set tempdir and the usual
stuff to contain them better.
2015-07-07 23:10:18 -04:00
David Pilato 7ec6dc9956 [build] Update to Apache Maven PMD Plugin 3.5
http://maven.apache.org/plugins/maven-pmd-plugin/

You should specify the version in your project's plugin configuration:

```xml
<plugin>
 <groupId>org.apache.maven.plugins</groupId>
 <artifactId>maven-pmd-plugin</artifactId>
 <version>3.5</version>
</plugin>
```

Release Notes - Apache Maven PMD Plugin - Version 3.5
---------------

Bug

* [MPMD-208] Warning about deprecated Rule name when using rulesets/maven.xml
* [MPMD-205] Javascript violations won't fail the build

Improvement

* [MPMD-211] Upgrade plexus-resources from 1.0-alpha-7 to 1.1
* [MPMD-209] Upgrade to PMD 5.3.2
* [MPMD-206] Make the sourceDirectories configurable

New Feature

* [MPMD-207] Support Javascript and JSP for CPD
2015-07-06 23:19:55 +02:00
Robert Muir 23dfb5f1a5 don't try to run tests for packaging=pom 2015-07-06 15:49:57 -04:00
Robert Muir 9a146b9e75 Get delete by query rest tests running 2015-07-06 15:13:50 -04:00
Robert Muir 3cb95d7595 Merge branch 'master' into plugin-integration-tests 2015-07-06 13:34:32 -04:00
Robert Muir a7b4a061be don't pull outdated/old packages. 2015-07-06 13:32:46 -04:00
Robert Muir 1c114110c5 integration tests need the zip 2015-07-06 12:29:30 -04:00
David Pilato 1f04d6c2bb [build] Update maven-invoker-plugin to 2.0.0
Release Notes - Apache Maven Invoker Plugin - Version 2.0.0

https://issues.apache.org/jira/secure/ReleaseNote.jspa?projectId=12317525&version=12332831

Important Note:

This Release is intended for the bug fix MINVOKER-187 and in this relationship
is was necessary to upgrade the JDK minimum to JDK 1.6. This was the reason for
the version upgrade to 2.0.0.

Bug:

* [MINVOKER-187] - Cloned IT project must be writable

Improvement:

* [MINVOKER-192] - Using fluido skin
2015-07-06 17:54:33 +02:00
Robert Muir 3f4b8df00d Merge pull request #12026 from rmuir/integ_tests
add integration test harness to maven build
2015-07-06 10:16:54 -04:00
Robert Muir 9d233aeaf0 use external test cluster for integration tests 2015-07-03 12:20:35 -04:00
Tanguy Leroux 9495816cb7 Remove sigar completely 2015-07-03 15:49:17 +02:00
Robert Muir 80871bae2b Add simple integ testing infra 2015-07-03 02:12:01 -04:00
Robert Muir f8fbf1fa0c Fail startup (and tests) on jar hell 2015-06-29 23:13:45 -04:00
Robert Muir 55c33b29f6 Merge pull request #11898 from rmuir/lockdown
steps to remove dangerous security permissions
2015-06-29 11:13:16 -04:00
Robert Muir a585ff54fd Link securemock into the build so we will have snapshots, etc 2015-06-29 10:23:42 -04:00
Robert Muir bda60d6d76 first stab at per-jar permissions for the scary stuff.
unfortunately finds a crab in pluginmanager
2015-06-26 20:40:42 -04:00
Colin Goodheart-Smithe 772d0cc6e7 Build: Make rest-spec-api a project so eclipse build works
The change makes rest-spec-api a project in the same way as we build dev-tools. it packages the tests and api in a bundle using the maven-remote-resources-plugin and uses the same plugin in the plugins and core pom to unpack the rest-api-spec into the target directory and references the rest tests there in the test resources.

The main stimulus for this change is that for those using Eclipse the current build does not work. After running `mvn eclipse:eclipse` the Eclipse IDE errors because the rest-api-spec is outside of the project scope, meaning that every time the command is run (required whenever any dependencies change), the class path of all the projects has to be manually fixed.
2015-06-22 11:41:44 +01:00
Simon Willnauer 5808102d19 Upgrade AWS dependency to 1.10.0
This also apgrades JodaTime to 2.8.0.
2015-06-18 14:04:38 +02:00
Simon Willnauer 90f9b5f60d Allow to opt-out of loading packaged REST tests
this is really just a workaround for plugins to run their own
REST tests instead of the core ones. It opts out of the rest test
loading from the core jar file and tries to load from the classpath instead.
Eventually we need to fix this infrastrucutre to move away from parameterized
tests such that subclasses can override behavior.

Closes #11721
2015-06-17 20:37:59 +02:00
Clinton Gormley 05d512f417 Packaging: Add LICENSE, NOTICE, and sha1 files and tests for all core dependencies
Added a licenses/ directory to core which contains a sha1 file for each JAR
dependency, and one or more LICENSE files and one NOTICE file for each
project.

Also adds dev-tools/src/main/resources/license-check/check_license_and_sha.pl
which checks that the licenses/ dir is up to date during a mvn verify,
and which can be used to update the sha1 files when upgrading dependencies.

Closes #2794
Closes #10684
Closes #11705
2015-06-17 18:06:00 +02:00
David Pilato fd6112171d Merge pull request #11622 from dadoonet/maven/shade-plugin
[build] update maven-shade-plugin to 2.4

## Release Notes - Apache Maven Shade - Version 2.4

https://issues.apache.org/jira/secure/ReleaseNote.jspa?projectId=12317921&version=12331393

### Bugs:

* [MSHADE-155] - dependency-reduced-pom should use shadedArtifactId
* [MSHADE-169] - Typos in warning message
* [MSHADE-172] - "java.lang.ArithmeticException: / by zero" in MinijarFilter
* [MSHADE-174] - Unable to shade Java 8 jarfiles with static interface methods using minimizeJar
* [MSHADE-183] - Getting "Error creating shaded jar: java.util.jar.Attributes cannot be
                 cast to java.lang.String" error when using ManifestResourceTransformer with Maven 3.2.5
* [MSHADE-185] - systemPath content is interpolated for system dependencies

### Improvements:

* [MSHADE-177] - MavenProject/MavenSession Injection as a paremeter instead as a component.
* [MSHADE-178] - Removing plexus-container-default dependency
* [MSHADE-179] - Fix RAT Report
* [MSHADE-180] - Upgrade plexus-utils to 3.0.18
* [MSHADE-188] - Upgrade maven-dependency-tree to 2.2
* [MSHADE-191] - Upgrade plexus-utils to 3.0.22
* [MSHADE-192] - Upgrade maven-invoker to 1.10
* [MSHADE-193] - Upgrade to fluido skin 1.4.0
2015-06-16 09:34:55 +02:00
David Pilato bd5c7d0ea2 [maven] clean pom.xml
In Maven parent project, in dependency management, we should only declare which versions of 3rd party jars we want to use but not force any scope.
It makes then more obvious in modules what is exactly the scope of any dependency.

For example, one could imagine importing `jimfs` as a `compile` dependency in another module/plugin with:

```xml
<dependency>
   <groupId>com.google.jimfs</groupId>
   <artifactId>jimfs</artifactId>
</dependency>
```

But it won't work as expected as the default maven `scope` should be `compile` but here it's `test` as defined in the parent project.

So, if you want to use this lib for tests, you should simply define:

```xml
<dependency>
   <groupId>com.google.jimfs</groupId>
   <artifactId>jimfs</artifactId>
   <scope>test</scope>
</dependency>
```

We also remove `maven-s3-wagon` from gce plugin as it's not used.
2015-06-15 17:08:15 +02:00
Adrien Grand ff279275b3 Upgrade to Lucene 5.2.1. 2015-06-15 10:51:05 +02:00
David Pilato a41f68f9fe [build] update maven-shade-plugin to 2.4
## Release Notes - Apache Maven Shade - Version 2.4

https://issues.apache.org/jira/secure/ReleaseNote.jspa?projectId=12317921&version=12331393

### Bugs:

* [MSHADE-155] - dependency-reduced-pom should use shadedArtifactId
* [MSHADE-169] - Typos in warning message
* [MSHADE-172] - "java.lang.ArithmeticException: / by zero" in MinijarFilter
* [MSHADE-174] - Unable to shade Java 8 jarfiles with static interface methods using minimizeJar
* [MSHADE-183] - Getting "Error creating shaded jar: java.util.jar.Attributes cannot be
                 cast to java.lang.String" error when using ManifestResourceTransformer with Maven 3.2.5
* [MSHADE-185] - systemPath content is interpolated for system dependencies

### Improvements:

* [MSHADE-177] - MavenProject/MavenSession Injection as a paremeter instead as a component.
* [MSHADE-178] - Removing plexus-container-default dependency
* [MSHADE-179] - Fix RAT Report
* [MSHADE-180] - Upgrade plexus-utils to 3.0.18
* [MSHADE-188] - Upgrade maven-dependency-tree to 2.2
* [MSHADE-191] - Upgrade plexus-utils to 3.0.22
* [MSHADE-192] - Upgrade maven-invoker to 1.10
* [MSHADE-193] - Upgrade to fluido skin 1.4.0
2015-06-12 13:56:35 +02:00
Colin Goodheart-Smithe 240155b4ea Adds eclipse settings files to set correct encoding and complier preferences
This is done in the maven-eclipse-plugin so now all eclipse files are generated by mV eclipse:eclipse and can safely be ignored
2015-06-08 14:54:44 +01:00
Robert Muir 1f3c4604a5 use released lucene 5.2 jar 2015-06-08 06:51:46 -04:00