Commit Graph

622 Commits

Author SHA1 Message Date
jaymode 3125ad6aae remove securemock from repository
The securemock has a new library, located at https://github.com/elastic/securemock
2015-07-16 12:56:13 -04:00
Martijn van Groningen 9687670e8d upgraded groovy from 2.4.0 to 2.4.4 2015-07-16 13:56:32 +02:00
jaymode a7d4a97325 integration tests should use the baseVersion when copying artifacts 2015-07-15 15:25:43 -04:00
Simon Willnauer 6faa19c3de Deploy artifacts to S3 as well as sonatype when running a release
This change allows when running a release build with:

```
 mvn -Prelease deploy
```

To deploy all artifacts to S3 as well as to sonatype at the same time.
Both sources will be consistent in terms of content and no further action
is required to publish our artifacts including .rpm, .deb, .zip, .tar.gz, .jar
etc. on to out S3 download service. Albeit this changes the structure of our
downloads to pretty much matching the maven repository layout, this makes
releaseing core as well as the plugins extremely simple. This will allow to
remove most of our python script used for release and it will automatically
allow to release and integrate new modules without further interaction.

This also allows us to bascially streamline our release process on CI such that
CI builds can simply run maven deploy which is all we do during a release.

With this commit only the git related modifications like tagging, version bumping
on our pom files and publishing RPM and .deb in their dedicated repository is left
for the python script.

With this change our artifact are available as follows:

```
http://download.elasticsearch.org/elasticsearch/release/org/elasticsearch/elasticsearch/2.0.0-beta1/elasticsearch-2.0.0-beta1.deb
http://download.elasticsearch.org/elasticsearch/release/org/elasticsearch/elasticsearch/2.0.0-beta1/elasticsearch-2.0.0-beta1.zip
http://download.elasticsearch.org/elasticsearch/release/org/elasticsearch/elasticsearch/2.0.0-beta1/elasticsearch-2.0.0-beta1.rpm
```

Plugins are deployed to URLs like this:

```
http://download.elasticsearch.org/elasticsearch/release/org/elasticsearch/plugin/elasticsearch-analysis-icu/2.0.0-beta1/elasticsearch-analysis-icu-2.0.0-beta1.zip
```

All artifacts like .jar as well as their checksums and gpg signatures are also available next to it.
2015-07-15 17:14:34 +02:00
Robert Muir 68143dedf2 Refactor integration tests
1. tests don't have a bogus test dependency on zips anymore,
   instead we handle this in pre-integration-test. This reduces
   lots of confusion for e.g. mvn clean test.
2. refactor integ logic so that core/ and plugin/ share it.
   previously they were duplicates but the above change simplifies life.
   it also makes it easier for doing more interesting stuff
2015-07-14 23:37:04 -04:00
Robert Muir ed2f8ace51 enable system assertions 2015-07-14 21:25:32 -04:00
Robert Muir 882b46d65f remove shadow m2e build 2015-07-14 20:31:12 -04:00
Robert Muir 38054ff4ee Merge pull request #12243 from rmuir/enforcer
Use enforcer:display-info to print version info
2015-07-14 15:01:14 -04:00
Simon Willnauer b159a36559 Merge pull request #12227 from s1monw/release_profile
Add a release profile to the parent pom.xml
2015-07-14 21:00:40 +02:00
Robert Muir 15f233f672 Use enforcer:display-info to print version info
Currently this is done with ant tasks, but this is simpler and
designed for this purpose.
2015-07-14 14:48:20 -04:00
Robert Muir 0808c98eeb enable enforcer plugin 2015-07-14 13:42:12 -04:00
Robert Muir 01601e9a3d Add permissions to all classpath elements in BootstrapForTesting
The previous strategy (target/xxx + .m2/repository) is obviously broken for multi-module
builds.

Includes hack for crazy jython, which "finds its own jar" then looks for a Lib/ beside it
2015-07-14 12:01:03 -04:00
Simon Willnauer 295f1971d2 Add a release profile to the parent pom.xml
This commit adds a release profile that runs additional checks like:
  * check for `norelease` in the source
  * check for `AwaitsFix` annotations
  * deploys only if all reactor builds where successful
  * signs all artifacts with the GPG key provided (required for release)
  * builds the RPM together with the zip, tar.gz and .deb during package phase

Note this is WIP of replacing major functionality of the release pythong script but
won't work by itself.
2015-07-14 12:04:16 +02:00
Ryan Ernst fd6bc9fc57 tests: explicitly disable security manager for maven integration tests 2015-07-13 17:09:06 -07:00
Ryan Ernst adfd55bf89 Tests: Explicitly use newer junit runner for maven integ tests
The older runner (junit 4.0?) hides exceptions in static initialization
(and runs the tests anyways!). With this change, the actual cause
of the recent windows integ test failures with security manager
are shown.
2015-07-13 15:33:02 -07:00
uboness 686413771c Removed duplication in parent pom
Found 2 definition for  `maven-install-plugin`
2015-07-13 22:40:18 +02:00
uboness b40186652c updated the elasticsearch versioning format
Moving to from `X.Y.Z.beta1`/`X.Y.Z.RC1` to `X.Y.Z-beta1`/`X.Y.Z-rc1`
2015-07-13 20:26:37 +02:00
Simon Willnauer 0eb9f07f0e Calculate artifact checksums in maven
Today we calculate the checksums in a python scritp to do the release.
We can also do this way simpler in maven using an ant task.
2015-07-10 13:30:31 +02:00
Ryan Ernst 5d059313dc appease maven warnings about using ${version} in pom 2015-07-09 13:38:15 -07:00
Simon Willnauer e0708813a9 Make 2.0.0.beta1-SNAPSHOT the current version.
Today everything is tight to having the next version as the latest.
In order to work towards 2.0.0.beta1 we need to fix all the usage of
2.0.0-SNAPSHOT to reflect the version we will release soon.
Usually we do this on the release branch but to simplify things I wanna
keep this on master for now and move to 2.1.0-SNAPSHOT on master once
we created a 2.0 branch.

Closes #12148
2015-07-09 21:24:32 +02:00
Robert Muir 8ef73b7dba Get integration tests working on windows.
We also run our license checker in `mvn verify`, but there
are problems with checksum calculation on windows there, so I've
disabled the license checker on windows to prevent those false fails.
2015-07-08 02:40:00 -04:00
Robert Muir 8f0b7164b7 Fix tophints noise.
previously this was done wrong, junit4 ant tasks were brought into
the test classpath. This created jar hell for tests, and also encouraged
people to use _internal_ stuff like its bundled guava in tests.

also the task was set to be lenient and ignore errors. And we were
passing in a messload of unnecessary classpaths to run this. It
only needs the module classpath: the explicit ant dependencies we declare.
2015-07-08 00:45:02 -04:00
Robert Muir fb62d945aa isolate integ tests a little better.
They are mostly just ES clients, but we can set tempdir and the usual
stuff to contain them better.
2015-07-07 23:10:18 -04:00
David Pilato 7ec6dc9956 [build] Update to Apache Maven PMD Plugin 3.5
http://maven.apache.org/plugins/maven-pmd-plugin/

You should specify the version in your project's plugin configuration:

```xml
<plugin>
 <groupId>org.apache.maven.plugins</groupId>
 <artifactId>maven-pmd-plugin</artifactId>
 <version>3.5</version>
</plugin>
```

Release Notes - Apache Maven PMD Plugin - Version 3.5
---------------

Bug

* [MPMD-208] Warning about deprecated Rule name when using rulesets/maven.xml
* [MPMD-205] Javascript violations won't fail the build

Improvement

* [MPMD-211] Upgrade plexus-resources from 1.0-alpha-7 to 1.1
* [MPMD-209] Upgrade to PMD 5.3.2
* [MPMD-206] Make the sourceDirectories configurable

New Feature

* [MPMD-207] Support Javascript and JSP for CPD
2015-07-06 23:19:55 +02:00
Robert Muir 23dfb5f1a5 don't try to run tests for packaging=pom 2015-07-06 15:49:57 -04:00
Robert Muir 9a146b9e75 Get delete by query rest tests running 2015-07-06 15:13:50 -04:00
Robert Muir 3cb95d7595 Merge branch 'master' into plugin-integration-tests 2015-07-06 13:34:32 -04:00
Robert Muir a7b4a061be don't pull outdated/old packages. 2015-07-06 13:32:46 -04:00
Robert Muir 1c114110c5 integration tests need the zip 2015-07-06 12:29:30 -04:00
David Pilato 1f04d6c2bb [build] Update maven-invoker-plugin to 2.0.0
Release Notes - Apache Maven Invoker Plugin - Version 2.0.0

https://issues.apache.org/jira/secure/ReleaseNote.jspa?projectId=12317525&version=12332831

Important Note:

This Release is intended for the bug fix MINVOKER-187 and in this relationship
is was necessary to upgrade the JDK minimum to JDK 1.6. This was the reason for
the version upgrade to 2.0.0.

Bug:

* [MINVOKER-187] - Cloned IT project must be writable

Improvement:

* [MINVOKER-192] - Using fluido skin
2015-07-06 17:54:33 +02:00
Robert Muir 3f4b8df00d Merge pull request #12026 from rmuir/integ_tests
add integration test harness to maven build
2015-07-06 10:16:54 -04:00
Robert Muir 9d233aeaf0 use external test cluster for integration tests 2015-07-03 12:20:35 -04:00
Tanguy Leroux 9495816cb7 Remove sigar completely 2015-07-03 15:49:17 +02:00
Robert Muir 80871bae2b Add simple integ testing infra 2015-07-03 02:12:01 -04:00
Robert Muir f8fbf1fa0c Fail startup (and tests) on jar hell 2015-06-29 23:13:45 -04:00
Robert Muir 55c33b29f6 Merge pull request #11898 from rmuir/lockdown
steps to remove dangerous security permissions
2015-06-29 11:13:16 -04:00
Robert Muir a585ff54fd Link securemock into the build so we will have snapshots, etc 2015-06-29 10:23:42 -04:00
Robert Muir bda60d6d76 first stab at per-jar permissions for the scary stuff.
unfortunately finds a crab in pluginmanager
2015-06-26 20:40:42 -04:00
Colin Goodheart-Smithe 772d0cc6e7 Build: Make rest-spec-api a project so eclipse build works
The change makes rest-spec-api a project in the same way as we build dev-tools. it packages the tests and api in a bundle using the maven-remote-resources-plugin and uses the same plugin in the plugins and core pom to unpack the rest-api-spec into the target directory and references the rest tests there in the test resources.

The main stimulus for this change is that for those using Eclipse the current build does not work. After running `mvn eclipse:eclipse` the Eclipse IDE errors because the rest-api-spec is outside of the project scope, meaning that every time the command is run (required whenever any dependencies change), the class path of all the projects has to be manually fixed.
2015-06-22 11:41:44 +01:00
Simon Willnauer 5808102d19 Upgrade AWS dependency to 1.10.0
This also apgrades JodaTime to 2.8.0.
2015-06-18 14:04:38 +02:00
Simon Willnauer 90f9b5f60d Allow to opt-out of loading packaged REST tests
this is really just a workaround for plugins to run their own
REST tests instead of the core ones. It opts out of the rest test
loading from the core jar file and tries to load from the classpath instead.
Eventually we need to fix this infrastrucutre to move away from parameterized
tests such that subclasses can override behavior.

Closes #11721
2015-06-17 20:37:59 +02:00
Clinton Gormley 05d512f417 Packaging: Add LICENSE, NOTICE, and sha1 files and tests for all core dependencies
Added a licenses/ directory to core which contains a sha1 file for each JAR
dependency, and one or more LICENSE files and one NOTICE file for each
project.

Also adds dev-tools/src/main/resources/license-check/check_license_and_sha.pl
which checks that the licenses/ dir is up to date during a mvn verify,
and which can be used to update the sha1 files when upgrading dependencies.

Closes #2794
Closes #10684
Closes #11705
2015-06-17 18:06:00 +02:00
David Pilato fd6112171d Merge pull request #11622 from dadoonet/maven/shade-plugin
[build] update maven-shade-plugin to 2.4

## Release Notes - Apache Maven Shade - Version 2.4

https://issues.apache.org/jira/secure/ReleaseNote.jspa?projectId=12317921&version=12331393

### Bugs:

* [MSHADE-155] - dependency-reduced-pom should use shadedArtifactId
* [MSHADE-169] - Typos in warning message
* [MSHADE-172] - "java.lang.ArithmeticException: / by zero" in MinijarFilter
* [MSHADE-174] - Unable to shade Java 8 jarfiles with static interface methods using minimizeJar
* [MSHADE-183] - Getting "Error creating shaded jar: java.util.jar.Attributes cannot be
                 cast to java.lang.String" error when using ManifestResourceTransformer with Maven 3.2.5
* [MSHADE-185] - systemPath content is interpolated for system dependencies

### Improvements:

* [MSHADE-177] - MavenProject/MavenSession Injection as a paremeter instead as a component.
* [MSHADE-178] - Removing plexus-container-default dependency
* [MSHADE-179] - Fix RAT Report
* [MSHADE-180] - Upgrade plexus-utils to 3.0.18
* [MSHADE-188] - Upgrade maven-dependency-tree to 2.2
* [MSHADE-191] - Upgrade plexus-utils to 3.0.22
* [MSHADE-192] - Upgrade maven-invoker to 1.10
* [MSHADE-193] - Upgrade to fluido skin 1.4.0
2015-06-16 09:34:55 +02:00
David Pilato bd5c7d0ea2 [maven] clean pom.xml
In Maven parent project, in dependency management, we should only declare which versions of 3rd party jars we want to use but not force any scope.
It makes then more obvious in modules what is exactly the scope of any dependency.

For example, one could imagine importing `jimfs` as a `compile` dependency in another module/plugin with:

```xml
<dependency>
   <groupId>com.google.jimfs</groupId>
   <artifactId>jimfs</artifactId>
</dependency>
```

But it won't work as expected as the default maven `scope` should be `compile` but here it's `test` as defined in the parent project.

So, if you want to use this lib for tests, you should simply define:

```xml
<dependency>
   <groupId>com.google.jimfs</groupId>
   <artifactId>jimfs</artifactId>
   <scope>test</scope>
</dependency>
```

We also remove `maven-s3-wagon` from gce plugin as it's not used.
2015-06-15 17:08:15 +02:00
Adrien Grand ff279275b3 Upgrade to Lucene 5.2.1. 2015-06-15 10:51:05 +02:00
David Pilato a41f68f9fe [build] update maven-shade-plugin to 2.4
## Release Notes - Apache Maven Shade - Version 2.4

https://issues.apache.org/jira/secure/ReleaseNote.jspa?projectId=12317921&version=12331393

### Bugs:

* [MSHADE-155] - dependency-reduced-pom should use shadedArtifactId
* [MSHADE-169] - Typos in warning message
* [MSHADE-172] - "java.lang.ArithmeticException: / by zero" in MinijarFilter
* [MSHADE-174] - Unable to shade Java 8 jarfiles with static interface methods using minimizeJar
* [MSHADE-183] - Getting "Error creating shaded jar: java.util.jar.Attributes cannot be
                 cast to java.lang.String" error when using ManifestResourceTransformer with Maven 3.2.5
* [MSHADE-185] - systemPath content is interpolated for system dependencies

### Improvements:

* [MSHADE-177] - MavenProject/MavenSession Injection as a paremeter instead as a component.
* [MSHADE-178] - Removing plexus-container-default dependency
* [MSHADE-179] - Fix RAT Report
* [MSHADE-180] - Upgrade plexus-utils to 3.0.18
* [MSHADE-188] - Upgrade maven-dependency-tree to 2.2
* [MSHADE-191] - Upgrade plexus-utils to 3.0.22
* [MSHADE-192] - Upgrade maven-invoker to 1.10
* [MSHADE-193] - Upgrade to fluido skin 1.4.0
2015-06-12 13:56:35 +02:00
Colin Goodheart-Smithe 240155b4ea Adds eclipse settings files to set correct encoding and complier preferences
This is done in the maven-eclipse-plugin so now all eclipse files are generated by mV eclipse:eclipse and can safely be ignored
2015-06-08 14:54:44 +01:00
Robert Muir 1f3c4604a5 use released lucene 5.2 jar 2015-06-08 06:51:46 -04:00
Simon Willnauer 4c981ff4bf [BUILD] Don't shade core artifacts
This commit adds an additioal jar that is shaded and keeps all the
artifacts that are used by default on the server-side unshaded. Users
that need a shaded jar can now use the `shaded` classifyer to pull
the shaded minimized jar in instead. Including the shaded jar in a
downstream project looks like this:

```XML
<dependency>
  <groupId>org.elasticsearch</groupId>
  <artifactId>elasticsearch</artifactId>
  <classifier>shaded</classifier>
</dependency>
```
2015-06-05 21:52:09 +02:00
David Pilato 02c9c8998e [maven] update maven-assembly-plugin to 2.5.5
```xml
<plugin>
 <groupId>org.apache.maven.plugins</groupId>
 <artifactId>maven-assembly-plugin</artifactId>
 <version>2.5.5</version>
</plugin>
```

## Release Notes - Maven Assembly Plugin - Version 2.5.5

* Bug
   - [MASSEMBLY-767] - Schema missing from the web site
   - [MASSEMBLY-768] - JarInputStream unable to find  manifest
created by version 2.5.4
   - [MASSEMBLY-769] - ZIP fileMode permissions not properly set with
dependencySet and unpackOptions
2015-06-05 19:17:21 +02:00
Robert Muir 49eae2bb8b Seriously make forbidden apis not picky until builds are stable 2015-06-05 08:47:07 -04:00
Adrien Grand 4efa134745 Build: Make the Eclipse build use cross-project dependencies. 2015-06-05 14:05:59 +02:00
Robert Muir 1ea9a7f028 Make forbidden apis lenient (temporary!) until builds are stable 2015-06-05 07:44:24 -04:00
Simon Willnauer 7662f4711f [BUILD] Run unshaded by default 2015-06-05 13:13:44 +02:00
Simon Willnauer 29d06605c0 add core module 2015-06-05 13:12:05 +02:00
Simon Willnauer 05db5dc2c8 create parent pom project from its original location 2015-06-05 13:12:05 +02:00
Tanguy Leroux 340b7ef6ef Add common SystemD file for RPM/DEB package 2015-05-27 11:51:58 +02:00
David Pilato 60519911b4 [maven] fix paths for final artifact
We need to define an absolute path (based on `${project.basedir}`) instead of using related paths.
2015-05-26 11:40:57 +02:00
Robert Muir c08481f523 allow third party license analysis to run (com.mycila vs org.codehaus.mojo mess) 2015-05-23 08:40:40 -04:00
Robert Muir 9e7595655b Ensure shaded jar has manifest too 2015-05-23 08:37:50 -04:00
Robert Muir b462fd712a factor out static analysis 2015-05-23 01:33:37 -04:00
Robert Muir 5b1ecbf07f add snapshot repo (so we know where to get elasticsearch-parent from if needed) 2015-05-23 00:27:28 -04:00
Robert Muir 20da601e92 factor out code coverage 2015-05-23 00:25:50 -04:00
Robert Muir 38496c8265 remove duplicate test logging configuration 2015-05-22 23:55:27 -04:00
Robert Muir 5330e3423f remove build duplication 2015-05-22 23:23:59 -04:00
Ryan Ernst 3ff839d35f Upgrade to lucene-5.2.0-snapshot-1681214 2015-05-22 13:39:58 -07:00
Shay Banon 2eaaef571c Merge pull request #11304 from kimchy/upgrade_netty_3_10_3
Upgrade to Netty 3.10.3
2015-05-22 18:42:16 +02:00
Shay Banon 2f00a27042 Upgrade to Netty 3.10.3 2015-05-22 17:37:44 +02:00
Adrien Grand 2bfac41c8e Upgrade to lucene-5.2.0-snapshot-1681024 2015-05-22 11:36:40 +02:00
Robert Muir 8abb8e4861 Remove useless outdated forbidden-api version in `-Pdev` config. 2015-05-21 02:11:11 -04:00
Adrien Grand 437910e882 Upgrade to lucene-5.2.0-snapshot-1680200. 2015-05-19 10:28:35 +02:00
Robert Muir fc73540aac include rest tests in test-framework.jar
Closes #11192 which I accidentally already closed.

Squashed commit of the following:

commit f23faccddc2a77a880841da4c89c494edaa2aa46
Author: Robert Muir <rmuir@apache.org>
Date:   Fri May 15 16:04:55 2015 -0400

    Simplify this FileUtils even more: its either from the filesystem, or the classpath,
    not both. Its already trying 4 different combinations of crazy paths for either of these anyway.

commit c7016c8a2b5a6043e2ded4b48b160821ba196974
Author: Robert Muir <rmuir@apache.org>
Date:   Fri May 15 14:21:37 2015 -0400

    include rest tests in test-framework jar
2015-05-15 16:16:24 -04:00
Adrien Grand 3e215e720a Merge pull request #11144 from jpountz/fix/remove_hppc_esoteric_dep
Internal: remove dependency on hppc:esoteric.
2015-05-15 16:25:51 +02:00
Robert Muir 30cdd4c03b Use our provided JNA library, versus one installed on the system
which might be older and not work.
2015-05-14 00:09:44 -04:00
Robert Muir 3dd706f1e6 Add these two props for consistency, so we aren't confused
when we deduplicate with elasticsearch-parent
2015-05-13 14:56:29 -04:00
Adrien Grand 5c9c4776cd Internal: remove dependency on hppc:esoteric.
The esoteric classifier contains in particular maps that take bytes or doubles
as keys. In the byte case, we can just use integer, and in the double case we
can use their long bits instead.
2015-05-13 14:13:10 +02:00
Dawid Weiss 491b00c4ec Updating to HPPC-0.7.1
Close #11035
2015-05-13 13:53:14 +02:00
Robert Muir 262555079c Merge pull request #11106 from rmuir/symlink
Improve path mgmt on init, better error messages, symlink support
2015-05-12 14:21:19 -04:00
Adrien Grand 290fefcdf8 Upgrade to lucene-5.2.0-snapshot-1678978. 2015-05-12 18:49:51 +02:00
Robert Muir 019a9410a3 SecurityBootstrap -> BootstrapForTesting, and make less things public 2015-05-12 00:34:02 -04:00
David Pilato 234716a1ef Remove Codehaus repository
Codehaus announced they are shutting down their services: https://www.codehaus.org/

We should remove their repository from our pom as it could cause some errors an useless HTTP calls.

Related to #10939
2015-05-06 17:51:17 +02:00
Robert Muir 2ed2c4f884 fix permissions bugs 2015-05-04 16:27:24 -04:00
Robert Muir fe046df125 hacky state 2015-05-04 15:38:46 -04:00
Robert Muir b8efa8ea21 Remove JNI permissions, improve JNI testing. 2015-05-04 12:30:03 -04:00
Shay Banon 16c7689355 Exclude jackson-databind dependency
the jackson yaml data format pulls in the databind dependency, its important that we exclude it so we won't use any of its classes by mistake
2015-05-01 19:03:14 +02:00
Ryan Ernst a0451a37cc Upgrade lucene snapshot to r1677039 2015-04-30 13:43:19 -07:00
Robert Muir d7b7614f6f Use http for lucene snapshot downloads 2015-04-28 08:38:57 -04:00
Colin Goodheart-Smithe b3723cdb28 Fix to let eclipse ignore the antigun plugin in it's maven integration
This plugin should be ignored as it will make the internal eclipse build fail when there are NOCOMMIT comments in files, which are expected during feature development. This change only affects eclipse users and only when they are using the m2e eclipse integration
2015-04-27 12:50:11 +01:00
Alexander Reelsen f64739788b Build: Update package repositories when creating a release
In order to automatically sign and and upload our debian and RPM
packages, this commit incorporates signing into the build process
and adds the necessary steps to the release process. In order to do this
the pom.xml has been adapted and the RPM and jdeb maven plugins have been
updated, so the packages are signed on build. However the repositories
need to signed as well.

Syncing the repos requires downloading the current repo, adding
the new packages and syncing it back.

The following environment variables are now required as part of the build

* GPG_KEY_ID - the key ID of the key used for signing
* GPG_PASSPHRASE - your GPG passphrase
* S3_BUCKET_SYNC_TO: S3 bucket to sync new repo into

The following environment variables are optional

* S3_BUCKET_SYNC_FROM: S3 bucket to get existing packages from
* GPG_KEYRING - home of gnupg, defaults to ~/.gnupg

The following command line tools are needed

* createrepo (creates RPM repositories)
* expect (used by the maven rpm plugin)
* apt-ftparchive (creates DEB repositories)
* gpg (signs packages and repo files)
* s3cmd (syncing between the different S3 buckets)

The current approach would also work for users who want to run their
own repositories, all they need to change are a couple of environment
variables.

Minor implementation detail: Right now the branch name is used as version
for the repositories (like 1.4/1.5/1.6) - if we ever change our branch naming
scheme, the script needs to be fixed.
2015-04-26 19:05:47 +02:00
Tanguy Leroux f7d4baacfb Remove working directory
This commit removes the working directory and its associated environment variable "WORK_DIR"
2015-04-25 13:08:36 +02:00
Ryan Ernst f27f5aaa22 Upgrade to lucene-5.2-snapshot-1675927 2015-04-24 12:38:32 -07:00
Lee Hinman 54cf885d14 Disable security manager when running with `mvn exec:exec` 2015-04-24 11:00:54 -06:00
Robert Muir 500c956b45 Remove policy config file, its a resource.
Remove exposed boolean to turn off security.
Add unit test
2015-04-23 22:02:57 -04:00
Robert Muir 7b6e470f5d Merge branch 'master' into put_me_in_coach 2015-04-23 08:34:27 -04:00
Robert Muir a0414599a8 Merge pull request #10711 from rmuir/java_home_only
Execute tests with $JAVA_HOME.
2015-04-22 14:51:36 -04:00
Adrien Grand dd679a3a38 Upgrade to lucene-5.2-snapshot-1675363.
This snapshot contains in particular LUCENE-6446 (refactored explanation API)
and LUCENE-6448 (better equals/hashcode for filters).
2015-04-22 17:23:17 +02:00
Robert Muir 9406d83441 fix typo 2015-04-22 08:46:14 -04:00
Robert Muir 270cb9f349 enable securitymanager 2015-04-22 03:04:50 -04:00
Robert Muir 730314fec1 Execute tests with $JAVA_HOME. 2015-04-21 20:27:27 -04:00
Robert Muir 9d6b1382e7 Fix JVM isolation in tests.
Currently security manager would allow for one JVM to muck
with the files (read, write, AND delete) of another JVM.

This is unnecessary.
2015-04-21 19:02:14 -04:00
Robert Muir d1c3ec6291 Upgrade to Lucene 5.2 r1675100
This upgrade is for https://issues.apache.org/jira/browse/LUCENE-6442

It should improve test reproducibility, especially if you are on a mac
and want to reproduce a jenkins failure that happened on linux.
2015-04-21 09:06:44 -04:00
Tanguy Leroux 867955188e Standardization of packages structure and install
The existing DEB/RPM packages have a lot of differences: they don't execute the same actions when installing or removing the package. They also don't declare exactly the same environment variables at the same place. At the end of the day the global behavior and configuration is *almost* the same but it's very difficult to maintain the scripts.

This commits unifies the package behavior:
- DEB/RPM use the same package scripts (pre installation, post installation etc) in order to execute exactly the same actions
- Use of a unique environment vars file that declares everything needed by scripts (the goal is to delete vars declaration in init.d and systemd scripts, this will be done in another PR)
- Variables like directory paths are centralized and replaced according to the target platform (using #10330)
- Move /etc/rc.d/init.d to standard /etc/init.d (RPM only)
- Add PID_DIR env var
- Always set ES_USER, ES_GROUP,MAX_MAP_COUNT and MAX_OPEN_FILES in env vars file
- Create log, data, work and plugins directories with DEB/RPM packaging system
- Change to elastic.co domain in copyright and control files
- Add Bats files to automate testing of DEB and RPM packages
- Update TESTING.asciidoc

More info on Bats here:  https://github.com/sstephenson/bats
2015-04-20 14:58:19 +02:00
Robert Muir 069e11ba40 set heartbeat to 10s 2015-04-19 18:44:14 -04:00
Robert Muir c153772fca ensure these two versions are always in sync 2015-04-19 09:12:15 -04:00
Robert Muir b728772fe4 more fine-grained @slow tuning, remove from many tests that got unlucky with fsync 2015-04-18 19:59:29 -04:00
Robert Muir c00f0ff08e upgrade to lucene r1674576 2015-04-18 16:04:28 -04:00
Robert Muir 5bcd599cb3 remove repo, latest randomizedtesting is on maven central now 2015-04-18 12:25:39 -04:00
Robert Muir 96f08a38e6 parallelize rest tests 2015-04-17 23:38:01 -04:00
Robert Muir 43b6cd2118 Merge branch 'master' into mockfilesystem 2015-04-17 19:15:25 -04:00
Robert Muir e71553556e remove tests.processors, this is a reproducibility nightmare 2015-04-17 18:32:52 -04:00
Robert Muir e3e4c02379 nobody wants to look at bytecode 2015-04-17 17:16:03 -04:00
Ryan Ernst 7faa9a045a Change nocommit checks to not happen on intellij files (where there can
be a TODO task named eg Nocommits).
2015-04-17 12:54:15 -07:00
Robert Muir 5718e5616a fail the build if you typo test name 2015-04-17 09:45:38 -04:00
Robert Muir 7afa241e19 make stacktraces reasonable 2015-04-17 09:22:28 -04:00
Robert Muir c421948300 upgrade to lucene 5.2 r1674278 2015-04-17 07:58:20 -04:00
Adrien Grand fd416d5ed5 Upgrade to Lucene-5.2-snapshot-1674183. 2015-04-17 10:05:30 +02:00
Robert Muir 41d0e60dc7 Add tests progress indicator
Looks like this:

    Suite: org.elasticsearch.search.aggregations.bucket.HistogramTests
    Completed [10/713] on J0 in 8.32s, 27 tests

    Suite: org.elasticsearch.indices.state.OpenCloseIndexTests
    Completed [11/713] on J2 in 16.23s, 20 tests

Upgrades randomizedtesting to 2.1.13.
2015-04-16 13:29:51 -04:00
Ryan Ernst a3f078985b Tests: Forbid tests from writing to CWD
Allowing tests writing to the working directory can mask problems.
For example, multiple tests running in the same jvm, and using the
same relative path, may cause issues if the first test to run
leaves data in the directory, and the second test does not remember
to cleanup the path before using it.

This change adds security manager rules to disallow tests writing
to the working directory. Instead, tests create a temp dir with
the existing test framework.

closes #10605
2015-04-15 12:45:20 -07:00
Robert Muir b1afcdb71d upgrade to 5.2 r1673726 2015-04-15 10:28:31 -04:00
Adrien Grand 22720a1abd Merge pull request #10562 from jpountz/upgrade/lucene-5.2.0-snapshot-1673124
Upgrade to lucene-5.2.0-snapshot-1673124.
2015-04-14 16:57:09 +02:00
Simon Willnauer c13e604697 [BUILD] Restrict read permission to project.basedir
This prevents reads from anywhere outside of the elasticsearch
clone when running tests with security manager enabled.
2015-04-13 16:44:31 +02:00
Adrien Grand 45afa47a6f Upgrade to lucene-5.2.0-snapshot-1673124. 2015-04-13 10:43:58 +02:00
Simon Willnauer d738b3d1e2 [BUILD] Use SuppressFrobidden annotation instead of class level excludes
Forbidden APIs 1.8 allows excludes based on annotations which can now be on methods etc. for more find grained control.

Closes #10560
2015-04-13 10:08:50 +02:00
Uwe Schindler df118214c4 Update forbiddenapis to version 1.8 2015-04-13 01:27:56 +02:00
David Pursehouse 97a9b4ec2f Fix m2e Eclipse Integration
Update Eclipse core prefs

Eclipse Luna overwrites the prefs file, putting all the settings
in alphabetical order and removing comments. This causes the prefs
files to be modified in the git workspace.

Update the file with the version generated by Eclipse to prevent it
from being modified every time.

No settings values are modified by this change.

This also  adds another plugin to the lifecycle mapping in the pom.xml which was missed in https://github.com/elastic/elasticsearch/pull/10524.
2015-04-10 16:48:04 +01:00
Simon Willnauer bdc4c7f84c [BUILD] Add sigar binaries when running unittests
the sigar binaries are not available when running tests today. This
commit adds the path to the test run.
2015-04-10 14:37:18 +02:00
Colin Goodheart-Smithe 7e353e0768 removed erroneous tab character 2015-04-10 10:37:26 +01:00
Colin Goodheart-Smithe 4a09a98561 Fix to pom.xml to allow eclipse maven integration using m2e 2015-04-10 08:34:01 +01:00
Tanguy Leroux e60f61d0a9 Fix maven-resources-plugin warnings
Commit 168238dab6 declared multiple maven-resources-plugin usages instead of declaring multiple executions for the same plugin... resulting to Maven warnings.

Closes #10433
2015-04-09 10:05:10 +02:00
Adrien Grand 89b97c1c8d Upgrade to lucene-5.1.0-snapshot-1671894.
This includes a fix for LUCENE-6406.
2015-04-07 20:04:54 +02:00
Robert Muir 1d56502e52 Upgrade to Lucene 5.1 r1671277.
Closes #10435.

Squashed commit of the following:

commit aa1935c790b2731fc2bbc7de6142b09e3fe8bd4a
Author: Ryan Ernst <ryan@iernst.net>
Date:   Mon Apr 6 13:44:40 2015 -0700

    fix index lookup

commit bb6373595ff62ffc56fdf0cba3ac9c0ebe679946
Merge: 916962b eb3a170
Author: Robert Muir <rmuir@apache.org>
Date:   Mon Apr 6 14:24:38 2015 -0400

    Merge branch 'lucene_r1671277' of github.com:elasticsearch/elasticsearch into lucene_r1671277

commit 916962b82d192a53add471b4cc4a1396bc30eb0e
Merge: 197b3a2 21f72fe
Author: Robert Muir <rmuir@apache.org>
Date:   Mon Apr 6 07:09:41 2015 -0400

    Merge branch 'master' into lucene_r1671277

commit eb3a1703f7932ddd0cf3e83bec0e86131d255407
Author: Ryan Ernst <ryan@iernst.net>
Date:   Sat Apr 4 11:06:03 2015 -0700

    re-enable index lookup tests

commit 80d65d5eab39062dd8364687da74ddbb87ebcb76
Author: Ryan Ernst <ryan@iernst.net>
Date:   Sat Apr 4 10:39:52 2015 -0700

    update pom to point to new snapshot repo

commit 197b3a21ac2c2d70c9f740fe53e58632a22d1aad
Author: Robert Muir <rmuir@apache.org>
Date:   Sat Apr 4 12:51:22 2015 -0400

    fix postingsenum usage

commit 0e2b7a00cd07d068f755c51185ac521aa1eb0326
Author: Robert Muir <rmuir@apache.org>
Date:   Sat Apr 4 12:21:23 2015 -0400

    upgrade to lucene r1671277 (have not yet run tests or looked at postings changes)
2015-04-07 06:09:35 -04:00
Tanguy Leroux 168238dab6 Add properties files to configure startup and installation scripts
Many scripts are used to start/stop and install/uninstall elasticsearch. These scripts share a lot of configuration properties like directory paths, max value for a setting, default user etc. Most of the values are identical but some of them are different depending of the platform (Debian-based or Redhat-based OS), depending of the way elasticsearch is started (shell script, systemd, sysv-init...) or the way it is installed (zip, rpm, deb...). Today the values are duplicated in multiple places, making it difficult to maintain the scripts or to update a value.

This pull request make this more uniform: values used in scripts must be defined in a common packaging.properties file. Each value can be overridden in another specific packaging.properties file for Debian or Redhat. All startup and installation scripts are filtered with the common then the custom packaging.properties files before being packaged as a zip/tar.gz/rpm/dpkf archive.
2015-04-02 18:09:43 +02:00
Lee Hinman 1cb169afb4 Add ability to launch bootstrapped ES using maven
This adds the exec-maven-plugin that allows a developer to run:

```
mvn exec:exec
```

To launch the `Bootstrap` process similar to the way that a Java IDE
would. All the logs go to logs/elasticsearch.log (or wherever
configured)
2015-03-30 11:07:21 -06:00
Simon Willnauer c87e1b41a4 [TEST] only use perm gen space on java 1.7.x 2015-03-24 10:04:32 +01:00
Simon Willnauer c017a400d5 [TEST] Add back permGen space tuning for now 2015-03-24 09:38:34 +01:00
Robert Muir 10e6fa1963 fix build with java 9
* add compiler workarounds for JDK bug JI-9019884
* remove permgen specification during tests (this results in an error on java 9)
* fix threadpool grow/shrink to call methods in the right order (this results in IAE with java 9)
2015-03-24 00:29:24 -04:00
Shay Banon 9f9ada4b7b Upgrade to Jackson 2.5.1
Note, Jackson 2.5 is less lenient when it comes to not starting an object before starting to add fields on a fresh builder, fixed where applicable.
closes #10210
2015-03-23 09:03:27 +01:00
Shay Banon 31b63b1a84 Better detection of CBOR
CBOR has a special header that is optional, if exists, allows for exact detection. Also, since we know which formats we support in ES, we can support the object major type case.
closes #7640
2015-03-20 22:06:03 +01:00
Simon Willnauer 73953142a7 [BUILD] fix snapshot URL 2015-03-16 22:59:49 -07:00
Clinton Gormley 4d78026ef1 Updated copyright years to include 2015 2015-02-28 03:18:45 +01:00
Robert Muir 4ca57ab10a Upgrade to Lucene r1662607
Closes #9915.

Squashed commit of the following:

commit cfa59f5a3f03d9d1b432980dcee6495447c1e7ea
Author: Robert Muir <rmuir@apache.org>
Date:   Fri Feb 27 12:10:16 2015 -0500

    add missing null check

commit 62fe5403068c730c0e0b6fd1ab1a0246eeef6220
Author: Robert Muir <rmuir@apache.org>
Date:   Fri Feb 27 11:31:53 2015 -0500

    Disable ExtrasFS for now, until we hook all this in properly in a separate issue.

commit 822795c57c5cf846423fad443c2327c4ed0094ac
Author: Adrien Grand <jpountz@gmail.com>
Date:   Fri Feb 27 10:12:02 2015 +0100

    Fix PercolatorTests.

commit 98b2a0a7d8298648125c9a367cb7e31b3ec7d51b
Author: Adrien Grand <jpountz@gmail.com>
Date:   Fri Feb 27 09:27:11 2015 +0100

    Fix ChildrenQueryTests.

commit 9b99656fc56bbd01c9afe22baffae3c37bb48a71
Author: Robert Muir <rmuir@apache.org>
Date:   Thu Feb 26 20:50:02 2015 -0500

    cutover apis, no work on test failures yet.
2015-02-27 12:51:36 -05:00
Robert Muir 30a4294a6a Upgrade to lucene r1660560
Squashed commit of the following:

commit 07391388715ed1f737e8acc391cea0bce5d79db9
Merge: a71cc45 b61b021
Author: Robert Muir <rmuir@apache.org>
Date:   Fri Feb 20 06:58:11 2015 -0500

    Git really sucks

    Merge branch 'lucene_r1660560' of github.com:elasticsearch/elasticsearch into lucene_r1660560

commit b61b02163f62ad8ddd9906cedb3d57fed75eb52d
Author: Adrien Grand <jpountz@gmail.com>
Date:   Wed Feb 18 19:03:49 2015 +0100

    Try to improve TopDocs.merge usage.

commit bf8e4ac46d7fdaf9ae128606d96328a59784f126
Author: Ryan Ernst <ryan@iernst.net>
Date:   Wed Feb 18 07:43:37 2015 -0800

    reenable scripting test for accessing postings pieces.  commented out
    parts that fail because of bad assumptions

commit 6d4d635b1a23b33c437a6bae70beea70ad52d91c
Author: Robert Muir <rmuir@apache.org>
Date:   Wed Feb 18 09:41:46 2015 -0500

    add some protection against broken asserts, but, also disable crappy test

commit c735bbb11f38782dfea9c4200fcf732564126bf5
Author: Robert Muir <rmuir@apache.org>
Date:   Wed Feb 18 02:21:30 2015 -0500

    cutover remaining stuff from old postings api

commit 11c9c2bea3db3ff1cd2807bd43e77b500b167aed
Author: Robert Muir <rmuir@apache.org>
Date:   Wed Feb 18 01:46:04 2015 -0500

    cut over most DocsEnum usage

commit bc18017662f6abddf3f074078f74e582494c88e2
Author: Robert Muir <rmuir@apache.org>
Date:   Wed Feb 18 01:19:35 2015 -0500

    upgrade to lucene_r1660560, modulo one test fail
2015-02-20 06:58:43 -05:00
David Pilato 3c2414128c Rename Maven repository id
We are using repository ids with spaces in our `pom.xml`. Although it's not forbidden, a common practice is to avoid space in id.

This commit changes codehaus snapshots and lucene snapshots to a consistent naming (using a dash, all lowercase).
We also add a name which is used by Maven when displaying some information about the repository.

This naming is also consistent with [elasticsearch-parent project](https://github.com/elasticsearch/elasticsearch-parent) which will be used in the next future in 1.x and master branch.

**Important note**: If you have trouble to compile elasticsearch or a plugin using `mvn compile` and hit a `Access denied to: [URL_HERE], ReasonPhrase: Forbidden. -> [Help 1]`, you can remove related maven files:

```sh
find ~/.m2/repository -name _remote.repositories -exec rm -v {} \;
find ~/.m2/repository -name _maven.repositories -exec rm -v {} \;
```

Another option is to tell Maven not using those files with `--llr`:

```sh
mvn compile --llr
```
2015-02-17 20:39:04 +01:00
Alexander Reelsen 5092904885 Packaging: Add antlr and asm dependencies 2015-02-13 23:55:35 +01:00
Victor Buldakov f500293ffb Dependencies: Update joda-time to v2.7
Closes #9338
Closes #9610
2015-02-12 15:06:18 +01:00
Alexander Reelsen 9cd14a5c29 CliTool: Add command to warn on permission/owner change
When using the CLI tool infrastructure, a command can potentially write
a new file. In case it overwrites an existing one, you may want to ensure
that the permissions, the owner and the group are kept the same and do not
accidentally change when overwriting those files.

This PR introduces a command that allows you to execute this check per path.

It also adds a new testing dependency, namely jimfs, which allows you to create
in-memory filesystems with certain properties (like supporting or not posix permissions
on this filesystem), so that you can test those features, without executing
tests on a certain operating system.
2015-02-12 10:10:11 +01:00
Robert Muir 9c9b5c27d3 Upgrade to Lucene r1657571.
Closes #9587

Squashed commit of the following:

commit 23ac91dca4b949638ca1d3842fd6db2e00ee1d36
Author: Adrien Grand <jpountz@gmail.com>
Date:   Thu Feb 5 18:42:28 2015 +0100

    Do not compute scores if aggregations do not need it (like top_hits) or use a script (which might compute scores).

commit 51262fe2681c067337ca41ab88096ef80a2e8ebb
Author: Adrien Grand <jpountz@gmail.com>
Date:   Thu Feb 5 15:58:38 2015 +0100

    Fix more compile errors.

commit a074895d55b8b3c898d23f7f5334e564d5271a56
Author: Robert Muir <rmuir@apache.org>
Date:   Thu Feb 5 09:31:22 2015 -0500

    fix a few more obvious ones

commit 399c41186cb3c9be70107f6c25b51fc4844f8fde
Author: Robert Muir <rmuir@apache.org>
Date:   Thu Feb 5 09:28:32 2015 -0500

    fix some collectors and queries

commit 5f46c2f846c5020d5749233b71cbe66ae534ba51
Author: Robert Muir <rmuir@apache.org>
Date:   Thu Feb 5 09:24:24 2015 -0500

    upgrade to lucene r1657571
2015-02-06 08:53:20 -05:00
Michael McCandless e29cf903c8 Core: upgrade to Lucene snapshot r1656366
* IndexWriter deadlock and DV update concurrency fix
  * BytesRef reuse bug with SortedSetDVTermsEnum
  * Int overflow skip data corruption bug
  * Compound file API cleanups
  * IndexWriter doesn't accept per-doc Analyzer anymore

Closes #9524
2015-02-02 13:37:45 -05:00
Lee Hinman 6c27f1242a Update Groovy dependency to 2.4.0 2015-02-02 09:27:41 -07:00
Robert Muir be3e60efc8 Upgrade to lucene r1654549 snapshot.
Closes #9402.

Squashed commit of the following:

commit 85c71b6478441a73738c81f02257193f9837f3ba
Author: Robert Muir <rmuir@apache.org>
Date:   Sat Jan 24 11:24:36 2015 -0500

    upgrade to lucene r1654549 snapshot
2015-01-25 15:01:45 -05:00