Commit Graph

435 Commits

Author SHA1 Message Date
Colin Goodheart-Smithe b3723cdb28 Fix to let eclipse ignore the antigun plugin in it's maven integration
This plugin should be ignored as it will make the internal eclipse build fail when there are NOCOMMIT comments in files, which are expected during feature development. This change only affects eclipse users and only when they are using the m2e eclipse integration
2015-04-27 12:50:11 +01:00
Alexander Reelsen f64739788b Build: Update package repositories when creating a release
In order to automatically sign and and upload our debian and RPM
packages, this commit incorporates signing into the build process
and adds the necessary steps to the release process. In order to do this
the pom.xml has been adapted and the RPM and jdeb maven plugins have been
updated, so the packages are signed on build. However the repositories
need to signed as well.

Syncing the repos requires downloading the current repo, adding
the new packages and syncing it back.

The following environment variables are now required as part of the build

* GPG_KEY_ID - the key ID of the key used for signing
* GPG_PASSPHRASE - your GPG passphrase
* S3_BUCKET_SYNC_TO: S3 bucket to sync new repo into

The following environment variables are optional

* S3_BUCKET_SYNC_FROM: S3 bucket to get existing packages from
* GPG_KEYRING - home of gnupg, defaults to ~/.gnupg

The following command line tools are needed

* createrepo (creates RPM repositories)
* expect (used by the maven rpm plugin)
* apt-ftparchive (creates DEB repositories)
* gpg (signs packages and repo files)
* s3cmd (syncing between the different S3 buckets)

The current approach would also work for users who want to run their
own repositories, all they need to change are a couple of environment
variables.

Minor implementation detail: Right now the branch name is used as version
for the repositories (like 1.4/1.5/1.6) - if we ever change our branch naming
scheme, the script needs to be fixed.
2015-04-26 19:05:47 +02:00
Tanguy Leroux f7d4baacfb Remove working directory
This commit removes the working directory and its associated environment variable "WORK_DIR"
2015-04-25 13:08:36 +02:00
Ryan Ernst f27f5aaa22 Upgrade to lucene-5.2-snapshot-1675927 2015-04-24 12:38:32 -07:00
Lee Hinman 54cf885d14 Disable security manager when running with `mvn exec:exec` 2015-04-24 11:00:54 -06:00
Robert Muir 500c956b45 Remove policy config file, its a resource.
Remove exposed boolean to turn off security.
Add unit test
2015-04-23 22:02:57 -04:00
Robert Muir 7b6e470f5d Merge branch 'master' into put_me_in_coach 2015-04-23 08:34:27 -04:00
Robert Muir a0414599a8 Merge pull request #10711 from rmuir/java_home_only
Execute tests with $JAVA_HOME.
2015-04-22 14:51:36 -04:00
Adrien Grand dd679a3a38 Upgrade to lucene-5.2-snapshot-1675363.
This snapshot contains in particular LUCENE-6446 (refactored explanation API)
and LUCENE-6448 (better equals/hashcode for filters).
2015-04-22 17:23:17 +02:00
Robert Muir 9406d83441 fix typo 2015-04-22 08:46:14 -04:00
Robert Muir 270cb9f349 enable securitymanager 2015-04-22 03:04:50 -04:00
Robert Muir 730314fec1 Execute tests with $JAVA_HOME. 2015-04-21 20:27:27 -04:00
Robert Muir 9d6b1382e7 Fix JVM isolation in tests.
Currently security manager would allow for one JVM to muck
with the files (read, write, AND delete) of another JVM.

This is unnecessary.
2015-04-21 19:02:14 -04:00
Robert Muir d1c3ec6291 Upgrade to Lucene 5.2 r1675100
This upgrade is for https://issues.apache.org/jira/browse/LUCENE-6442

It should improve test reproducibility, especially if you are on a mac
and want to reproduce a jenkins failure that happened on linux.
2015-04-21 09:06:44 -04:00
Tanguy Leroux 867955188e Standardization of packages structure and install
The existing DEB/RPM packages have a lot of differences: they don't execute the same actions when installing or removing the package. They also don't declare exactly the same environment variables at the same place. At the end of the day the global behavior and configuration is *almost* the same but it's very difficult to maintain the scripts.

This commits unifies the package behavior:
- DEB/RPM use the same package scripts (pre installation, post installation etc) in order to execute exactly the same actions
- Use of a unique environment vars file that declares everything needed by scripts (the goal is to delete vars declaration in init.d and systemd scripts, this will be done in another PR)
- Variables like directory paths are centralized and replaced according to the target platform (using #10330)
- Move /etc/rc.d/init.d to standard /etc/init.d (RPM only)
- Add PID_DIR env var
- Always set ES_USER, ES_GROUP,MAX_MAP_COUNT and MAX_OPEN_FILES in env vars file
- Create log, data, work and plugins directories with DEB/RPM packaging system
- Change to elastic.co domain in copyright and control files
- Add Bats files to automate testing of DEB and RPM packages
- Update TESTING.asciidoc

More info on Bats here:  https://github.com/sstephenson/bats
2015-04-20 14:58:19 +02:00
Robert Muir 069e11ba40 set heartbeat to 10s 2015-04-19 18:44:14 -04:00
Robert Muir c153772fca ensure these two versions are always in sync 2015-04-19 09:12:15 -04:00
Robert Muir b728772fe4 more fine-grained @slow tuning, remove from many tests that got unlucky with fsync 2015-04-18 19:59:29 -04:00
Robert Muir c00f0ff08e upgrade to lucene r1674576 2015-04-18 16:04:28 -04:00
Robert Muir 5bcd599cb3 remove repo, latest randomizedtesting is on maven central now 2015-04-18 12:25:39 -04:00
Robert Muir 96f08a38e6 parallelize rest tests 2015-04-17 23:38:01 -04:00
Robert Muir 43b6cd2118 Merge branch 'master' into mockfilesystem 2015-04-17 19:15:25 -04:00
Robert Muir e71553556e remove tests.processors, this is a reproducibility nightmare 2015-04-17 18:32:52 -04:00
Robert Muir e3e4c02379 nobody wants to look at bytecode 2015-04-17 17:16:03 -04:00
Ryan Ernst 7faa9a045a Change nocommit checks to not happen on intellij files (where there can
be a TODO task named eg Nocommits).
2015-04-17 12:54:15 -07:00
Robert Muir 5718e5616a fail the build if you typo test name 2015-04-17 09:45:38 -04:00
Robert Muir 7afa241e19 make stacktraces reasonable 2015-04-17 09:22:28 -04:00
Robert Muir c421948300 upgrade to lucene 5.2 r1674278 2015-04-17 07:58:20 -04:00
Adrien Grand fd416d5ed5 Upgrade to Lucene-5.2-snapshot-1674183. 2015-04-17 10:05:30 +02:00
Robert Muir 41d0e60dc7 Add tests progress indicator
Looks like this:

    Suite: org.elasticsearch.search.aggregations.bucket.HistogramTests
    Completed [10/713] on J0 in 8.32s, 27 tests

    Suite: org.elasticsearch.indices.state.OpenCloseIndexTests
    Completed [11/713] on J2 in 16.23s, 20 tests

Upgrades randomizedtesting to 2.1.13.
2015-04-16 13:29:51 -04:00
Ryan Ernst a3f078985b Tests: Forbid tests from writing to CWD
Allowing tests writing to the working directory can mask problems.
For example, multiple tests running in the same jvm, and using the
same relative path, may cause issues if the first test to run
leaves data in the directory, and the second test does not remember
to cleanup the path before using it.

This change adds security manager rules to disallow tests writing
to the working directory. Instead, tests create a temp dir with
the existing test framework.

closes #10605
2015-04-15 12:45:20 -07:00
Robert Muir b1afcdb71d upgrade to 5.2 r1673726 2015-04-15 10:28:31 -04:00
Adrien Grand 22720a1abd Merge pull request #10562 from jpountz/upgrade/lucene-5.2.0-snapshot-1673124
Upgrade to lucene-5.2.0-snapshot-1673124.
2015-04-14 16:57:09 +02:00
Simon Willnauer c13e604697 [BUILD] Restrict read permission to project.basedir
This prevents reads from anywhere outside of the elasticsearch
clone when running tests with security manager enabled.
2015-04-13 16:44:31 +02:00
Adrien Grand 45afa47a6f Upgrade to lucene-5.2.0-snapshot-1673124. 2015-04-13 10:43:58 +02:00
Simon Willnauer d738b3d1e2 [BUILD] Use SuppressFrobidden annotation instead of class level excludes
Forbidden APIs 1.8 allows excludes based on annotations which can now be on methods etc. for more find grained control.

Closes #10560
2015-04-13 10:08:50 +02:00
Uwe Schindler df118214c4 Update forbiddenapis to version 1.8 2015-04-13 01:27:56 +02:00
David Pursehouse 97a9b4ec2f Fix m2e Eclipse Integration
Update Eclipse core prefs

Eclipse Luna overwrites the prefs file, putting all the settings
in alphabetical order and removing comments. This causes the prefs
files to be modified in the git workspace.

Update the file with the version generated by Eclipse to prevent it
from being modified every time.

No settings values are modified by this change.

This also  adds another plugin to the lifecycle mapping in the pom.xml which was missed in https://github.com/elastic/elasticsearch/pull/10524.
2015-04-10 16:48:04 +01:00
Simon Willnauer bdc4c7f84c [BUILD] Add sigar binaries when running unittests
the sigar binaries are not available when running tests today. This
commit adds the path to the test run.
2015-04-10 14:37:18 +02:00
Colin Goodheart-Smithe 7e353e0768 removed erroneous tab character 2015-04-10 10:37:26 +01:00
Colin Goodheart-Smithe 4a09a98561 Fix to pom.xml to allow eclipse maven integration using m2e 2015-04-10 08:34:01 +01:00
Tanguy Leroux e60f61d0a9 Fix maven-resources-plugin warnings
Commit 168238dab6 declared multiple maven-resources-plugin usages instead of declaring multiple executions for the same plugin... resulting to Maven warnings.

Closes #10433
2015-04-09 10:05:10 +02:00
Adrien Grand 89b97c1c8d Upgrade to lucene-5.1.0-snapshot-1671894.
This includes a fix for LUCENE-6406.
2015-04-07 20:04:54 +02:00
Robert Muir 1d56502e52 Upgrade to Lucene 5.1 r1671277.
Closes #10435.

Squashed commit of the following:

commit aa1935c790b2731fc2bbc7de6142b09e3fe8bd4a
Author: Ryan Ernst <ryan@iernst.net>
Date:   Mon Apr 6 13:44:40 2015 -0700

    fix index lookup

commit bb6373595ff62ffc56fdf0cba3ac9c0ebe679946
Merge: 916962b eb3a170
Author: Robert Muir <rmuir@apache.org>
Date:   Mon Apr 6 14:24:38 2015 -0400

    Merge branch 'lucene_r1671277' of github.com:elasticsearch/elasticsearch into lucene_r1671277

commit 916962b82d192a53add471b4cc4a1396bc30eb0e
Merge: 197b3a2 21f72fe
Author: Robert Muir <rmuir@apache.org>
Date:   Mon Apr 6 07:09:41 2015 -0400

    Merge branch 'master' into lucene_r1671277

commit eb3a1703f7932ddd0cf3e83bec0e86131d255407
Author: Ryan Ernst <ryan@iernst.net>
Date:   Sat Apr 4 11:06:03 2015 -0700

    re-enable index lookup tests

commit 80d65d5eab39062dd8364687da74ddbb87ebcb76
Author: Ryan Ernst <ryan@iernst.net>
Date:   Sat Apr 4 10:39:52 2015 -0700

    update pom to point to new snapshot repo

commit 197b3a21ac2c2d70c9f740fe53e58632a22d1aad
Author: Robert Muir <rmuir@apache.org>
Date:   Sat Apr 4 12:51:22 2015 -0400

    fix postingsenum usage

commit 0e2b7a00cd07d068f755c51185ac521aa1eb0326
Author: Robert Muir <rmuir@apache.org>
Date:   Sat Apr 4 12:21:23 2015 -0400

    upgrade to lucene r1671277 (have not yet run tests or looked at postings changes)
2015-04-07 06:09:35 -04:00
Tanguy Leroux 168238dab6 Add properties files to configure startup and installation scripts
Many scripts are used to start/stop and install/uninstall elasticsearch. These scripts share a lot of configuration properties like directory paths, max value for a setting, default user etc. Most of the values are identical but some of them are different depending of the platform (Debian-based or Redhat-based OS), depending of the way elasticsearch is started (shell script, systemd, sysv-init...) or the way it is installed (zip, rpm, deb...). Today the values are duplicated in multiple places, making it difficult to maintain the scripts or to update a value.

This pull request make this more uniform: values used in scripts must be defined in a common packaging.properties file. Each value can be overridden in another specific packaging.properties file for Debian or Redhat. All startup and installation scripts are filtered with the common then the custom packaging.properties files before being packaged as a zip/tar.gz/rpm/dpkf archive.
2015-04-02 18:09:43 +02:00
Lee Hinman 1cb169afb4 Add ability to launch bootstrapped ES using maven
This adds the exec-maven-plugin that allows a developer to run:

```
mvn exec:exec
```

To launch the `Bootstrap` process similar to the way that a Java IDE
would. All the logs go to logs/elasticsearch.log (or wherever
configured)
2015-03-30 11:07:21 -06:00
Simon Willnauer c87e1b41a4 [TEST] only use perm gen space on java 1.7.x 2015-03-24 10:04:32 +01:00
Simon Willnauer c017a400d5 [TEST] Add back permGen space tuning for now 2015-03-24 09:38:34 +01:00
Robert Muir 10e6fa1963 fix build with java 9
* add compiler workarounds for JDK bug JI-9019884
* remove permgen specification during tests (this results in an error on java 9)
* fix threadpool grow/shrink to call methods in the right order (this results in IAE with java 9)
2015-03-24 00:29:24 -04:00
Shay Banon 9f9ada4b7b Upgrade to Jackson 2.5.1
Note, Jackson 2.5 is less lenient when it comes to not starting an object before starting to add fields on a fresh builder, fixed where applicable.
closes #10210
2015-03-23 09:03:27 +01:00