Commit Graph

757 Commits

Author SHA1 Message Date
Adrien Grand abe3880dd9 Merge pull request #12274 from smflorentino/smflorentino-fix-winjvm-detect
Improve JVM Arch Detection
2015-08-11 16:17:26 +02:00
Nik Everett 3c76e15aab Merge pull request #12785 from nik9000/remove_conf
Fix rpm -e removing /etc/elasticsearch
2015-08-11 05:08:07 -07:00
Nik Everett 7223f7c75a Packaging: fix rpm -e removing /etc/elasticsearch
When we fixed rpm creating the /etc/elasticsearch/scripts directory we
broke removing the rpm - it lef the /etc/elasticsearch directory behind.
This fixes that.
2015-08-10 14:32:41 -07:00
Ryan Ernst 40f119d85a This method on settings loaded a class, based on a setting value, using
the default classloader. It had all kinds of leniency in how the
classname was found, and simply cannot work with plugins having isolated
classloaders.

This change removes that method. Some of the uses of it were for custom
extension points, like custom repository or discovery types. A lot were
just there to plugin mock implementations for tests. For the settings
that were legitimate, all now support plugins adding the given setting
via onModule. For those that were specific to tests for mocks, they now
use Classes.loadClass (a helper around Class.forName). This is a
temporary measure until (in a future PR) tests can change the
implementation via package private statics.

I also removed a number of unnecessary intermediate modules, added a
"jvm-example" plugin that can be filled in in the future as a smoke test
for breaking plugins, and gave some documentation to "spawn" modules
interface.

closes #12643
closes #12656
2015-08-10 14:04:45 -07:00
Scott Florentino e7aa2a1fb2 Closing #12256 Update service.bat with a safer RAM value 2015-08-10 13:22:29 -07:00
Colin Goodheart-Smithe 6abc69c488 Packaging: Makes sure all POMs contain a description
Adds an explicit description the RPM package so it doesn't inherit the description from the POM.

Closes #12550

Also, modified descriptions for deb and rpm packages to be the same and to reference the documentation rather than listing features that are out of date.
2015-08-10 15:21:24 +01:00
Nik Everett cde2502732 Merge pull request #12706 from nik9000/osx_can_build_rpm
Let OSX build rpms for linux
2015-08-10 05:48:36 -04:00
Nik Everett f84552dc98 [Packaging] Run tests in vagrant
This creates a module in qa called vagrant that can be run if you have
vagrant and virtualbox installed and will run the packaging tests in trusty
and centos-7.0. You can ask it to run tests in other linuxes. This is the full
list:
* precise aka Ubuntu 12.04
* trusty aka Ubuntu 14.04
* vivid aka Ubuntun 15.04
* wheezy aka Debian 7, the current debian oldstable distribution
* jessie aka Debian 8, the current debina stable distribution
* centos-6
* centos-7
* fedora-22
* oel-7

There is lots of documentation on how to do this in the TESTING.asciidoc.

Closes #12611
2015-08-10 05:31:42 -04:00
Nik Everett d6087667bf Merge pull request #12704 from nik9000/create_scripts
rpm and deb create scripts directory
2015-08-10 05:27:45 -04:00
Tanguy Leroux b1fd0a61da Do not kill process on service shutdown
When installed as a service with a DEB or RPM package, we should gently wait for elasticsearch to stop (flushing indices on closing can take some time) and never kill the process.

Closes #11248
2015-08-10 10:04:56 +02:00
Clinton Gormley 205262c737 Enable the license checker over distribution/* and plugins/*
Moved the license checker config into the parent pom, and overrede
the license dir/target-to-check in distributions/pom.

Disabled the license checker explicitly for projects which run integration
tests but have no licenses dir:

* core
* distribution
* qa
* plugins/delete-by-query
* plugins/mapper-size
* plugins/site-example

Closes #12752
Closes #12754
2015-08-09 18:18:59 +02:00
Clinton Gormley 7efc419041 Revert "Revert "disable license checker on windows for now, jenkins boxes have perl problems""
This reverts commit e2cbec185b.
2015-08-07 16:47:52 +02:00
Alexander Reelsen b6016b6e1e Bats testing: Remove useless systemctl check
Bats testing uncovered a useless systemctl check, that resulted in an
error, because the systemctl file was uninstalled, but we hoped to
check for an explicetely configured SystemExitCode.

In addition we did not reload the systemctl configuration when uninstalling
elasticsearch, which now is fixed as well.

Closes #12682
2015-08-07 14:42:38 +02:00
Clinton Gormley e2cbec185b Revert "disable license checker on windows for now, jenkins boxes have perl problems"
This reverts commit 6e9d8671bc.
2015-08-07 14:27:24 +02:00
Nik Everett 8e6e8ff234 Let OSX build rpms for linux
Closes #12701
2015-08-07 08:11:34 -04:00
Nik Everett f1fc2efcc8 Packaging: rpm and deb create scripts directory
Elasticsearch will create this if it doesn't exist if it cant but because
it doesn't own /etc/elasticsearch when installed by rpm and deb it can't
create /etc/elasticsearch/scripts.

Closes #12702
2015-08-07 08:10:53 -04:00
Robert Muir f50531c403 Revert "Revert "disable license checker on windows for now, jenkins boxes have perl problems""
This reverts commit 633b4868d9.
2015-08-06 23:38:24 -04:00
Robert Muir 633b4868d9 Revert "disable license checker on windows for now, jenkins boxes have perl problems"
This reverts commit 6e9d8671bc.

Lets see if they are up for the task now.
2015-08-06 22:17:45 -04:00
Robert Muir 6e7d1a828d fix totally broken batch file. thank you jenkins. windows you still suck. 2015-08-06 18:27:06 -04:00
Robert Muir 6e9d8671bc disable license checker on windows for now, jenkins boxes have perl problems 2015-08-06 18:01:04 -04:00
Robert Muir 7414d19d28 Support jenkins randomization in integration tests 2015-08-06 14:48:27 -04:00
Robert Muir 354a32b05d remove annoying maven warning 2015-08-05 22:08:12 -04:00
Robert Muir 5fe49e4420 cleanup skipping everywhere 2015-08-05 21:45:09 -04:00
Alexander Reelsen 9028fb8fbe Release: Update build release script to reflect latest changes
As the script now deploys to S3 and several things in master have
changed, this script needs to reflect the latest changes

* An unsigned RPM is built by default, so that users of older
  RPM based distros can download and use that RPM by default
* In addition a signed RPM is built, that is used for the repositories
* Paths for the new distributions have been fixed
* The check for the number of jars has been removed, as this is done
  as part of the license checking in `mvn verify`
* Checksum generation has been removed, as this is done as part of the
  mvn build
* Publishing artifacts of S3 has been removed
* Repostitory creation script has been updated
2015-08-05 09:38:42 +02:00
Robert Muir 6f9a067197 Change master branch back to 2.0-beta1 2015-08-04 15:38:21 -04:00
Robert Muir 29d73accdd Merge pull request #12639 from rmuir/improve_integ
Don't use port 9200/9300 for integration tests
2015-08-04 13:09:47 -04:00
David Pilato c61dccd189 Packaging: mvn install renames artifacts when copying
This PR:

* renames all distribution artifacts to `elasticsearch` so maven plugins will pick up the correct finalName without needing any hack.
* changes the groupId for every single distribution module as we can't have more than one module using the same groupId:artifactId
* does not attach anymore empty jar files for tar/zip/... modules as they don't contain any `src/main/java` stuff.

When you build it, you end up with:

```
$ tree ~/.m2/repository/org/elasticsearch/distribution
distribution
├── deb
│   └── elasticsearch
│       ├── 2.0.0-beta1-SNAPSHOT
│       │   ├── elasticsearch-2.0.0-beta1-SNAPSHOT.deb
│       │   ├── elasticsearch-2.0.0-beta1-SNAPSHOT.deb.md5
│       │   ├── elasticsearch-2.0.0-beta1-SNAPSHOT.deb.sha1
│       │   ├── elasticsearch-2.0.0-beta1-SNAPSHOT.pom
│       │   ├── elasticsearch-2.0.0-beta1-SNAPSHOT.pom.md5
│       │   └── elasticsearch-2.0.0-beta1-SNAPSHOT.pom.sha1
├── elasticsearch-distribution
│   ├── 2.0.0-beta1-SNAPSHOT
│   │   ├── elasticsearch-distribution-2.0.0-beta1-SNAPSHOT.pom
│   │   ├── elasticsearch-distribution-2.0.0-beta1-SNAPSHOT.pom.md5
│   │   └── elasticsearch-distribution-2.0.0-beta1-SNAPSHOT.pom.sha1
├── fully-loaded
│   └── elasticsearch
│       ├── 2.0.0-beta1-SNAPSHOT
│       │   ├── elasticsearch-2.0.0-beta1-SNAPSHOT.pom
│       │   ├── elasticsearch-2.0.0-beta1-SNAPSHOT.pom.md5
│       │   └── elasticsearch-2.0.0-beta1-SNAPSHOT.pom.sha1
├── rpm
│   └── elasticsearch
│       ├── 2.0.0-beta1-SNAPSHOT
│       │   ├── elasticsearch-2.0.0-beta1-SNAPSHOT.pom
│       │   ├── elasticsearch-2.0.0-beta1-SNAPSHOT.pom.md5
│       │   ├── elasticsearch-2.0.0-beta1-SNAPSHOT.pom.sha1
│       │   ├── elasticsearch-2.0.0-beta1-SNAPSHOT.rpm
│       │   ├── elasticsearch-2.0.0-beta1-SNAPSHOT.rpm.md5
│       │   └── elasticsearch-2.0.0-beta1-SNAPSHOT.rpm.sha1
├── shaded
│   └── elasticsearch
│       ├── 2.0.0-beta1-SNAPSHOT
│       │   ├── elasticsearch-2.0.0-beta1-SNAPSHOT.jar
│       │   ├── elasticsearch-2.0.0-beta1-SNAPSHOT.jar.md5
│       │   ├── elasticsearch-2.0.0-beta1-SNAPSHOT.jar.sha1
│       │   ├── elasticsearch-2.0.0-beta1-SNAPSHOT.pom
│       │   ├── elasticsearch-2.0.0-beta1-SNAPSHOT.pom.md5
│       │   └── elasticsearch-2.0.0-beta1-SNAPSHOT.pom.sha1
├── tar
│   └── elasticsearch
│       ├── 2.0.0-beta1-SNAPSHOT
│       │   ├── elasticsearch-2.0.0-beta1-SNAPSHOT.pom
│       │   ├── elasticsearch-2.0.0-beta1-SNAPSHOT.pom.md5
│       │   ├── elasticsearch-2.0.0-beta1-SNAPSHOT.pom.sha1
│       │   ├── elasticsearch-2.0.0-beta1-SNAPSHOT.tar.gz
│       │   ├── elasticsearch-2.0.0-beta1-SNAPSHOT.tar.gz.md5
│       │   └── elasticsearch-2.0.0-beta1-SNAPSHOT.tar.gz.sha1
└── zip
    └── elasticsearch
        └── 2.0.0-beta1-SNAPSHOT
            ├── elasticsearch-2.0.0-beta1-SNAPSHOT.pom
            ├── elasticsearch-2.0.0-beta1-SNAPSHOT.pom.md5
            ├── elasticsearch-2.0.0-beta1-SNAPSHOT.pom.sha1
            ├── elasticsearch-2.0.0-beta1-SNAPSHOT.zip
            ├── elasticsearch-2.0.0-beta1-SNAPSHOT.zip.md5
            ├── elasticsearch-2.0.0-beta1-SNAPSHOT.zip.sha1
            └──
```

Closes #12536
2015-08-04 15:34:11 +02:00
Robert Muir cf6fe6cdcd Don't use port 9200/9300 for integration tests 2015-08-04 09:25:44 -04:00
Nik Everett e8ccdf91fc Merge pull request #12610 from nik9000/bug/12504_2.0
Plugin script: Fix spaces
2015-08-04 09:05:38 -04:00
Robert Muir 5059c978e3 Fix distribution checks to pass unzipped directory to license checker 2015-08-04 07:54:55 -04:00
Simon Willnauer 6753f7f03e Cut over master to 2.0.0-SNAPSHOT 2015-08-04 10:54:12 +02:00
Robert Muir c059101fbb Fix missed test class 2015-08-03 21:56:43 -04:00
Ryan Ernst 1e12d03252 Tests: Rename base tests cases to use "TestCase" suffix
Most of the abstract base test classes we have were previously @Ignored.
However, there were also some other tests ignored. Having two ways to
quiet tests is confusing, and clearly it has caused some tests
to get lost in the fold.

This change moves all base test classes to use the "TestCase" suffix,
which is not picked up by the test class name pattern. It also removes
@Ignore from (almost) all tests, and adds it to forbidden apis.
And since we were renaming, I shorted base test class names to use
"ES" instead of "Elasticsearch". I type this a lot of types a day,
and I have heard others express a similar desire for a shorter name.

closes #10659
2015-08-03 17:43:00 -07:00
Scott Florentino 21a7518271 Merge branch 'master' into smflorentino-fix-winjvm-detect 2015-08-03 17:26:25 -07:00
Nik Everett 1b5bf17ae0 Plugin script: Fix spaces
Fixes ES_HOME with spaces and installing plugins from a local directory
with spaces.

Closes #12504
2015-08-03 15:40:18 -04:00
Robert Muir 0e027d82c9 Merge pull request #12602 from rmuir/core_integ_tests
Add integration test phase for elasticsearch core/
2015-08-03 10:15:37 -04:00
Robert Muir 379ffdb8ee Add integration test phase for elasticsearch core/ 2015-08-03 09:43:11 -04:00
Robert Muir e1886a9b12 Test that shaded jar contains only shaded classes 2015-08-03 08:56:45 -04:00
Robert Muir a5a27a5cd9 Add jar hell check test for the shaded jar 2015-08-03 05:54:55 -04:00
Robert Muir aaf0976cb7 clear up dependency hell a bit more 2015-08-01 14:02:14 -04:00
Robert Muir eaaa06b6b3 fix shading to create correct POM and not drag in wacky dependencies 2015-08-01 12:44:00 -04:00
Robert Muir 6f33ea4f62 move shading to distribution/shaded 2015-08-01 11:37:43 -04:00
Robert Muir fef55348f9 clean up .zip handling so its not uselessly downloaded in the build 2015-08-01 09:10:03 -04:00
Robert Muir 8d5b5ad862 restore my lost code changes and add measures to detect/prevent this in the future 2015-07-31 13:00:24 -04:00
Robert Muir 3d7da5a826 remove unused properties sections from distribution modules 2015-07-31 10:20:50 -04:00
Robert Muir f804ec3cd9 cleanup 2015-07-31 10:05:11 -04:00
Robert Muir 2068fbfd1e Merge pull request #12549 from rmuir/distribution_tests_1
add integration tests for tar/deb/rpm packaging
2015-07-30 09:18:09 -04:00
Alexander Reelsen 69d7f1a78a Startup: Remove getopt parsing in shell script, use java CLITool
In order to ensure, we have the same experience across operating systems
and shells, this commit uses the java CLI parser instead of the shell
getopt parsing to parse arguments.

This also allows for support for paths, which contain spaces.

Also commons-cli depdency was upgraded to 1.3.1 and tests have been added.

Changes

* new exit code, OK_AND_EXIT, allowing to tell the caller to exit, as everything
  went as expected (e.g. when running a version output)

BWC breaking:

* execute() returns an ExitStatus instead of an integer, otherwise there is no
  possibility to signal by a command, if the JVM should be exited after a run.
  This affects plugins, that have command line tools
* -v used to be version, but is a verbose flag by default in the current CLI infra,
  must be -V or --version now
* -X has been removed - the current implementation was useless anyway, as
  it prefixed those properties with "es.". You should use
  ES_JAVA_OPTS/JAVA_OPTS for JVM configuration
2015-07-30 13:20:29 +02:00
Adrien Grand 6893643bef Build: Set the correct sha1 for HDRHistogram. 2015-07-30 10:24:55 +02:00
Robert Muir ddfea366de Add integration tests for distribution/rpm 2015-07-29 22:26:45 -04:00
Robert Muir 2ed8e697d5 Add integration tests for distribution/deb 2015-07-29 21:34:46 -04:00
Robert Muir 1c5a558164 Add integration tests for distribution/tar 2015-07-29 15:33:29 -04:00
Robert Muir c6d6eaef4d Add a run.sh to run from current source code with debugger
The maven magic needed for this is now heavier
2015-07-29 14:21:43 -04:00
Alexander Reelsen c423319dda Build: Remove profile to create attached RPM
This is no longer necessary, as the RPM is now built as
a primary artifact, so no need to deploy it as a secondary
artifact anymore.

Closes #12529
2015-07-29 15:54:26 +02:00
Robert Muir c6abf28273 Disable license checker temporarily to get regression builds working again
x
2015-07-29 08:39:25 -04:00
Chris Earle e1c99bc8b4 Add -XX:+PrintGCDateStamps when using GC Logs
Now that Elasticsearch requires Java 7 or later, it's safe to add `-XX:+PrintGCDateStamps` to get human readable times alongside JVM times.
2015-07-27 15:17:54 -04:00
Alexander Reelsen 9628d2632f Build: Split packages into submodules
This change creates a proper `distribution` modules in which we have today packaging for
all of our four current packages:

* zip
* tar.gz
* rpm
* deb

Licenes have moved into the distribution project as well. So have the config/ and the bin/ directory
from the core/ project.

The RPM package is now built, if rpmbuild exists.

The bats tests have been moved as well.

Also the zip distribution now executes the REST integration tests.
2015-07-27 17:50:54 +02:00