Commit Graph

260 Commits

Author SHA1 Message Date
Luca Cavanna 63cbc84393 removed rest-spec submodule and prepared project for same files added directly to the codebase (no submodule) within rest-api-spec
(temporarily disabled FileUtilsTests & REST tests as there's temporarily no rest-spec dir)

Relates to #4540 #4376
2013-12-27 20:36:12 +01:00
Shay Banon e5f52ce778 update to netty 3.9.0 2013-12-23 20:06:45 +01:00
Luca Cavanna d97a00d4a7 added REST test suites runner
The REST layer can now be tested through tests that are shared between all the elasticsearch official clients.
The tests are based on REST specification that can be found on the elasticsearch-rest-api-spec project and consist of YAML files that describe the operations to be executed and the obtained results that need to be tested.

REST tests can be executed through the ElasticsearchRestTests class, which relies on the rest-spec git submodule that contains the rest spec and tests pulled from the elasticsearch-rest-spec-api project. The rest-spec submodule gets automatically initialized and updated through maven (generate-test-resources phase).

The REST runner and the needed classes are distributed within the test artifact.

The following are the options supported by the REST tests runner:

- tests.rest[true|false|host:port]: determines whether the REST tests need to be run and if so whether to rely on an external cluster (providing host and port) or fire a test cluster (default)
- tests.rest.suite: comma separated paths of the test suites to be run (by default loaded from /rest-spec/test classpath). it is possible to run only a subset of the tests providing a sub-folder or even a single yaml file (the default /rest-spec/test prefix is optional when files are loaded from classpath) e.g. -Dtests.rest.suite=index,get,create/10_with_id
- tests.rest.spec: REST spec path (default /rest-spec/api from classpath)
- tests.iters: runs multiple iterations
- tests.seed: seed to base the random behaviours on
- tests.appendseed[true|false]: enables adding the seed to each test section's description (default false)
- tests.cluster_seed: seed used to create the test cluster (if enabled)

Closes #4469
2013-12-17 15:36:16 +01:00
Simon Willnauer 8f85d63b67 Enable incremental compilation using a workaround for the maven-compiler-plugin 3.1 bug 2013-12-14 21:56:01 +01:00
Shay Banon 59d85bc22a upgrade to Jackson 2.3.0 2013-12-14 00:53:52 +01:00
Alexander Reelsen 81e13a870b Packaging: Ensure setting of sysctl vm.max_map_count
In order to be sure that memory mapped lucene directories are working
one can configure the kernel about how many memory mapped areas
a process may have. This setting ensure for the debian and redhat initscripts
as well as the systemd startup, that this setting is set high enough.

Closes #4397
2013-12-11 09:19:22 +01:00
Simon Willnauer 2b42a0f94a Override DefaultExceptionHandler to filter out certain exceptions
We have the situation that some tests fail since they don't handle
EsRejectedExecutionException which gets thrown when a node shuts
down. That is ok to ignore this exception and not fail.
We also suffer from OOMs that can't create native threads but don't
get threaddumps for those failures. This patch prints the thread
stacks once we catch a OOM which can' create native threads.
2013-12-04 14:18:52 +00:00
mrsolo 3494ac252e removing test.jvm.argline declaration
<properties> declaration was overwriting environment setting.
2013-12-02 11:44:51 -08:00
Simon Willnauer 2c8ee3fbbe Moving to 1.0.0RC1 snap 2013-12-02 17:10:07 +01:00
Kevin Kluge 296cfbe390 release [1.0.0.Beta2] 2013-12-02 15:45:30 +00:00
David Pilato fa762f09fa OOM when building with java6
The maven-compiler-plugin upgrade from 2.3.2 to 3.1 (see #4279) could cause out of memory issue when building the project with Maven and JDK6 and default memory settings (no `MAVEN_OPTS`).

This issue does not appear with JDK7.
2013-11-28 18:22:29 +01:00
David Pilato baaa1a6aa2 Upgrade Maven Surefire Plugin to 2.16
Closes #4275.
2013-11-28 14:56:51 +01:00
David Pilato 90c3fce4fe Upgrade Maven Source Plugin to 2.2.1
Closes #4276.
2013-11-28 14:56:34 +01:00
David Pilato 975c43f6d5 Upgrade RPM Maven Plugin to 2.1-alpha-3
Closes #4282.
2013-11-28 14:56:20 +01:00
David Pilato d18f0a28d9 Upgrade Maven Resources Plugin to 2.6
Closes #4280.
2013-11-28 14:56:07 +01:00
David Pilato fa0a5e5844 Upgrade Maven Jar Plugin to 2.4
Closes #4281.
2013-11-28 14:55:53 +01:00
David Pilato 2b32121e34 Upgrade Maven Eclipse Plugin to 2.9
Closes #4277.
2013-11-28 14:55:40 +01:00
David Pilato eac602ec70 Upgrade Maven Dependency Plugin to 2.8
Closes #4274.
2013-11-28 14:52:58 +01:00
David Pilato b1c62b02b0 Upgrade Maven Compiler Plugin to 3.1
Closes #4279.
2013-11-28 14:52:32 +01:00
David Pilato 79d92dffa6 Upgrade Maven Assembly Plugin to 2.4
Closes #4278.
2013-11-28 14:49:51 +01:00
David Pilato 5ec0bf32d6 Remove randomizedtesting-runner explicit dependency
Was added by mistake with #4266
2013-11-28 12:07:21 +01:00
David Pilato a9655ba812 Update to shade plugin 2.2 to shade test artifact as well
When we want to use test artifact in other projects, dependencies
are not shaded as for core artifact.

Issue opened in maven shade project: [MSHADE-158](http://jira.codehaus.org/browse/MSHADE-158)

When using it in other projects, you basically need to change your `pom.xml` file:

```xml
<dependency>
    <groupId>org.elasticsearch</groupId>
    <artifactId>elasticsearch</artifactId>
    <version>${elasticsearch.version}</version>
    <type>test-jar</type>
    <scope>test</scope>
</dependency>
```

You can also define some properties:

```xml
<properties>
    <tests.jvms>1</tests.jvms>
    <tests.shuffle>true</tests.shuffle>
    <tests.output>onerror</tests.output>
    <tests.client.ratio></tests.client.ratio>
    <es.logger.level>INFO</es.logger.level>
</properties>
```

Closes #4266
2013-11-28 10:54:48 +01:00
Simon Willnauer 5f7146aab8 Run tests with reduced stack size as we run in production 2013-11-26 11:16:02 +01:00
Simon Willnauer cf3ba7c51c Only use -Dtests.jvm.argline instead of numbered options 2013-11-26 11:11:19 +01:00
Simon Willnauer 46ab6a1533 Include JVM Arg line in reproduce line 2013-11-25 14:35:34 +01:00
Simon Willnauer 8e17d636ef Upgrade to Lucene 4.6
This commit upgrades to Lucene 4.6 and contains the following improvements:

 * Remove XIndexWriter in favor of the fixed IndexWriter
 * Removes patched XLuceneConstantScoreQuery
 * Now uses Lucene passage formatters contributed from Elasticsearch in PostingsHighlighter
 * Upgrades to Lucene46 Codec from Lucene45 Codec
 * Fixes problem in CommonTermsQueryParser where close was never called.

Closes #4241
2013-11-24 21:08:38 +01:00
Simon Willnauer eb55458e44 Upgrade RandomizedRunner Maven Plugin to 2.0.14 2013-11-22 15:00:12 +01:00
Simon Willnauer a949a8056b Upgrade to forbiddenapis 1.4 2013-11-22 08:01:30 +01:00
Simon Willnauer 00562d3a6f allow passing JVM args via env variables if not defined via the cmd 2013-11-21 17:58:43 +01:00
Simon Willnauer 02522dac06 Allow passing additional parameters to the test JVM like GC settings etc. 2013-11-21 17:39:11 +01:00
colings86 76c5f53dfa Changed pom to allow import and running from eclipse
Currently when importing projects into eclipse you need to run 'mvn
eclipse:eclipse' on the command line to generate the poject files. This
means that when the pom changes you need to re-run the command on the
command line to reflect those changes in the project in eclipse.  This
commit allows the developer to import the project as an existing maven
project (can be shared using git after import) and then allows the
application to be run inside eclipse using the .launch file in
/dev-tools enabling easy debugging of the application within eclipse
without requiring a maven build.
2013-11-21 10:56:52 +01:00
Simon Willnauer c48c8fd974 Include inner classes in the test package as well 2013-11-14 13:49:37 +01:00
Simon Willnauer 16ee742682 Cleanup test framework in order to release it as a jar file
This commit adds javadocs and removed unused methods from central
classes like ElasticsearchIntegrationTest. It also changes visibility
of many methods and classes that are only needed inside the test infrastructure.
2013-11-12 11:54:55 +01:00
Martijn van Groningen 9fdcb0ad05 Upgraded to hppc version 0.5.3 2013-11-11 14:27:52 +01:00
Shay Banon b17732ea56 upgrade to Netty 3.8.0 2013-11-09 15:41:24 +01:00
Simon Willnauer 985916038e Just filter test jar and not the main jar file 2013-11-08 23:14:56 +01:00
Simon Willnauer a824f69d21 Package test-framework as individual jar
This commit causes all classes under 'org.elasticsearch.test.*'
to be included in a 'elasticsearch-${version}-test.jar' that can be
inclued by 3rd party projects or plugins via:

```
 <dependency>
    <groupId>org.elasticsearch</groupId>
    <artifactId>elasticsearch</artifactId>
    <version>${elasticsearch.version}</version>
    <type>test-jar</type>
    <scope>test</scope>
 </dependency>
```
2013-11-08 15:08:31 +01:00
Simon Willnauer c946a69e28 Set MaxDirectMemorySize for tests to have consistent upper bounds 2013-11-07 09:44:19 +01:00
Simon Willnauer e1b6988886 move to [1.0.0.Beta2] SNAP 2013-11-06 16:19:50 +01:00
Simon Willnauer 77bc5d5ecf release [1.0.0.Beta1] 2013-11-06 15:32:43 +01:00
Shay Banon 75eda6e957 upgrade randomized testing to 2.0.12 2013-10-31 10:02:03 +01:00
Simon Willnauer 3a34aa735e Upgrade to Lucene 4.5.1 2013-10-24 18:37:44 +02:00
Simon Willnauer 0eea6e8183 Enable Random TransportClients in tests 2013-10-10 13:23:02 +02:00
Adrien Grand 6b02611971 Update Lucene to version 4.5.0. 2013-10-08 15:44:03 +02:00
Martijn van Groningen 088e05b368 Migrate from Trove to Hppc. 2013-10-03 23:10:57 +02:00
Simon Willnauer 8f087e802d Print Java Version during Validate 2013-09-27 14:39:07 +02:00
Simon Willnauer 5a365795ec Added support for dynamic transport client ratio in tests.
Currently tests only run with node clients but eventually we want to
run all tests with randomly choosen node / transport clients. To enable
this during development and on test servers as a transition phase this
commit adds the ability to allow a fraction of the clients used in
tests to be transport clients. By default this is still disabled.
To enable transport clients in tests pass '-Dtests.client.ratio=[0..1]'
where '1.0' will force transport clients and '0.0' completely disables
them. If an empty string is passed the ratio is chosen at random for
each test.
2013-09-22 22:25:30 +02:00
Simon Willnauer cabbf7805b Make TestCluster based integration tests more repoducible
While testing an async system providing reproducible tests that
use randomized components is a hard task we should at least try to
reestablish the enviroment of a failing test as much as possible.
This commit allows to re-establish the shared 'TestCluster' by
resetting the cluster to a predefined shared state before each test.

Before this commit a tests that is executed in isolation was likely
using a entirely different node enviroment as the failing test since
the 'TestCluster' kept intermediate nodes started by other tests around.
2013-09-17 23:07:29 +02:00
Shay Banon e110d53b0c change default number of test jvms to 1
the auto default can cause a lot of pressure on a machine when running the tests (each process actually runs a multi node cluster)
2013-09-17 21:30:27 +02:00
Costin Leau dcc45070bd Merge pull request #3702 from costin/master
add elasticsearch as a service for Windows platforms
2013-09-17 05:16:29 -07:00
Costin Leau 08bf131899 rework script to handle path with spaces
use service id for pid name
disable filtering on *.exe (caused corruption)
rename exe names and add more options to .bat
start/stop operations are now supported (and expected to be called) by service.bat
add more variables from the env to customize default behavior prior to installing the service
add manager option
fixes regarding batch flow
specify service id in description
minor readability improvement
include .exe only in ZIP archive
rename x64 service id to make it work out of the box
add elasticsearch as a service for Windows platforms
based on Apace Commons Daemon
supports both x64 and x86
2013-09-17 15:01:09 +03:00
Shay Banon ae0489266e Add `node.mode` with `local` or `network`
Compared to setting node.local to true, would be nicer to support node.mode with values of local or network.

Note, node.local is still supported.

closes #3713
2013-09-17 00:34:10 +02:00
Shay Banon 88173d762a upgrade to jackson 2.2.3 2013-09-16 15:38:47 +02:00
Shay Banon 31097691af upgrade to netty 3.7.0 2013-09-16 15:05:28 +02:00
Shay Banon cd90382964 upgrade to google guava v 15 2013-09-16 09:14:00 +02:00
Simon Willnauer 0e936c99e3 Add -Dtests.integration support to pom.xml 2013-09-11 22:16:43 +02:00
Simon Willnauer e68303d6a6 Allow test output to be configurable via CMD 2013-09-04 12:46:43 +02:00
Adrien Grand 5b6be0c456 Use a separate build directory for Eclipse.
The fact that Maven and Eclipse share the same build directories can trigger
race conditions when both are trying to build at the same time, eg. if you run
`mvn clean test` while Eclipse is up and running: Eclipse will notice that some
class files are missing and start compiling in parallel with Maven.
2013-08-29 10:29:26 +02:00
Shay Banon f3a35ccc90 upgrade to joda 2.3 2013-08-24 00:07:57 +02:00
Boaz Leskes 9869427ef6 Allow to configure root logging level using system properties. Ex. -Des.logger.level=DEBUG . Defaults to INFO as before. 2013-08-15 14:36:00 +02:00
Simon Willnauer ba13930b32 Fix test include pattern to include *Test.class
We missed *Test.class which is not our convention but we could miss
some tests. We should better include the *Test.class tests as well.
2013-08-13 17:46:48 +02:00
Simon Willnauer 59be83f9fc Remove accidentially committed default values
`-Dtests.maxFailures` and `-Dtests.failfast` should not be enabled
by default.
2013-08-12 14:32:04 +02:00
Simon Willnauer dbed36a13f Added support for `tests.failfast` and `tests.maxiters`
This commit adds support for failing fast when running a test
case with `-Dtests.iters=N` and uses some goodness from LuceneTestCase
in a new base `AbstractRandomizedTest`. This class checks among other
things if a tests doesn't call `super.setup` / `super.tearDown` when it
should do and checks if a large static resources are not cleaned up
after the tests ie. a running node.
2013-08-12 13:18:56 +02:00
Alexander Reelsen 68b77c1ae3 Included only runtime dependencies when copying
This makes sure, that no test dependencies are placed in the distribution
2013-08-06 15:13:25 +02:00
Shay Banon 976152b23e enable sys out check + remove unused code 2013-07-26 18:57:21 +02:00
Shay Banon 70bbcb4c48 Add Git build info when we build a distribution
closes #3370
2013-07-24 20:24:23 +02:00
Simon Willnauer ed473e272d Cut over to JUnit & Randomized Runner from TestNG
For better integration with the Lucene Test Framework and the
availabilty of RandomizedRunner / Randommized Testing this commit
moves over from TestNG to JUnit.
This commit also moves relevant places over to RandomzedRunner for
reproduceability and removes copied classes from the Lucene Test
Framework.
2013-07-24 16:59:36 +02:00
Simon Willnauer 2e9851138e Upgrade to Lucene 4.4 2013-07-23 13:55:15 +02:00
Alexander Reelsen 3162f5b725 Updated maven shade plugin to version 2.1
The currently used maven shade plugin still keeps references to the
original classes in their constant pools around. This is never a problem
at runtime, but for dependency tools which try to use the constant pool
for determining dependencies will get confused (OSGI for example). This
patch simply bumps the version and will implicetely fix
fix http://jira.codehaus.org/browse/MSHADE-105

Closes #3254
Closes #3255
2013-07-04 15:57:13 +02:00
Alexander Reelsen 455bc32460 Moving forbidden-api checks to compile phase instead of test phase (fail fast) 2013-06-28 13:12:52 +02:00
Shay Banon 0114fb0f58 add running the tests in headless mode in maven 2013-06-27 23:23:13 +01:00
Adrien Grand 1954f770a1 Put Eclipse settings in the root directory.
This enforces that settings are taken into account whichever mean is used to
import the project into Eclipse (manual import, m2e, mvn eclipse:eclipse, ...).
2013-06-26 16:51:47 +02:00
Simon Willnauer a388588b1f Upgrade to Lucene 4.3.1 2013-06-18 22:15:31 +02:00
Shay Banon f155525cad upgrade jackson to 2.2.2, netty to 3.6.6 2013-06-11 20:36:08 +02:00
Alexander Reelsen a5f9173e14 Making deb installable by being lintian compatible
According to #2515 the ubuntu software center does not allow to install
debian packages which are not lintian compatible

I worked on the package and made it lintian compatible by doing

* Ignoring errors about arch dependent binaries as we will not split
  this package. The arch dependent libraries are used correctly.
* Added a copyright file pointing to the apache license in debian

Closes #2515
Closes #2320
2013-06-07 13:53:14 +02:00
Alexander Reelsen 183bb76371 Ensure config files are not overwritten in RPM upgrade
In order to ensure that configuration files do not get overwritten when
upgrading an RPM, it is not sufficient to mark them as configuration. You
have to use the 'noreplace' parameter to make sure, they are never
overwritten. Added this parameter for the /etc/elasticsearch directory
as well as the /etc/sysconfig/elasticsearch file.

In addition, the post remove script now only deletes the user in case of
a package removal (and does nothing on package upgrade).

Closes #3123
2013-05-31 16:26:10 +02:00
Adrien Grand c16a46e15c Make it easier to get started with Eclipse.
This patch makes mvn eclipse:eclipse generate additional eclipse configuration
files so that Eclipse:
 - uses Java 1.6 compliance level,
 - truncates lines after 140 chars,
 - uses 4 spaces for indentation,
 - automatically adds a license header when creating a new class file,
 - organizes imports the same way as Intellij Idea (which makes sense I guess
   since most of the code bas has been written with Intellij, this will prevent
   from having large diffs due to the fact that the order of imports has
   changed).
2013-05-30 16:48:58 +02:00
Simon Willnauer 31f0aca65d Integrate forbiddenAPI checks into Maven build.
This commit integrates the forbiddenAPI checks that checks
Java byte code against a list of "forbidden" API signatures.
The commit also contains the fixes of the current source code
that didn't pass the default API checks.

See https://code.google.com/p/forbidden-apis/ for details.

Closes #3059
2013-05-19 23:25:44 +02:00
Shay Banon 2ab72da7d6 update to joda 2.2 2013-05-11 23:37:56 +02:00
Shay Banon e09e3eb73b upgrade to mvel 2.1.5 2013-05-11 23:17:00 +02:00
Shay Banon 522ca7e889 upgrade to jackson 2.2.1 2013-05-09 18:00:23 +02:00
Simon Willnauer 2219925485 Upgrade to Lucene 4.3.0
This Lucene Release introduced a new API on DocIdSetIterator that requires each
implementation to return a `cost` upperbound as a function of the iterated documents.
This API allows for several optimizations during query execution especially in
Conjunction and Disjunction Queries with min_should_match set.

Closes #2990
2013-05-06 18:03:16 +02:00
Shay Banon 6c3bb4dcdd move to 1.0.0.Beta1 snap 2013-04-29 13:51:09 +02:00
Shay Banon cb75ce0caa release 0.90.0 GA 2013-04-29 13:41:43 +02:00
Shay Banon 80fc55a01d upgrade to netty 3.6.5 2013-04-09 09:44:49 -07:00
Shay Banon 3120457bfe move to 0.90.0.RC3 snap 2013-04-08 05:48:29 -07:00
Shay Banon 3a8cba4d50 release 0.90.0.RC2 2013-04-08 05:46:26 -07:00
Simon Willnauer eb8b38d027 Upgrade to Lucene 4.2.1 2013-04-03 12:22:39 +02:00
Alexander Reelsen 0a466352cd Add support for creating a fedora RPM package with maven
Note: This has been disabled by default and is therefore not included in a
standard build. The main reason for this is, that you need to have a RPM
binary and the rpm development packages installed, which is not the case
on many systems.

The package contains an init.d-script as well as systemd configurations.

You can build your own RPM package simply by running 'maven rpm:rpm'
2013-04-02 16:19:45 +02:00
Shay Banon ea698add72 move to 0.90.0.RC2 snap 2013-03-20 19:06:30 +01:00
Shay Banon a2f14b68e8 release 0.90.0.RC1 2013-03-20 19:05:08 +01:00
Shay Banon bea7bdde4c upgrade to guava 14.0.1 2013-03-18 23:10:14 +01:00
Simon Willnauer 11bf7a8b1a Upgrade to Lucene 4.2 2013-03-11 08:23:01 +01:00
Shay Banon 1d45edd856 upgrade to jackson 2.1.4 2013-02-28 09:10:43 +01:00
Shay Banon f02c3ec39a upgrade to guava 14.0 2013-02-27 18:31:13 +01:00
Shay Banon 31c273231a remove compress flag, as its no longer relevant 2013-02-27 18:13:48 +01:00
Shay Banon bd75b731c6 move to 0.90.0.Beta2 snap 2013-02-26 10:33:57 +01:00
Shay Banon ab3a59e0bf release 0.90.0.Beta1 2013-02-26 10:32:50 +01:00
Shay Banon 358c0e35fb upgrade to latest jackson 2013-02-25 11:15:45 +01:00
Shay Banon 7787901a2d cleanup the pom 2013-02-23 10:52:30 +01:00