Refactor cache recycling so that it only caches large arrays (pages) that can
later be used to build more complex data-structures such as hash tables.
- QueueRecycler now takes a limit like other non-trivial recyclers.
- New PageCacheRecycler (inspired of CacheRecycler) has the ability to cache
byte[], int[], long[], double[] or Object[] arrays using a fixed amount of
memory (either globally or per-thread depending on the Recycler impl, eg.
queue is global while thread_local is per-thread).
- Paged arrays in o.e.common.util can now optionally take a PageCacheRecycler
to reuse existing pages.
- All aggregators' data-structures now use PageCacheRecycler:
- for all arrays (counts, mins, maxes, ...)
- LongHash can now take a PageCacheRecycler
- there is a new BytesRefHash (inspired from Lucene but quite different,
still; for instance it cheats on BytesRef comparisons by using Unsafe)
that also takes a PageCacheRecycler
Close#4557
Test can be run with `-Dtests.assertion.disabled=org.elasticsearch`
to run the tests without assertions to make sure assertions
don't hide any assignements etc. that introduce bugs in production.
We are mocking out some functionality to add assertions etc. or
randomize store types. We should randomly run with our defaults to make
sure we don't hide any potential problems.
The REST layer can now be tested through tests that are shared between all the elasticsearch official clients.
The tests are based on REST specification that can be found on the elasticsearch-rest-api-spec project and consist of YAML files that describe the operations to be executed and the obtained results that need to be tested.
REST tests can be executed through the ElasticsearchRestTests class, which relies on the rest-spec git submodule that contains the rest spec and tests pulled from the elasticsearch-rest-spec-api project. The rest-spec submodule gets automatically initialized and updated through maven (generate-test-resources phase).
The REST runner and the needed classes are distributed within the test artifact.
The following are the options supported by the REST tests runner:
- tests.rest[true|false|host:port]: determines whether the REST tests need to be run and if so whether to rely on an external cluster (providing host and port) or fire a test cluster (default)
- tests.rest.suite: comma separated paths of the test suites to be run (by default loaded from /rest-spec/test classpath). it is possible to run only a subset of the tests providing a sub-folder or even a single yaml file (the default /rest-spec/test prefix is optional when files are loaded from classpath) e.g. -Dtests.rest.suite=index,get,create/10_with_id
- tests.rest.spec: REST spec path (default /rest-spec/api from classpath)
- tests.iters: runs multiple iterations
- tests.seed: seed to base the random behaviours on
- tests.appendseed[true|false]: enables adding the seed to each test section's description (default false)
- tests.cluster_seed: seed used to create the test cluster (if enabled)
Closes#4469
In order to be sure that memory mapped lucene directories are working
one can configure the kernel about how many memory mapped areas
a process may have. This setting ensure for the debian and redhat initscripts
as well as the systemd startup, that this setting is set high enough.
Closes#4397
We have the situation that some tests fail since they don't handle
EsRejectedExecutionException which gets thrown when a node shuts
down. That is ok to ignore this exception and not fail.
We also suffer from OOMs that can't create native threads but don't
get threaddumps for those failures. This patch prints the thread
stacks once we catch a OOM which can' create native threads.
The maven-compiler-plugin upgrade from 2.3.2 to 3.1 (see #4279) could cause out of memory issue when building the project with Maven and JDK6 and default memory settings (no `MAVEN_OPTS`).
This issue does not appear with JDK7.
When we want to use test artifact in other projects, dependencies
are not shaded as for core artifact.
Issue opened in maven shade project: [MSHADE-158](http://jira.codehaus.org/browse/MSHADE-158)
When using it in other projects, you basically need to change your `pom.xml` file:
```xml
<dependency>
<groupId>org.elasticsearch</groupId>
<artifactId>elasticsearch</artifactId>
<version>${elasticsearch.version}</version>
<type>test-jar</type>
<scope>test</scope>
</dependency>
```
You can also define some properties:
```xml
<properties>
<tests.jvms>1</tests.jvms>
<tests.shuffle>true</tests.shuffle>
<tests.output>onerror</tests.output>
<tests.client.ratio></tests.client.ratio>
<es.logger.level>INFO</es.logger.level>
</properties>
```
Closes#4266
This commit upgrades to Lucene 4.6 and contains the following improvements:
* Remove XIndexWriter in favor of the fixed IndexWriter
* Removes patched XLuceneConstantScoreQuery
* Now uses Lucene passage formatters contributed from Elasticsearch in PostingsHighlighter
* Upgrades to Lucene46 Codec from Lucene45 Codec
* Fixes problem in CommonTermsQueryParser where close was never called.
Closes#4241
Currently when importing projects into eclipse you need to run 'mvn
eclipse:eclipse' on the command line to generate the poject files. This
means that when the pom changes you need to re-run the command on the
command line to reflect those changes in the project in eclipse. This
commit allows the developer to import the project as an existing maven
project (can be shared using git after import) and then allows the
application to be run inside eclipse using the .launch file in
/dev-tools enabling easy debugging of the application within eclipse
without requiring a maven build.
This commit adds javadocs and removed unused methods from central
classes like ElasticsearchIntegrationTest. It also changes visibility
of many methods and classes that are only needed inside the test infrastructure.
This commit causes all classes under 'org.elasticsearch.test.*'
to be included in a 'elasticsearch-${version}-test.jar' that can be
inclued by 3rd party projects or plugins via:
```
<dependency>
<groupId>org.elasticsearch</groupId>
<artifactId>elasticsearch</artifactId>
<version>${elasticsearch.version}</version>
<type>test-jar</type>
<scope>test</scope>
</dependency>
```
Adrien Grand
Simon Willnauer
David Pilato
Luca Cavanna
Shay Banon
Alexander Reelsen
mrsolo
Kevin Kluge
colings86
Martijn van Groningen