In the bats test ES_CLEAN_BEFORE_TEST was used to clean the environment
before running the tests. Unfortunately the tests don't work unless you
specify it every time. This removes that option and always runs the clean.
This creates a module in qa called vagrant that can be run if you have
vagrant and virtualbox installed and will run the packaging tests in trusty
and centos-7.0. You can ask it to run tests in other linuxes. This is the full
list:
* precise aka Ubuntu 12.04
* trusty aka Ubuntu 14.04
* vivid aka Ubuntun 15.04
* wheezy aka Debian 7, the current debian oldstable distribution
* jessie aka Debian 8, the current debina stable distribution
* centos-6
* centos-7
* fedora-22
* oel-7
There is lots of documentation on how to do this in the TESTING.asciidoc.
Closes#12611
This change creates a proper `distribution` modules in which we have today packaging for
all of our four current packages:
* zip
* tar.gz
* rpm
* deb
Licenes have moved into the distribution project as well. So have the config/ and the bin/ directory
from the core/ project.
The RPM package is now built, if rpmbuild exists.
The bats tests have been moved as well.
Also the zip distribution now executes the REST integration tests.
The existing DEB/RPM packages have a lot of differences: they don't execute the same actions when installing or removing the package. They also don't declare exactly the same environment variables at the same place. At the end of the day the global behavior and configuration is *almost* the same but it's very difficult to maintain the scripts.
This commits unifies the package behavior:
- DEB/RPM use the same package scripts (pre installation, post installation etc) in order to execute exactly the same actions
- Use of a unique environment vars file that declares everything needed by scripts (the goal is to delete vars declaration in init.d and systemd scripts, this will be done in another PR)
- Variables like directory paths are centralized and replaced according to the target platform (using #10330)
- Move /etc/rc.d/init.d to standard /etc/init.d (RPM only)
- Add PID_DIR env var
- Always set ES_USER, ES_GROUP,MAX_MAP_COUNT and MAX_OPEN_FILES in env vars file
- Create log, data, work and plugins directories with DEB/RPM packaging system
- Change to elastic.co domain in copyright and control files
- Add Bats files to automate testing of DEB and RPM packages
- Update TESTING.asciidoc
More info on Bats here: https://github.com/sstephenson/bats
This adds the exec-maven-plugin that allows a developer to run:
```
mvn exec:exec
```
To launch the `Bootstrap` process similar to the way that a Java IDE
would. All the logs go to logs/elasticsearch.log (or wherever
configured)
`-Dtests.filter` allows to pass filter expressions to the elasticsearch
tests. This allows to filter test annotaged with TestGroup annotations
like @Slow, @Nightly, @Backwards, @Integration with a boolean expresssion like:
* to run only backwards tests run:
`mvn -Dtests.bwc.version=X.Y.Z -Dtests.filter="@backwards"`
* to run all integration tests but skip slow tests run:
`mvn -Dtests.filter="@integration and not @slow"
* to take defaults into account ie run all test as well as backwards:
`mvn -Dtests.filter="default and @backwards"
This feature is a more powerful alternative to flags like
`-Dtests.nighly=true|false` etc.
Closes#6703
All settings should be passes as settings and the enviroment should not
influence the test cluster settings. The settings we care about ie.
`es.node.mode` and `es.logger.level` should be passed via settings.
This allows tests to override these settings if they for instance need
`network` transport to operate at all.
Closes#6663
This commit add a basic infrastructure as well as primitive tests
to ensure version backwards compatibility between the current
development trunk and an arbitrary previous version. The compatibility
tests are simple unit tests derived from a base class that starts
and manages nodes from a provided elasticsearch release package.
Use the following commandline executes all backwards compatiblity tests
in isolation:
```
mvn test -Dtests.bwc=true -Dtests.bwc.version=1.2.1 -Dtests.class=org.elasticsearch.bwcompat.*
```
These tests run basic checks like rolling upgrades and
routing/searching/get etc. against the specified version. The version
must be present in the `./backwards` folder as
`./backwards/elasticsearch-x.y.z`
Needed for REST backwards compatibility tests, since we need to run older tests with the latest runner, which randomizes shards and replicas, but the tests rely on defaults (5,1).
Done in a generic way based on compatibility versions e.g. `-Dtests.compatibility=1.0.0` allows to run tests in a special manner that is compatibile with 1.0.0 version.
Also moved back randomIndexTemplate to ElasticsearchIntegrationTest (from ImmutableCluster) where all the randomized aspects should be.
Closes#5897
The blacklist can be provided through -Dtests.rest.blacklist and supports a comma separated list of globs
e.g. -Dtests.rest.blacklist=get/10_basic/*,index/*/*
Also added some missing docs and made it clearer that the suite/test descriptions effectively contains their (relative) path (api/yaml_file/test section)
Closes#5881
ElasticsearchRestTests extends now ElasticsearchIntegrationTest and makes use of our ordinary test infrastructure, in particular all randomized aspects now come for free instead of having to maintain a separate (custom) tests runner
We previously parsed only the tests that needed to be run given the version of the cluster the tests are running against. This doesn't happen anymore as it didn't buy much and it would be harder to support as the tests get now parsed before the test cluster gets started. Thus all the tests are now parsed regardless of their skip sections, afterwards the ones that don't need to be run will be skipped through assume directives.
Fixed REST tests that rely on a specific number of shards as this change introduces also random number of shards and replicas (through randomIndexTemplate)
Closes#5654
Multiple nodes are now started when running REST tests against the `TestCluster` (default randomized settings are now used instead of the hardcoded `1`)
Added also randomized round-robin based on all available nodes, and ability to provide multiple addresses when running tests against an external cluster to have the same behaviour
The REST layer can now be tested through tests that are shared between all the elasticsearch official clients.
The tests are based on REST specification that can be found on the elasticsearch-rest-api-spec project and consist of YAML files that describe the operations to be executed and the obtained results that need to be tested.
REST tests can be executed through the ElasticsearchRestTests class, which relies on the rest-spec git submodule that contains the rest spec and tests pulled from the elasticsearch-rest-spec-api project. The rest-spec submodule gets automatically initialized and updated through maven (generate-test-resources phase).
The REST runner and the needed classes are distributed within the test artifact.
The following are the options supported by the REST tests runner:
- tests.rest[true|false|host:port]: determines whether the REST tests need to be run and if so whether to rely on an external cluster (providing host and port) or fire a test cluster (default)
- tests.rest.suite: comma separated paths of the test suites to be run (by default loaded from /rest-spec/test classpath). it is possible to run only a subset of the tests providing a sub-folder or even a single yaml file (the default /rest-spec/test prefix is optional when files are loaded from classpath) e.g. -Dtests.rest.suite=index,get,create/10_with_id
- tests.rest.spec: REST spec path (default /rest-spec/api from classpath)
- tests.iters: runs multiple iterations
- tests.seed: seed to base the random behaviours on
- tests.appendseed[true|false]: enables adding the seed to each test section's description (default false)
- tests.cluster_seed: seed used to create the test cluster (if enabled)
Closes#4469
David Pilato
Nik Everett
Robert Muir
Ryan Ernst
Dave Parfitt
Alexander Reelsen
Britta Weber
Tanguy Leroux
Simon Willnauer
Lee Hinman
javanna
Dawid Weiss
Shay Banon
Boaz Leskes