honeymoose/OpenSearch
1
0
Fork 0
mirror of https://github.com/honeymoose/OpenSearch.git synced 2025-02-23 13:26:02 +00:00
Code Issues Packages Projects Releases Wiki Activity

Docs: Update integration tests documentation

Browse Source
This commit is contained in:
Alexander Reelsen 2015-09-10 19:04:35 +02:00
parent 2b4bcaeee3
commit 5d6ee7e82e
1 changed files with 42 additions and 32 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

74
docs/reference/testing/testing-framework.asciidoc
View File

@ -18,17 +18,11 @@ All of the tests are run using a custom junit runner, the `RandomizedRunner` pro
[[using-elasticsearch-test-classes]]
=== Using the elasticsearch test classes
First, you need to include the testing dependency in your project. If you use maven and its `pom.xml` file, it looks like this
First, you need to include the testing dependency in your project, along with the elasticsearch dependency you have already added. If you use maven and its `pom.xml` file, it looks like this
[[source,xml]]
--------------------------------------------------
<dependencies>
<dependency>
<groupId>com.carrotsearch.randomizedtesting</groupId>
<artifactId>randomizedtesting-runner</artifactId>
<version>${randomizedtesting-runner.version}</version>
<scope>test</scope>
</dependency>
<dependency>
<groupId>org.apache.lucene</groupId>
<artifactId>lucene-test-framework</artifactId>
@ -42,25 +36,22 @@ First, you need to include the testing dependency in your project. If you use ma
<scope>test</scope>
<type>test-jar</type>
</dependency>
<dependency>
<groupId>org.elasticsearch</groupId>
<artifactId>elasticsearch</artifactId>
<version>${elasticsearch.version}</version>
<scope>test</scope>
</dependency>
</dependencies>
--------------------------------------------------
Replace the elasticsearch version and the lucene versions with the current elasticsearch version and its accompanying lucene release.
And replace the "randomizedtesting version" with the version that the current elasticsearch uses.
Replace the elasticsearch version and the lucene version with the corresponding elasticsearch version and its accompanying lucene release.
There are already have a couple of classes, you can inherit from in your own test classes. The advantages of doing so is having already defined loggers, the whole randomized infrastructure is set up already.
We provide a few classes that you can inherit from in your own test classes which provide:
* pre-defined loggers
* randomized testing infrastructure
* a number of helper methods
[[unit-tests]]
=== unit tests
In case you only need to execute a unit test, because your implementation can be isolated that well and does not require an up and running elasticsearch cluster, you can use the `ElasticsearchTestCase`. If you are testing lucene features, use `ElasticsearchLuceneTestCase` and if you are testing concrete token streams, use the `ElasticsearchTokenStreamTestCase` class. Those specific classes execute additional checks, which ensure that no resources leaks are happening, after the test has run.
If your test is a well isolated unit test which doesn't need a running elasticsearch cluster, you can use the `ESTestCase`. If you are testing lucene features, use `ESTestCase` and if you are testing concrete token streams, use the `ESTokenStreamTestCase` class. Those specific classes execute additional checks which ensure that no resources leaks are happening, after the test has run.
[[integration-tests]]
@ -68,18 +59,20 @@ In case you only need to execute a unit test, because your implementation can be
These kind of tests require firing up a whole cluster of nodes, before the tests can actually be run. Compared to unit tests they are obviously way more time consuming, but the test infrastructure tries to minimize the time cost by only restarting the whole cluster, if this is configured explicitly.
The class your tests have to inherit from is `ElasticsearchIntegrationTest`. As soon as you inherit, there is no need for you to start any elasticsearch nodes manually in your test anymore, though you might need to ensure that at least a certain number of nodes is up.
The class your tests have to inherit from is `ESIntegTestCase`. By inheriting from this class, you will no longer need to start elasticsearch nodes manually in your test, although you might need to ensure that at least a certain number of nodes are up. The integration test behaviour can be configured heavily by specifying different system properties on test runs. See the `TESTING.asciidoc` documentation in the https://github.com/elastic/elasticsearch/blob/master/TESTING.asciidoc[source repository] for more information.
[[number-of-shards]]
==== number of shards
The number of shards used for indices created during integration tests is randomized between `1` and `10` unless overwritten upon index creation via index settings.
Rule of thumb is not to specify the number of shards unless needed, so that each test will use a different one all the time.
The rule of thumb is not to specify the number of shards unless needed, so that each test will use a different one all the time. Alternatively you can override the `numberOfShards()` method. The same applies to the `numberOfReplicas()` method.
[[helper-methods]]
==== generic helper methods
There are a couple of helper methods in `ElasticsearchIntegrationTest`, which will make your tests shorter and more concise.
There are a couple of helper methods in `ESIntegTestCase`, which will make your tests shorter and more concise.
[horizontal]
`refresh()`:: Refreshes all indices in a cluster
@ -98,7 +91,7 @@ There are a couple of helper methods in `ElasticsearchIntegrationTest`, which wi
[[test-cluster-methods]]
==== test cluster methods
The `TestCluster` class is the heart of the cluster functionality in a randomized test and allows you to configure a specific setting or replay certain types of outages to check, how your custom code reacts.
The `InternalTestCluster` class is the heart of the cluster functionality in a randomized test and allows you to configure a specific setting or replay certain types of outages to check, how your custom code reacts.
[horizontal]
`ensureAtLeastNumNodes(n)`:: Ensure at least the specified number of nodes is running in the cluster
@ -112,10 +105,30 @@ The `TestCluster` class is the heart of the cluster functionality in a randomize
`startNode(settings)`:: Create and start a new elasticsearch node
[[changing-node-settings]]
==== Changing node settings
If you want to ensure a certain configuration for the nodes, which are started as part of the `EsIntegTestCase`, you can override the `nodeSettings()` method
[source,java]
-----------------------------------------
public class Mytests extends ESIntegTestCase {
@Override
protected Settings nodeSettings(int nodeOrdinal) {
return settingsBuilder().put(super.nodeSettings(nodeOrdinal))
.put("node.mode", "network")
.build();
}
}
-----------------------------------------
[[accessing-clients]]
==== Accessing clients
In order to execute any actions, you have to use a client. You can use the `ElasticsearchIntegrationTest.client()` method to get back a random client. This client can be a `TransportClient` or a `NodeClient` - and usually you do not need to care as long as the action gets executed. There are several more methods for client selection inside of the `TestCluster` class, which can be accessed using the `ElasticsearchIntegrationTest.cluster()` method.
In order to execute any actions, you have to use a client. You can use the `ESIntegTestCase.client()` method to get back a random client. This client can be a `TransportClient` or a `NodeClient` - and usually you do not need to care as long as the action gets executed. There are several more methods for client selection inside of the `InternalTestCluster` class, which can be accessed using the `ESIntegTestCase.internalCluster()` method.
[horizontal]
`iterator()`:: An iterator over all available clients
@ -136,7 +149,7 @@ You can use the `@ClusterScope` annotation at class level to configure this beha
[source,java]
-----------------------------------------
@ClusterScope(scope=TEST, numNodes=1)
public class CustomSuggesterSearchTests extends ElasticsearchIntegrationTest {
public class CustomSuggesterSearchTests extends ESIntegTestCase {
// ... tests go here
}
-----------------------------------------
@ -145,7 +158,7 @@ The above sample configures the test to use a new cluster for each test method.
[[changing-node-configuration]]
==== Changing node configuration
==== Changing plugins via configuration
As elasticsearch is using JUnit 4, using the `@Before` and `@After` annotations is not a problem. However you should keep in mind, that this does not have any effect in your cluster setup, as the cluster is already up and running when those methods are run. So in case you want to configure settings - like loading a plugin on node startup - before the node is actually running, you should overwrite the `nodePlugins()` method from the `ESIntegTestCase` class and return the plugin classes each node should load.
@ -157,13 +170,6 @@ protected Collection<Class<? extends Plugin>> nodePlugins() {
}
-----------------------------------------
[[parametrized-tests]]
=== parameterized tests
It is possible to write parameterized tests, that get run multiple times with different parameters. `RandomizedRunner` needs to be used rather than the `Parameterized` runner provided with junit (all the base test classes already use `RandomizedRunner` though). The method that provides the parameters
needs to be annotated with the `@ParametersFactory` annotation and must be `static`, `public`, without arguments and must have a return type assignable to `Iterable<Object[]>`. The iterable must return arrays conforming to the suite class's constructor with respect to the number and types of parameters.
The constructor's parameters can be annotated with the `@Name` annotation to provide more descriptive parameter names for test descriptions. Have a look at `ElasticsearchRestTests` for an example.
[[randomized-testing]]
=== Randomized testing
@ -200,6 +206,8 @@ So, how can you create random data. The most important thing to know is, that yo
`randomLocale()`:: Returns a random locale
`randomTimeZone()`:: Returns a random timezone
`randomFrom()`:: Returns a random element from a list/array
In addition, there are a couple of helper methods, allowing you to create random ASCII and Unicode strings, see methods beginning with `randomAscii`, `randomUnicode`, and `randomRealisticUnicode` in the random test class. The latter one tries to create more realistic unicode string by not being arbitrary random.
If you want to debug a specific problem with a specific random seed, you can use the `@Seed` annotation to configure a specific seed for a test. If you want to run a test more than once, instead of starting the whole test suite over and over again, you can use the `@Repeat` annotation with an arbitrary value. Each iteration than gets run with a different seed.
@ -208,7 +216,7 @@ If you want to debug a specific problem with a specific random seed, you can use
[[assertions]]
=== Assertions
As many elasticsearch tests are checking for a similar output, like the amount of hits or the first hit or special highlighting, a couple of predefined assertions have been created. Those have been put into the `ElasticsearchAssertions` class.
As many elasticsearch tests are checking for a similar output, like the amount of hits or the first hit or special highlighting, a couple of predefined assertions have been created. Those have been put into the `ElasticsearchAssertions` class. There is also a specific geo assertions in `ElasticsearchGeoAssertions`.
[horizontal]
`assertHitCount()`:: Checks hit count of a search or count request
@ -232,6 +240,8 @@ Common matchers
`hasId()`:: Matcher to check for a search hit id
`hasType()`:: Matcher to check for a search hit type
`hasIndex()`:: Matcher to check for a search hit index
`hasScore()`:: Matcher to check for a certain score of a hit
`hasStatus()`:: Matcher to check for a certain `RestStatus` of a response
Usually, you would combine assertions and matchers in your test like this