nifi/nifi-nar-bundles/nifi-elasticsearch-bundle

Integration Tests

Overview

The integration tests use Testcontainers to provide a sane default for developers who have installed Docker. Testcontainers support can be disabled by setting the system property elasticsearch.testcontainers.enabled to something other than true. If Testcontainers are disabled, the endpoint will need to be configured. It can be set manually with the system property elasticsearch.endpoint. The default value is http://localhost:9200.

Maven Profiles

• integration-tests
• elasticsearch6
• elasticsearch7

Configurable System Properties

• elasticsearch.endpoint - Manually configure the endpoint root for a non-Docker version of Elasticsearch,
• elasticsearch.testcontainers.enabled - Set to anything other than true to disable Testcontainers and use a non-Docker version of Elasticsearch.
• elasticsearch.elastic_user.password - Set the Elasticsearch elastic user's password. When Testcontainers are enabled, this sets up the Docker container and the rest clients for accessing it within the tests. When Testcontainers are disabled, it needs to be set to whatever password is used on the external Elasticsearch node or cluster.

Maven Run Examples

Elasticsearch 8.X is the current default version of Elasticsearch when Testcontainers are used. An example run of the integration tests with Elasticsearch 7 support would be like this:

mvn -Pintegration-tests,elasticsearch7 --fail-at-end clean install

An example using a non-Docker version of Elasticsearch:

mvn -Pintegration-tests --fail-at-end -Delasticsearch.testcontainers.enabled=false -Delasticsearch.elastic_user.password=s3cret1234 clean install

Bash Script Example

Execute the following script from the nifi-elasticsearch-bundle directory:

mvn --fail-at-end -Pcontrib-check clean install

es_versions=(elasticsearch6 elasticsearch7 elasticsearch8)
it_modules=(nifi-elasticsearch-client-service nifi-elasticsearch-restapi-processors)
for v in "${es_versions[@]}"; do
    for m in "${it_modules[@]}"; do
        pushd "${m}"
        if ! mvn -P "integration-tests,${v}" --fail-at-end failsafe:integration-test failsafe:verify; then
            echo; echo; echo "Integration Tests failed for ${v} in ${m}, see Maven logs for details"
            exit 1
        fi
        popd
    done
done

Modules with Integration Tests (using Testcontainers)

• Elasticsearch Client Service
• Elasticsearch REST API Processors

Misc

Integration Tests with Testcontainers currently only uses the amd64 Docker Images.

elasticsearch6 is known to experience some problems with arm64 machines (e.g. Mac M1/M2), but other Elasticsearch images (e.g. 7.x and 8.x) appear to work. Settings have been altered for the Elasticsearch containers in order to try and enable them on different architectures, but there may still be some inconsistencies.

Explicit arm64 architecture support may be added in future where the Elasticsearch images exist.