Added a new revised IT group BackwardCompatibilityMain. The idea is to catch potential backward compatibility issues that may arise during rolling upgrade.
This test group runs a docker-compose cluster with Overlord & Coordinator service on the previous druid version.
Following env vars are required in the GHA file .github/workflows/unit-and-integration-tests-unified.yml to run this test
DRUID_PREVIOUS_VERSION -> Previous druid version to test backward incompatibility.
DRUID_PREVIOUS_VERSION_DOWNLOAD_URL -> URL to fetch the tar.
* Upgrade actions/checkout from v3 to v4.
* Upgrade actions/setup-java from v3 to v4.
* Upgrade dorny/paths-filter, actions/cdache/restore, actions/stale to v3, v4 and v9 respectively.
* Add a GHA label for .github/** and skip UT/IT on .github files.
* remove skipping UT/IT on .github/** changes.
With this PR, docker and druid service logs are uploaded as artifacts onto GitHub when an IT job fails so that we can later download them for investigation.
* Fix read timed out failures and remove containers before test
* remove containers before loading images
* add labels to IT docker containers, download stable minio docker image release instead of latest
Currently, there is no way to run ITs if unit-tests fail on coverage. This PR allows Revised, Standard ITs to run even when unit-tests fail on coverage errors, still failing the workflow. This PR also fixes existing GHA workflow naming.
Instead of downloading jdk everytime we run CI, we're using inbuilt temurin jdk distributions 8, 11, 17 by settiing JAVA_HOME variable. This is not working as expected since we were not setting this as global environment variable as a result all CI builds are running on jdk11. This PR fixes the issue.
*
1. Handling deletion/creation of container created during the previously run test in AzureTestUtil.java.
2. Adding/updating log messages and comments in Azure and GCS deep storage tests.
The FiniteFirehoseFactory and InputRowParser classes were deprecated in 0.17.0 (#8823) in favor of InputSource & InputFormat. This PR removes the FiniteFirehoseFactory and all its implementations along with classes solely used by them like Fetcher (Used by PrefetchableTextFilesFirehoseFactory). Refactors classes including tests using FiniteFirehoseFactory to use InputSource instead.
Removing InputRowParser may not be as trivial as many classes that aren't deprecated depends on it (with no alternatives), like EventReceiverFirehoseFactory. Hence FirehoseFactory, EventReceiverFirehoseFactory, and Firehose are marked deprecated.
Create the docker image in case of maven dependencies cache restore failure too as env.sh file is removed on maven rebuild.
Increase java heap size for security IT failing with error
* rebuild maven project or docker image in case of cache hit miss
* rebuild maven project in case of docker cache hit miss too
* fix docker-restore cache hit fail issue
Rishabh Singh
Abhishek Radhakrishnan
Xavier Léauté
Tejaswini Bandlamudi
abhagraw
somu-imply
Elliott Freis