Commit Graph

95 Commits

Author SHA1 Message Date
Abhishek Agarwal 9eebeead44
Tune stale bot to pick older issues first (#14267) 2023-05-12 11:45:29 +05:30
Tejaswini Bandlamudi 8ef99f091a
Fix jdk setup in GHA (#14091)
Instead of downloading jdk everytime we run CI, we're using inbuilt temurin jdk distributions 8, 11, 17 by settiing JAVA_HOME variable. This is not working as expected since we were not setting this as global environment variable as a result all CI builds are running on jdk11. This PR fixes the issue.
2023-05-12 10:36:59 +05:30
Abhishek Agarwal f3ff36a004
Move the stale bot to a GHA action (#14238)
Move the stale bot to a GHA action
2023-05-11 11:31:28 +05:30
AmatyaAvadhanula ac7181bbda
Persist supervisor spec only after successful start (#14150)
* Persist spec after successful start

* Fix checkstyle.

* checkstyle after mvn install
2023-05-03 18:27:39 +05:30
Tejaswini Bandlamudi 774073b2e7
Update Hadoop3 as default build version (#14005)
Hadoop 2 often causes red security scans on Druid distribution because of the dependencies it brings. We want to move away from Hadoop 2 and provide Hadoop 3 distribution available. Switch druid to building with Hadoop 3 by default. Druid will still be compatible with Hadoop 2 and users can build hadoop-2 compatible distribution using hadoop2 profile.
2023-04-26 12:52:51 +05:30
Tejaswini Bandlamudi f86ea5cbc4
Separate web-checks from static-checks to improve build time (#14071)
* Moves web-checks to separate job
2023-04-12 20:17:03 -07:00
Tejaswini Bandlamudi 966cae1c94
Fix GHA CI branch trigger patterns (#14067)
* fix matching pattern on release branches

* test
2023-04-11 20:43:40 -07:00
Tejaswini Bandlamudi 5a9c13293b
remove duplicate trigger on Cron Job ITs workflow (#14013) 2023-04-04 09:39:48 +05:30
abhagraw eb31207402
Using MinIO to run S3DeepStorage ITs (#13997)
* Using MinIO to S3DeepStorage ITs

* Adding S3DeepStorageTest to github actions revised ITs
2023-03-30 12:15:53 -07:00
Tejaswini Bandlamudi f715887172
Debug docker logs on ITs failure. (#13978) 2023-03-29 09:06:41 -07:00
Tejaswini Bandlamudi 3c096c01a2
cache mvn dependencies across tests without building (#13962) 2023-03-29 16:27:36 +05:30
abhagraw c52d15d65d
Fixing security vulnerability check errors (#13956)
* Fixing security vulnerability check errors

* Updating javax.el to jakarta.el

* Adding cron job trigger on changes to suppressions file
2023-03-23 11:10:06 +05:30
Tejaswini Bandlamudi 1c250a0bc0
Fix error in cron job ITs workflow (#13945) 2023-03-17 17:29:45 +05:30
Tejaswini Bandlamudi da197c9273
Migrate existing jdk11 ITs to cron job (#13918)
This cron job runs on the latest commit of the master branch by default daily at 3:00 AM UTC.
2023-03-16 15:30:07 +05:30
abhagraw c7d864d3bc
Update container creation in AzureTestUtil.java (#13911)
*
1. Handling deletion/creation of container created during the previously run test in AzureTestUtil.java.
2. Adding/updating log messages and comments in Azure and GCS deep storage tests.
2023-03-16 11:04:43 +05:30
somu-imply 7ce3371730
Fixing a github workflow to resolve conflicts and use the correct tag for jdk (#13933) 2023-03-14 16:06:27 -07:00
Elliott Freis 8a1dc2f51c
We want to tag the container based on the build jdk version, not the runtime version (#13917)
Co-authored-by: Elliott Freis <elliottfreis@Elliott-Freis.earth.dynamic.blacklight.net>
2023-03-10 11:35:33 -08:00
Clint Wylie 68db39d08a
fix ci (#13901)
This PR is #13899 plus spotbugs fix to fix the failures introduced by #13815
2023-03-08 16:55:47 +05:30
Elliott Freis d93fdb2632
Bump CycloneDX module to address POM errors (#13878)
* Bump CycloneDX module to address POM errors

* Including web-console in the PR

---------

Co-authored-by: Elliott Freis <elliottfreis@Elliott-Freis.earth.dynamic.blacklight.net>
2023-03-03 15:39:15 +05:30
Abhishek Agarwal 81356f7667
do not run non sql compatible tests for all jdk flavours (#13875) 2023-03-03 11:26:57 +05:30
Tejaswini Bandlamudi 7103cb4b9d
Removes FiniteFirehoseFactory and its implementations (#12852)
The FiniteFirehoseFactory and InputRowParser classes were deprecated in 0.17.0 (#8823) in favor of InputSource & InputFormat. This PR removes the FiniteFirehoseFactory and all its implementations along with classes solely used by them like Fetcher (Used by PrefetchableTextFilesFirehoseFactory). Refactors classes including tests using FiniteFirehoseFactory to use InputSource instead.
Removing InputRowParser may not be as trivial as many classes that aren't deprecated depends on it (with no alternatives), like EventReceiverFirehoseFactory. Hence FirehoseFactory, EventReceiverFirehoseFactory, and Firehose are marked deprecated.
2023-03-02 18:07:17 +05:30
Elliott Freis d046cee3e4
Workaround for CycloneDX is causing POM build errors (#13867)
Co-authored-by: Elliott Freis <elliottfreis@Elliott-Freis.earth.dynamic.blacklight.net>
2023-03-01 16:47:13 -08:00
Elliott Freis faf602108b
We never want to restore caches that aren't an exact match to the commit SHA (#13863)
Co-authored-by: Elliott Freis <elliottfreis@Elliott-Freis.earth.dynamic.blacklight.net>
2023-02-28 13:59:49 -08:00
Kashif Faraz 54da38b508
Add missing license for jakarta.activation against module druid-avro-extensions (#13845) 2023-02-26 17:06:23 +05:30
Elliott Freis 471cb82af3
Use built-in java rather than spending time to install (#13834)
Co-authored-by: Elliott Freis <elliottfreis@Elliott-Freis.earth.dynamic.blacklight.net>
2023-02-23 08:50:04 -08:00
Abhishek Radhakrishnan 786172947e
Skip tests when there are markdown only changes. (#13836) 2023-02-23 11:39:53 +05:30
Tejaswini Bandlamudi e788f1ae6b
Add option to run standard & revised ITs manually on PRs (#13814)
Create the docker image in case of maven dependencies cache restore failure too as env.sh file is removed on maven rebuild.
Increase java heap size for security IT failing with error
2023-02-20 16:15:15 +05:30
Clint Wylie 08b5951cc5
merge druid-core, extendedset, and druid-hll into druid-processing to simplify everything (#13698)
* merge druid-core, extendedset, and druid-hll into druid-processing to simplify everything
* fix poms and license stuff
* mockito is evil
* allow reset of JvmUtils RuntimeInfo if tests used static injection to override
2023-02-17 14:27:41 -08:00
Abhishek Agarwal 8d03ace1b4
Use K3S instead of minikube for integration tests (#13782)
We are seeing failures on GHA while using minikube so switching to K3S instead.
2023-02-17 23:06:30 +05:30
Abhishek Agarwal ddae7974c2
Don't run UTs, ITs when there are only docs, helm and console changes. (#13812)
* Don't run UTs, ITs for docs, helm and console changes

* Test

* right place

* Revert test

* Remove ITs from ignore list

* Update unit-and-integration-tests-unified.yml

* Update unit-and-integration-tests-unified.yml
2023-02-17 09:56:17 +05:30
317brian add2081f7c
gha: add auto labeler for doc prs (#13791)
This adds an autolabeler GitHub action that will label any changes to the docs directories (docs and the Jupyter nb directory) as Area - Documentation.

The GHA will also remove the label if the PR changes and no longer touches files in those directories.
2023-02-16 12:05:34 +05:30
Elliott Freis e5ecb4d6aa
Flip boolean logic so the test description makes sense (#13805)
Co-authored-by: Elliott Freis <elliottfreis@Elliott-Freis.local>
2023-02-15 16:07:49 +05:30
Tejaswini Bandlamudi 9ffaba9c7f
Fix MySQL drivers setup for Revised ITs (#13800)
* download both mysql drivers and use org.mariadb.jdbc.Driver for now

* use com.mysql.jdbc.Driver
2023-02-15 11:03:25 +05:30
Tejaswini Bandlamudi 477bc424d9
Fix GHA tests cache hit miss scenario (#13772)
* rebuild maven project or docker image in case of cache hit miss
* rebuild maven project in case of docker cache hit miss too
* fix docker-restore cache hit fail issue
2023-02-10 08:57:45 -08:00
Elliott Freis e16639121f
Local pathing for tests (#13753)
Co-authored-by: Elliott Freis <elliottfreis@Elliott-Freis.earth.dynamic.blacklight.net>
2023-02-03 20:11:17 -08:00
Elliott Freis c06631037d
Moving to SHA based cache key (#13751)
Co-authored-by: Elliott Freis <elliottfreis@Elliott-Freis.earth.dynamic.blacklight.net>
2023-02-03 15:49:17 -08:00
Tejaswini Bandlamudi 6cb842e76e
update snapshots if cache restore failed otherwise run test normally (not all test mvn dependencies are downloaded during build phase due to skipTests) (#13740) 2023-02-02 08:09:56 -08:00
Tejaswini Bandlamudi 440212c5f9
Fix GHA workflow maven build erroring incase of version updates (#13735)
* build maven sequentially

* run mvn tests in offline mode after retrieving cache
2023-02-01 20:32:57 -08:00
Tejaswini Bandlamudi c95a26cae3
Migrate ITs from Travis to GHA (#13681) 2023-02-01 03:31:29 -08:00
Tejaswini Bandlamudi 7a54524076
install node on runners (#13690) 2023-01-18 16:19:57 +05:30
Tejaswini Bandlamudi 4368b3a071
Migrate jdk8 unit tests from Travis to GHA (#13518)
* migrate UTs form Travis to GHA

* update permissions

* rename file

* set fetch depth to 1

* debugs remote branches

* test with github.ref variable

* fetch github.base_ref for diff

* nit

* test git diff

* run tests

* test code coverage failure scenario

* nit

* nit

* revert code changes

* revert code changes

* Setup diff-test-coverage before tests

* build distribution module at end in packaging check

* nit

* remove redundant steps in static-checks workflow

* drop jdk8 unit tests from Travis
2023-01-13 14:46:58 +05:30
Peter Stöckli df55768535
Add CodeQL workflow (#13477)
* workflower: Add CodeQL workflow

* add modified CodeQL build config
2022-12-21 09:24:39 +05:30
Tejaswini Bandlamudi 136322d13b
clean install before license checks (#13502) 2022-12-05 22:38:03 -08:00
Tejaswini Bandlamudi 30498c1f98
Update gha & travis checks (#13412)
* update static-checks GHA to run sequentially
remove static-checks from travis.yml
move docs, web-console, packaging checks from travis to GHA

* nit

* nit

* groups all checks, runs on 8, 11, 17 jdks

* nit

* adds license info

* update permissions on scripts folder

* nit

* nit

* fix packaging check

* changes naming, cleans repo before license checks

* simulate failure

* bump up license checks

* test license checks failure

* test license checks failure

* test license checks failure

* verify gha script run exit code

* fail fast in case of shell script

* verified fail fast in case of shell script
2022-12-02 15:06:31 +05:30
Tejaswini Bandlamudi 8e9e46b519
Add static-checks Github Action (#13347)
* Adds static-checks github action

* bug fixes

* bug fixes

* adds maven install step

* update permissions on scripts folder

* fix license check errors

* attempt for parallelization

* fix concurrency

* fix naming

* remove intellij inspections to add in different CI pipeline

* minimize naming, add new lines

* setting hadoop profile through matrix

* also runs on push triggers to master and release branches

* changes on review
2022-11-17 17:20:16 +05:30