* Make resources an ordered collection so it's deterministic.
* test cleanup
* fixup docs.
* Replace deprecated ObjectNode#put() calls with ObjectNode#set().
Currently, there is no way to run ITs if unit-tests fail on coverage. This PR allows Revised, Standard ITs to run even when unit-tests fail on coverage errors, still failing the workflow. This PR also fixes existing GHA workflow naming.
Co-authored-by: Charles Smith <techdocsmith@gmail.com>
Co-authored-by: Victoria Lim <vtlim@users.noreply.github.com>
Co-authored-by: Victoria Lim <lim.t.victoria@gmail.com>
* Fixing an issue with filtering on a single dimension by converting In filter to a selector filter as needed with Filters.toFilter
* Adding a test so that any future refactoring does not break this behavior
* Made comment a bit more meaningful
The "new" IT framework provides a convenient way to package and run integration tests (ITs), but only for core modules. We have a use case to run an IT for a contrib extension: the proposed gRPC query extension. This PR provides the IT framework functionality to allow non-core ITs.
* Be able to load segments on Peons
This change introduces a new config on WorkerConfig
that indicates how many bytes of each storage
location to use for storage of a task. Said config
is divided up amongst the locations and slots
and then used to set TaskConfig.tmpStorageBytesPerTask
The Peons use their local task dir and
tmpStorageBytesPerTask as their StorageLocations for
the SegmentManager such that they can accept broadcast
segments.
Changes:
- Replace `OverlordHelper` with `OverlordDuty` to align with `CoordinatorDuty`
- Each duty has a `run()` method and defines a `Schedule` with an initial delay and period.
- Update existing duties `TaskLogAutoCleaner` and `DurableStorageCleaner`
- Add utility class `Configs`
- Update log, error messages and javadocs
- Other minor style improvements
* queue tasks if all slots in use
* Declare hamcrest-core dependency
* Use AtomicBoolean for shutdown requested
* Use AtomicReference for peon lifecycle state
* fix uninitialized read error
* fix indentations
* Make tasks protected
* fix KubernetesTaskRunnerConfig deserialization
* ensure k8s task runner max capacity is Integer.MAX_VALUE
* set job duration as task status duration
* Address pr comments
---------
Co-authored-by: George Shiqi Wu <george.wu@imply.io>
Instead of downloading jdk everytime we run CI, we're using inbuilt temurin jdk distributions 8, 11, 17 by settiing JAVA_HOME variable. This is not working as expected since we were not setting this as global environment variable as a result all CI builds are running on jdk11. This PR fixes the issue.
Changes:
- Do not allow retention rules for any datasource or cluster to be null
- Allow empty rules at the datasource level but not at the cluster level
- Add validation to ensure that `druid.manager.rules.defaultRule` is always set correctly
- Minor style refactors
Abhishek Radhakrishnan
Victoria Lim
Nhi Pham
George Shiqi Wu
Tejaswini Bandlamudi
317brian
Katya Macedo
Vadim Ogievetsky
imply-cheddar
Charles Smith
Clint Wylie
Soumyava
Adarsh Sanjeev
Peter Marshall
Paul Rogers
AmatyaAvadhanula
Kashif Faraz
Nicholas Lippis
Abhishek Agarwal