druid/integration-tests-ex/cases
Kashif Faraz fd7864ae33
Improve run time of coordinator duty MarkAsUnusedOvershadowedSegments (#13287)
In clusters with a large number of segments, the duty `MarkAsUnusedOvershadowedSegments`
can take a long very long time to finish. This is because of the costly invocation of 
`timeline.isOvershadowed` which is done for every used segment in every coordinator run.

Changes
- Use `DataSourceSnapshot.getOvershadowedSegments` to get all overshadowed segments
- Iterate over this set instead of all used segments to identify segments that can be marked as unused
- Mark segments as unused in the DB in batches rather than one at a time
- Refactor: Add class `SegmentTimeline` for ease of use and readability while using a
`VersionedIntervalTimeline` of segments.
2022-11-01 20:19:52 +05:30
..
assets Add the new integration test framework (#12368) 2022-08-24 17:03:23 +05:30
cluster Add IT for MSQ task engine using the new IT framework (#12992) 2022-09-22 16:09:47 +05:30
resources Add the new integration test framework (#12368) 2022-08-24 17:03:23 +05:30
src/test Improve run time of coordinator duty MarkAsUnusedOvershadowedSegments (#13287) 2022-11-01 20:19:52 +05:30
.gitignore Add the new integration test framework (#12368) 2022-08-24 17:03:23 +05:30
cluster.sh Add the new integration test framework (#12368) 2022-08-24 17:03:23 +05:30
pom.xml Add IT for MSQ task engine using the new IT framework (#12992) 2022-09-22 16:09:47 +05:30