druid/docs/api-reference
Abhishek Radhakrishnan 7463589b07
Support for bootstrap segments (#16609)
* Initial support for bootstrap segments.

  - Adds a new API in the coordinator.
  - All processes that have storage locations configured (including tasks)
    talk to the coordinator if they can, and fetch bootstrap segments from it.
  - Then load the segments onto the segment cache as part of startup.
  - This addresses the segment bootstrapping logic required by processes before
    they can start serving queries or ingesting.

    This patch also lays the foundation to speed up upgrades.

* Fail open by default if there are any errors talking to the coordinator.

* Add test for failure scenario and cleanup logs.

* Cleanup and add debug log

* Assert the events so we know the list exactly.

* Revert RunRules test.

The rules aren't evaluated if there are no clusters.

* Revert RunRulesTest too.

* Remove debug info.

* Make the API POST and update log.

* Fix up UTs.

* Throw 503 from MetadataResource; clean up exception handling and DruidException.

* Remove unused logger, add verification of metrics and docs.

* Update error message

* Update server/src/main/java/org/apache/druid/server/coordination/SegmentLoadDropHandler.java

Co-authored-by: Kashif Faraz <kashif.faraz@gmail.com>

* Apply suggestions from code review

Co-authored-by: Kashif Faraz <kashif.faraz@gmail.com>

* Adjust test metric expectations with the rename.

* Add BootstrapSegmentResponse container in the response for future extensibility.

* Rename to BootstrapSegmentsInfo for internal consistency.

* Remove unused log.

* Use a member variable for broadcast segments instead of segmentAssigner.

* Minor cleanup

* Add test for loadable bootstrap segments and clarify comment.

* Review suggestions.

---------

Co-authored-by: Kashif Faraz <kashif.faraz@gmail.com>
2024-06-24 09:27:17 -07:00
..
api-reference.md API reference refactor (#14372) 2023-06-26 15:48:54 -07:00
automatic-compaction-api.md [Docs] Fix API placeholder formatting (#16240) 2024-04-12 09:19:13 -07:00
data-management-api.md [Docs] Fix API placeholder formatting (#16240) 2024-04-12 09:19:13 -07:00
dynamic-configuration-api.md [Docs] Fix API placeholder formatting (#16240) 2024-04-12 09:19:13 -07:00
json-querying-api.md [Docs] Fix API placeholder formatting (#16240) 2024-04-12 09:19:13 -07:00
legacy-metadata-api.md Support for bootstrap segments (#16609) 2024-06-24 09:27:17 -07:00
lookups-api.md [Docs] Fix API placeholder formatting (#16240) 2024-04-12 09:19:13 -07:00
retention-rules-api.md [Docs] Fix API placeholder formatting (#16240) 2024-04-12 09:19:13 -07:00
service-status-api.md [Docs] Fix API placeholder formatting (#16240) 2024-04-12 09:19:13 -07:00
sql-api.md [Docs] Fix API placeholder formatting (#16240) 2024-04-12 09:19:13 -07:00
sql-ingestion-api.md [Docs] Fix API placeholder formatting (#16240) 2024-04-12 09:19:13 -07:00
sql-jdbc.md [Docs] Fix API placeholder formatting (#16240) 2024-04-12 09:19:13 -07:00
supervisor-api.md Docs: Clarify that all supervisors can support early handoff (#16588) 2024-06-13 08:43:22 +05:30
tasks-api.md Simplify serialized form of JsonInputFormat. (#15691) 2024-06-05 20:01:14 -07:00