druid/docs/operations
Rishabh Singh 8c802e4c9b
Relocating Table Schema Building: Shifting from Brokers to Coordinator for Improved Efficiency (#14985)
In the current design, brokers query both data nodes and tasks to fetch the schema of the segments they serve. The table schema is then constructed by combining the schemas of all segments within a datasource. However, this approach leads to a high number of segment metadata queries during broker startup, resulting in slow startup times and various issues outlined in the design proposal.

To address these challenges, we propose centralizing the table schema management process within the coordinator. This change is the first step in that direction. In the new arrangement, the coordinator will take on the responsibility of querying both data nodes and tasks to fetch segment schema and subsequently building the table schema. Brokers will now simply query the Coordinator to fetch table schema. Importantly, brokers will still retain the capability to build table schemas if the need arises, ensuring both flexibility and resilience.
2023-11-04 19:33:25 +05:30
..
alerts.md Docusaurus build framework + ingestion doc refresh. (#8311) 2019-08-20 21:48:59 -07:00
auth-ldap.md docs: LDAP trust store property clarification (#15028) 2023-11-02 13:00:08 -07:00
auth.md Update Ingestion section (#14023) 2023-05-19 09:42:27 -07:00
basic-cluster-tuning.md remove group-by v1 (#14866) 2023-08-23 12:44:06 -07:00
clean-metadata-store.md Docusaurus2 upgrade for master (#14411) 2023-08-16 19:01:21 -07:00
deep-storage-migration.md cleaning up and fixing links (#10528) 2020-12-17 13:37:43 -08:00
dump-segment.md Improve memory efficiency of WrappedRoaringBitmap. (#13889) 2023-03-09 15:48:02 -08:00
durable-storage.md docs: durable storage azure cleanup (#15120) 2023-10-31 15:20:38 -07:00
dynamic-config-provider.md Docs: Splits Kafka topic. Adds detailed example for kafka inputFormat (#11912) 2021-11-12 13:02:23 -08:00
export-metadata.md Update Kafka loading docs to use the streaming data loader (#8544) 2019-09-22 15:00:52 -07:00
high-availability.md fixes web console introduction and addresses linking issues (#11609) 2021-08-18 08:37:05 -07:00
http-compression.md De-incubation cleanup in code, docs, packaging (#9108) 2020-01-03 12:33:19 -05:00
insert-segment-to-db.md De-incubation cleanup in code, docs, packaging (#9108) 2020-01-03 12:33:19 -05:00
java.md Claim full support for Java 17. (#14384) 2023-07-07 12:52:35 -07:00
kubernetes.md Update kubernetes.md (#13858) 2023-02-28 11:20:24 -08:00
metadata-migration.md Clean up after add kill bufferPeriod (#14868) 2023-08-19 00:00:04 +05:30
metrics.md Relocating Table Schema Building: Shifting from Brokers to Coordinator for Improved Efficiency (#14985) 2023-11-04 19:33:25 +05:30
migrate-from-firehose-ingestion.md Update Ingestion section (#14023) 2023-05-19 09:42:27 -07:00
mixed-workloads.md Docusaurus2 upgrade for master (#14411) 2023-08-16 19:01:21 -07:00
other-hadoop.md Removes support for Hadoop 2 (#14763) 2023-08-09 17:47:52 +05:30
password-provider.md Security overview documentation (#10339) 2020-11-19 15:24:58 -08:00
pull-deps.md Docusaurus2 upgrade for master (#14411) 2023-08-16 19:01:21 -07:00
request-logging.md Add sql planning time metric (#12923) 2022-08-22 11:09:44 +05:30
reset-cluster.md Update reset-cluster.md (#10990) 2021-03-29 20:38:35 -07:00
rolling-updates.md add links to release notes, light refactor of landing page (#11051) 2021-05-07 14:26:47 -07:00
rule-configuration.md Docusaurus2 upgrade for master (#14411) 2023-08-16 19:01:21 -07:00
security-overview.md remove references to Jupyter notebooks within the Druid repo (#15143) 2023-11-01 13:17:06 -07:00
security-user-auth.md Update Ingestion section (#14023) 2023-05-19 09:42:27 -07:00
segment-optimization.md Docusaurus2 upgrade for master (#14411) 2023-08-16 19:01:21 -07:00
single-server.md Docs: reword single server page (#13659) 2023-01-11 21:12:52 -08:00
tls-support.md Update Ingestion section (#14023) 2023-05-19 09:42:27 -07:00
use_sbt_to_build_fat_jar.md Docusaurus build framework + ingestion doc refresh. (#8311) 2019-08-20 21:48:59 -07:00
web-console.md Docusaurus2 upgrade for master (#14411) 2023-08-16 19:01:21 -07:00