mirror of https://github.com/apache/druid.git
Docs - Add links to Basic Tuning guide in process pages (#12741)
Added link to the relevant section of the Basic Cluster Tuning page on each process page. This is in order to improve access to this information, which is not easy to find through search or nav.
This commit is contained in:
parent
adbebc174a
commit
f665a0c077
|
@ -27,6 +27,8 @@ title: "Broker"
|
||||||
|
|
||||||
For Apache Druid Broker Process Configuration, see [Broker Configuration](../configuration/index.md#broker).
|
For Apache Druid Broker Process Configuration, see [Broker Configuration](../configuration/index.md#broker).
|
||||||
|
|
||||||
|
For basic tuning guidance for the Broker process, see [Basic cluster tuning](../operations/basic-cluster-tuning.md#broker).
|
||||||
|
|
||||||
### HTTP endpoints
|
### HTTP endpoints
|
||||||
|
|
||||||
For a list of API endpoints supported by the Broker, see [Broker API](../operations/api-reference.md#broker).
|
For a list of API endpoints supported by the Broker, see [Broker API](../operations/api-reference.md#broker).
|
||||||
|
|
|
@ -27,6 +27,8 @@ title: "Coordinator Process"
|
||||||
|
|
||||||
For Apache Druid Coordinator Process Configuration, see [Coordinator Configuration](../configuration/index.md#coordinator).
|
For Apache Druid Coordinator Process Configuration, see [Coordinator Configuration](../configuration/index.md#coordinator).
|
||||||
|
|
||||||
|
For basic tuning guidance for the Coordinator process, see [Basic cluster tuning](../operations/basic-cluster-tuning.md#coordinator).
|
||||||
|
|
||||||
### HTTP endpoints
|
### HTTP endpoints
|
||||||
|
|
||||||
For a list of API endpoints supported by the Coordinator, see [Coordinator API](../operations/api-reference.md#coordinator).
|
For a list of API endpoints supported by the Coordinator, see [Coordinator API](../operations/api-reference.md#coordinator).
|
||||||
|
|
|
@ -27,6 +27,8 @@ title: "Historical Process"
|
||||||
|
|
||||||
For Apache Druid Historical Process Configuration, see [Historical Configuration](../configuration/index.md#historical).
|
For Apache Druid Historical Process Configuration, see [Historical Configuration](../configuration/index.md#historical).
|
||||||
|
|
||||||
|
For basic tuning guidance for the Historical process, see [Basic cluster tuning](../operations/basic-cluster-tuning.md#historical).
|
||||||
|
|
||||||
### HTTP endpoints
|
### HTTP endpoints
|
||||||
|
|
||||||
For a list of API endpoints supported by the Historical, please see the [API reference](../operations/api-reference.md#historical).
|
For a list of API endpoints supported by the Historical, please see the [API reference](../operations/api-reference.md#historical).
|
||||||
|
@ -39,21 +41,21 @@ org.apache.druid.cli.Main server historical
|
||||||
|
|
||||||
### Loading and serving segments
|
### Loading and serving segments
|
||||||
|
|
||||||
Each Historical process copies or "pulls" segment files from Deep Storage to local disk in an area called the *segment cache*. Set the `druid.segmentCache.locations` to configure the size and location of the segment cache on each Historical process. See [Historical general configuration](../configuration/index.html#historical-general-configuration).
|
Each Historical process copies or "pulls" segment files from Deep Storage to local disk in an area called the *segment cache*. Set the `druid.segmentCache.locations` to configure the size and location of the segment cache on each Historical process. See [Historical general configuration](../configuration/index.md#historical-general-configuration).
|
||||||
|
|
||||||
See the [Tuning Guide](../operations/basic-cluster-tuning.html#segment-cache-size) for more information.
|
See the [Tuning Guide](../operations/basic-cluster-tuning.md#segment-cache-size) for more information.
|
||||||
|
|
||||||
The [Coordinator](../design/coordinator.html) controls the assignment of segments to Historicals and the balance of segments between Historicals. Historical processes do not communicate directly with each other, nor do they communicate directly with the Coordinator. Instead, the Coordinator creates ephemeral entries in Zookeeper in a [load queue path](../configuration/index.html#path-configuration). Each Historical process maintains a connection to Zookeeper, watching those paths for segment information.
|
The [Coordinator](../design/coordinator.html) controls the assignment of segments to Historicals and the balance of segments between Historicals. Historical processes do not communicate directly with each other, nor do they communicate directly with the Coordinator. Instead, the Coordinator creates ephemeral entries in Zookeeper in a [load queue path](../configuration/index.md#path-configuration). Each Historical process maintains a connection to Zookeeper, watching those paths for segment information.
|
||||||
|
|
||||||
For more information about how the Coordinator assigns segments to Historical processes, see [Coordinator](../design/coordinator.html).
|
For more information about how the Coordinator assigns segments to Historical processes, see [Coordinator](../design/coordinator.md).
|
||||||
|
|
||||||
When a Historical process detects a new entry in the Zookeeper load queue, it checks its own segment cache. If no information about the segment exists there, the Historical process first retrieves metadata from Zookeeper about the segment, including where the segment is located in Deep Storage and how it needs to decompress and process it.
|
When a Historical process detects a new entry in the Zookeeper load queue, it checks its own segment cache. If no information about the segment exists there, the Historical process first retrieves metadata from Zookeeper about the segment, including where the segment is located in Deep Storage and how it needs to decompress and process it.
|
||||||
|
|
||||||
For more information about segment metadata and Druid segments in general, see [Segments](../design/segments.html).
|
For more information about segment metadata and Druid segments in general, see [Segments](../design/segments.md).
|
||||||
|
|
||||||
After a Historical process pulls down and processes a segment from Deep Storage, Druid advertises the segment as being available for queries from the Broker. This announcement by the Historical is made via Zookeeper, in a [served segments path](../configuration/index.html#path-configuration).
|
After a Historical process pulls down and processes a segment from Deep Storage, Druid advertises the segment as being available for queries from the Broker. This announcement by the Historical is made via Zookeeper, in a [served segments path](../configuration/index.md#path-configuration).
|
||||||
|
|
||||||
For more information about how the Broker determines what data is available for queries, please see [Broker](broker.html).
|
For more information about how the Broker determines what data is available for queries, please see [Broker](broker.md).
|
||||||
|
|
||||||
To make data from the segment cache available for querying as soon as possible, Historical services search the local segment cache upon startup and advertise the segments found there.
|
To make data from the segment cache available for querying as soon as possible, Historical services search the local segment cache upon startup and advertise the segments found there.
|
||||||
|
|
||||||
|
@ -63,7 +65,7 @@ The segment cache uses [memory mapping](https://en.wikipedia.org/wiki/Mmap). The
|
||||||
|
|
||||||
At query time, if the required part of a segment file is available in the memory mapped cache or "page cache", the Historical re-uses it and reads it directly from memory. If it is not in the memory-mapped cache, the Historical reads that part of the segment from disk. In this case, there is potential for new data to flush other segment data from memory. This means that if free operating system memory is close to `druid.server.maxSize`, the more likely that segment data will be available in memory and reduce query times. Conversely, the lower the free operating system memory, the more likely a Historical is to read segments from disk.
|
At query time, if the required part of a segment file is available in the memory mapped cache or "page cache", the Historical re-uses it and reads it directly from memory. If it is not in the memory-mapped cache, the Historical reads that part of the segment from disk. In this case, there is potential for new data to flush other segment data from memory. This means that if free operating system memory is close to `druid.server.maxSize`, the more likely that segment data will be available in memory and reduce query times. Conversely, the lower the free operating system memory, the more likely a Historical is to read segments from disk.
|
||||||
|
|
||||||
Note that this memory-mapped segment cache is in addition to other [query-level caches](../querying/caching.html).
|
Note that this memory-mapped segment cache is in addition to other [query-level caches](../querying/caching.md).
|
||||||
|
|
||||||
### Querying segments
|
### Querying segments
|
||||||
|
|
||||||
|
|
|
@ -27,6 +27,8 @@ title: "MiddleManager Process"
|
||||||
|
|
||||||
For Apache Druid MiddleManager Process Configuration, see [Indexing Service Configuration](../configuration/index.md#middlemanager-and-peons).
|
For Apache Druid MiddleManager Process Configuration, see [Indexing Service Configuration](../configuration/index.md#middlemanager-and-peons).
|
||||||
|
|
||||||
|
For basic tuning guidance for the MiddleManager process, see [Basic cluster tuning](../operations/basic-cluster-tuning.md#middlemanager).
|
||||||
|
|
||||||
### HTTP endpoints
|
### HTTP endpoints
|
||||||
|
|
||||||
For a list of API endpoints supported by the MiddleManager, please see the [API reference](../operations/api-reference.md#middlemanager).
|
For a list of API endpoints supported by the MiddleManager, please see the [API reference](../operations/api-reference.md#middlemanager).
|
||||||
|
|
|
@ -27,6 +27,8 @@ title: "Overlord Process"
|
||||||
|
|
||||||
For Apache Druid Overlord Process Configuration, see [Overlord Configuration](../configuration/index.md#overlord).
|
For Apache Druid Overlord Process Configuration, see [Overlord Configuration](../configuration/index.md#overlord).
|
||||||
|
|
||||||
|
For basic tuning guidance for the Overlord process, see [Basic cluster tuning](../operations/basic-cluster-tuning.md#overlord).
|
||||||
|
|
||||||
### HTTP endpoints
|
### HTTP endpoints
|
||||||
|
|
||||||
For a list of API endpoints supported by the Overlord, please see the [API reference](../operations/api-reference.md#overlord).
|
For a list of API endpoints supported by the Overlord, please see the [API reference](../operations/api-reference.md#overlord).
|
||||||
|
|
|
@ -27,6 +27,8 @@ title: "Peons"
|
||||||
|
|
||||||
For Apache Druid Peon Configuration, see [Peon Query Configuration](../configuration/index.md#peon-query-configuration) and [Additional Peon Configuration](../configuration/index.md#additional-peon-configuration).
|
For Apache Druid Peon Configuration, see [Peon Query Configuration](../configuration/index.md#peon-query-configuration) and [Additional Peon Configuration](../configuration/index.md#additional-peon-configuration).
|
||||||
|
|
||||||
|
For basic tuning guidance for MiddleManager tasks, see [Basic cluster tuning](../operations/basic-cluster-tuning.md#task-configurations).
|
||||||
|
|
||||||
### HTTP endpoints
|
### HTTP endpoints
|
||||||
|
|
||||||
For a list of API endpoints supported by the Peon, please see the [Peon API reference](../operations/api-reference.md#peon).
|
For a list of API endpoints supported by the Peon, please see the [Peon API reference](../operations/api-reference.md#peon).
|
||||||
|
|
|
@ -36,6 +36,8 @@ In addition to query routing, the Router also runs the [Druid console](../operat
|
||||||
|
|
||||||
For Apache Druid Router Process Configuration, see [Router Configuration](../configuration/index.md#router).
|
For Apache Druid Router Process Configuration, see [Router Configuration](../configuration/index.md#router).
|
||||||
|
|
||||||
|
For basic tuning guidance for the Router process, see [Basic cluster tuning](../operations/basic-cluster-tuning.md#router).
|
||||||
|
|
||||||
### HTTP endpoints
|
### HTTP endpoints
|
||||||
|
|
||||||
For a list of API endpoints supported by the Router, see [Router API](../operations/api-reference.md#router).
|
For a list of API endpoints supported by the Router, see [Router API](../operations/api-reference.md#router).
|
||||||
|
|
Loading…
Reference in New Issue