From 95a5ae0373ada64529144304ada9ad3ea4a8cbbb Mon Sep 17 00:00:00 2001 From: Oleg Zaezdny Date: Sun, 1 Nov 2015 21:56:28 +0200 Subject: [PATCH] Docs improved by adding more details about local cache and memory for segments on historicals. --- docs/content/configuration/caching.md | 2 +- docs/content/operations/performance-faq.md | 2 +- 2 files changed, 2 insertions(+), 2 deletions(-) diff --git a/docs/content/configuration/caching.md b/docs/content/configuration/caching.md index f1ca264ae10..ddee7c49736 100644 --- a/docs/content/configuration/caching.md +++ b/docs/content/configuration/caching.md @@ -24,7 +24,7 @@ for both broker and historical nodes, when defined in the common properties file #### Local Cache -A simple in-memory LRU cache. +A simple in-memory LRU cache. Local cache resides in JVM heap memory, so if you enable it, make sure you increase heap size accordingly. |Property|Description|Default| |--------|-----------|-------| diff --git a/docs/content/operations/performance-faq.md b/docs/content/operations/performance-faq.md index 37b04447647..e6f6ffff6d3 100644 --- a/docs/content/operations/performance-faq.md +++ b/docs/content/operations/performance-faq.md @@ -10,7 +10,7 @@ The size of the JVM heap really depends on the type of Druid node you are runnin [Broker nodes](../design/broker.html) uses the JVM heap mainly to merge results from historicals and real-times. Brokers also use off-heap memory and processing threads for groupBy queries. We recommend 20G-30G of heap here. -[Historical nodes](../design/historical.html) use off-heap memory to store intermediate results, and by default, all segments are memory mapped before they can be queried. Typically, the more memory is available on a historical node, the more segments can be served without the possibility of data being paged on to disk. On historicals, the JVM heap is used for [GroupBy queries](../querying/groupbyquery.html), some data structures used for intermediate computation, and general processing. One way to calculate how much space there is for segments is: memory_for_segments = total_memory - heap - direct_memory - jvm_overhead. +[Historical nodes](../design/historical.html) use off-heap memory to store intermediate results, and by default, all segments are memory mapped before they can be queried. Typically, the more memory is available on a historical node, the more segments can be served without the possibility of data being paged on to disk. On historicals, the JVM heap is used for [GroupBy queries](../querying/groupbyquery.html), some data structures used for intermediate computation, and general processing. One way to calculate how much space there is for segments is: memory_for_segments = total_memory - heap - direct_memory - jvm_overhead. Note that total_memory here refers to the memory available to the cgroup (if running on Linux), which for default cases is going to be all the system memory. We recommend 250mb * (processing.numThreads) for the heap.