mirror of https://github.com/apache/druid.git
182c4fad29
* Kinesis: More robust default fetch settings. 1) Default recordsPerFetch and recordBufferSize based on available memory rather than using hardcoded numbers. For this, we need an estimate of record size. Use 10 KB for regular records and 1 MB for aggregated records. With 1 GB heaps, 2 processors per task, and nonaggregated records, recordBufferSize comes out to the same as the old default (10000), and recordsPerFetch comes out slightly lower (1250 instead of 4000). 2) Default maxRecordsPerPoll based on whether records are aggregated or not (100 if not aggregated, 1 if aggregated). Prior default was 100. 3) Default fetchThreads based on processors divided by task count on Indexers, rather than overall processor count. 4) Additionally clean up the serialized JSON a bit by adding various JsonInclude annotations. * Updates for tests. * Additional important verify. |
||
|---|---|---|
| .. | ||
| _bin | ||
| assets | ||
| comparisons | ||
| configuration | ||
| data-management | ||
| dependencies | ||
| design | ||
| development | ||
| ingestion | ||
| misc | ||
| multi-stage-query | ||
| operations | ||
| querying | ||
| tutorials | ||