mirror of https://github.com/apache/druid.git
docs: Migration guide for subquery limit (#16519)
Adds a migration guide for Druid 30 to help users understand the new byte-based subquery limit property maxSubqueryBytes
This commit is contained in:
parent
92e565e3b8
commit
3c72ec8413
|
@ -0,0 +1,60 @@
|
|||
---
|
||||
id: migr-subquery-limit
|
||||
title: "Migration guide: Subquery limit"
|
||||
sidebar_label: Subquery limit
|
||||
---
|
||||
|
||||
<!--
|
||||
~ Licensed to the Apache Software Foundation (ASF) under one
|
||||
~ or more contributor license agreements. See the NOTICE file
|
||||
~ distributed with this work for additional information
|
||||
~ regarding copyright ownership. The ASF licenses this file
|
||||
~ to you under the Apache License, Version 2.0 (the
|
||||
~ "License"); you may not use this file except in compliance
|
||||
~ with the License. You may obtain a copy of the License at
|
||||
~
|
||||
~ http://www.apache.org/licenses/LICENSE-2.0
|
||||
~
|
||||
~ Unless required by applicable law or agreed to in writing,
|
||||
~ software distributed under the License is distributed on an
|
||||
~ "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
|
||||
~ KIND, either express or implied. See the License for the
|
||||
~ specific language governing permissions and limitations
|
||||
~ under the License.
|
||||
-->
|
||||
|
||||
Druid now allows you to set a byte-based limit on subquery size, to prevent brokers from running out of memory when handling large subqueries.
|
||||
Druid uses subqueries as joins as well as in common table expressions, such as WITH.
|
||||
|
||||
The byte-based subquery limit overrides Druid's row-based subquery limit.
|
||||
|
||||
:::info
|
||||
We recommend that you move towards using byte-based limits starting in Druid 30.0.
|
||||
:::
|
||||
|
||||
For queries that generate a large number of rows (5 million or more), we recommend that you don't use `maxSubqueryBytes` from the outset.
|
||||
You can increase `maxSubqueryRows` and then configure the byte-based limit if you find that Druid needs it to process the query.
|
||||
|
||||
## Row-based subquery limit
|
||||
|
||||
Druid uses the `maxSubqueryRows` property to limit the number of rows Druid returns in a subquery.
|
||||
Because this is a row-based limit, it doesn't restrict the overall size of the returned data.
|
||||
|
||||
The `maxSubqueryRows` property is set to 100,000 by default.
|
||||
|
||||
## Enable a byte-based subquery limit
|
||||
|
||||
Set the optional property `maxSubqueryBytes` to set a maximum number of returned bytes.
|
||||
This property takes precedence over `maxSubqueryRows`.
|
||||
|
||||
## Usage considerations
|
||||
|
||||
You can set both `maxSubqueryRows` and `maxSubqueryBytes` at cluster level and override them in individual queries.
|
||||
See [Overriding default query context values](../configuration#overriding-default-query-context-values) for more information.
|
||||
|
||||
## Learn more
|
||||
|
||||
See the following topics for more information:
|
||||
|
||||
- [Query context](../querying/query-context.md) for information on setting query context parameters.
|
||||
- [Broker configuration reference](../configuration#guardrails-for-materialization-of-subqueries) for more information on `maxSubqueryRows` and `maxSubqueryBytes`.
|
|
@ -363,9 +363,10 @@
|
|||
"id": "release-info/migration-guide"
|
||||
},
|
||||
"items": [
|
||||
"release-info/migr-subquery-limit"
|
||||
]
|
||||
}
|
||||
]},
|
||||
"misc/papers-and-talks"
|
||||
]
|
||||
}
|
||||
}
|
||||
|
|
Loading…
Reference in New Issue