From ced8be30446d85c8e861f51ffd609ae85dcb3d61 Mon Sep 17 00:00:00 2001 From: Katya Macedo <38017980+ektravel@users.noreply.github.com> Date: Mon, 4 Mar 2024 10:58:52 -0600 Subject: [PATCH] docs: Add upgrade notes for Druid 29.0.0 (#16022) --- docs/release-info/upgrade-notes.md | 63 ++++++++++++++++++++++++++++++ 1 file changed, 63 insertions(+) diff --git a/docs/release-info/upgrade-notes.md b/docs/release-info/upgrade-notes.md index 46e5ed6fc1a..9cb58e42bf2 100644 --- a/docs/release-info/upgrade-notes.md +++ b/docs/release-info/upgrade-notes.md @@ -26,6 +26,69 @@ The upgrade notes assume that you are upgrading from the Druid version that imme For the full release notes for a specific version, see the [releases page](https://github.com/apache/druid/releases). +## 29.0.0 + +### Upgrade notes + +#### Changed `equals` filter for native queries + +The [equality filter](https://druid.apache.org/docs/latest/querying/filters#equality-filter) on mixed type `auto` columns that contain arrays must now be filtered as their presenting type. This means that if any rows are arrays (for example, the segment metadata and `information_schema` reports the type as some array type), then the native queries must also filter as if they are some array type. + +This change impacts mixed type `auto` columns that contain both scalars and arrays. It doesn't impact SQL, which already has this limitation due to how the type presents itself. + +[#15503](https://github.com/apache/druid/pull/15503) + +#### Console automatically sets `arrayIngestMode` for MSQ queries + +Druid console now configures the `arrayIngestMode` parameter in the data loading flow, and its value can persist across the SQL tab unless manually updated. When loading multi-value dimensions or arrays in the Druid console, note the value of the `arrayIngestMode` parameter to prevent mixing multi-value dimensions and arrays in the same column of a data source. + +[#15588](https://github.com/apache/druid/pull/15588) + +#### Improved concurrent append and replace (experimental) + +You no longer have to manually determine the task lock type for concurrent append and replace (experimental) with the `taskLockType` task context. Instead, Druid can now determine it automatically for you. You can use the context parameter `"useConcurrentLocks": true` for individual tasks and datasources or enable concurrent append and replace at a cluster level using `druid.indexer.task.default.context`. + +[#15684](https://github.com/apache/druid/pull/15684) + +#### Enabled empty ingest queries + +The MSQ task engine now allows empty ingest queries by default. For queries that don't generate any output rows, the MSQ task engine reports zero values for `numTotalRows` and `totalSizeInBytes` instead of null. Previously, ingest queries that produced no data would fail with the `InsertCannotBeEmpty` MSQ fault. + +To revert to the original behavior, set the MSQ query parameter `failOnEmptyInsert` to `true`. + +[#15495](https://github.com/apache/druid/pull/15495) [#15674](https://github.com/apache/druid/pull/15674) + +#### Enabled query request queuing by default when total laning is turned on + +When query scheduler threads are less than server HTTP threads, total laning turns on. +This reserves some HTTP threads for non-query requests such as health checks. +The total laning previously would reject any query request that exceeds the lane capacity. +Now, excess requests will instead be queued with a timeout equal to `MIN(Integer.MAX_VALUE, druid.server.http.maxQueryTimeout)`. + +[#15440](https://github.com/apache/druid/pull/15440) + +#### Changed how empty or null array columns are stored + +Columns ingested with the auto column indexer that contain only empty or null arrays are now stored as `ARRAY` instead of `COMPLEX`. + +[#15505](https://github.com/apache/druid/pull/15505) + +#### Changed how Druid allocates weekly segments + +When the requested granularity is a month or larger but a segment can't be allocated, Druid resorts to day partitioning. +Unless explicitly specified, Druid skips week-granularity segments for data partitioning because these segments don't align with the end of the month or more coarse-grained intervals. + +Previously, if Druid couldn't allocate segments by month, it tried allocating them by week next. +In the new behavior, Druid skips partitioning by week and goes directly to day. Week segments can only be allocated if the chosen partitioning in the append task is WEEK. + +[#15589](https://github.com/apache/druid/pull/15589) + +#### Removed the `auto` search strategy + +Removed the `auto` search strategy from the native search query. Setting `searchStrategy` to `auto` is now equivalent to `useIndexes`. + +[#15550](https://github.com/apache/druid/pull/15550) + ## 28.0.0 ### Upgrade notes