druid/extensions-core
Akshat Jain fbd305af0f
MSQ WF: Batch multiple PARTITION BY keys for processing (#16823)
Currently, if we have a query with window function having PARTITION BY xyz, and we have a million unique values for xyz each having 1 row, we'd end up creating a million individual RACs for processing, each having a single row. This is unnecessary, and we can batch the PARTITION BY keys together for processing, and process them only when we can't batch further rows to adhere to maxRowsMaterialized config.

The previous iteration of this PR was simplifying WindowOperatorQueryFrameProcessor to run all operators on all the rows instead of creating smaller RACs per partition by key. That approach was discarded in favor of the batching approach, and the details are summarized here: #16823 (comment).
2024-08-28 11:32:47 +05:30
..
avro-extensions Better error handling when retrieving Avro schemas from registry (#16684) 2024-07-02 16:48:34 -07:00
azure-extensions remove Firehose and FirehoseFactory (#16758) 2024-07-19 14:37:21 -07:00
datasketches generic block compressed complex columns (#16863) 2024-08-27 00:34:41 -07:00
druid-aws-rds-extensions Prepare master branch for 31.0.0 release (#16333) 2024-04-26 09:22:43 +05:30
druid-basic-security Fix excessive logging from druid-basic-security (#16767) 2024-07-22 08:33:00 +05:30
druid-bloom-filter generic block compressed complex columns (#16863) 2024-08-27 00:34:41 -07:00
druid-catalog Quidem record (#16624) 2024-08-05 14:58:32 +02:00
druid-kerberos Prepare master branch for 31.0.0 release (#16333) 2024-04-26 09:22:43 +05:30
druid-pac4j Remove index_realtime and index_realtime_appenderator tasks (#16602) 2024-06-24 20:13:33 -07:00
druid-ranger-security Prepare master branch for 31.0.0 release (#16333) 2024-04-26 09:22:43 +05:30
ec2-extensions Prepare master branch for 31.0.0 release (#16333) 2024-04-26 09:22:43 +05:30
google-extensions Prepare master branch for 31.0.0 release (#16333) 2024-04-26 09:22:43 +05:30
hdfs-storage Prepare master branch for 31.0.0 release (#16333) 2024-04-26 09:22:43 +05:30
histogram generic block compressed complex columns (#16863) 2024-08-27 00:34:41 -07:00
kafka-extraction-namespace Prepare master branch for 31.0.0 release (#16333) 2024-04-26 09:22:43 +05:30
kafka-indexing-service Track IngestionState more accurately in realtime tasks. (#16934) 2024-08-22 11:43:46 +05:30
kinesis-indexing-service Track IngestionState more accurately in realtime tasks. (#16934) 2024-08-22 11:43:46 +05:30
kubernetes-extensions #16717 defer provider instantiation in Kubernetes Module (#16726) 2024-07-16 13:05:28 -07:00
lookups-cached-global upgrade mysql:mysql-connector-java to 8.2.0 (#16024) 2024-05-06 21:58:37 +08:00
lookups-cached-single upgrade mysql:mysql-connector-java to 8.2.0 (#16024) 2024-05-06 21:58:37 +08:00
multi-stage-query MSQ WF: Batch multiple PARTITION BY keys for processing (#16823) 2024-08-28 11:32:47 +05:30
mysql-metadata-storage remove Firehose and FirehoseFactory (#16758) 2024-07-19 14:37:21 -07:00
orc-extensions Prepare master branch for 31.0.0 release (#16333) 2024-04-26 09:22:43 +05:30
parquet-extensions Refactor: Clean up `DecimalParquetInputTest` using Assume (#16436) 2024-05-14 21:13:07 +05:30
postgresql-metadata-storage remove Firehose and FirehoseFactory (#16758) 2024-07-19 14:37:21 -07:00
protobuf-extensions Prepare master branch for 31.0.0 release (#16333) 2024-04-26 09:22:43 +05:30
s3-extensions Reduce logging in RetryableS3OutputStream (#16853) 2024-08-08 10:42:40 +05:30
simple-client-sslcontext Prepare master branch for 31.0.0 release (#16333) 2024-04-26 09:22:43 +05:30
stats generic block compressed complex columns (#16863) 2024-08-27 00:34:41 -07:00
testing-tools Quidem record (#16624) 2024-08-05 14:58:32 +02:00