Apache
/
druid
mirror of https://github.com/apache/druid.git
1
0
Fork 0
Code Issues Packages Projects Releases Wiki Activity
14,426 Commits 111 Branches 503 Tags 728 MiB
Commit Graph

1118 Commits

Author SHA1 Message Date
Zoltan Haindrich 090f937d58 Merge branch 'quidem-record' into quidem-msq 2024-08-05 09:03:53 +00:00
Zoltan Haindrich e6add9ea84 Merge remote-tracking branch 'apache/master' into quidem-record 2024-08-05 07:04:02 +00:00
Sree Charan Manamala c7eacd079e
fallback SQL IN filter to expression filter when VirtualColumnRegistry is null (#16836) 2024-08-05 11:27:51 +05:30
Abhishek Radhakrishnan 31b43753fb
Add `druid.indexing.formats.stringMultiValueHandlingMode` system config (#16822) 
This patch introduces an optional cluster configuration, druid.indexing.formats.stringMultiValueHandlingMode, allowing operators to override the default mode SORTED_SET for string dimensions. The possible values for the config are SORTED_SET, SORTED_ARRAY, or ARRAY (SORTED_SET is the default). Case insensitive values are allowed.
While this cluster property allows users to manage the multi-value handling mode for string dimension types, it's recommended to migrate to using real array types instead of MVDs.
 
This fixes a long-standing issue where compaction will honor the configured cluster wide property instead of rewriting it as the default SORTED_ARRAY always, even if the data was originally ingested with ARRAY or SORTED_SET.
 2024-08-03 10:23:44 -07:00
Zoltan Haindrich 5e81a026e9 Merge branch 'quidem-record' into quidem-msq 2024-07-31 15:27:59 +00:00
Zoltan Haindrich a03fb49f4f remove exception 2024-07-30 16:34:06 +00:00
Zoltan Haindrich 5f6290eb54 use updated hook class 2024-07-30 16:11:57 +00:00
Zoltan Haindrich de207c7295 move key 2024-07-30 16:04:11 +00:00
Zoltan Haindrich b1ab252b31 Merge branch 'quidem-record' into quidem-msq 2024-07-30 16:03:33 +00:00
Zoltan Haindrich eb2a047e4b Merge remote-tracking branch 'apache/master' into quidem-record 2024-07-30 14:24:37 +00:00
Zoltan Haindrich b345dd9d03 updates/fix style/etc 2024-07-30 13:25:40 +00:00
Zoltan Haindrich df42245685 add apidoc/etc 2024-07-30 13:14:51 +00:00
Zoltan Haindrich 9ac26e3a89 wire-in hookdispatcher thru connection/etc 2024-07-30 12:29:36 +00:00
Zoltan Haindrich 78b75d3e8e move more to non-static 2024-07-30 10:42:41 +00:00
Zoltan Haindrich f6cc540368 use druidhookdispatcherr#1 2024-07-30 10:33:57 +00:00
Zoltan Haindrich ce667eeb5e move stuff around / prepare to unglobalize 2024-07-30 10:23:35 +00:00
Zoltan Haindrich c7cde31a89
HAVING clauses may not contain window functions (#16742) 
Rejects having clauses if they contain windowed expressions.
Also added a check to produce a more descriptive error if an OVER expression
reaches the filter translation layer.

---------

Co-authored-by: Benedict Jin <asdf2014@apache.org>
 2024-07-29 04:11:36 -04:00
Sree Charan Manamala 9b76d13ff8
Check for Aggregation inside a window clause when syntax used as - WINDOW W AS DEF (#16801) 2024-07-26 11:18:35 +02:00
Zoltan Haindrich ed9ef1f635 checkstryle 2024-07-26 03:39:55 +00:00
Clint Wylie 14954c7eb9
serialize legacy as false for scan query for rolling downgrade/upgrade (#16793) 
Fixes rolling downgrades/upgrades after #16659 by hard coding scan query "legacy":false since it is a required property during deserialization.
 2024-07-25 14:51:58 +05:30
Zoltan Haindrich 8bb38a04a5 fix FIMXE 2024-07-25 03:33:33 +00:00
Zoltan Haindrich 7e3fab5bf9
Make WindowFrames more specific (#16741) 
Changes the WindowFrame internals / representation a bit; introduces dedicated frametypes for rows and groups which corresponds to the implemented processing methods
 2024-07-25 04:57:36 +02:00
Zoltan Haindrich d010b488a7 cleanup 2024-07-24 17:23:15 +00:00
Zoltan Haindrich 7428da51de cleanup 2024-07-24 17:20:42 +00:00
Akshat Jain a0437b6c93
MSQ window functions: Fix partition boundary issues for arrays (#16780) 
* MSQ window functions: Fix partition boundary issues for arrays

* Address review comments

* Cache type strategies

* Trigger Build

* Convert typeStrategies from list to array
 2024-07-24 18:47:04 +05:30
Zoltan Haindrich a9dcb2da46 Merge branch 'quidem-record' into quidem-msq 2024-07-24 10:59:41 +00:00
Sree Charan Manamala 3f4d66c399
Check for Unsupported Aggregation with Distinct when useApproxCountDistinct is enabled (#16770) 
* init

* add NativelySupportsDistinct

* refactor

* javadoc

* refactor

* fix tests

* fix drill tests

* comments

* Update sql/src/test/java/org/apache/druid/sql/calcite/DrillWindowQueryTest.java

---------

Co-authored-by: Benedict Jin <asdf2014@apache.org>
 2024-07-24 11:13:22 +08:00
Laksh Singla 11bb40981e
Deduce type from the aggregators when materializing subquery results (#16703) 
For aggregators like StringFirst/Last, whose intermediate type isn't the same as the final type, using them in GroupBy, TopN or Timeseries subqueries causes a fallback when maxSubqueryBytes is set. This is because we assume that the finalization is not known, due to which the row signature cannot determine whether to use the intermediate or the final type, and it puts it as null. This PR figures out the finalization from the query context and uses the intermediate or the final type appropriately.
 2024-07-23 11:52:39 +05:30
Akshat Jain c45d4fdbca
MSQ window functions: Minor cleanup for empty over clause related flows + Exhaustive tests (#16754) 
* MSQ window functions: Revamp logic to create separate window stages when empty over() clause is present

* Fix tests

* Revert changes of creating separate stages for empty over clause

* Address review comments
 2024-07-23 11:37:34 +05:30
Akshat Jain 6a2348b78b
Preemptive restriction for queries with approximate count distinct on complex columns of unsupported type (#16682) 
This PR aims to check if the complex column being queried aligns with the supported types in the aggregator and aggregator factories, and throws a user-friendly error message if they don't.
 2024-07-22 21:34:06 +05:30
Sree Charan Manamala 149d7c5207
Throw exceptions in SqlValidator when DISTINCT used over WINDOW (#16738) 
* Throw exception if DISTINCT used with window functions aggregate call
* Improve error message when unsupported aggregations are used with window functions
 2024-07-22 16:29:46 +02:00
Sree Charan Manamala c9aae9d8e6
Enable WINDOW_LEAF_OPERATOR for native engine to support queries without group by (#16753) 2024-07-22 12:31:55 +02:00
Zoltan Haindrich d227029b6b undo unrealted change 2024-07-19 19:16:46 +00:00
Zoltan Haindrich f7247e1bb7 use entryset 2024-07-19 15:13:17 +00:00
Zoltan Haindrich b38935a450 add test; fb 2024-07-19 11:44:23 +00:00
Zoltan Haindrich e2a54b5758 update 2024-07-19 08:42:58 +00:00
Clint Wylie 35b876436b
remove native scan query legacy mode (#16659) 2024-07-18 23:33:27 -07:00
Zoltan Haindrich 76ff3f26e1 add supress 2024-07-18 07:25:19 +00:00
Zoltan Haindrich 47aeb016df Merge branch 'quidem-record' into quidem-msq 2024-07-18 05:48:32 +00:00
Zoltan Haindrich 06b68b6c89 Merge remote-tracking branch 'apache/master' into quidem-record 2024-07-18 05:48:13 +00:00
Akshat Jain b53c26f5c5
Fix issues with partitioning boundaries for MSQ window functions (#16729) 
* Fix issues with partitioning boundaries for MSQ window functions

* Address review comments

* Address review comments

* Add test for coverage check failure

* Address review comment

* Remove DruidWindowQueryTest and WindowQueryTestBase, move those tests to DrillWindowQueryTest

* Update extensions-core/multi-stage-query/src/main/java/org/apache/druid/msq/querykit/WindowOperatorQueryKit.java

* Address review comments

* Add test for equals and hashcode for WindowOperatorQueryFrameProcessorFactory

* Address review comment

* Fix checkstyle

---------

Co-authored-by: Benedict Jin <asdf2014@apache.org>
 2024-07-18 10:05:09 +08:00
Zoltan Haindrich 70ff2a3e97 add exploratory msqPlan cmd 2024-07-17 19:48:08 +00:00
Zoltan Haindrich b100e982a4 make/etc 2024-07-17 16:40:30 +00:00
Zoltan Haindrich 0811d801fb make query run 2024-07-17 16:33:10 +00:00
Zoltan Haindrich 97c32ca3de less crappy way to run it 2024-07-17 16:19:08 +00:00
Zoltan Haindrich 6790f9cf8b move stuff 2024-07-17 16:08:32 +00:00
Zoltan Haindrich 51d465df6d make engine load via injector for msqdrill 2024-07-17 16:04:14 +00:00
Zoltan Haindrich 0eaf4c61b9 removePrint 2024-07-17 15:52:19 +00:00
Zoltan Haindrich f3cf778115 some stuff 2024-07-17 15:48:36 +00:00
Zoltan Haindrich 42b3086512 msq-test-0 2024-07-17 15:38:50 +00:00