druid/docs/querying
Gian Merlino cccf13ea82
Reverse, pull up lookups in the SQL planner. (#15626)
* Reverse, pull up lookups in the SQL planner.

Adds two new rules:

1) ReverseLookupRule, which eliminates calls to LOOKUP by doing
   reverse lookups.

2) AggregatePullUpLookupRule, which pulls up calls to LOOKUP above
   GROUP BY, when the lookup is injective.

Adds configs `sqlReverseLookup` and `sqlPullUpLookup` to control whether
these rules fire. Both are enabled by default.

To minimize the chance of performance problems due to many keys mapping to
the same value, ReverseLookupRule refrains from reversing a lookup if there
are more keys than `inSubQueryThreshold`. The rationale for using this setting
is that reversal works by generating an IN, and the `inSubQueryThreshold`
describes the largest IN the user wants the planner to create.

* Add additional line.

* Style.

* Remove commented-out lines.

* Fix tests.

* Add test.

* Fix doc link.

* Fix docs.

* Add one more test.

* Fix tests.

* Logic, test updates.

* - Make FilterDecomposeConcatRule more flexible.

- Make CalciteRulesManager apply reduction rules til fixpoint.

* Additional tests, simplify code.
2024-01-12 00:06:31 -08:00
..
aggregations.md Native doc update (#15456) 2023-11-30 10:37:23 +05:30
arrays.md explicit outputType for ExpressionPostAggregator, better documentation for the differences between arrays and mvds (#15245) 2023-11-02 00:31:37 -07:00
caching.md remove group-by v1 (#14866) 2023-08-23 12:44:06 -07:00
datasource.md SQL: Plan non-equijoin conditions as cross join followed by filter (#15302) 2023-11-29 13:46:11 +05:30
datasourcemetadataquery.md Docusaurus2 upgrade for master (#14411) 2023-08-16 19:01:21 -07:00
dimensionspecs.md Reverse, pull up lookups in the SQL planner. (#15626) 2024-01-12 00:06:31 -08:00
filters.md document arrayContainsElement filter (#15455) 2023-12-07 00:14:00 -08:00
geo.md Docusaurus2 upgrade for master (#14411) 2023-08-16 19:01:21 -07:00
granularities.md Docusaurus2 upgrade for master (#14411) 2023-08-16 19:01:21 -07:00
groupbyquery.md Fix dictionarySize overrides in tests (#15354) 2023-11-28 18:49:09 +05:30
having.md Docusaurus2 upgrade for master (#14411) 2023-08-16 19:01:21 -07:00
hll-old.md De-incubation cleanup in code, docs, packaging (#9108) 2020-01-03 12:33:19 -05:00
joins.md Sort-merge join and hash shuffles for MSQ. (#13506) 2023-03-08 14:19:39 -08:00
limitspec.md Docusaurus2 upgrade for master (#14411) 2023-08-16 19:01:21 -07:00
lookups.md Reverse, pull up lookups in the SQL planner. (#15626) 2024-01-12 00:06:31 -08:00
math-expr.md New handling for COALESCE, SEARCH, and filter optimization. (#15609) 2024-01-03 08:56:22 -08:00
multi-value-dimensions.md explicit outputType for ExpressionPostAggregator, better documentation for the differences between arrays and mvds (#15245) 2023-11-02 00:31:37 -07:00
multitenancy.md Docs: Fix some typos. (#14663) 2023-07-26 21:24:18 +05:30
nested-columns.md consolidate json and auto indexers, remove v4 nested column serializer (#14456) 2023-08-22 18:50:11 -07:00
post-aggregations.md explicit outputType for ExpressionPostAggregator, better documentation for the differences between arrays and mvds (#15245) 2023-11-02 00:31:37 -07:00
query-context.md Reverse, pull up lookups in the SQL planner. (#15626) 2024-01-12 00:06:31 -08:00
query-execution.md Docusaurus2 upgrade for master (#14411) 2023-08-16 19:01:21 -07:00
query-from-deep-storage.md Query from deep storage doc fixes. (#15382) 2023-11-16 14:05:20 +05:30
query-processing.md Revamp design page (#15486) 2023-12-08 11:40:24 -08:00
querying.md Docusaurus2 upgrade for master (#14411) 2023-08-16 19:01:21 -07:00
scan-query.md Docusaurus2 upgrade for master (#14411) 2023-08-16 19:01:21 -07:00
searchquery.md remove search auto strategy, estimateSelectivity of BitmapColumnIndex (#15550) 2023-12-13 16:30:01 -08:00
segmentmetadataquery.md Docusaurus2 upgrade for master (#14411) 2023-08-16 19:01:21 -07:00
select-query.md Add "offset" parameter to the Scan query. (#10233) 2020-08-13 14:56:24 -07:00
sorting-orders.md Docusaurus2 upgrade for master (#14411) 2023-08-16 19:01:21 -07:00
sql-aggregations.md Enabling aggregateMultipleValues in all StringAnyAggregators (#15434) 2023-11-29 14:32:49 -08:00
sql-array-functions.md fixup array and mvd sql docs (#14928) 2023-09-05 16:17:00 -07:00
sql-data-types.md docs: Anchor link checker (#15624) 2024-01-08 15:19:05 -08:00
sql-functions.md minor doc adjustments (#15531) 2023-12-11 18:22:44 -08:00
sql-json-functions.md minor doc adjustments (#15531) 2023-12-11 18:22:44 -08:00
sql-metadata-tables.md docs: Anchor link checker (#15624) 2024-01-08 15:19:05 -08:00
sql-multivalue-string-functions.md fixup array and mvd sql docs (#14928) 2023-09-05 16:17:00 -07:00
sql-operators.md Add IS [NOT] DISTINCT FROM to SQL and join matchers. (#14976) 2023-09-20 10:44:32 -07:00
sql-query-context.md Reverse, pull up lookups in the SQL planner. (#15626) 2024-01-12 00:06:31 -08:00
sql-scalar.md [Docs] Document decode_base64_complex and decode_base64_utf8 functions (#15444) 2023-12-11 09:12:06 -08:00
sql-translation.md Docusaurus2 upgrade for master (#14411) 2023-08-16 19:01:21 -07:00
sql-window-functions.md window functions docs (#14739) 2023-11-06 11:34:42 -08:00
sql.md explicit outputType for ExpressionPostAggregator, better documentation for the differences between arrays and mvds (#15245) 2023-11-02 00:31:37 -07:00
timeboundaryquery.md Docusaurus2 upgrade for master (#14411) 2023-08-16 19:01:21 -07:00
timeseriesquery.md update timeseries to reflect NULL filling (#15512) 2023-12-07 14:41:27 -08:00
tips-good-queries.md remove references to Jupyter notebooks within the Druid repo (#15143) 2023-11-01 13:17:06 -07:00
topnmetricspec.md Docusaurus2 upgrade for master (#14411) 2023-08-16 19:01:21 -07:00
topnquery.md Docusaurus2 upgrade for master (#14411) 2023-08-16 19:01:21 -07:00
troubleshooting.md remove group-by v1 (#14866) 2023-08-23 12:44:06 -07:00
using-caching.md Update Ingestion section (#14023) 2023-05-19 09:42:27 -07:00
virtual-columns.md Docusaurus2 upgrade for master (#14411) 2023-08-16 19:01:21 -07:00