mirror of https://github.com/apache/druid.git
4b1ffbc452
* Various changes and fixes to UNNEST. Native changes: 1) UnnestDataSource: Replace "column" and "outputName" with "virtualColumn". This enables pushing expressions into the datasource. This in turn allows us to do the next thing... 2) UnnestStorageAdapter: Logically apply query-level filters and virtual columns after the unnest operation. (Physically, filters are pulled up, when possible.) This is beneficial because it allows filters and virtual columns to reference the unnested column, and because it is consistent with how the join datasource works. 3) Various documentation updates, including declaring "unnest" as an experimental feature for now. SQL changes: 1) Rename DruidUnnestRel (& Rule) to DruidUnnestRel (& Rule). The rel is simplified: it only handles the UNNEST part of a correlated join. Constant UNNESTs are handled with regular inline rels. 2) Rework DruidCorrelateUnnestRule to focus on pulling Projects from the left side up above the Correlate. New test testUnnestTwice verifies that this works even when two UNNESTs are stacked on the same table. 3) Include ProjectCorrelateTransposeRule from Calcite to encourage pushing mappings down below the left-hand side of the Correlate. 4) Add a new CorrelateFilterLTransposeRule and CorrelateFilterRTransposeRule to handle pulling Filters up above the Correlate. New tests testUnnestWithFiltersOutside and testUnnestTwiceWithFilters verify this behavior. 5) Require a context feature flag for SQL UNNEST, since it's undocumented. As part of this, also cleaned up how we handle feature flags in SQL. They're now hooked into EngineFeatures, which is useful because not all engines support all features. |
||
|---|---|---|
| .. | ||
| cluster.md | ||
| docker.md | ||
| index.md | ||
| tutorial-batch-hadoop.md | ||
| tutorial-batch-native.md | ||
| tutorial-batch.md | ||
| tutorial-compaction.md | ||
| tutorial-delete-data.md | ||
| tutorial-ingestion-spec.md | ||
| tutorial-jdbc.md | ||
| tutorial-jupyter-index.md | ||
| tutorial-kafka.md | ||
| tutorial-kerberos-hadoop.md | ||
| tutorial-msq-convert-spec.md | ||
| tutorial-msq-extern.md | ||
| tutorial-query.md | ||
| tutorial-retention.md | ||
| tutorial-rollup.md | ||
| tutorial-sketches-theta.md | ||
| tutorial-sql-query-view.md | ||
| tutorial-transform-spec.md | ||
| tutorial-unnest-datasource.md | ||
| tutorial-update-data.md | ||