druid/extensions-core
Gian Merlino d4cace385f
SQL: Allow Scans to be used as outer queries. (#11831)
* SQL: Allow Scans to be used as outer queries.

This has been possible in the native query system for a while, but the capability
hasn't yet propagated into the SQL layer. One example of where this is useful is
a query like:

  SELECT * FROM (... LIMIT X) WHERE <filter>

Because this expands the kinds of subquery structures the SQL layer will consider,
it was also necessary to improve the cost calculations. These changes appear in
PartialDruidQuery and DruidOuterQueryRel. The ideas are:

- Attach per-column penalties to the output signature of each query, instead of to
  the initial projection that starts a query. This encourages moving projections
  into subqueries instead of leaving them on outer queries.
- Only attach penalties to projections if there are actually expressions happening.
  So, now, projections that simply reorder or remove fields are free.
- Attach a constant penalty to every outer query. This discourages creating them
  when they are not needed.

The changes are generally beneficial to the test cases we have in CalciteQueryTest.
Most plans are unchanged, or are changed in purely cosmetic ways. Two have changed
for the better:

- testUsingSubqueryWithLimit now returns a constant from the subquery, instead of
  returning every column.
- testJoinOuterGroupByAndSubqueryHasLimit returns a minimal set of columns from
  the innermost subquery; two unnecessary columns are no longer there.

* Fix various DS operator conversions.

These were all implemented as direct conversions, which isn't appropriate
because they do not actually map onto native functions. These are only
usable as post-aggregations.

* Test case adjustment.
2021-10-23 17:18:43 -07:00
..
avro-extensions bump version to 0.23.0-SNAPSHOT (#11670) 2021-09-08 15:56:04 -07:00
azure-extensions bump version to 0.23.0-SNAPSHOT (#11670) 2021-09-08 15:56:04 -07:00
datasketches SQL: Allow Scans to be used as outer queries. (#11831) 2021-10-23 17:18:43 -07:00
druid-aws-rds-extensions bump version to 0.23.0-SNAPSHOT (#11670) 2021-09-08 15:56:04 -07:00
druid-basic-security refactor sql authorization to get resource type from schema, resource type to be string (#11692) 2021-09-17 09:53:25 -07:00
druid-bloom-filter better types (#11713) 2021-10-19 01:47:25 -07:00
druid-kerberos bump version to 0.23.0-SNAPSHOT (#11670) 2021-09-08 15:56:04 -07:00
druid-pac4j bump version to 0.23.0-SNAPSHOT (#11670) 2021-09-08 15:56:04 -07:00
druid-ranger-security refactor sql authorization to get resource type from schema, resource type to be string (#11692) 2021-09-17 09:53:25 -07:00
ec2-extensions bump version to 0.23.0-SNAPSHOT (#11670) 2021-09-08 15:56:04 -07:00
google-extensions bump version to 0.23.0-SNAPSHOT (#11670) 2021-09-08 15:56:04 -07:00
hdfs-storage bump version to 0.23.0-SNAPSHOT (#11670) 2021-09-08 15:56:04 -07:00
histogram add output type information to ExpressionPostAggregator (#11818) 2021-10-22 13:52:51 -07:00
kafka-extraction-namespace Update Apache Kafka client libraries to 3.0.0 (#11735) 2021-10-05 10:23:19 -07:00
kafka-indexing-service Kafka Input Format for headers, key and payload parsing (#11630) 2021-10-07 08:56:27 -07:00
kinesis-indexing-service bump version to 0.23.0-SNAPSHOT (#11670) 2021-09-08 15:56:04 -07:00
kubernetes-extensions Remove CloseQuietly and migrate its usages to other methods. (#10247) 2021-10-23 17:03:21 -07:00
lookups-cached-global bump version to 0.23.0-SNAPSHOT (#11670) 2021-09-08 15:56:04 -07:00
lookups-cached-single bump version to 0.23.0-SNAPSHOT (#11670) 2021-09-08 15:56:04 -07:00
mysql-metadata-storage bump version to 0.23.0-SNAPSHOT (#11670) 2021-09-08 15:56:04 -07:00
orc-extensions Upgrade ORC to 1.7.0 (#11726) 2021-09-27 13:20:09 -07:00
parquet-extensions bump version to 0.23.0-SNAPSHOT (#11670) 2021-09-08 15:56:04 -07:00
postgresql-metadata-storage bump version to 0.23.0-SNAPSHOT (#11670) 2021-09-08 15:56:04 -07:00
protobuf-extensions bump version to 0.23.0-SNAPSHOT (#11670) 2021-09-08 15:56:04 -07:00
s3-extensions bump version to 0.23.0-SNAPSHOT (#11670) 2021-09-08 15:56:04 -07:00
simple-client-sslcontext bump version to 0.23.0-SNAPSHOT (#11670) 2021-09-08 15:56:04 -07:00
stats add output type information to ExpressionPostAggregator (#11818) 2021-10-22 13:52:51 -07:00
testing-tools better types (#11713) 2021-10-19 01:47:25 -07:00