druid/extensions-core
Gian Merlino e40b96e026
Reverse lookup fixes and enhancements. (#15611)
* Reverse lookup fixes and enhancements.

1) Add a "mayIncludeUnknown" parameter to DimFilter#optimize. This is important
   because otherwise the reverse-lookup optimization is done improperly when
   the "in" filter appears under a "not", and the lookup extractionFn may return
   null for some possible values of the filtered column. The "includeUnknown" test
   cases in InDimFilterTest illustrate the difference in behavior.

2) Enhance InDimFilter#optimizeLookup to handle "mayIncludeUnknown", and to be able
   to do a reverse lookup in a wider variety of cases.

3) Make "unapply" protected in LookupExtractor, and move callers to "unapplyAll".
   The main reason is that MapLookupExtractor, a common implementation, lacks a
   reverse mapping and therefore does a scan of the map for each call to "unapply".
   For performance sake these calls need to be batched.

* Remove optimize call from BloomDimFilter.

* Follow the law.

* Fix tests.

* Fix imports.

* Switch function.

* Fix tests.

* More tests.
2024-01-03 13:28:44 -08:00
..
avro-extensions update confluent's dependencies to common, supported version (#15441) 2023-11-28 21:35:22 -08:00
azure-extensions Add system fields to input sources. (#15276) 2023-11-02 10:31:28 -07:00
datasketches Add null checks for HllSketchHolder (#15502) 2023-12-08 11:43:04 +05:30
druid-aws-rds-extensions Prepare master for Druid 29 (#15121) 2023-10-11 10:33:45 +05:30
druid-basic-security Add logging implementation for AuditManager and audit more endpoints (#15480) 2023-12-19 13:14:04 +05:30
druid-bloom-filter Reverse lookup fixes and enhancements. (#15611) 2024-01-03 13:28:44 -08:00
druid-catalog Prepare master for Druid 29 (#15121) 2023-10-11 10:33:45 +05:30
druid-kerberos Prepare master for Druid 29 (#15121) 2023-10-11 10:33:45 +05:30
druid-pac4j Upgrade pac4j-oidc to 4.5.7 to address CVE-2021-44878 (#15522) 2023-12-13 10:44:05 -08:00
druid-ranger-security add gson to dependencyManagement (#15488) 2023-12-05 11:50:32 -08:00
ec2-extensions Prepare master for Druid 29 (#15121) 2023-10-11 10:33:45 +05:30
google-extensions Add MSQ Durable Storage Connector for Google Cloud Storage and change current Google Cloud Storage client library (#15398) 2023-12-14 07:34:49 +05:30
hdfs-storage Add system fields to input sources. (#15276) 2023-11-02 10:31:28 -07:00
histogram + Switching Comparison from String to JSON (#15364) 2023-11-14 08:07:19 -08:00
kafka-extraction-namespace Reverse lookup fixes and enhancements. (#15611) 2024-01-03 13:28:44 -08:00
kafka-indexing-service Fix log typos, clean up some kill messages in SeekableStreamSupervisor (#15424) 2023-11-24 16:09:10 +05:30
kinesis-indexing-service Better automatic offset reset for Kinesis ingestion (#15338) 2023-12-13 12:03:17 +05:30
kubernetes-extensions update kubernetes java client to 19.0.0 and docker-java to 3.3.4 (#15449) 2023-12-12 14:27:57 -08:00
lookups-cached-global Refactor lookups behavior while loading/dropping the containers (#14806) 2023-11-07 10:07:28 -08:00
lookups-cached-single Reverse lookup fixes and enhancements. (#15611) 2024-01-03 13:28:44 -08:00
multi-stage-query Allow empty inserts and replaces in MSQ. (#15495) 2024-01-02 13:05:51 -08:00
mysql-metadata-storage Add logging implementation for AuditManager and audit more endpoints (#15480) 2023-12-19 13:14:04 +05:30
orc-extensions add optional 'castToType' parameter to 'auto' column schema (#15417) 2023-11-28 17:19:23 -08:00
parquet-extensions add optional 'castToType' parameter to 'auto' column schema (#15417) 2023-11-28 17:19:23 -08:00
postgresql-metadata-storage Prepare master for Druid 29 (#15121) 2023-10-11 10:33:45 +05:30
protobuf-extensions update kubernetes java client to 19.0.0 and docker-java to 3.3.4 (#15449) 2023-12-12 14:27:57 -08:00
s3-extensions Fixed a flaky test in `S3DataSegmentPusherConfigTest#testSerialization` by changing string to key:value pair (#15207) 2023-11-15 09:05:55 -08:00
simple-client-sslcontext Prepare master for Druid 29 (#15121) 2023-10-11 10:33:45 +05:30
stats Add null check for VarianceAggregatorCollector 2023-12-04 22:26:44 +05:30
testing-tools Prepare master for Druid 29 (#15121) 2023-10-11 10:33:45 +05:30