druid/extensions-core
Clint Wylie f8301a314f
generic block compressed complex columns (#16863)
changes:
* Adds new `CompressedComplexColumn`, `CompressedComplexColumnSerializer`, `CompressedComplexColumnSupplier` based on `CompressedVariableSizedBlobColumn` used by JSON columns
* Adds `IndexSpec.complexMetricCompression` which can be used to specify compression for the generic compressed complex column. Defaults to uncompressed because compressed columns are not backwards compatible.
* Adds new definition of `ComplexMetricSerde.getSerializer` which accepts an `IndexSpec` argument when creating a serializer. The old signature has been marked `@Deprecated` and has a default implementation that returns `null`, but it will be used by the default implementation of the new version if it is implemented to return a non-null value. The default implementation of the new method will use a `CompressedComplexColumnSerializer` if `IndexSpec.complexMetricCompression` is not null/none/uncompressed, or will use `LargeColumnSupportedComplexColumnSerializer` otherwise.
* Removed all duplicate generic implementations of `ComplexMetricSerde.getSerializer` and `ComplexMetricSerde.deserializeColumn` into default implementations `ComplexMetricSerde` instead of being copied all over the place. The default implementation of `deserializeColumn` will check if the first byte indicates that the new compression was used, otherwise will use the `GenericIndexed` based supplier.
* Complex columns with custom serializers/deserializers are unaffected and may continue doing whatever it is they do, either with specialized compression or whatever else, this new stuff is just to provide generic implementations built around `ObjectStrategy`.
* add ObjectStrategy.readRetainsBufferReference so CompressedComplexColumn only copies on read if required
* add copyValueOnRead flag down to CompressedBlockReader to avoid buffer duplicate if the value needs copied anyway
2024-08-27 00:34:41 -07:00
..
avro-extensions Better error handling when retrieving Avro schemas from registry (#16684) 2024-07-02 16:48:34 -07:00
azure-extensions remove Firehose and FirehoseFactory (#16758) 2024-07-19 14:37:21 -07:00
datasketches generic block compressed complex columns (#16863) 2024-08-27 00:34:41 -07:00
druid-aws-rds-extensions Prepare master branch for 31.0.0 release (#16333) 2024-04-26 09:22:43 +05:30
druid-basic-security Fix excessive logging from druid-basic-security (#16767) 2024-07-22 08:33:00 +05:30
druid-bloom-filter generic block compressed complex columns (#16863) 2024-08-27 00:34:41 -07:00
druid-catalog Quidem record (#16624) 2024-08-05 14:58:32 +02:00
druid-kerberos Prepare master branch for 31.0.0 release (#16333) 2024-04-26 09:22:43 +05:30
druid-pac4j Remove index_realtime and index_realtime_appenderator tasks (#16602) 2024-06-24 20:13:33 -07:00
druid-ranger-security Prepare master branch for 31.0.0 release (#16333) 2024-04-26 09:22:43 +05:30
ec2-extensions Prepare master branch for 31.0.0 release (#16333) 2024-04-26 09:22:43 +05:30
google-extensions Prepare master branch for 31.0.0 release (#16333) 2024-04-26 09:22:43 +05:30
hdfs-storage Prepare master branch for 31.0.0 release (#16333) 2024-04-26 09:22:43 +05:30
histogram generic block compressed complex columns (#16863) 2024-08-27 00:34:41 -07:00
kafka-extraction-namespace Prepare master branch for 31.0.0 release (#16333) 2024-04-26 09:22:43 +05:30
kafka-indexing-service Track IngestionState more accurately in realtime tasks. (#16934) 2024-08-22 11:43:46 +05:30
kinesis-indexing-service Track IngestionState more accurately in realtime tasks. (#16934) 2024-08-22 11:43:46 +05:30
kubernetes-extensions #16717 defer provider instantiation in Kubernetes Module (#16726) 2024-07-16 13:05:28 -07:00
lookups-cached-global upgrade mysql:mysql-connector-java to 8.2.0 (#16024) 2024-05-06 21:58:37 +08:00
lookups-cached-single upgrade mysql:mysql-connector-java to 8.2.0 (#16024) 2024-05-06 21:58:37 +08:00
multi-stage-query generic block compressed complex columns (#16863) 2024-08-27 00:34:41 -07:00
mysql-metadata-storage remove Firehose and FirehoseFactory (#16758) 2024-07-19 14:37:21 -07:00
orc-extensions Prepare master branch for 31.0.0 release (#16333) 2024-04-26 09:22:43 +05:30
parquet-extensions Refactor: Clean up `DecimalParquetInputTest` using Assume (#16436) 2024-05-14 21:13:07 +05:30
postgresql-metadata-storage remove Firehose and FirehoseFactory (#16758) 2024-07-19 14:37:21 -07:00
protobuf-extensions Prepare master branch for 31.0.0 release (#16333) 2024-04-26 09:22:43 +05:30
s3-extensions Reduce logging in RetryableS3OutputStream (#16853) 2024-08-08 10:42:40 +05:30
simple-client-sslcontext Prepare master branch for 31.0.0 release (#16333) 2024-04-26 09:22:43 +05:30
stats generic block compressed complex columns (#16863) 2024-08-27 00:34:41 -07:00
testing-tools Quidem record (#16624) 2024-08-05 14:58:32 +02:00