druid/extensions-core
Karan Kumar 2f2d8ded5a
Introducing Storage connector Interface (#12874)
In the current druid code base, we have the interface DataSegmentPusher which allows us to push segments to the appropriate deep storage without the extension being worried about the semantics of how to push too deep storage.

While working on #12262, whose some part of the code will go as an extension, I realized that we do not have an interface that allows us to do basic "write, get, delete, deleteAll" operations on the appropriate deep storage without let's say pulling the s3-storage-extension dependency in the custom extension.

Hence, the idea of StorageConnector was born where the storage connector sits inside the druid core so all extensions have access to it.

Each deep storage implementation, for eg s3, GCS, will implement this interface.
Now with some Jackson magic, we bind the implementation of the correct deep storage implementation on runtime using a type variable.
2022-08-12 16:11:49 +05:30
..
avro-extensions Bump up the versions (#12480) 2022-04-27 14:28:20 +05:30
azure-extensions Able to filter Cloud objects with glob notation. (#12659) 2022-06-24 11:40:08 +05:30
datasketches Adding withName implementation to AggregatorFactory (#12862) 2022-08-08 18:31:56 +05:30
druid-aws-rds-extensions Bump up the versions (#12480) 2022-04-27 14:28:20 +05:30
druid-basic-security Improve build performance of modules (#12486) 2022-05-01 22:43:11 +08:00
druid-bloom-filter Adding withName implementation to AggregatorFactory (#12862) 2022-08-08 18:31:56 +05:30
druid-kerberos Bump up the versions (#12480) 2022-04-27 14:28:20 +05:30
druid-pac4j Mark specific nimbus.lang.tag.version. (#12751) 2022-07-07 09:58:35 +05:30
druid-ranger-security Bump up the versions (#12480) 2022-04-27 14:28:20 +05:30
ec2-extensions Bump up the versions (#12480) 2022-04-27 14:28:20 +05:30
google-extensions Able to filter Cloud objects with glob notation. (#12659) 2022-06-24 11:40:08 +05:30
hdfs-storage Add authentication call before cleaning up intermediate files in hadoop ingestions (#12030) 2022-05-02 08:40:44 -05:00
histogram Adding withName implementation to AggregatorFactory (#12862) 2022-08-08 18:31:56 +05:30
kafka-extraction-namespace Change Kafka Lookup Extractor to not register consumer group (#12842) 2022-08-09 16:14:22 +05:30
kafka-indexing-service Kinesis ingestion with empty shards (#12792) 2022-08-05 22:38:58 +05:30
kinesis-indexing-service Kinesis ingestion with empty shards (#12792) 2022-08-05 22:38:58 +05:30
kubernetes-extensions Bump up the versions (#12480) 2022-04-27 14:28:20 +05:30
lookups-cached-global Bump up the versions (#12480) 2022-04-27 14:28:20 +05:30
lookups-cached-single Bump up the versions (#12480) 2022-04-27 14:28:20 +05:30
mysql-metadata-storage deps: upgrade mysql-connector-java to v5.1.49 (#12704) 2022-06-29 23:15:46 +08:00
orc-extensions Improved Java 17 support and Java runtime docs. (#12839) 2022-08-03 23:16:05 -07:00
parquet-extensions Perform lazy initialization of parquet extensions module (#12827) 2022-08-02 13:41:12 +05:30
postgresql-metadata-storage Optimize overlord GET /tasks memory usage (#12404) 2022-06-16 22:30:37 +05:30
protobuf-extensions Bump up the versions (#12480) 2022-04-27 14:28:20 +05:30
s3-extensions Introducing Storage connector Interface (#12874) 2022-08-12 16:11:49 +05:30
simple-client-sslcontext Tidy up construction of the Guice Injectors (#12816) 2022-08-04 00:05:07 -07:00
stats Adding withName implementation to AggregatorFactory (#12862) 2022-08-08 18:31:56 +05:30
testing-tools Tidy up construction of the Guice Injectors (#12816) 2022-08-04 00:05:07 -07:00