Apache
/
druid
mirror of https://github.com/apache/druid.git
1
0
Fork 0
Code Issues Packages Projects Releases Wiki Activity
druid/extensions-core
History
AmatyaAvadhanula 1f63b447c4
Mitigate Kinesis stream LimitExceededException by using listShards API (#12161) 
Makes kinesis ingestion resilient to `LimitExceededException` caused by resharding.
Replace `describeStream` with `listShards` (recommended) to get shard related info.
`describeStream` has a limit (100) to the number of shards returned per call and a low default TPS limit of 10.
`listShards` returns the info for at most 1000 shards and has a higher TPS limit of 100 as well.

Key changed/added classes in this PR
 * `KinesisRecordSupplier`
 * `KinesisAdminClient`
 		2022-01-21 10:15:51 +05:30
..
avro-extensions Throw parse exceptions on schema get errors for SchemaRegistryBasedAvroBytesDecoder (#12080) 2022-01-13 12:36:51 -06:00
azure-extensions Migrate File.mkdirs to FileUtils.mkdirp. (#11879) 2021-11-09 11:10:49 -08:00
datasketches Add a small LRU cache and use utf8 bytes in ArrayOfDoubles (#12130) 2022-01-11 13:04:11 -08:00
druid-aws-rds-extensions bump version to 0.23.0-SNAPSHOT (#11670) 2021-09-08 15:56:04 -07:00
druid-basic-security Fix classNotFoundException when connecting to secure LDAP (#11978) 2021-12-03 12:08:19 +05:30
druid-bloom-filter restore and deprecate AggregatorFactory methods (#11917) 2021-11-19 15:59:35 -08:00
druid-kerberos Support for hadoop 3 via maven profiles (#11794) 2021-10-30 22:46:24 +05:30
druid-pac4j bump version to 0.23.0-SNAPSHOT (#11670) 2021-09-08 15:56:04 -07:00
druid-ranger-security Support for hadoop 3 via maven profiles (#11794) 2021-10-30 22:46:24 +05:30
ec2-extensions bump version to 0.23.0-SNAPSHOT (#11670) 2021-09-08 15:56:04 -07:00
google-extensions Migrate File.mkdirs to FileUtils.mkdirp. (#11879) 2021-11-09 11:10:49 -08:00
hdfs-storage Migrate File.mkdirs to FileUtils.mkdirp. (#11879) 2021-11-09 11:10:49 -08:00
histogram restore and deprecate AggregatorFactory methods (#11917) 2021-11-19 15:59:35 -08:00
kafka-extraction-namespace TestKafkaExtractionCluster: Shut down Kafka, ZK in @After. (#11963) 2021-11-20 15:17:05 -08:00
kafka-indexing-service Fix infinite retrying during task pausing (#12167) 2022-01-19 09:03:36 +05:30
kinesis-indexing-service Mitigate Kinesis stream LimitExceededException by using listShards API (#12161) 2022-01-21 10:15:51 +05:30
kubernetes-extensions Remove CloseQuietly and migrate its usages to other methods. (#10247) 2021-10-23 17:03:21 -07:00
lookups-cached-global Fix bad lookup config fails task (#12021) 2021-12-07 00:55:34 -05:00
lookups-cached-single bump version to 0.23.0-SNAPSHOT (#11670) 2021-09-08 15:56:04 -07:00
mysql-metadata-storage MySqlFirehoseDatabaseConnector uses configured driver class name (#12049) 2021-12-09 20:58:55 -08:00
orc-extensions Support JsonPath functions in JsonPath expressions (#11722) 2021-12-10 10:53:23 +08:00
parquet-extensions Support JsonPath functions in JsonPath expressions (#11722) 2021-12-10 10:53:23 +08:00
postgresql-metadata-storage bump version to 0.23.0-SNAPSHOT (#11670) 2021-09-08 15:56:04 -07:00
protobuf-extensions Add parse error list API for stream supervisors, use structured object for parse exceptions, simplify parse exception message (#11961) 2021-12-09 15:42:55 -06:00
s3-extensions AWS "Data read has a different length than the expected" error should reset stream and try again (#11941) 2021-11-26 12:45:34 -07:00
simple-client-sslcontext bump version to 0.23.0-SNAPSHOT (#11670) 2021-09-08 15:56:04 -07:00
stats restore and deprecate AggregatorFactory methods (#11917) 2021-11-19 15:59:35 -08:00
testing-tools complex typed expressions (#11853) 2021-11-08 00:33:06 -08:00