druid/extensions-core
Agustin Gonzalez 9efa6cc9c8
Make persists concurrent with adding rows in batch ingestion (#11536)
* Make persists concurrent with ingestion

* Remove semaphore but keep concurrent persists (with add) and add push in the backround as well

* Go back to documented default persists (zero)

* Move to debug

* Remove unnecessary Atomics

* Comments on synchronization (or not) for sinks & sinkMetadata

* Some cleanup for unit tests but they still need further work

* Shutdown & wait for persists and push on close

* Provide support for three existing batch appenderators using batchProcessingMode flag

* Fix reference to wrong appenderator

* Fix doc typos

* Add BatchAppenderators class test coverage

* Add log message to batchProcessingMode final value, fix typo in enum name

* Another typo and minor fix to log message

* LEGACY->OPEN_SEGMENTS, Edit docs

* Minor update legacy->open segments log message

* More code comments, mostly small adjustments to naming etc

* fix spelling

* Exclude BtachAppenderators from Jacoco since it is fully tested but Jacoco still refuses to ack coverage

* Coverage for Appenderators & BatchAppenderators, name change of a method that was still using "legacy" rather than "openSegments"

Co-authored-by: Clint Wylie <cjwylie@gmail.com>
2021-09-08 13:31:52 -07:00
..
avro-extensions Add DynamicConfigProvider for Schema Registry (#11362) 2021-08-03 13:24:52 -07:00
azure-extensions option to use deep storage for storing shuffle data (#11507) 2021-08-13 16:40:25 -04:00
datasketches Configurable maxStreamLength for doubles sketches (#11574) 2021-08-31 14:56:37 -07:00
druid-aws-rds-extensions Bump dev version to 0.22.0-SNAPSHOT (#10759) 2021-01-15 13:16:23 -08:00
druid-basic-security Fix expiration logic for ldap internal credential cache (#11395) 2021-07-01 14:34:29 -07:00
druid-bloom-filter fix count and average SQL aggregators on constant virtual columns (#11208) 2021-05-10 13:41:48 -07:00
druid-kerberos update error-prone to 2.8.0 with fix for crashing check (#11494) 2021-07-29 09:13:46 -07:00
druid-pac4j Bump dev version to 0.22.0-SNAPSHOT (#10759) 2021-01-15 13:16:23 -08:00
druid-ranger-security Bump dev version to 0.22.0-SNAPSHOT (#10759) 2021-01-15 13:16:23 -08:00
ec2-extensions Bump dev version to 0.22.0-SNAPSHOT (#10759) 2021-01-15 13:16:23 -08:00
google-extensions option to use deep storage for storing shuffle data (#11507) 2021-08-13 16:40:25 -04:00
hdfs-storage option to use deep storage for storing shuffle data (#11507) 2021-08-13 16:40:25 -04:00
histogram add single input string expression dimension vector selector and better expression planning (#11213) 2021-07-06 11:20:49 -07:00
kafka-extraction-namespace remove ZooKeeper 3.4 support + pass tests with Java 15 (#11073) 2021-05-25 12:49:49 -07:00
kafka-indexing-service Make persists concurrent with adding rows in batch ingestion (#11536) 2021-09-08 13:31:52 -07:00
kinesis-indexing-service Make persists concurrent with adding rows in batch ingestion (#11536) 2021-09-08 13:31:52 -07:00
kubernetes-extensions k8s discovery module: fix issue for druid.host being more than 63chars not permitted as k8s resource label value (#10961) 2021-04-07 17:45:28 -07:00
lookups-cached-global Better logging for lookups (#11539) 2021-08-04 16:44:54 -07:00
lookups-cached-single removes unnecessary checks (#11431) 2021-07-12 18:21:16 -07:00
mysql-metadata-storage MySQL extension with MariaDB connector docs (#11608) 2021-08-19 01:52:26 -07:00
orc-extensions Upgrade ORC to 1.6.9 (#11518) 2021-07-31 23:33:03 -07:00
parquet-extensions Bump parquet.version from 1.11.1 to 1.12.0 (#11346) 2021-08-13 19:17:57 -07:00
postgresql-metadata-storage fix pom version reference (#11435) 2021-07-13 08:37:57 +08:00
protobuf-extensions Add DynamicConfigProvider for Schema Registry (#11362) 2021-08-03 13:24:52 -07:00
s3-extensions option to use deep storage for storing shuffle data (#11507) 2021-08-13 16:40:25 -04:00
simple-client-sslcontext Bump dev version to 0.22.0-SNAPSHOT (#10759) 2021-01-15 13:16:23 -08:00
stats Fix bug in Variance Buffer Aggregator resulting in intermittent NaN when druid.generic.useDefaultValueForNull=false (#11617) 2021-08-20 09:13:51 -07:00
testing-tools Put sleep in an extension (#11632) 2021-08-25 01:27:45 -07:00