druid/extensions-core
Abhishek Radhakrishnan 9c7d7fc777
Allow empty inserts and replaces in MSQ. (#15495)
* Allow empty inserts and replace.

- Introduce a new query context failOnEmptyInsert which defaults to false.
- When this context is false (default), MSQE will now allow empty inserts and replaces.
- When this context is true, MSQE will throw the existing InsertCannotBeEmpty MSQ fault.
- For REPLACE ALL over an ALL grain segment, the query will generate a tombstone spanning eternity
which will be removed eventually be the coordinator.
- Add unit tests in MSQInsertTest, MSQReplaceTest to test the new default behavior (i.e., when failOnEmptyInsert = false)
- Update unit tests in MSQFaultsTest to test the non-default behavior (i.e., when failOnEmptyInsert = true)

* Ignore test to see if it's the culprit for OOM

* Add heap dump config

* Bump up -Xmx from 1500 MB to 2048 MB

* Add steps to tarball and collect hprof dump to GHA action

* put back mx to 1500MB to trigger the failure

* add the step to reusable unit test workflow as well

* Revert the temp heap dump & @Ignore changes since max heap size is increased

* Minor updates

* Review comments

1. Doc suggestions
2. Add tests for empty insert and replace queries with ALL grain and limit in the
   default failOnEmptyInsert mode (=false). Add similar tests to MSQFaultsTest with
   failOnEmptyInsert = true, so the query does fail with an InsertCannotBeEmpty fault.
3. Nullable annotation and javadocs

* Add comment
	replace_limit.patch
2024-01-02 13:05:51 -08:00
..
avro-extensions update confluent's dependencies to common, supported version (#15441) 2023-11-28 21:35:22 -08:00
azure-extensions Add system fields to input sources. (#15276) 2023-11-02 10:31:28 -07:00
datasketches Add null checks for HllSketchHolder (#15502) 2023-12-08 11:43:04 +05:30
druid-aws-rds-extensions Prepare master for Druid 29 (#15121) 2023-10-11 10:33:45 +05:30
druid-basic-security Add logging implementation for AuditManager and audit more endpoints (#15480) 2023-12-19 13:14:04 +05:30
druid-bloom-filter Relocating Table Schema Building: Shifting from Brokers to Coordinator for Improved Efficiency (#14985) 2023-11-04 19:33:25 +05:30
druid-catalog Prepare master for Druid 29 (#15121) 2023-10-11 10:33:45 +05:30
druid-kerberos Prepare master for Druid 29 (#15121) 2023-10-11 10:33:45 +05:30
druid-pac4j Upgrade pac4j-oidc to 4.5.7 to address CVE-2021-44878 (#15522) 2023-12-13 10:44:05 -08:00
druid-ranger-security add gson to dependencyManagement (#15488) 2023-12-05 11:50:32 -08:00
ec2-extensions Prepare master for Druid 29 (#15121) 2023-10-11 10:33:45 +05:30
google-extensions Add MSQ Durable Storage Connector for Google Cloud Storage and change current Google Cloud Storage client library (#15398) 2023-12-14 07:34:49 +05:30
hdfs-storage Add system fields to input sources. (#15276) 2023-11-02 10:31:28 -07:00
histogram + Switching Comparison from String to JSON (#15364) 2023-11-14 08:07:19 -08:00
kafka-extraction-namespace Refactor lookups behavior while loading/dropping the containers (#14806) 2023-11-07 10:07:28 -08:00
kafka-indexing-service Fix log typos, clean up some kill messages in SeekableStreamSupervisor (#15424) 2023-11-24 16:09:10 +05:30
kinesis-indexing-service Better automatic offset reset for Kinesis ingestion (#15338) 2023-12-13 12:03:17 +05:30
kubernetes-extensions update kubernetes java client to 19.0.0 and docker-java to 3.3.4 (#15449) 2023-12-12 14:27:57 -08:00
lookups-cached-global Refactor lookups behavior while loading/dropping the containers (#14806) 2023-11-07 10:07:28 -08:00
lookups-cached-single Refactor lookups behavior while loading/dropping the containers (#14806) 2023-11-07 10:07:28 -08:00
multi-stage-query Allow empty inserts and replaces in MSQ. (#15495) 2024-01-02 13:05:51 -08:00
mysql-metadata-storage Add logging implementation for AuditManager and audit more endpoints (#15480) 2023-12-19 13:14:04 +05:30
orc-extensions add optional 'castToType' parameter to 'auto' column schema (#15417) 2023-11-28 17:19:23 -08:00
parquet-extensions add optional 'castToType' parameter to 'auto' column schema (#15417) 2023-11-28 17:19:23 -08:00
postgresql-metadata-storage Prepare master for Druid 29 (#15121) 2023-10-11 10:33:45 +05:30
protobuf-extensions update kubernetes java client to 19.0.0 and docker-java to 3.3.4 (#15449) 2023-12-12 14:27:57 -08:00
s3-extensions Fixed a flaky test in `S3DataSegmentPusherConfigTest#testSerialization` by changing string to key:value pair (#15207) 2023-11-15 09:05:55 -08:00
simple-client-sslcontext Prepare master for Druid 29 (#15121) 2023-10-11 10:33:45 +05:30
stats Add null check for VarianceAggregatorCollector 2023-12-04 22:26:44 +05:30
testing-tools Prepare master for Druid 29 (#15121) 2023-10-11 10:33:45 +05:30