Commit Graph

14560 Commits

Author SHA1 Message Date
Abhishek Agarwal 421aae39ad
Upgrade avro - minor version (#17230) 2024-10-03 18:02:11 +05:30
Gian Merlino 316f8c81d2
RunWorkOrder: Account for two simultaneous statistics collectors. (#17216)
* RunWorkOrder: Account for two simultaneous statistics collectors.

As a follow up to #17057, divide the amount of partitionStatsMemory
by two, to account for the fact that there are at some times going to
be two copies of the full collector. First there will be one for processors
and one for the accumulated collector. Then, after the processor ones are
GCed, a snapshot of the accumulated collector will be created.

Also includes an optimization to "addAll" for the two KeyCollectors,
for the case where we're adding into an empty collector. This is always
going to happen once per stage due to the "withAccumulation" call.

* Fix missing variable.

* Don't divide by numProcessingThreads twice.

* Fix test.
2024-10-03 16:25:01 +05:30
Akshat Jain edc235cfe1
WindowOperatorQueryFrameProcessor: Avoid unnecessary re-runs of runIncrementally() (#17211) 2024-10-03 15:33:50 +05:30
Vadim Ogievetsky 8c4db8aeed
explore QA (#17225) 2024-10-02 23:05:19 -07:00
Akshat Jain 135ca8f6a7
WindowOperatorQueryFrameProcessor: Fix frame writer capacity issues + adhere to FrameProcessor's contract (#17209)
This PR fixes the above issue by maintaining the state of last rowId flushed to output channel, and triggering another iteration of runIncrementally() method if frame writer has rows pending flush to the output channel.

The above is done keeping in mind FrameProcessor's contract which enforces that we should write only a single frame to each output channel in any given iteration of runIncrementally().
2024-10-03 10:39:22 +05:30
Gian Merlino fbc1221837
DartTableInputSpecSlicer: Fix for TLS workers. (#17224)
We should use getHost(), which returns TLS if configured or plaintext
otherwise. getHostAndPort() returns plaintext only.
2024-10-03 11:01:11 +08:00
Vadim Ogievetsky 715ae5ece0
Web console: misc fixes to the Explore view (#17213)
* make record table able to hide column

* stickyness

* refactor query log

* fix measure drag

* start nested column dialog

* nested expand

* fix filtering on Measures

* use output name

* fix scrolling

* select all / none

* use ARRAY_CONCAT_AGG

* no need to limit if aggregating

* remove magic number

* better search

* update arg list

* add, don't replace
2024-10-02 08:52:08 -07:00
Arun Ramani e5d027ee1c
Skip generating task context reports for sub tasks (#17219)
* Skip task context for sub tasks

* DRY a little + skip context for live report
2024-10-02 09:32:50 -04:00
Zoltan Haindrich 65277b17a9
Decoupled planning: add support for unnest (#17177)
* adds support for `UNNEST` expressions
* introduces `LogicalUnnestRule` to transform a `Correlate` doing UNNEST into a `LogicalUnnest`
* `UnnestInputCleanupRule` could move the final unnested expr into the `LogicalUnnest` itself (usually its an `mv_to_array` expression)
* enhanced source unwrapping to utilize `FilteredDataSource` if it looks right
2024-10-02 08:54:56 +02:00
Vadim Ogievetsky c8529294eb
Web console: add support for Dart engine (#17147)
* add console support for Dart engine

This reverts commit 6e46edf15dd55e5c51a1a4068e83deba4f22529b.

* feedback fixes

* surface new fields

* prioratize error over results

* better metadata refresh

* feedback fixes
2024-10-01 17:53:36 -07:00
317brian 1fc82a96bd
docs: update future development blurbs (#16939)
Co-authored-by: Victoria Lim <vtlim@users.noreply.github.com>
2024-10-01 15:02:05 -07:00
Gian Merlino 878adff9aa
MSQ profile for Brokers and Historicals. (#17140)
This patch adds a profile of MSQ named "Dart" that runs on Brokers and
Historicals, and which is compatible with the standard SQL query API.
For more high-level description, and notes on future work, refer to #17139.

This patch contains the following changes, grouped into packages.

Controller (org.apache.druid.msq.dart.controller):

The controller runs on Brokers. Main classes are,

- DartSqlResource, which serves /druid/v2/sql/dart/.
- DartSqlEngine and DartQueryMaker, the entry points from SQL that actually
  run the MSQ controller code.
- DartControllerContext, which configures the MSQ controller.
- DartMessageRelays, which sets up relays (see "message relays" below) to read
  messages from workers' DartControllerClients.
- DartTableInputSpecSlicer, which assigns work based on a TimelineServerView.

Worker (org.apache.druid.msq.dart.worker)

The worker runs on Historicals. Main classes are,

- DartWorkerResource, which supplies the regular MSQ WorkerResource, plus
  Dart-specific APIs.
- DartWorkerRunner, which runs MSQ worker code.
- DartWorkerContext, which configures the MSQ worker.
- DartProcessingBuffersProvider, which provides processing buffers from
  sliced-up merge buffers.
- DartDataSegmentProvider, which provides segments from the Historical's
  local cache.

Message relays (org.apache.druid.messages):

To avoid the need for Historicals to contact Brokers during a query, which
would create opportunities for queries to get stuck, all connections are
opened from Broker to Historical. This is made possible by a message relay
system, where the relay server (worker) has an outbox of messages.

The relay client (controller) connects to the outbox and retrieves messages.
Code for this system lives in the "server" package to keep it separate from
the MSQ extension and make it easier to maintain. The worker-to-controller
ControllerClient is implemented using message relays.

Other changes:

- Controller: Added the method "hasWorker". Used by the ControllerMessageListener
  to notify the appropriate controllers when a worker fails.
- WorkerResource: No longer tries to respond more than once in the
  "httpGetChannelData" API. This comes up when a response due to resolved future
  is ready at about the same time as a timeout occurs.
- MSQTaskQueryMaker: Refactor to separate out some useful functions for reuse
  in DartQueryMaker.
- SqlEngine: Add "queryContext" to "resultTypeForSelect" and "resultTypeForInsert".
  This allows the DartSqlEngine to modify result format based on whether a "fullReport"
  context parameter is set.
- LimitedOutputStream: New utility class. Used when in "fullReport" mode.
- TimelineServerView: Add getDruidServerMetadata as a performance optimization.
- CliHistorical: Add SegmentWrangler, so it can query inline data, lookups, etc.
- ServiceLocation: Add "fromUri" method, relocating some code from ServiceClientImpl.
- FixedServiceLocator: New locator for a fixed set of service locations. Useful for
  URI locations.
2024-10-01 14:38:55 -07:00
Hardik Bajaj 3d56fa6f56
Improve logging to include taskId in segment handoff notifier thread (#17185) 2024-10-01 15:34:39 +05:30
Vadim Ogievetsky f33f60b32e
fix input step typo (#17202) 2024-09-30 21:26:22 -07:00
George Shiqi Wu 5ad6ed0b73
Refactor pod template logic to make it easier to test (#17178)
* Refactoring of pod template logic

* fix javadoc

* Fix intellij

* remove unneeded throw

* PR comments

* fix style

* Fix unit tests
2024-09-30 15:34:13 -07:00
Kashif Faraz 28fead58b4
MSQ: Use task context flag useConcurrentLocks to determine task lock type (#17193) 2024-09-30 21:15:25 +05:30
Abhishek Radhakrishnan 15987f51f1
Update Delta Kernel to 3.2.1 (#17179)
Updated Delta Kernel from 3.2.0 to 3.2.1. This upstream version bump contains fixes to reading long columns, class loader and better retry mechanism when reading checkpoint files.
2024-09-30 07:22:49 -07:00
Vadim Ogievetsky d982727a29
Web console: revamp the experimental explore view (#17180)
* explore revamp

* remove ToDo

* fix CodeQL

* add tooltips

* show issue on echart chars

* fix: browser back does not refresh chart

* fix maxRows 0

* be more resiliant to missing __time
2024-09-29 23:15:21 -07:00
Adarsh Sanjeev 95fb813b6d
Fix bug with while adding DistinctSketches (#17184)
* Fix bug with checking the incorrect key

* Add tests
2024-09-30 11:35:32 +05:30
Shivam Garg ab361747a8
Migrated commons-lang usages to commons-lang3 (#17156) 2024-09-28 10:28:11 +02:00
Sree Charan Manamala 661614129e
Window Functions : Context Parameter to Enable Transfer of RACs over wire (#17150) 2024-09-28 08:04:22 +02:00
Clint Wylie f8a72b987a
read metadata in SimpleQueryableIndex if available to compute segment ordering (#17181) 2024-09-27 19:39:03 -07:00
Vadim Ogievetsky 7417ead3af
Web console: Fixed sampling for delta source in classic data loader and MSQ (#17160)
Added the delta tile to MSQ and made sure it works there and classic data loader. It is special because it always assumes a parquet format so it has to skip some steps in the code. Now the framework is there to add more tiles that force a specific format
2024-09-27 07:51:24 -07:00
Gian Merlino dc223f22db
SQL: Use regular filters for time filtering in subqueries. (#17173)
* SQL: Use regular filters for time filtering in subqueries.

Using the "intervals" feature on subqueries, or any non-table, should be
avoided because it isn't a meaningful optimization in those cases, and
it's simpler for runtime implementations if they can assume all filters
are located in the regular filter object.

Two changes:

1) Fix the logic in DruidQuery.canUseIntervalFiltering. It was intended
   to return false for QueryDataSource, but actually returned true.

2) Add a validation to ScanQueryFrameProcessor to ensure that when running
   on an input channel (which would include any subquery), the query has
   "intervals" set to ONLY_ETERNITY.

Prior to this patch, the new test case in testTimeFilterOnSubquery would
throw a "Can only handle a single interval" error in the native engine,
and "QueryNotSupported" in the MSQ engine.

* Mark new case as having extra columns in decoupled mode.

* Adjust test.
2024-09-27 10:32:30 +05:30
Clint Wylie 157fe1bc1f
fix a mistake in CursorGranularizer to check doneness after advance (#17175)
Fixes a mistake introduced in #16533 which can result in CursorGranularizer incorrectly trying to get values from a selector after calling cursor.advance because of a missing check for cursor.isDone
2024-09-27 09:36:05 +05:30
Clint Wylie 6ee9e42f03
fix issue with ScanQueryFrameProcessor cursor build not adjusting intervals (#17168)
* fix issue with ScanQueryFrameProcessor cursor build not adjusting intervals

* all hail the robot overlords

* style bot
2024-09-26 21:09:14 +05:30
Zoltan Haindrich ca96629207
Add kttm-nested dataset and some usage sessions of it (#16867)
* Add kttm-nested dataset and some usage sessions of it

Squashed commit of the following:

commit 3075269bdefbbfeeb3c7958751d9c44dd017944f
Author: Zoltan Haindrich <kirk@rxd.hu>
Date:   Thu Aug 8 13:48:19 2024 +0000

    rename files

commit ad8d37a235a89cdf92ea3c1e9fdda9dff87fa7ce
Author: Zoltan Haindrich <kirk@rxd.hu>
Date:   Thu Aug 8 13:44:09 2024 +0000

    rename

commit 8fc4efe19eba10e7d6a82042aedec0ec7c04c77c
Author: Zoltan Haindrich <kirk@rxd.hu>
Date:   Thu Aug 8 13:43:31 2024 +0000

    cleanup kttm loader

commit 113a4b6f4c0a450581600aa7483a5163ee00e937
Author: Zoltan Haindrich <kirk@rxd.hu>
Date:   Thu Aug 8 13:38:10 2024 +0000

    weed out infoschema stuff

commit 1eb9b5070c16a42a19ab9774d2b0378e090097df
Author: Zoltan Haindrich <kirk@rxd.hu>
Date:   Thu Aug 8 13:17:27 2024 +0000

    reduce crap

commit 1745ececc77cae93ed9130fd6e9e83fddd44471a
Merge: 9dd2bff6cc 1f8473d82b
Author: Zoltan Haindrich <kirk@rxd.hu>
Date:   Thu Aug 8 09:49:52 2024 +0000

    Merge branch 'quidem-msq' into quidem-qa

commit 9dd2bff6ccdb30c927a256ff7ab6ebf491508008
Merge: 7120f84e36 fda0d63e44
Author: Zoltan Haindrich <kirk@rxd.hu>
Date:   Thu Aug 8 09:49:08 2024 +0000

    Merge branch 'quidem-record' into quidem-qa

commit 1f8473d82be23a57e929fed796e5d5b2dbaccf67
Merge: 4e5d7c9c21 6cd8c6be22
Author: Zoltan Haindrich <kirk@rxd.hu>
Date:   Thu Aug 8 09:46:40 2024 +0000

    Merge remote-tracking branch 'apache/master' into quidem-msq

commit 4e5d7c9c21
Author: Zoltan Haindrich <kirk@rxd.hu>
Date:   Wed Aug 7 15:58:32 2024 +0000

    remove CompositeDruidModule

commit 6cef3eb6e8
Author: Zoltan Haindrich <kirk@rxd.hu>
Date:   Wed Aug 7 07:30:25 2024 +0000

    disable differently

commit 73904b520d
Author: Zoltan Haindrich <kirk@rxd.hu>
Date:   Wed Aug 7 05:48:27 2024 +0000

    nested

commit ad5768e512
Author: Zoltan Haindrich <kirk@rxd.hu>
Date:   Wed Aug 7 05:43:20 2024 +0000

    migrate assume

commit 039d9c88f8
Author: Zoltan Haindrich <kirk@rxd.hu>
Date:   Tue Aug 6 20:15:16 2024 +0000

    mask 2 failing testcases

commit 7a6ba55fa2
Author: Zoltan Haindrich <kirk@rxd.hu>
Date:   Tue Aug 6 20:05:00 2024 +0000

    Revert "explore failure"

    This reverts commit 5bc32d5678.

commit 5bc32d5678
Author: Zoltan Haindrich <kirk@rxd.hu>
Date:   Tue Aug 6 20:04:58 2024 +0000

    explore failure

commit 8a92ec6b2e
Merge: dbe2674971 ebea34a814
Author: Zoltan Haindrich <kirk@rxd.hu>
Date:   Tue Aug 6 18:46:39 2024 +0000

    Merge remote-tracking branch 'apache/master' into quidem-msq

commit dbe2674971
Author: Zoltan Haindrich <kirk@rxd.hu>
Date:   Tue Aug 6 16:45:43 2024 +0000

    remove/cleanup/etc

commit 23c6f3bd96
Author: Zoltan Haindrich <kirk@rxd.hu>
Date:   Tue Aug 6 16:33:16 2024 +0000

    use other authmapper

commit 7740832995
Author: Zoltan Haindrich <kirk@rxd.hu>
Date:   Tue Aug 6 16:14:44 2024 +0000

    undo

commit 3e2b59f808
Author: Zoltan Haindrich <kirk@rxd.hu>
Date:   Tue Aug 6 16:14:16 2024 +0000

    Revert "shared tries"

    This reverts commit 34651e70e5.

commit 34651e70e5
Author: Zoltan Haindrich <kirk@rxd.hu>
Date:   Tue Aug 6 16:14:12 2024 +0000

    shared tries

commit 449a7f3a73
Author: Zoltan Haindrich <kirk@rxd.hu>
Date:   Tue Aug 6 16:13:56 2024 +0000

    move stuff

commit 69a39a42da
Author: Zoltan Haindrich <kirk@rxd.hu>
Date:   Tue Aug 6 15:28:58 2024 +0000

    Revert "connection supplies properties approach"

    This reverts commit 2700557a55.

commit 2700557a55
Author: Zoltan Haindrich <kirk@rxd.hu>
Date:   Tue Aug 6 15:28:56 2024 +0000

    connection supplies properties approach

commit 42cc5d62a8
Author: Zoltan Haindrich <kirk@rxd.hu>
Date:   Tue Aug 6 15:28:36 2024 +0000

    apidoc

commit d5b82af4a9
Author: Zoltan Haindrich <kirk@rxd.hu>
Date:   Tue Aug 6 14:38:30 2024 +0000

    cleanup

commit 21016a28a2
Author: Zoltan Haindrich <kirk@rxd.hu>
Date:   Tue Aug 6 14:21:28 2024 +0000

    remove ifs

commit 3f507d8648
Author: Zoltan Haindrich <kirk@rxd.hu>
Date:   Tue Aug 6 14:20:42 2024 +0000

    ignore duplicate queries

commit 19fe5867b1
Author: Zoltan Haindrich <kirk@rxd.hu>
Date:   Tue Aug 6 14:13:50 2024 +0000

    fix one fixme

commit 941c39aae5
Author: Zoltan Haindrich <kirk@rxd.hu>
Date:   Tue Aug 6 13:59:13 2024 +0000

    it does work

commit 00ee182d75
Author: Zoltan Haindrich <kirk@rxd.hu>
Date:   Tue Aug 6 13:55:23 2024 +0000

    add fixme/etc

commit 91d5d14bf1
Author: Zoltan Haindrich <kirk@rxd.hu>
Date:   Tue Aug 6 11:44:02 2024 +0000

    remove builtintypes

commit 29b2b559d9
Merge: 6d38e8f075 c3aa033e14
Author: Zoltan Haindrich <kirk@rxd.hu>
Date:   Tue Aug 6 11:42:04 2024 +0000

    Merge remote-tracking branch 'apache/master' into quidem-msq

commit 6d38e8f075
Author: Zoltan Haindrich <kirk@rxd.hu>
Date:   Tue Aug 6 11:41:54 2024 +0000

    clenaup

commit f867db774a
Author: Zoltan Haindrich <kirk@rxd.hu>
Date:   Tue Aug 6 10:31:32 2024 +0000

    fix pom

commit 61c3b16b17
Author: Zoltan Haindrich <kirk@rxd.hu>
Date:   Tue Aug 6 09:44:27 2024 +0000

    checkstyle

commit 130252bb5e
Author: Zoltan Haindrich <kirk@rxd.hu>
Date:   Tue Aug 6 09:20:13 2024 +0000

    fix; move class

commit 3b77784e6e
Author: Zoltan Haindrich <kirk@rxd.hu>
Date:   Tue Aug 6 08:13:25 2024 +0000

    add test

commit 22d8a4b872
Author: Zoltan Haindrich <kirk@rxd.hu>
Date:   Tue Aug 6 07:37:24 2024 +0000

    add compositemodule

commit 34ab911399
Author: Zoltan Haindrich <kirk@rxd.hu>
Date:   Tue Aug 6 07:25:42 2024 +0000

    fix intellij errors

commit 5e5c94d6d8
Author: Zoltan Haindrich <kirk@rxd.hu>
Date:   Tue Aug 6 07:23:08 2024 +0000

    cleanup

commit 93457c6b3e
Author: Zoltan Haindrich <kirk@rxd.hu>
Date:   Tue Aug 6 05:50:39 2024 +0000

    cleanup

commit 12cfde805e
Author: Zoltan Haindrich <kirk@rxd.hu>
Date:   Tue Aug 6 05:50:11 2024 +0000

    update

commit c8f9147810
Author: Zoltan Haindrich <kirk@rxd.hu>
Date:   Mon Aug 5 14:23:54 2024 +0000

    minor fixes

commit 6d339d1706
Author: Zoltan Haindrich <kirk@rxd.hu>
Date:   Mon Aug 5 14:21:25 2024 +0000

    rename class

commit 181458c873
Author: Zoltan Haindrich <kirk@rxd.hu>
Date:   Mon Aug 5 14:19:52 2024 +0000

    mask more

commit 4c722f271f
Author: Zoltan Haindrich <kirk@rxd.hu>
Date:   Mon Aug 5 14:15:06 2024 +0000

    Revert "this doesnt work"

    This reverts commit 1063948749.

commit 1063948749
Author: Zoltan Haindrich <kirk@rxd.hu>
Date:   Mon Aug 5 14:15:02 2024 +0000

    this doesnt work

commit c40474285c
Author: Zoltan Haindrich <kirk@rxd.hu>
Date:   Mon Aug 5 13:49:08 2024 +0000

    updates

commit f4af51ef7f
Author: Zoltan Haindrich <kirk@rxd.hu>
Date:   Mon Aug 5 13:41:53 2024 +0000

    extend/cleanup/etc

commit bc70443c7f
Author: Zoltan Haindrich <kirk@rxd.hu>
Date:   Mon Aug 5 13:20:13 2024 +0000

    update few more

commit 841ab462dd
Merge: 929e68c11a fda0d63e44
Author: Zoltan Haindrich <kirk@rxd.hu>
Date:   Mon Aug 5 13:00:59 2024 +0000

    Merge branch 'quidem-record' into quidem-msq

commit fda0d63e44
Merge: bb23ace518 26e3c44f4b
Author: Zoltan Haindrich <kirk@rxd.hu>
Date:   Mon Aug 5 13:00:50 2024 +0000

    Merge remote-tracking branch 'apache/master' into quidem-record

commit 929e68c11a
Author: Zoltan Haindrich <kirk@rxd.hu>
Date:   Mon Aug 5 12:59:50 2024 +0000

    undo unrelated

commit 436ba18815
Author: Zoltan Haindrich <kirk@rxd.hu>
Date:   Mon Aug 5 12:59:19 2024 +0000

    x

commit 70e46eadb9
Author: Zoltan Haindrich <kirk@rxd.hu>
Date:   Mon Aug 5 09:07:46 2024 +0000

    update

commit 090f937d58
Merge: 5e81a026e9 bb23ace518
Author: Zoltan Haindrich <kirk@rxd.hu>
Date:   Mon Aug 5 09:03:53 2024 +0000

    Merge branch 'quidem-record' into quidem-msq

commit bb23ace518
Author: Zoltan Haindrich <kirk@rxd.hu>
Date:   Mon Aug 5 08:59:48 2024 +0000

    builtintypes instead nesteddata

commit e6add9ea84
Merge: a03fb49f4f c7eacd079e
Author: Zoltan Haindrich <kirk@rxd.hu>
Date:   Mon Aug 5 07:04:02 2024 +0000

    Merge remote-tracking branch 'apache/master' into quidem-record

commit 5e81a026e9
Merge: 5f6290eb54 a03fb49f4f
Author: Zoltan Haindrich <kirk@rxd.hu>
Date:   Wed Jul 31 15:27:59 2024 +0000

    Merge branch 'quidem-record' into quidem-msq

commit a03fb49f4f
Author: Zoltan Haindrich <kirk@rxd.hu>
Date:   Tue Jul 30 16:34:06 2024 +0000

    remove exception

commit 5f6290eb54
Author: Zoltan Haindrich <kirk@rxd.hu>
Date:   Tue Jul 30 16:11:57 2024 +0000

    use updated hook class

commit de207c7295
Author: Zoltan Haindrich <kirk@rxd.hu>
Date:   Tue Jul 30 16:04:11 2024 +0000

    move key

commit b1ab252b31
Merge: 4157a8f105 ca121aa083
Author: Zoltan Haindrich <kirk@rxd.hu>
Date:   Tue Jul 30 16:03:33 2024 +0000

    Merge branch 'quidem-record' into quidem-msq

commit ca121aa083
Author: Zoltan Haindrich <kirk@rxd.hu>
Date:   Tue Jul 30 14:29:02 2024 +0000

    remove firehose

commit eb2a047e4b
Merge: 57980066b3 954aaafe0c
Author: Zoltan Haindrich <kirk@rxd.hu>
Date:   Tue Jul 30 14:24:37 2024 +0000

    Merge remote-tracking branch 'apache/master' into quidem-record

commit 57980066b3
Author: Zoltan Haindrich <kirk@rxd.hu>
Date:   Tue Jul 30 13:30:52 2024 +0000

    rename module

    (cherry picked from commit 8d40cca50a3e4c8098f49f5d588c7b7220b76788)

commit 7f99ee24d7
Author: Zoltan Haindrich <kirk@rxd.hu>
Date:   Tue Jul 30 14:03:46 2024 +0000

    fix copy-pasted crap

commit b345dd9d03
Author: Zoltan Haindrich <kirk@rxd.hu>
Date:   Tue Jul 30 13:25:40 2024 +0000

    updates/fix style/etc

commit df42245685
Author: Zoltan Haindrich <kirk@rxd.hu>
Date:   Tue Jul 30 13:14:51 2024 +0000

    add apidoc/etc

commit 9ac26e3a89
Author: Zoltan Haindrich <kirk@rxd.hu>
Date:   Tue Jul 30 12:29:36 2024 +0000

    wire-in hookdispatcher thru connection/etc

commit 78b75d3e8e
Author: Zoltan Haindrich <kirk@rxd.hu>
Date:   Tue Jul 30 10:42:41 2024 +0000

    move more to non-static

commit f6cc540368
Author: Zoltan Haindrich <kirk@rxd.hu>
Date:   Tue Jul 30 10:33:38 2024 +0000

    use druidhookdispatcherr#1

commit ce667eeb5e
Author: Zoltan Haindrich <kirk@rxd.hu>
Date:   Tue Jul 30 10:22:28 2024 +0000

    move stuff around / prepare to unglobalize

commit 4157a8f105
Author: Zoltan Haindrich <kirk@rxd.hu>
Date:   Tue Jul 30 10:16:03 2024 +0000

    add/.etc

commit ed9ef1f635
Author: Zoltan Haindrich <kirk@rxd.hu>
Date:   Fri Jul 26 03:37:28 2024 +0000

    checkstryle

commit 8bb38a04a5
Author: Zoltan Haindrich <kirk@rxd.hu>
Date:   Thu Jul 25 03:33:33 2024 +0000

    fix FIMXE

commit d705c2759b
Author: Zoltan Haindrich <kirk@rxd.hu>
Date:   Thu Jul 25 03:05:04 2024 +0000

    cleanup

commit a489e19242
Author: Zoltan Haindrich <kirk@rxd.hu>
Date:   Wed Jul 24 17:26:07 2024 +0000

    move to new file

commit d010b488a7
Author: Zoltan Haindrich <kirk@rxd.hu>
Date:   Wed Jul 24 17:23:15 2024 +0000

    cleanup

commit 7428da51de
Author: Zoltan Haindrich <kirk@rxd.hu>
Date:   Wed Jul 24 17:20:42 2024 +0000

    cleanup

commit 0be1f81d7e
Author: Zoltan Haindrich <kirk@rxd.hu>
Date:   Wed Jul 24 17:17:15 2024 +0000

    remove druidPrettyprinter

commit 7cfbfdc3ee
Author: Zoltan Haindrich <kirk@rxd.hu>
Date:   Wed Jul 24 17:14:30 2024 +0000

    add DruidPrettyPrinter

commit e60a200d95
Author: Zoltan Haindrich <kirk@rxd.hu>
Date:   Wed Jul 24 15:16:39 2024 +0000

    format/etc

commit a9dcb2da46
Merge: 31e97324ce d227029b6b
Author: Zoltan Haindrich <kirk@rxd.hu>
Date:   Wed Jul 24 10:59:41 2024 +0000

    Merge branch 'quidem-record' into quidem-msq

commit d227029b6b
Author: Zoltan Haindrich <kirk@rxd.hu>
Date:   Fri Jul 19 19:16:46 2024 +0000

    undo unrealted change

commit f7247e1bb7
Author: Zoltan Haindrich <kirk@rxd.hu>
Date:   Fri Jul 19 15:13:17 2024 +0000

    use entryset

commit b38935a450
Author: Zoltan Haindrich <kirk@rxd.hu>
Date:   Fri Jul 19 11:44:23 2024 +0000

    add test; fb

commit 31e97324ce
Author: Zoltan Haindrich <kirk@rxd.hu>
Date:   Fri Jul 19 11:36:51 2024 +0000

    x

commit e2a54b5758
Author: Zoltan Haindrich <kirk@rxd.hu>
Date:   Fri Jul 19 08:42:58 2024 +0000

    update

commit 361149b097
Author: Zoltan Haindrich <kirk@rxd.hu>
Date:   Fri Jul 19 07:29:50 2024 +0000

    m

commit bc7174cb6a
Author: Zoltan Haindrich <kirk@rxd.hu>
Date:   Fri Jul 19 04:30:15 2024 +0000

    cleanup

commit 9cf723adae
Author: Zoltan Haindrich <kirk@rxd.hu>
Date:   Fri Jul 19 04:29:05 2024 +0000

    rename

commit 7a34b6e092
Author: Zoltan Haindrich <kirk@rxd.hu>
Date:   Fri Jul 19 04:28:02 2024 +0000

    cleanup

commit d216b934fc
Merge: 76ff3f26e1 e388140b2a
Author: Zoltan Haindrich <kirk@rxd.hu>
Date:   Thu Jul 18 11:41:21 2024 +0000

    Merge remote-tracking branch 'kgyrtkirk/quidem-record' into quidem-record

commit 76ff3f26e1
Author: Zoltan Haindrich <kirk@rxd.hu>
Date:   Thu Jul 18 07:25:19 2024 +0000

    add supress

commit eb4fd9f66c
Author: Zoltan Haindrich <kirk@rxd.hu>
Date:   Thu Jul 18 07:24:56 2024 +0000

    removedup

commit e388140b2a
Author: Benedict Jin <asdf2014@apache.org>
Date:   Thu Jul 18 15:06:59 2024 +0800

    Apply suggestions from code review

commit 47aeb016df
Merge: 70ff2a3e97 06b68b6c89
Author: Zoltan Haindrich <kirk@rxd.hu>
Date:   Thu Jul 18 05:48:32 2024 +0000

    Merge branch 'quidem-record' into quidem-msq

commit 06b68b6c89
Merge: 82436df585 b53c26f5c5
Author: Zoltan Haindrich <kirk@rxd.hu>
Date:   Thu Jul 18 05:48:13 2024 +0000

    Merge remote-tracking branch 'apache/master' into quidem-record

commit 70ff2a3e97
Author: Zoltan Haindrich <kirk@rxd.hu>
Date:   Wed Jul 17 19:48:08 2024 +0000

    add exploratory msqPlan cmd

commit 8b26e490e9
Author: Zoltan Haindrich <kirk@rxd.hu>
Date:   Wed Jul 17 19:30:33 2024 +0000

    fix types/resultset/etc

commit c59f1adcc8
Author: Zoltan Haindrich <kirk@rxd.hu>
Date:   Wed Jul 17 16:42:22 2024 +0000

    updates

commit 95ca0a9f5d
Author: Zoltan Haindrich <kirk@rxd.hu>
Date:   Wed Jul 17 16:41:09 2024 +0000

    cleanup

commit b100e982a4
Author: Zoltan Haindrich <kirk@rxd.hu>
Date:   Wed Jul 17 16:40:30 2024 +0000

    make/etc

commit 0811d801fb
Author: Zoltan Haindrich <kirk@rxd.hu>
Date:   Wed Jul 17 16:33:10 2024 +0000

    make query run

commit 97c32ca3de
Author: Zoltan Haindrich <kirk@rxd.hu>
Date:   Wed Jul 17 16:19:08 2024 +0000

    less crappy way to run it

commit 6790f9cf8b
Author: Zoltan Haindrich <kirk@rxd.hu>
Date:   Wed Jul 17 16:08:32 2024 +0000

    move stuff

commit 51d465df6d
Author: Zoltan Haindrich <kirk@rxd.hu>
Date:   Wed Jul 17 16:04:14 2024 +0000

    make engine load via injector for msqdrill

commit 0eaf4c61b9
Author: Zoltan Haindrich <kirk@rxd.hu>
Date:   Wed Jul 17 15:51:57 2024 +0000

    removePrint

commit f3cf778115
Author: Zoltan Haindrich <kirk@rxd.hu>
Date:   Wed Jul 17 15:48:36 2024 +0000

    some stuff

commit 42b3086512
Author: Zoltan Haindrich <kirk@rxd.hu>
Date:   Wed Jul 17 15:38:50 2024 +0000

    msq-test-0

commit 8ada2ff238
Author: Zoltan Haindrich <kirk@rxd.hu>
Date:   Wed Jul 17 14:44:27 2024 +0000

    picked akshat's 3e0202811e05dcd07db5ab47791151fab5dd5772

commit 82436df585
Author: Zoltan Haindrich <kirk@rxd.hu>
Date:   Wed Jul 17 14:34:33 2024 +0000

    fix test;disable dep-check for module

commit 2a590eb3ae
Merge: 20c543dfc2 01d67ae543
Author: Zoltan Haindrich <kirk@rxd.hu>
Date:   Wed Jul 17 13:27:54 2024 +0000

    Merge commit 'apache/master^^^' into quidem-record

commit 7120f84e368b1fd7045ddbf7fe638565480faefa
Author: Zoltan Haindrich <kirk@rxd.hu>
Date:   Wed Jul 10 15:48:52 2024 +0000

    add resultsets

commit ec50ad4f8df7903e621de85c5d48201290c477be
Author: Zoltan Haindrich <kirk@rxd.hu>
Date:   Wed Jul 10 15:46:24 2024 +0000

    x

commit 400c797401e519dcae7817e76e78769b198be8e9
Author: Zoltan Haindrich <kirk@rxd.hu>
Date:   Wed Jul 10 13:42:30 2024 +0000

    expand

commit 4ec1f121bb6ed7bfbe883f3e6cf1f2a478218c34
Author: Zoltan Haindrich <kirk@rxd.hu>
Date:   Wed Jul 10 13:41:30 2024 +0000

    from sai

commit 3ff292a00f9ecf17fe75af2f5a7a2cb7c4a85635
Author: Zoltan Haindrich <kirk@rxd.hu>
Date:   Wed Jun 26 13:01:41 2024 +0000

    use kttm dataset stored in maven repo

commit a56c7534a74b76bdc6dbcccd45f14d87ebf58644
Author: Zoltan Haindrich <kirk@rxd.hu>
Date:   Wed Jun 26 12:28:39 2024 +0000

    update readme

commit 20c543dfc2
Author: Zoltan Haindrich <kirk@rxd.hu>
Date:   Wed Jun 26 12:27:15 2024 +0000

    Revert "Revert "Revert "Revert "Revert "add kttm tx"""""

    This reverts commit c559bc3422.

commit cc4e0adcbf
Author: Zoltan Haindrich <kirk@rxd.hu>
Date:   Wed Jun 26 12:06:50 2024 +0000

    make synch

commit b8992434a9
Author: Zoltan Haindrich <kirk@rxd.hu>
Date:   Wed Jun 26 11:13:32 2024 +0000

    x

commit c559bc3422
Author: Zoltan Haindrich <kirk@rxd.hu>
Date:   Wed Jun 26 10:08:01 2024 +0000

    Revert "Revert "Revert "Revert "add kttm tx""""

    This reverts commit 9a4d04a818.

commit e7141e2080
Author: Zoltan Haindrich <kirk@rxd.hu>
Date:   Wed Jun 26 10:07:48 2024 +0000

    Revert "stuff"

    This reverts commit 1b7dd8fd3c.

commit 1b7dd8fd3c
Author: Zoltan Haindrich <kirk@rxd.hu>
Date:   Wed Jun 26 10:07:47 2024 +0000

    stuff

commit 1a5faf1afb
Author: Zoltan Haindrich <kirk@rxd.hu>
Date:   Tue Jun 25 08:13:31 2024 +0000

    more pomxml stuff

commit 3dfe5c4a05
Author: Zoltan Haindrich <kirk@rxd.hu>
Date:   Tue Jun 25 07:08:01 2024 +0000

    add reflections

commit 6c02cbdf4d
Author: Zoltan Haindrich <kirk@rxd.hu>
Date:   Tue Jun 25 06:38:13 2024 +0000

    fixes

commit 0d76a73c4c
Author: Zoltan Haindrich <kirk@rxd.hu>
Date:   Tue Jun 25 06:33:45 2024 +0000

    remove final

commit 9a4d04a818
Author: Zoltan Haindrich <kirk@rxd.hu>
Date:   Thu Jun 20 17:45:08 2024 +0000

    Revert "Revert "Revert "add kttm tx"""

    This reverts commit 26a16fb4fe.

commit 0af3b910f1
Author: Zoltan Haindrich <kirk@rxd.hu>
Date:   Thu Jun 20 17:45:01 2024 +0000

    update readme

commit cefdf96a26
Author: Zoltan Haindrich <kirk@rxd.hu>
Date:   Thu Jun 20 16:45:39 2024 +0000

    prep

commit 724212381c
Author: Zoltan Haindrich <kirk@rxd.hu>
Date:   Thu Jun 20 16:40:06 2024 +0000

    close stuff

commit 26a16fb4fe
Author: Zoltan Haindrich <kirk@rxd.hu>
Date:   Thu Jun 20 16:02:09 2024 +0000

    Revert "Revert "add kttm tx""

    This reverts commit 82f24e61f2.

commit 82f24e61f2
Author: Zoltan Haindrich <kirk@rxd.hu>
Date:   Thu Jun 20 16:01:53 2024 +0000

    Revert "add kttm tx"

    This reverts commit eda48497e2.

commit d329686d5c
Author: Zoltan Haindrich <kirk@rxd.hu>
Date:   Thu Jun 20 15:37:13 2024 +0000

    fix msq test

commit 6e48cb86d5
Author: Zoltan Haindrich <kirk@rxd.hu>
Date:   Thu Jun 20 14:41:45 2024 +0000

    move annotation

commit ebb27cf462
Author: Zoltan Haindrich <kirk@rxd.hu>
Date:   Thu Jun 20 14:40:44 2024 +0000

    add extension to disabel when not sql compat

commit 604910cead
Author: Zoltan Haindrich <kirk@rxd.hu>
Date:   Thu Jun 20 14:26:48 2024 +0000

    cleanup

commit b2be5abdd5
Author: Zoltan Haindrich <kirk@rxd.hu>
Date:   Thu Jun 20 14:21:56 2024 +0000

    fix md

commit eda48497e2
Author: Zoltan Haindrich <kirk@rxd.hu>
Date:   Thu Jun 20 12:55:15 2024 +0000

    add kttm tx

commit 4bd8039715
Author: Zoltan Haindrich <kirk@rxd.hu>
Date:   Wed Jun 19 16:49:24 2024 +0000

    fix delegate

commit 1a0ab2c3b1
Merge: fdf115940b 169a8dbd1a
Author: Zoltan Haindrich <kirk@rxd.hu>
Date:   Wed Jun 19 12:59:26 2024 +0000

    Merge remote-tracking branch 'apache/master' into quidem-record

commit fdf115940b
Author: Zoltan Haindrich <kirk@rxd.hu>
Date:   Tue Jun 18 16:43:10 2024 +0000

    disable record by default

commit 61dd670784
Author: Zoltan Haindrich <kirk@rxd.hu>
Date:   Tue Jun 18 16:36:41 2024 +0000

    remove not needed

commit dffa331018
Author: Zoltan Haindrich <kirk@rxd.hu>
Date:   Tue Jun 18 16:33:22 2024 +0000

    updates/etc

commit 64a971f893
Author: Zoltan Haindrich <kirk@rxd.hu>
Date:   Tue Jun 18 16:20:53 2024 +0000

    fix autostart

commit d14b7374ba
Author: Zoltan Haindrich <kirk@rxd.hu>
Date:   Tue Jun 18 15:59:29 2024 +0000

    add

commit 74c488679b
Author: Zoltan Haindrich <kirk@rxd.hu>
Date:   Tue Jun 18 15:43:48 2024 +0000

    cleanup

commit 475f5b5830
Author: Zoltan Haindrich <kirk@rxd.hu>
Date:   Tue Jun 18 15:43:46 2024 +0000

    cleanup

commit e725df7110
Author: Zoltan Haindrich <kirk@rxd.hu>
Date:   Tue Jun 18 15:43:35 2024 +0000

    fix loader

commit 9df6f3341a
Author: Zoltan Haindrich <kirk@rxd.hu>
Date:   Tue Jun 18 14:29:02 2024 +0000

    add test-jar ref

commit 0366192554
Author: Zoltan Haindrich <kirk@rxd.hu>
Date:   Tue Jun 18 13:07:48 2024 +0000

    updates

commit 75c578ae33
Author: Zoltan Haindrich <kirk@rxd.hu>
Date:   Tue Jun 18 12:41:21 2024 +0000

    cleanup

commit e81591499c
Author: Zoltan Haindrich <kirk@rxd.hu>
Date:   Tue Jun 18 08:18:32 2024 +0000

    update/etc

commit 6dd62fb7c6
Author: Zoltan Haindrich <kirk@rxd.hu>
Date:   Mon Jun 17 16:25:03 2024 +0000

    updates

commit f5720ce97d
Author: Zoltan Haindrich <kirk@rxd.hu>
Date:   Mon Jun 17 16:05:29 2024 +0000

    u

commit 7eccf5b518
Author: Zoltan Haindrich <kirk@rxd.hu>
Date:   Mon Jun 17 15:42:48 2024 +0000

    add some validation/etc

commit e06e54631e
Author: Zoltan Haindrich <kirk@rxd.hu>
Date:   Mon Jun 17 14:26:48 2024 +0000

    fix style; emitter

commit a9952292f2
Author: Zoltan Haindrich <kirk@rxd.hu>
Date:   Mon Jun 17 13:44:01 2024 +0000

    cleanup

commit 47696a6108
Author: Zoltan Haindrich <kirk@rxd.hu>
Date:   Mon Jun 17 13:05:11 2024 +0000

    updates

commit 7a65938fd6
Author: Zoltan Haindrich <kirk@rxd.hu>
Date:   Tue Jun 11 15:55:55 2024 +0000

    use druidhook instead Hook

commit 6c6053eee0
Author: Zoltan Haindrich <kirk@rxd.hu>
Date:   Tue Jun 11 15:22:52 2024 +0000

    cleanup

commit 002c3056d4
Author: Zoltan Haindrich <kirk@rxd.hu>
Date:   Tue Jun 11 15:14:08 2024 +0000

    make test indep from java11

commit 298a470713
Author: Zoltan Haindrich <kirk@rxd.hu>
Date:   Tue Jun 11 14:18:47 2024 +0000

    try w/o upgrade guice

commit 2fbd7fe47b
Author: Zoltan Haindrich <kirk@rxd.hu>
Date:   Tue Jun 11 14:08:36 2024 +0000

    add

commit 0bcfec4c53
Author: Zoltan Haindrich <kirk@rxd.hu>
Date:   Mon Jun 10 14:05:03 2024 +0000

    trial

commit d75dcea4dc
Author: Zoltan Haindrich <kirk@rxd.hu>
Date:   Mon Jun 10 08:44:08 2024 +0000

    undo integration-tests module changes

commit 90c88aeab0
Author: Zoltan Haindrich <kirk@rxd.hu>
Date:   Tue Jun 4 10:16:10 2024 +0000

    some pom

commit fb4b32e5b7
Author: Zoltan Haindrich <kirk@rxd.hu>
Date:   Wed May 29 16:10:38 2024 +0000

    updates

commit d1b4587eae
Author: Zoltan Haindrich <kirk@rxd.hu>
Date:   Tue May 28 16:38:47 2024 +0000

    uu

commit 88da8d9659
Author: Zoltan Haindrich <kirk@rxd.hu>
Date:   Tue May 28 16:21:04 2024 +0000

    rename

commit ff7abeb0f1
Author: Zoltan Haindrich <kirk@rxd.hu>
Date:   Tue May 28 15:54:29 2024 +0000

    small cleanup

commit 295c09a03c
Author: Zoltan Haindrich <kirk@rxd.hu>
Date:   Tue May 28 15:44:22 2024 +0000

    launcher-x

commit 07e05a664d
Author: Zoltan Haindrich <kirk@rxd.hu>
Date:   Tue May 28 15:30:06 2024 +0000

    cleasnup

commit ff31c14dba
Author: Zoltan Haindrich <kirk@rxd.hu>
Date:   Tue May 28 15:27:18 2024 +0000

    clkeaup

commit a713b663b0
Author: Zoltan Haindrich <kirk@rxd.hu>
Date:   Tue May 28 15:23:04 2024 +0000

    undo

commit 6c264f2977
Author: Zoltan Haindrich <kirk@rxd.hu>
Date:   Tue May 28 15:22:42 2024 +0000

    move stuff

commit ee195719b7
Author: Zoltan Haindrich <kirk@rxd.hu>
Date:   Tue May 28 15:17:07 2024 +0000

    inline class

commit a1b7f981fb
Author: Zoltan Haindrich <kirk@rxd.hu>
Date:   Tue May 28 15:09:42 2024 +0000

    cleanup

commit b20ee99371
Author: Zoltan Haindrich <kirk@rxd.hu>
Date:   Tue May 28 15:07:09 2024 +0000

    clean

commit 9ae80f05de
Merge: ec2ecde235 206363c255
Author: Zoltan Haindrich <kirk@rxd.hu>
Date:   Mon May 27 10:52:01 2024 +0000

    Merge remote-tracking branch 'kgyrtkirk/quidem-runner-extension-submit' into quidem-record

commit ec2ecde235
Author: Zoltan Haindrich <kirk@rxd.hu>
Date:   Mon May 27 10:49:38 2024 +0000

    updates

commit 206363c255
Merge: e4ca09d756 4e1de50e30
Author: Zoltan Haindrich <kirk@rxd.hu>
Date:   Mon May 27 07:01:43 2024 +0000

    Merge remote-tracking branch 'apache/master' into quidem-runner-extension-submit

commit e4ca09d756
Merge: ad64784a29 44ea4e1c51
Author: Zoltan Haindrich <kirk@rxd.hu>
Date:   Wed May 22 18:43:41 2024 +0000

    Merge remote-tracking branch 'apache/master' into quidem-runner-extension-submit

commit 4595b0c128
Author: Zoltan Haindrich <kirk@rxd.hu>
Date:   Tue May 21 14:18:55 2024 +0000

    u[pdate

commit ba09e7d1de
Author: Zoltan Haindrich <kirk@rxd.hu>
Date:   Tue May 21 14:00:02 2024 +0000

    add

commit 08b73d1969
Author: Zoltan Haindrich <kirk@rxd.hu>
Date:   Tue May 21 12:02:46 2024 +0000

    Revert "some stuff"

    This reverts commit 52598d3bca.

commit 52598d3bca
Author: Zoltan Haindrich <kirk@rxd.hu>
Date:   Tue May 21 12:02:44 2024 +0000

    some stuff

commit ad64784a29
Author: Zoltan Haindrich <kirk@rxd.hu>
Date:   Tue May 21 06:57:26 2024 +0000

    fix test expectation

commit ecdcd0f621
Merge: 453c11f9f5 c948201507
Author: Zoltan Haindrich <kirk@rxd.hu>
Date:   Tue May 21 06:53:46 2024 +0000

    Merge remote-tracking branch 'apache/master' into quidem-runner-extension-submit

commit 453c11f9f5
Author: Zoltan Haindrich <kirk@rxd.hu>
Date:   Fri May 17 07:12:12 2024 +0000

    update exceptions;test output

commit 652cc20597
Author: Zoltan Haindrich <kirk@rxd.hu>
Date:   Fri May 17 07:00:57 2024 +0000

    use lower_camel for config names

commit 94f93865e8
Merge: b7b73fa7fe 435b58f101
Author: Zoltan Haindrich <kirk@rxd.hu>
Date:   Thu May 16 14:22:03 2024 +0000

    Merge remote-tracking branch 'apache/master' into quidem-runner-extension-submit

commit e7e119b559
Author: Zoltan Haindrich <kirk@rxd.hu>
Date:   Thu May 16 13:33:27 2024 +0000

    reduce copypaste

commit fc9a6c7740
Author: Zoltan Haindrich <kirk@rxd.hu>
Date:   Thu May 16 13:23:45 2024 +0000

    move/etc

commit cabf2a31c3
Author: Zoltan Haindrich <kirk@rxd.hu>
Date:   Thu May 16 13:19:30 2024 +0000

    fix

commit 1d2a79f5be
Author: Zoltan Haindrich <kirk@rxd.hu>
Date:   Thu May 16 13:01:53 2024 +0000

    cleanup

commit 1fb9fac159
Author: Zoltan Haindrich <kirk@rxd.hu>
Date:   Thu May 16 12:59:20 2024 +0000

    remove cl

commit 76ffbfb7cf
Author: Zoltan Haindrich <kirk@rxd.hu>
Date:   Thu May 16 12:50:38 2024 +0000

    cl

commit e2986ae612
Author: Zoltan Haindrich <kirk@rxd.hu>
Date:   Thu May 16 12:49:10 2024 +0000

    cleanup

commit f4c73e1499
Author: Zoltan Haindrich <kirk@rxd.hu>
Date:   Thu May 16 11:31:28 2024 +0000

    make old defaults to overides

commit 59be71c068
Author: Zoltan Haindrich <kirk@rxd.hu>
Date:   Thu May 16 11:23:26 2024 +0000

    add other

commit 607ef174c5
Author: Zoltan Haindrich <kirk@rxd.hu>
Date:   Thu May 16 11:18:54 2024 +0000

    indent

commit 3658ab24c3
Author: Zoltan Haindrich <kirk@rxd.hu>
Date:   Thu May 16 11:18:24 2024 +0000

    remove f

commit bec1f38a0e
Author: Zoltan Haindrich <kirk@rxd.hu>
Date:   Thu May 16 11:17:05 2024 +0000

    move sqlmodule down

commit 688611eab3
Author: Zoltan Haindrich <kirk@rxd.hu>
Date:   Thu May 16 11:11:55 2024 +0000

    undo

commit 93892b6524
Author: Zoltan Haindrich <kirk@rxd.hu>
Date:   Thu May 16 11:11:03 2024 +0000

    undo some

commit b63a80e5b7
Author: Zoltan Haindrich <kirk@rxd.hu>
Date:   Thu May 16 11:01:39 2024 +0000

    passes basic test

commit 118eb61939
Author: Zoltan Haindrich <kirk@rxd.hu>
Date:   Thu May 16 10:31:38 2024 +0000

    there - with 1 boot

commit 28ea884e19
Author: Zoltan Haindrich <kirk@rxd.hu>
Date:   Thu May 16 10:01:22 2024 +0000

    almost ready?

commit 27735f2621
Author: Zoltan Haindrich <kirk@rxd.hu>
Date:   Thu May 16 09:50:10 2024 +0000

    move disco

commit cab3d945be
Author: Zoltan Haindrich <kirk@rxd.hu>
Date:   Thu May 16 09:48:18 2024 +0000

    up

commit c9638b7836
Author: Zoltan Haindrich <kirk@rxd.hu>
Date:   Thu May 16 09:44:16 2024 +0000

    update

commit 7e10df1ffa
Author: Zoltan Haindrich <kirk@rxd.hu>
Date:   Thu May 16 09:33:51 2024 +0000

    ...

commit 4a47b0229e
Author: Zoltan Haindrich <kirk@rxd.hu>
Date:   Thu May 16 09:31:21 2024 +0000

    no roles

commit 5f552a2997
Author: Zoltan Haindrich <kirk@rxd.hu>
Date:   Thu May 16 09:30:41 2024 +0000

    c

commit 074161dfde
Author: Zoltan Haindrich <kirk@rxd.hu>
Date:   Thu May 16 05:53:42 2024 +0000

    add some service crap

commit 55b2051f9d
Author: Zoltan Haindrich <kirk@rxd.hu>
Date:   Wed May 15 16:23:11 2024 +0000

    workinhg stuff

commit 8ee41f58d0
Author: Zoltan Haindrich <kirk@rxd.hu>
Date:   Wed May 15 15:14:43 2024 +0000

    it does work

commit d4b052a579
Author: Zoltan Haindrich <kirk@rxd.hu>
Date:   Wed May 15 11:57:13 2024 +0000

    stuff

commit 73011267af
Author: Zoltan Haindrich <kirk@rxd.hu>
Date:   Wed May 15 10:34:48 2024 +0000

    triaks

commit a16f982699
Author: Zoltan Haindrich <kirk@rxd.hu>
Date:   Tue May 14 16:04:19 2024 +0000

    remove crap

commit 43fd8af63c
Author: Zoltan Haindrich <kirk@rxd.hu>
Date:   Tue May 14 09:39:04 2024 +0000

    Revert "add"

    This reverts commit 3fbb3cb853.

commit 3fbb3cb853
Author: Zoltan Haindrich <kirk@rxd.hu>
Date:   Tue May 14 09:39:02 2024 +0000

    add

commit b7b73fa7fe
Author: Zoltan Haindrich <kirk@rxd.hu>
Date:   Tue May 14 08:53:01 2024 +0000

    fix context key order

commit 9578953678
Merge: 3132c12781 18a4722d11
Author: Zoltan Haindrich <kirk@rxd.hu>
Date:   Tue May 14 07:36:48 2024 +0000

    Merge remote-tracking branch 'apache/master' into quidem-runner-extension-submit

commit 3132c12781
Author: Zoltan Haindrich <kirk@rxd.hu>
Date:   Tue May 14 07:36:07 2024 +0000

    remove unnecessary \\

commit e36c46a85a
Author: Zoltan Haindrich <kirk@rxd.hu>
Date:   Mon May 13 14:11:19 2024 +0000

    fix import

    style fixes

    clenaup

commit e13d560b6e
Author: Zoltan Haindrich <kirk@rxd.hu>
Date:   Fri May 10 13:27:47 2024 +0000

    Enable quidem shadowing for decoupled testcases

    * Altered `QueryTestBuilder` to be able to switch to a backing quidem test
    * added a small crc to ensure that the shadow testcase does not deviate from the original one
    * Packaged all decoupled related things into a a single `DecoupledExtension` to reduce copy-paste
    * `DecoupledTestConfig#quidemReason` must describe why its being used
    * `DecoupledTestConfig#separateDefaultModeTest` can be used to make multiple case files based on `NullHandling` state
    * fixed a cosmetic bug during decoupled join translation
    * enhanced `!druidPlan` to report the final logical plan in non-decoupled mode as well
    * add check to ensure that only supported params are present in a druidtest uri
    * enabled shadow testcases for previously disabled testcases

* correct resultset

* update column types

* add dataset scaler

* scaled?

* disable scaler

* missed updates

* update README

* remove ScaledResoureInputDataSource for now

* Revert "remove ScaledResoureInputDataSource for now"

This reverts commit dd1905d2ef.

* use fraction

* use fraction+maxRows

* remove maxRows

* Revert "remove maxRows"

This reverts commit 4d4de51f4f.

* remove NoopRowIngestionMeters.INSTANCE

* update test

* Revert "Revert "remove maxRows""

This reverts commit e366192fb5.

* more-wood/etc

* instead of scaled+etc; load and smoke test the supplier for defaults mode

* Update quidem-ut/README.md

Co-authored-by: Abhishek Agarwal <1477457+abhishekagarwal87@users.noreply.github.com>

* add comment to all iq files; move notes into a README file next to the iq files

* add hdr to readme

* Update quidem-ut/src/test/quidem/org.apache.druid.quidem.QTest/README.generated

Co-authored-by: Abhishek Agarwal <1477457+abhishekagarwal87@users.noreply.github.com>

---------

Co-authored-by: Abhishek Agarwal <1477457+abhishekagarwal87@users.noreply.github.com>
2024-09-26 15:24:23 +05:30
Clint Wylie d77637344d
log.warn anytime a column is relying on ArrayIngestMode.MVD (#17164)
* log.warn anytime a column is relying on ArrayIngestMode.MVD
2024-09-26 13:44:37 +05:30
Cece Mei a2b011cdcd
Incorporate `estimatedComputeCost` into all `BitmapColumnIndex` classes. (#17125)
changes:
* filter index processing is now automatically ordered based on estimated 'cost', which is approximated based on how many expected bitmap operations are required to construct the bitmap used for the 'offset'
* cursorAutoArrangeFilters context flag now defaults to true, but can be set to false to disable cost based filter index sorting
2024-09-25 23:11:26 -07:00
Clint Wylie 1b5b61ef7f
add multi-value string object vector matcher and expression vector object selectors (#17162) 2024-09-25 22:57:29 -07:00
Gian Merlino 2e531bb724
ScanQueryFrameProcessor: Close CursorHolders as we go along. (#17152)
* ScanQueryFrameProcessor: Close CursorHolders as we go along.

The change in #16533 added CursorHolders to the processor-level Closer.
This is problematic when running on an input channel: it means we started
keeping around all CursorHolders for all frames we process and closing them
when the channel is complete, rather than closing them as we go along.

This patch restores the prior behavior, where resources are closed as we go.

* Fix other call sites.

* Fix reference.

* Improvements.
2024-09-25 15:51:52 -07:00
Victoria Lim 203d6345af
docs: Separate section on ingesting MVDs in migration guide (#17109) 2024-09-25 14:45:25 -07:00
Zoltan Haindrich 6f7e8ca74a
Decoupled planning: improve join predicate handling (#17105)
* enforces to only allow supported predicates in join conditions
* fixed a recursive query building issue by caching the `source` in `DruidQueryGenerator`
* moved `DruidAggregateRemoveRedundancyRule.instance` higher up; as if `CoreRules.AGGREGATE_EXPAND_DISTINCT_AGGREGATES` runs earlier the resulting `GROUPING` might become invalid
2024-09-25 16:00:25 +02:00
Abhishek Radhakrishnan 9132a65a48
Add `StreamSupervisor` interface (#17151)
Follow up to #17137.

Instead of moving the streaming-only methods to the SeekableStreamSupervisor abstract class, this patch moves them to a separate StreamSupervisor interface. The reason is that the SeekableStreamSupervisor abstract class also has many other abstract methods. The StreamSupervisor interface on the other hand provides a minimal set of functions offering a good middle ground for any custom concrete implementation that doesn't require all the goodies from SeekableStreamSupervisor.
2024-09-25 14:52:39 +05:30
Akshat Jain 446ffc325c
MSQ window functions: Fix boost column not being written to the frame in window stage (#17141)
In WindowOperatorQueryFrameProcessor, currently we were not adding the partition boost column to the ColumnSelectorFactory used to create the FrameWriter. Because of this, the boost column was not being written to the frame.
2024-09-25 09:50:42 +05:30
Adarsh Sanjeev 464b153850
Make sketch encoding configurable (#17086)
Makes sketch encoding in MSQ configurable by the user. This would allow a user to configure the sketch encoding method for a specific query.

The default is octet stream encoding.
2024-09-25 09:15:32 +05:30
Clint Wylie 0ec7eb3ae5
use CastToObjectVectorProcessor for cast to string (#17148) 2024-09-24 15:21:33 -07:00
Abhishek Radhakrishnan 83299e9882
Miscellaneous cleanup in the supervisor API flow. (#17144)
Extracting a few miscellaneous non-functional changes from the batch supervisor branch:

- Replace anonymous inner classes with lambda expressions in the SQL supervisor manager layer
- Add explicit @Nullable annotations in DynamicConfigProviderUtils to make IDE happy
- Small variable renames (copy-paste error perhaps) and fix typos
- Add table name for this exception message: Delete the supervisor from the table[%s] in the database...
- Prefer CollectionUtils.isEmptyOrNull() over list == null || list.size() > 0. We can change the Precondition checks to throwing DruidException separately for a batch of APIs at a time.
2024-09-24 13:06:23 -07:00
George Shiqi Wu d1bfabbf4d
inter-Extension dependency support (#16973)
* update docs for kafka lookup extension to specify correct extension ordering

* fix first line

* test with extension dependencies

* save work on dependency management

* working dependency graph

* working pull

* fix style

* fix style

* remove name

* load extension dependencies recursively

* generate depenencies on classloader creation

* add check for circular dependencies

* fix style

* revert style changes

* remove mutable class loader

* clean up class heirarchy

* extensions loader test working

* add unit tests

* pr comments

* fix unit tests
2024-09-24 14:17:33 -04:00
Abhishek Radhakrishnan 5c862f6ed9
Refactor: Move streaming supervisor methods to `SeekableStreamSupervisor` (#17137)
The current Supervisor interface is primarily focused on streaming use cases. However, as we introduce supervisors for non-streaming use cases, such as the recently added CompactionSupervisor (and the upcoming BatchSupervisor), certain operations like resetting offsets, checkpointing, task group handoff, etc., are not really applicable to non-streaming use cases.

So the methods are split between:

1. Supervisor: common methods that are applicable to both streaming and non-streaming use cases
2. SeekableStreamSupervisor: Supervisor + streaming-only operations. The existing streaming-only overrides exist along with the new abstract method public abstract LagStats computeLagStats(), for which custom implementations already exist in the concrete types

This PR is primarily a refactoring change with minimal functional adjustments (e.g., throwing an exception in a few places in SupervisorManager when the supervisor isn't the expected SeekableStreamSupervisor type).
2024-09-24 10:46:37 -07:00
Vadim Ogievetsky 4872f0457a
switch to using arrays by default (#17133) 2024-09-24 09:20:04 -07:00
Vadim Ogievetsky 1a13bd5485
Make the tooltip better and improve structure (#17132) 2024-09-24 09:19:52 -07:00
Kashif Faraz 9670305669
Cleanup Coordinator logs, add duty status API (#16959)
Description
-----------
Coordinator logs are fairly noisy and don't give much useful information (see example below).
Even when the Coordinator misbehaves, these logs are not very useful.

Main changes
------------
- Add API `GET /druid/coordinator/v1/duties` that returns a status list of all duty groups currently running on the Coordinator
- Emit metrics `segment/poll/time`, `segment/pollWithSchema/time`, `segment/buildSnapshot/time`
- Remove redundant logs that indicate normal operation of well-tested aspects of the Coordinator

Refactors
---------
- Move some logic from `DutiesRunnable` to `CoordinatorDutyGroup`
- Move stats collection from `CollectSegmentAndServerStats` to `PrepareBalancerAndLoadQueues`
- Minor cleanup of class `DruidCoordinator`
- Clean up class `DruidCoordinatorRuntimeParams`
  - Remove field `coordinatorStartTime`. Maintain start time in `MarkOvershadowedSegmentsAsUnused` instead.
  - Remove field `MetadataRuleManager`. Pass supplier to constructor of applicable duties instead.
  - Make `usedSegmentsNewestFirst` and `datasourcesSnapshot` as non-nullable as they are always required.
2024-09-24 19:46:22 +05:30
Vishesh Garg f576e299db
Allow MSQ engine only for compaction supervisors (#17033)
#16768 added the functionality to run compaction as a supervisor on the overlord.
This patch builds on top of that to restrict MSQ engine to compaction in the supervisor-mode only.
With these changes, users can no longer add MSQ engine as part of datasource compaction config,
or as the default cluster-level compaction engine, on the Coordinator. 

The patch also adds an Overlord runtime property `druid.supervisor.compaction.engine=<msq/native>`
to specify the default engine for compaction supervisors.

Since these updates require major changes to existing MSQ compaction integration tests,
this patch disables MSQ-specific compaction integration tests -- they will be taken up in a follow-up PR.

Key changed/added classes in this patch:
* CompactionSupervisor
* CompactionSupervisorSpec
* CoordinatorCompactionConfigsResource
* OverlordCompactionScheduler
2024-09-24 17:19:16 +05:30
Clint Wylie 77a362c555
various fixes and improvements to vectorization fallback (#17098)
changes:
* add `ApplyFunction` support to vectorization fallback, allowing many of the remaining expressions to be vectorized
* add `CastToObjectVectorProcessor` so that vector engine can correctly cast any type
* add support for array and complex vector constants
* reduce number of cases which can block vectorization in expression planner to be unknown inputs (such as unknown multi-valuedness)
* fix array constructor expression, apply map expression to make actual evaluated type match the output type inference
* fix bug in array_contains where something like array_contains([null], 'hello') would return true if the array was a numeric array since the non-null string value would cast to a null numeric
* fix isNull/isNotNull to correctly handle any type of input argument
2024-09-24 04:29:08 -07:00
Atul Mohan c1f8ae25b5
Support Iceberg ingestion from REST based catalogs (#17124)
Adds support to the iceberg input source to read from Iceberg REST Catalogs.
2024-09-23 22:13:24 -07:00
Vadim Ogievetsky ed33dbb76d
Web console: add stage graph (#17135)
* add stage graph

* update snapshot
2024-09-23 21:19:13 -07:00
PANKAJ KUMAR 36dfff4b1a
Adding extra debug logs for the checkpoint logic (#16321)
Logging to understand checkpointing better in streaming ingestion
2024-09-24 09:38:46 +05:30
Adithya Chakilam 8eaac2c051
cgroup monitors: Add mem/disk/cpu usage metrics for V2 (#16905)
* cgroup monitors: Add mem/disk/cpu usage metrics for V2

* intellij inspection

* docs and checks

* fix-dos

* add comments

* comments
2024-09-23 20:32:01 -07:00
zachjsh ba8245f114
Properly url encode schema and table names when syncing catalog (#17131)
* SQL syntax error should target USER persona

* * revert change to queryHandler and related tests, based on review comments

* * add test

* Properly encode schema and table names when syncing catalog

Previously table names and schema names were not being properly url
encododed when requested. Now they are.
2024-09-23 23:23:26 -04:00