Apache
/
druid
mirror of https://github.com/apache/druid.git
1
0
Fork 0
Code Issues Packages Projects Releases Wiki Activity
14,336 Commits 111 Branches 503 Tags 723 MiB
Commit Graph

14336 Commits

Author SHA1 Message Date
Zoltan Haindrich d705c2759b cleanup 2024-07-25 03:05:04 +00:00
Zoltan Haindrich a489e19242 move to new file 2024-07-24 17:26:07 +00:00
Zoltan Haindrich d010b488a7 cleanup 2024-07-24 17:23:15 +00:00
Zoltan Haindrich 7428da51de cleanup 2024-07-24 17:20:42 +00:00
Zoltan Haindrich 0be1f81d7e remove druidPrettyprinter 2024-07-24 17:17:15 +00:00
Zoltan Haindrich 7cfbfdc3ee add DruidPrettyPrinter 2024-07-24 17:14:30 +00:00
Zoltan Haindrich e60a200d95 format/etc 2024-07-24 15:16:39 +00:00
Zoltan Haindrich a9dcb2da46 Merge branch 'quidem-record' into quidem-msq 2024-07-24 10:59:41 +00:00
Zoltan Haindrich d227029b6b undo unrealted change 2024-07-19 19:16:46 +00:00
Zoltan Haindrich f7247e1bb7 use entryset 2024-07-19 15:13:17 +00:00
Zoltan Haindrich b38935a450 add test; fb 2024-07-19 11:44:23 +00:00
Zoltan Haindrich 31e97324ce x 2024-07-19 11:36:51 +00:00
Zoltan Haindrich e2a54b5758 update 2024-07-19 08:42:58 +00:00
Zoltan Haindrich 361149b097 m 2024-07-19 07:29:50 +00:00
Zoltan Haindrich bc7174cb6a cleanup 2024-07-19 04:30:15 +00:00
Zoltan Haindrich 9cf723adae rename 2024-07-19 04:29:05 +00:00
Zoltan Haindrich 7a34b6e092 cleanup 2024-07-19 04:28:02 +00:00
Zoltan Haindrich d216b934fc Merge remote-tracking branch 'kgyrtkirk/quidem-record' into quidem-record 2024-07-18 11:41:21 +00:00
Zoltan Haindrich 76ff3f26e1 add supress 2024-07-18 07:25:19 +00:00
Zoltan Haindrich eb4fd9f66c removedup 2024-07-18 07:24:56 +00:00
Benedict Jin e388140b2a
Apply suggestions from code review 2024-07-18 15:06:59 +08:00
Zoltan Haindrich 47aeb016df Merge branch 'quidem-record' into quidem-msq 2024-07-18 05:48:32 +00:00
Zoltan Haindrich 06b68b6c89 Merge remote-tracking branch 'apache/master' into quidem-record 2024-07-18 05:48:13 +00:00
Akshat Jain b53c26f5c5
Fix issues with partitioning boundaries for MSQ window functions (#16729) 
* Fix issues with partitioning boundaries for MSQ window functions

* Address review comments

* Address review comments

* Add test for coverage check failure

* Address review comment

* Remove DruidWindowQueryTest and WindowQueryTestBase, move those tests to DrillWindowQueryTest

* Update extensions-core/multi-stage-query/src/main/java/org/apache/druid/msq/querykit/WindowOperatorQueryKit.java

* Address review comments

* Add test for equals and hashcode for WindowOperatorQueryFrameProcessorFactory

* Address review comment

* Fix checkstyle

---------

Co-authored-by: Benedict Jin <asdf2014@apache.org>
 2024-07-18 10:05:09 +08:00
Vadim Ogievetsky 44b3f8e588
Web console: fix a few console bugs (#16735) 
* remove __time from min max query shortcut

* fix scrolling in retention rules dialog

* actions menus should have titles

* change term

* correctly name sort/shuffle
 2024-07-17 14:51:17 -07:00
Zoltan Haindrich 70ff2a3e97 add exploratory msqPlan cmd 2024-07-17 19:48:08 +00:00
Zoltan Haindrich 8b26e490e9 fix types/resultset/etc 2024-07-17 19:30:33 +00:00
Kashif Faraz 89066b72cf
Fix bug in TaskStorageQueryAdapter (#16750) 
Changes:
- Do not hold a reference to `TaskQueue` in `TaskStorageQueryAdapter`
- Use `TaskStorage` instead of `TaskStorageQueryAdapter` in `IndexerMetadataStorageAdapter`
- Rename `TaskStorageQueryAdapter` to `TaskQueryTool`
- Fix newly added task actions `RetrieveUpgradedFromSegmentIds` and `RetrieveUpgradedToSegmentIds`
by removing `isAudited` method.
 2024-07-17 23:17:41 +05:30
Zoltan Haindrich c59f1adcc8 updates 2024-07-17 16:42:22 +00:00
Zoltan Haindrich 95ca0a9f5d cleanup 2024-07-17 16:41:09 +00:00
Zoltan Haindrich b100e982a4 make/etc 2024-07-17 16:40:30 +00:00
Zoltan Haindrich 0811d801fb make query run 2024-07-17 16:33:10 +00:00
Zoltan Haindrich 97c32ca3de less crappy way to run it 2024-07-17 16:19:08 +00:00
Zoltan Haindrich 6790f9cf8b move stuff 2024-07-17 16:08:32 +00:00
Zoltan Haindrich 51d465df6d make engine load via injector for msqdrill 2024-07-17 16:04:14 +00:00
Zoltan Haindrich 0eaf4c61b9 removePrint 2024-07-17 15:52:19 +00:00
Zoltan Haindrich f3cf778115 some stuff 2024-07-17 15:48:36 +00:00
Zoltan Haindrich 42b3086512 msq-test-0 2024-07-17 15:38:50 +00:00
Zoltan Haindrich 8ada2ff238 picked akshat's 3e0202811e05dcd07db5ab47791151fab5dd5772 2024-07-17 14:44:27 +00:00
Zoltan Haindrich 82436df585 fix test;disable dep-check for module 2024-07-17 14:34:33 +00:00
Zoltan Haindrich 2a590eb3ae Merge commit 'apache/master^^^' into quidem-record 2024-07-17 13:27:54 +00:00
Sree Charan Manamala 40ef9fc4ec
Bug fix for array type selector causing array aggregation over window frame fail (#16653) 2024-07-17 14:09:56 +02:00
Kashif Faraz 9f6ce6ddc0
Remove task action audit logging and druid_taskLog metadata table (#16309) 
Description:
Task action audit logging was first deprecated and disabled by default in Druid 0.13, #6368.

As called out in the original discussion #5859, there are several drawbacks to persisting task action audit logs. 
- Only usage of the task audit logs is to serve the API `/indexer/v1/task/{taskId}/segments`
which returns the list of segments created by a task.
- The use case is really narrow and no prod clusters really use this information.
- There can be better ways of obtaining this information, such as the metric
`segment/added/bytes` which reports both the segment ID and task ID
when a segment is committed by a task. We could also include committed segment IDs in task reports.
- A task persisting several segments would bloat up the audit logs table putting unnecessary strain
on metadata storage.

Changes:
- Remove `TaskAuditLogConfig`
- Remove method `TaskAction.isAudited()`. No task action is audited anymore.
- Remove `SegmentInsertAction` as it is not used anymore. `SegmentTransactionalInsertAction`
is the new incarnation which has been in use for a while.
- Deprecate `MetadataStorageActionHandler.addLog()` and `getLogs()`. These are not used anymore
but need to be retained for backward compatibility of extensions.
- Do not create `druid_taskLog` metadata table anymore.
 2024-07-17 17:09:00 +05:30
trompa ebf216829d
#16717 defer provider instantiation in Kubernetes Module (#16726) 
* #16717 defer provider instatiation

* add license header

* fix style, ignore new class in jacoco as it is still initialization code

---------

Co-authored-by: Alberto Lago Alvarado <albl@sitecore.net>
 2024-07-16 13:05:28 -07:00
Kashif Faraz 01d67ae543
Allow CompactionSegmentIterator to have custom priority (#16737) 
Changes:
- Break `NewestSegmentFirstIterator` into two parts
  - `DatasourceCompactibleSegmentIterator` - this contains all the code from `NewestSegmentFirstIterator`
  but now handles a single datasource and allows a priority to be specified
  - `PriorityBasedCompactionSegmentIterator` - contains separate iterator for each datasource and
  combines the results into a single queue to be used by a compaction search policy
- Update `NewestSegmentFirstPolicy` to use the above new classes
- Cleanup `CompactionStatistics` and `AutoCompactionSnapshot`
- Cleanup `CompactSegments`
- Remove unused methods from `Tasks`
- Remove unneeded `TasksTest`
- Move tests from `NewestSegmentFirstIteratorTest` to `CompactionStatusTest`
and `DatasourceCompactibleSegmentIteratorTest`
 2024-07-16 19:54:49 +05:30
Adithya Chakilam 6cf6838eb9
kubernetes-overlord-extension: Fix tasks not being shutdown (#16711) 2024-07-15 14:35:11 -07:00
AmatyaAvadhanula 6891866c43
Process retrieval of parent and child segment ids in batches (#16734) 2024-07-15 18:24:23 +05:30
Sree Charan Manamala 78a4a09d01
Window Function offset correction for RAC (#16718) 
* When an ArrayList RAC creates a child RAC, the start and end offsets need to have the offset of parent's start offset
* Defaults the 2nd window bound to CURRENT ROW when only a single bound is specified
* Removes the windowingStrictValidation warning and throws a hard exception when Order By alongside RANGE clause is not provided with UNBOUNDED or CURRENT ROW as both bounds
 2024-07-15 12:43:27 +02:00
Rishabh Singh 64104533ac
Enable querying entirely cold datasources (#16676) 
Add ability to query entirely cold datasources.
 2024-07-15 15:02:59 +05:30
Laksh Singla 209f8a9546
Deserialize complex dimensions in group by queries to their respective types when reading from spilled files and cached results (#16620) 
Like #16511, but for keys that have been spilled or cached during the grouping process
 2024-07-15 15:00:17 +05:30