Commit Graph

606 Commits

Author SHA1 Message Date
Misha 354a3bea0b
The default `WHERE' filter for automatically generated SQL queries is returned (#16608)
* Returned the default `WHERE` filter for auto-generated SQL queries

* Checkstyle fix

---------

Co-authored-by: sviatahorau <mikhail.sviatahorau@deep.bi>
2024-06-24 08:52:35 -07:00
Vadim Ogievetsky 51c73b5a4e
Web console: show formatted JSON value (#16632)
* show formatted json value

* update snapshot

* window functions

* count star can also have a window

* better edit query context
2024-06-21 18:33:15 -07:00
Vadim Ogievetsky efe9079f0a
Web console: fix pagination and filtering regression in supervisor view (#16571)
* fix pagination and filtering in supervisor view

* update snapshot
2024-06-07 21:09:51 +05:30
Andreas Maechler e9f723344b
Disable event hubs when kafka extensions isn't loaded (#16559) 2024-06-06 16:59:26 -07:00
Vadim Ogievetsky 0b4ac78a7b
Web console: fix delta sorting in the explore view table (#16542)
* more robust query naming

* make order by delta work

* fix tests

* fix type imports

* tidy up
2024-06-04 10:15:35 -07:00
Sensor 6bc29534a7
[Web Console] Datasource page support search datasource by keyword (#16371)
* Frontend segment_timeline support filter by datasource

* add dependency

* fix eslint issues

* resolve code style issue, update snapshot

* fix comment

* update licence

* update package-lock.json

* update snapshot

* Update segment-timeline.tsx

* Update segment-timeline.tsx
2024-05-24 11:54:26 -07:00
Vadim Ogievetsky 10ea88e5bf
Web console: more robust durable storage setting detection (#16493)
* more robust durable storage setting

* add test
2024-05-22 15:47:20 -07:00
Vadim Ogievetsky 0ab3b34117
Web console: enable copy data as inline SQL (via VALUES) (#16458)
* copy as values

* address NULL issue

* add decription

* extend test

* fix json

* more types

* fix braces with nulls

* fix test

* update functions to scan
2024-05-22 08:33:07 -07:00
Vadim Ogievetsky 435b58f101
Web console: fix Druid doctor check to accept Java 17 (#16250)
* fix Druid doctor check

* fix doc link

* Update web-console/src/dialogs/doctor-dialog/doctor-checks.tsx

Co-authored-by: Abhishek Radhakrishnan <abhishek.rb19@gmail.com>

---------

Co-authored-by: Abhishek Radhakrishnan <abhishek.rb19@gmail.com>
2024-05-15 20:37:15 -07:00
Vadim Ogievetsky c419ae5f73
use objectGlob (#16452)
Catching up to a change introduced in #13027
2024-05-15 15:11:11 +05:30
Vadim Ogievetsky 760e449875
Web console: Fix order-by-delta in explore view table (#16417)
* change to using measure name

* Implment order by delta

* less paring, stricter types

* safeDivide0

* fix no query

* new DTQ alows parsing JSON_VALUE(...RETURNING...)
2024-05-13 19:03:46 -07:00
Abhishek Radhakrishnan 2a638d77d9
Remove stale references to coordinator dynamic config killAllDataSources. (#16387)
This parameter has been removed for awhile now as of Druid 0.23.0
https://github.com/apache/druid/pull/12187.

The code was only used in tests to verify that serialization works.
Now remove all references to avoid any confusion.
2024-05-05 08:48:56 +05:30
Vadim Ogievetsky 4d62c4a917
Web console: concat data when doing a durable storage download (#16375)
* concat data

* fix silly console.error
2024-05-03 08:00:32 -07:00
Abhishek Radhakrishnan 3717554e16
Web console changes for https://github.com/apache/druid/pull/16288 (#16379)
Adds a text box for delta filter that can accept an optional json
object.
2024-05-02 15:50:17 -07:00
Vadim Ogievetsky 39ada8b9ad
Web console: surface more info on the supervisor view (#16318)
* add rate and stats

* better tabs

* detail

* add recent errors

* update tests

* don't let people hide the actions column because why

* don't sort on actions

* better way to agg

* add timeouts

* show error only once

* fix tests and Explain showing up

* only consider active tasks

* refresh

* fix tests

* better formatting
2024-05-02 08:50:27 -07:00
John Gozde 834b0eddeb
web-console: ACE editor refactoring (#16359)
* Move druid-sql completions to dsql mode

* Use font-size 12

* Convert ace-modes to typescript

* Move aceCompleters to class member

* Use namespace imports
2024-04-30 11:53:39 -07:00
Adarsh Sanjeev 9a2d7c28bc
Prepare master branch for 31.0.0 release (#16333) 2024-04-26 09:22:43 +05:30
Vadim Ogievetsky 3e42ebbaea
Web console: Fix the supervisor offset reset dialog. (#16298)
* Add host to query output

* Init fixes for reset offsets

* fix the supervisor offset reset dialog

* Update web-console/src/views/load-data-view/load-data-view.tsx

Co-authored-by: Katya Macedo  <38017980+ektravel@users.noreply.github.com>

* Update web-console/src/views/load-data-view/load-data-view.tsx

Co-authored-by: Katya Macedo  <38017980+ektravel@users.noreply.github.com>

* Update web-console/src/views/load-data-view/load-data-view.tsx

Co-authored-by: Katya Macedo  <38017980+ektravel@users.noreply.github.com>

* reformat code

* &apos;

* fix conflict

---------

Co-authored-by: Katya Macedo <38017980+ektravel@users.noreply.github.com>
2024-04-19 17:25:46 -07:00
Abhishek Radhakrishnan 041d0bff5e
Set default `KillUnusedSegments` duty to coordinator's indexing period & `killTaskSlotRatio` to 0.1 (#16247)
The default value for druid.coordinator.kill.period (if unspecified) has changed from P1D to the value of druid.coordinator.period.indexingPeriod. Operators can choose to override druid.coordinator.kill.period and that will take precedence over the default behavior.
The default value for the coordinator dynamic config killTaskSlotRatio is updated from 1.0 to 0.1. This ensures that that kill tasks take up only 1 task slot right out-of-the-box instead of taking up all the task slots.

* Remove stale comment and inline canDutyRun()

* druid.coordinator.kill.period defaults to druid.coordinator.period.indexingPeriod if not set.

- Remove the default P1D value for druid.coordinator.kill.period. Instead default
  druid.coordinator.kill.period to whatever value druid.coordinator.period.indexingPeriod is set
  to if the former config isn't specified.
- If druid.coordinator.kill.period is set, the value will take precedence over
  druid.coordinator.period.indexingPeriod

* Update server/src/test/java/org/apache/druid/server/coordinator/DruidCoordinatorConfigTest.java

* Fix checkstyle error

* Clarify comment

* Update server/src/main/java/org/apache/druid/server/coordinator/DruidCoordinatorConfig.java

* Put back canDutyRun()

* Default killTaskSlotsRatio to 0.1 instead of 1.0 (all slots)

* Fix typo DEFAULT_MAX_COMPACTION_TASK_SLOTS

* Remove unused test method.

* Update default value of killTaskSlotsRatio in docs and web-console default mock

* Move initDuty() after params and config setup.
2024-04-14 18:56:17 -07:00
Vadim Ogievetsky 4ff7e2c6c9
Web console: Better manual capabilities detection indication (#16191)
* Better forced mode indication

* more robust

* Update web-console/src/components/header-bar/header-bar.tsx

Co-authored-by: Charles Smith <techdocsmith@gmail.com>

* Update web-console/src/components/header-bar/header-bar.tsx

Co-authored-by: Charles Smith <techdocsmith@gmail.com>

* Update web-console/src/components/header-bar/restricted-mode/__snapshots__/restricted-mode.spec.tsx.snap

Co-authored-by: Charles Smith <techdocsmith@gmail.com>

* Update web-console/src/components/header-bar/restricted-mode/__snapshots__/restricted-mode.spec.tsx.snap

Co-authored-by: Charles Smith <techdocsmith@gmail.com>

* Update web-console/src/components/header-bar/restricted-mode/restricted-mode.tsx

Co-authored-by: Charles Smith <techdocsmith@gmail.com>

* Update web-console/src/components/header-bar/restricted-mode/restricted-mode.tsx

Co-authored-by: Charles Smith <techdocsmith@gmail.com>

* Update web-console/src/components/header-bar/restricted-mode/restricted-mode.tsx

Co-authored-by: Charles Smith <techdocsmith@gmail.com>

* Update web-console/src/components/header-bar/restricted-mode/restricted-mode.tsx

Co-authored-by: Charles Smith <techdocsmith@gmail.com>

* Update web-console/src/components/header-bar/restricted-mode/restricted-mode.tsx

Co-authored-by: Charles Smith <techdocsmith@gmail.com>

* Update web-console/src/components/header-bar/restricted-mode/restricted-mode.tsx

Co-authored-by: Charles Smith <techdocsmith@gmail.com>

* Update web-console/src/components/header-bar/restricted-mode/restricted-mode.tsx

Co-authored-by: Charles Smith <techdocsmith@gmail.com>

* reformat

* forced => manual capability detection

* typo

* typo2

---------

Co-authored-by: Charles Smith <techdocsmith@gmail.com>
2024-04-08 10:07:21 -07:00
Vadim Ogievetsky 3ba878f21b
don't send lookups to sampler (#16234) 2024-04-04 21:17:42 -07:00
Vadim Ogievetsky 9658e1ad7f
Web console: fix query timer issues (#16235)
* fix timer issues

* wording
2024-04-04 13:13:31 -07:00
Vadim Ogievetsky 06268bf060
only pick kafka input format by default when needed (#16180) 2024-04-01 13:47:49 -07:00
dependabot[bot] 27b4028782
Bump webpack-dev-middleware from 5.3.3 to 5.3.4 in /web-console (#16195)
Bumps [webpack-dev-middleware](https://github.com/webpack/webpack-dev-middleware) from 5.3.3 to 5.3.4.
- [Release notes](https://github.com/webpack/webpack-dev-middleware/releases)
- [Changelog](https://github.com/webpack/webpack-dev-middleware/blob/v5.3.4/CHANGELOG.md)
- [Commits](https://github.com/webpack/webpack-dev-middleware/compare/v5.3.3...v5.3.4)

---
updated-dependencies:
- dependency-name: webpack-dev-middleware
  dependency-type: indirect
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2024-03-29 09:51:51 -07:00
dependabot[bot] bf88ddb2d7
Bump express from 4.18.2 to 4.19.2 in /web-console (#16204)
Bumps [express](https://github.com/expressjs/express) from 4.18.2 to 4.19.2.
- [Release notes](https://github.com/expressjs/express/releases)
- [Changelog](https://github.com/expressjs/express/blob/master/History.md)
- [Commits](https://github.com/expressjs/express/compare/4.18.2...4.19.2)

---
updated-dependencies:
- dependency-name: express
  dependency-type: indirect
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2024-03-29 09:50:53 -07:00
Vadim Ogievetsky 195221ca59
Web console: update druid-toolkit to get bug fixes (#16213)
* update druid-toolkit to get bug fixes

* update

* fix test
2024-03-29 08:31:35 -07:00
Vadim Ogievetsky e769ec7b96
Web console: fix Azure icon not rendering (#16173)
* do not lowercase the icon name

* missing icon

* update test
2024-03-20 13:37:39 -07:00
dependabot[bot] 7a0d6c53c8
Bump follow-redirects from 1.15.5 to 1.15.6 in /web-console (#16134)
Bumps [follow-redirects](https://github.com/follow-redirects/follow-redirects) from 1.15.5 to 1.15.6.
- [Release notes](https://github.com/follow-redirects/follow-redirects/releases)
- [Commits](https://github.com/follow-redirects/follow-redirects/compare/v1.15.5...v1.15.6)

---
updated-dependencies:
- dependency-name: follow-redirects
  dependency-type: indirect
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2024-03-15 15:30:41 -07:00
Vadim Ogievetsky ccae19a546
Web console: Make array ingest mode ux better (#15927)
* only set arrayIngestMode: array when needed (still do the queries the correct way)

* arrayIngestMode control

* update wording

* feedback fixes
2024-03-13 13:04:22 -07:00
Vadim Ogievetsky f7c0e425a9
fix URI wording (#16111) 2024-03-13 11:38:51 -07:00
Vadim Ogievetsky 8ef3eebd30
Web console: upgrade axios and follow-redirects (#16087)
* upgrade axios

* upgrade jest
2024-03-11 18:57:00 -07:00
Parth Agrawal 3aec90563e
Fix Jest and Prettify Checks (#15544)
* Fix jest and prettify checks

* Remove defaultQueryContext and run jest again

---------

Co-authored-by: Ghazanfar-CFLT <mghazanfar@confluent.io>
2024-03-08 14:36:26 -08:00
Vadim Ogievetsky 2816121ef0
wait for the coordinator (and proxy service to start) (#16088) 2024-03-08 13:31:29 -08:00
Vadim Ogievetsky 19a8af866b
make detail archive opening more robust (#16071) 2024-03-08 10:55:31 -08:00
Vadim Ogievetsky acb5124679
make double detection better (#15998) 2024-02-29 15:45:58 -08:00
Vadim Ogievetsky c5b032799c
Web console: add table and column search (#15990)
* Make a search

* fix snapshot

* added message when not found
2024-02-29 15:45:50 -08:00
Vadim Ogievetsky 6e222d47c8
fix crash when parsing data in data loader that can not be parsed (#15983) 2024-02-28 14:37:24 -08:00
Vadim Ogievetsky a81429746d
Web console: fix typos in Kinesis suggestions, add regions and groups (#15900)
* fix typo

* update regions

* add China

* Update web-console/src/druid-models/ingestion-spec/ingestion-spec.tsx

Co-authored-by: Benedict Jin <asdf2014@apache.org>

* add ,

---------

Co-authored-by: Benedict Jin <asdf2014@apache.org>
2024-02-27 10:00:02 +08:00
Vadim Ogievetsky bf3139562c
Web console: support for the export execution state (#15969)
* init

* add CSV keyword
2024-02-26 11:28:25 -08:00
Vadim Ogievetsky 28b3e117cf
Web console: Add input format props (#15950)
* fix typo

* add Protobuf

* better padding
2024-02-26 11:28:09 -08:00
Vadim Ogievetsky c52ddd0b86
make flattenSpec location adaptive (#15946) 2024-02-22 14:07:04 -08:00
Vadim Ogievetsky 66f54f2066
allow compaction config slots to drop to 0 (#15877) 2024-02-15 15:27:15 +08:00
Vadim Ogievetsky 26815d425b
Web console: add system fields UI (#15858)
This PR adds console support for configuring system fields in the batch data loader.
2024-02-08 11:08:55 +05:30
Vadim Ogievetsky f2b242b6e6
update console to core Druid changes (#15854) 2024-02-07 19:44:25 +05:30
George Shiqi Wu edb1ac1b71
Update azure console tile (#15820)
* Save web console changes

* Working new input type

* fix tests
2024-02-05 13:11:39 -08:00
Vadim Ogievetsky fcd65c9801
Web console: use arrayIngestMode: array (#15588)
* Adapt to new array mode

* Feedback fixes

* fixing type detection and highlighting

* goodies

* add docs

* feedback fixes

* finish array work

* update snapshots

* typo fix

* color fixes

* small fix

* make MVDs default for now

* better sqlStringifyArrays default

* fix spec converter

* fix tests
2024-01-31 20:19:29 -08:00
Vadim Ogievetsky 0089f6b905
Web console: Don't force waitUntilSegmentLoad to true (#15781)
* Don't force setting waitUntilSegmentsLoad

* delete irrelevant code
2024-01-31 16:16:36 +05:30
Abhishek Radhakrishnan 9f95a691f7
Extension to read and ingest Delta Lake tables (#15755)
* something

* test commit

* compilation fix

* more compilation fixes (fixme placeholders)

* Comment out druid-kereberos build since it conflicts with newly added transitive deps from delta-lake

Will need to sort out the dependencies later.

* checkpoint

* remove snapshot schema since we can get schema from the row

* iterator bug fix

* json json json

* sampler flow

* empty impls for read(InputStats) and sample()

* conversion?

* conversion, without timestamp

* Web console changes to show Delta Lake

* Asset bug fix and tile load

* Add missing pieces to input source info, etc.

* fix stuff

* Use a different delta lake asset

* Delta lake extension dependencies

* Cleanup

* Add InputSource, module init and helper code to process delta files.

* Test init

* Checkpoint changes

* Test resources and updates

* some fixes

* move to the correct package

* More tests

* Test cleanup

* TODOs

* Test updates

* requirements and javadocs

* Adjust dependencies

* Update readme

* Bump up version

* fixup typo in deps

* forbidden api and checkstyle checks

* Trim down dependencies

* new lines

* Fixup Intellij inspections.

* Add equals() and hashCode()

* chain splits, intellij inspections

* review comments and todo placeholder

* fix up some docs

* null table path and test dependencies. Fixup broken link.

* run prettify

* Different test; fixes

* Upgrade pyspark and delta-spark to latest (3.5.0 and 3.0.0) and regenerate tests

* yank the old test resource.

* add a couple of sad path tests

* Updates to readme based on latest.

* Version support

* Extract Delta DateTime converstions to DeltaTimeUtils class and add test

* More comprehensive split tests.

* Some test renames.

* Cleanup and update instructions.

* add pruneSchema() optimization for table scans.

* Oops, missed the parquet files.

* Update default table and rename schema constants.

* Test setup and misc changes.

* Add class loader logic as the context class loader is unaware about extension classes

* change some table client creation logic.

* Add hadoop-aws, hadoop-common and related exclusions.

* Remove org.apache.hadoop:hadoop-common

* Apply suggestions from code review

Co-authored-by: Victoria Lim <vtlim@users.noreply.github.com>

* Add entry to .spelling to fix docs static check

---------

Co-authored-by: abhishekagarwal87 <1477457+abhishekagarwal87@users.noreply.github.com>
Co-authored-by: Laksh Singla <lakshsingla@gmail.com>
Co-authored-by: Victoria Lim <vtlim@users.noreply.github.com>
2024-01-30 21:53:50 -08:00
Vadim Ogievetsky 497e2123f0
Web console: Make table driven query modification actions work with slices. (#15779)
* Make table driven query modification actions work with slices.

* cleanup found query prefix

* fix regex complexity
2024-01-29 20:09:46 -08:00
Vadim Ogievetsky 45ad47cc66
allow segment table to sort on start and end when grouped (#15720) 2024-01-26 18:59:23 +08:00