Commit Graph

459 Commits

Author SHA1 Message Date
Vadim Ogievetsky ca4df85941
fix SQL in segment card (#13895) 2023-03-07 13:08:04 -08:00
Vadim Ogievetsky 38b6373bf7
Web console: Compaction history dialog (#13861)
* initial renames

* add comaction history diff

* final fixes

* update snapshots

* use maps

* update test
2023-03-06 11:52:25 -08:00
Elliott Freis d93fdb2632
Bump CycloneDX module to address POM errors (#13878)
* Bump CycloneDX module to address POM errors

* Including web-console in the PR

---------

Co-authored-by: Elliott Freis <elliottfreis@Elliott-Freis.earth.dynamic.blacklight.net>
2023-03-03 15:39:15 +05:30
Abhishek Radhakrishnan 775f89c75b
Include workaround for CycloneDX is causing POM build errors to web-console as well (#13874) 2023-03-02 00:06:49 -08:00
Vadim Ogievetsky 13721f5998
upgrade druid query toolkit (#13848) 2023-02-28 14:34:21 +05:30
Vadim Ogievetsky e4e6c7ed01
make completions smarter (#13830) 2023-02-23 10:17:10 -08:00
Vadim Ogievetsky 1ca0edb8c9
Web console: Fixes query cancel NPE and more (#13786)
* add null icon

* empty string table cell

* enable views only if they will work

* make sure method exists

* use SQL compatible nulls for e2e tests
2023-02-15 15:02:50 -08:00
John Gozde b33962cab7
Upgrade typescript and other dependencies (#13762)
* Bump zustand, licenses

* Bump TypeScript, Eslint, use type imports

* Switch to react-shallow-renderer from enzyme

* Update ts-loader
2023-02-06 23:12:54 -08:00
Vadim Ogievetsky 3b62d7929c
Web console: Data loader should allow for multiline JSON messages in kafka (#13709)
* stricter

* data loader should allow for mulit-line json

* add await

* kinesis also
2023-01-25 21:23:18 -08:00
Vadim Ogievetsky 4ee4d99b8d
better error reporting (#13636) 2023-01-05 20:00:33 -08:00
Vadim Ogievetsky fdc8aa2833
better show totals when grouping (#13631) 2023-01-05 18:06:32 -08:00
Vadim Ogievetsky 8773d619a2
Web console: tidy up stage UI (#13615)
* show the right info

* sort indicator

* nicer marker

* move error icon
2022-12-22 12:52:16 -08:00
Vadim Ogievetsky e23abc710a
Web console: default max workers to cluster capacity and simplify live reports (#13577)
* step

* better capacity

* start with capacity

* more compressed stats display

* better rule editor

* fix SQL data loader also

* update snapshots

* new line

* better formatting
2022-12-16 15:13:32 -08:00
Vadim Ogievetsky 639decdf2e
fix preview droping out of MSQ mode (#13586) 2022-12-16 15:13:07 -08:00
Rishabh Singh 4ebdfe226d
Druid automated quickstart (#13365)
* Druid automated quickstart

* remove conf/druid/single-server/quickstart/_common/historical/jvm.config

* Minor changes in python script

* Add lower bound memory for some services

* Additional runtime properties for services

* Update supervise script to accept command arguments, corresponding changes in druid-quickstart.py

* File end newline

* Limit the ability to start multiple instances of a service, documentation changes

* simplify script arguments

* restore changes in medium profile

* run-druid refactor

* compute and pass middle manager runtime properties to run-druid
supervise script changes to process java opts array
use argparse, leave free memory, logging

* Remove extra quotes from mm task javaopts array

* Update logic to compute minimum memory

* simplify run-druid

* remove debug options from run-druid

* resolve the config_path provided

* comment out service specific runtime properties which are computed in the code

* simplify run-druid

* clean up docs, naming changes

* Throw ValueError exception on illegal state

* update docs

* rename args, compute_only -> compute, run_zk -> zk

* update help documentation

* update help documentation

* move task memory computation into separate method

* Add validation checks

* remove print

* Add validations

* remove start-druid bash script, rename start-druid-main

* Include tasks in lower bound memory calculation

* Fix test

* 256m instead of 256g

* caffeine cache uses 5% of heap

* ensure min task count is 2, task count is monotonic

* update configs and documentation for runtime props in conf/druid/single-server/quickstart

* Update docs

* Specify memory argument for each profile in single-server.md

* Update middleManager runtime.properties

* Move quickstart configs to conf/druid/base, add bash launch script, support python2

* Update supervise script

* rename base config directory to auto

* rename python script, changes to pass repeated args to supervise

* remove exmaples/conf/druid/base dir

* add docs

* restore changes in conf dir

* update start-druid-auto

* remove hashref for commands in supervise script

* start-druid-main java_opts array is comma separated

* update entry point script name in python script

* Update help docs

* documentation changes

* docs changes

* update docs

* add support for running indexer

* update supported services list

* update help

* Update python.md

* remove dir

* update .spelling

* Remove dependency on psutil and pathlib

* update docs

* Update get_physical_memory method

* Update help docs

* update docs

* update method to get physical memory on python

* udpate spelling

* update .spelling

* minor change

* Minor change

* memory comptuation for indexer

* update start-druid

* Update python.md

* Update single-server.md

* Update python.md

* run python3 --version to check if python is installed

* Update supervise script

* start-druid: echo message if python not found

* update anchor text

* minor change

* Update condition in supervise script

* JVM not jvm in docs
2022-12-09 11:04:02 -08:00
Vadim Ogievetsky d8e27eaab4
update error anchors (#13527) 2022-12-08 13:18:35 -08:00
Vadim Ogievetsky d85fb8cc4e
Web console: improve compaction status display (#13523)
* improve compaction status display

* even more accurate

* fix snapshot
2022-12-07 21:03:59 -08:00
Vadim Ogievetsky 9679f6a9b5
Web console: add arrayOfDoublesSketch and other small fixes (#13486)
* add padding and keywords

* add arrayOfDoubles

* Update docs/development/extensions-core/datasketches-tuple.md

Co-authored-by: Charles Smith <techdocsmith@gmail.com>

* Update docs/development/extensions-core/datasketches-tuple.md

Co-authored-by: Charles Smith <techdocsmith@gmail.com>

* Update docs/development/extensions-core/datasketches-tuple.md

Co-authored-by: Charles Smith <techdocsmith@gmail.com>

* Update docs/development/extensions-core/datasketches-tuple.md

Co-authored-by: Charles Smith <techdocsmith@gmail.com>

* Update docs/development/extensions-core/datasketches-tuple.md

Co-authored-by: Charles Smith <techdocsmith@gmail.com>

* partiton int

* fix docs

Co-authored-by: Charles Smith <techdocsmith@gmail.com>
2022-12-06 21:21:49 -08:00
Vadim Ogievetsky 2fdcfffe40
don't render duration if aggregated (#13455) 2022-11-30 19:21:07 -08:00
Vadim Ogievetsky d8f4353c43
Web console: be more robust to aux queries failing and improve kill tasks (#13431)
* be more robust to aux queries failing

* feedback fixes

* remove empty block

* fix spelling

* remove killAllDataSources from the console
2022-11-28 16:50:38 -08:00
Vadim Ogievetsky a2d5e335f3
Web console: Index spec dialog (#13425)
* add index spec dialog

* add sanpshot
2022-11-28 11:40:45 -08:00
dependabot[bot] 16385c7101
Bump minimatch and replace in /web-console (#13396)
Bumps [minimatch](https://github.com/isaacs/minimatch) to 3.0.5 and updates ancestor dependency [replace](https://github.com/ALMaclaine/replace). These dependencies need to be updated together.


Updates `minimatch` from 3.0.4 to 3.0.5
- [Release notes](https://github.com/isaacs/minimatch/releases)
- [Commits](https://github.com/isaacs/minimatch/compare/v3.0.4...v3.0.5)

Updates `replace` from 1.2.1 to 1.2.2
- [Release notes](https://github.com/ALMaclaine/replace/releases)
- [Commits](https://github.com/ALMaclaine/replace/commits)

---
updated-dependencies:
- dependency-name: minimatch
  dependency-type: indirect
- dependency-name: replace
  dependency-type: direct:development
...

Signed-off-by: dependabot[bot] <support@github.com>

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2022-11-23 12:16:00 -08:00
Kashif Faraz 7cf761cee4
Prepare master branch for next release, 26.0.0 (#13401)
* Prepare master branch for next release, 26.0.0

* Use docker image for druid 24.0.1

* Fix version in druid-it-cases pom.xml
2022-11-22 15:31:01 +05:30
Vadim Ogievetsky fe34ecc5e3
add ability to make inputFormat part of the example datasets (#13402) 2022-11-21 12:50:44 -08:00
Kashif Faraz 133054bf27
Make batched segment sampling the default, minor cleanup of coordinator config (#13391)
The batch segment sampling performs significantly better than the older method
of sampling if there are a large number of used segments. It also avoids duplicates.

Changes:
- Make batch segment sampling the default
- Deprecate the property `useBatchedSegmentSampler`
- Remove unused coordinator config `druid.coordinator.loadqueuepeon.repeatDelay`
- Cleanup `KillUnusedSegments`
- Simplify `KillUnusedSegmentsTest`, add better tests, remove redundant tests
2022-11-21 20:31:46 +05:30
Vadim Ogievetsky 08fa0383b9
add supported indexSpec options (#13388) 2022-11-19 00:00:32 -08:00
Vadim Ogievetsky c628947c31
Web console: streaming json input format specifics (#13381)
* streaming json input format specifics

* goodies
2022-11-18 14:15:16 -08:00
Vadim Ogievetsky a3d45f6086
treat user cancelation seriously (#13376) 2022-11-18 14:04:16 -08:00
Kashif Faraz 71b133f3ff
Add `RoundRobinServerSelector` to speed up segment assignments (#13367)
Segment assignments can take very long due to the strategy cost computation
for a large number of segments. This commit allows segment assignments to be
done in a round-robin fashion within a tier. Only segment balancing takes cost-based
decisions to move segments around.

Changes
- Add dynamic config `useRoundRobinSegmentAssignment` with default value false
- Add `RoundRobinServerSelector`. This does not implement the `BalancerStrategy`
as it does not conform to that contract and may also be used in conjunction with a
strategy (round-robin for `RunRules` and a cost strategy for `BalanceSegments`)
- Drops are still cost-based even when round-robin assignment is enabled.
2022-11-16 20:05:17 +05:30
dependabot[bot] 61a7199f2b
Bump loader-utils from 1.4.0 to 1.4.2 in /web-console (#13372)
Bumps [loader-utils](https://github.com/webpack/loader-utils) from 1.4.0 to 1.4.2.
- [Release notes](https://github.com/webpack/loader-utils/releases)
- [Changelog](https://github.com/webpack/loader-utils/blob/v1.4.2/CHANGELOG.md)
- [Commits](https://github.com/webpack/loader-utils/compare/v1.4.0...v1.4.2)

---
updated-dependencies:
- dependency-name: loader-utils
  dependency-type: indirect
...

Signed-off-by: dependabot[bot] <support@github.com>

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2022-11-16 20:00:33 +08:00
Vadim Ogievetsky f6aca21e82
Web console: update DQT to version 0.17 (#13323)
* update to DQT 17

* update licenses

* after npm i
2022-11-07 17:47:11 -08:00
Vadim Ogievetsky 176934e849
Web console: expose in the UI that a query is stuck waiting for task slots (#13291)
* add pending info

* update tests

* fixes after review
2022-11-01 09:40:55 -07:00
Margaret Brewster e40c7f2a67
Update data loader parse screen help text (#13241)
Co-authored-by: Margaret Brewster <margaretbrewster@Maggie-Brewster.local>
2022-10-29 11:51:37 +08:00
Vadim Ogievetsky 32020247d1
Web console: Update dqt to a version that adds quotes by default (#13243)
* update dqt

* auto quote by default
2022-10-28 01:06:45 -07:00
Vadim Ogievetsky 573e12c75f
Web console: making the cell filter menu more functional, removing the old query view, and updating d3 (#13169)
* remove old query view

* update tests

* add filter

* fix test

* bump d3 things to latest versions

* rent too far into the future with d3

* make config dialogs load

* goodies

* update snapshots

* only compute duration when running or pending
2022-10-07 12:44:40 -07:00
Vadim Ogievetsky ebfe1c0c90
Web console: fix DQT import (#13159)
* fix dqt import

* update licenses

* update tests
2022-09-30 09:31:06 -07:00
Vadim Ogievetsky a910764e41
better spec conversion with issues (#13136) 2022-09-22 10:46:57 -07:00
Vadim Ogievetsky 6c1dc6589e
initialize all counters for stages with input (#13137) 2022-09-22 08:10:50 -07:00
Vadim Ogievetsky f1d3728371
append to exisitng callout (#13130) 2022-09-21 19:39:28 -07:00
Vadim Ogievetsky b9edfe34a4
be consistent about referring to the web console by its name (#13118) 2022-09-19 15:02:17 -07:00
Vadim Ogievetsky de8f229bed
Web console: correctly escape path based flatten specs (#13105)
* fix path generation

* do escape

* fix replace

* fix replace for good
2022-09-17 14:02:42 -07:00
Vadim Ogievetsky c62a822121
support kafka lookups (#13098) 2022-09-16 15:25:25 -07:00
Vadim Ogievetsky 078b50ebe1
link to error docs (#13094) 2022-09-15 15:06:08 -07:00
Vadim Ogievetsky 08d6aca528
Web console: better detection for arrays containing objects (#13077)
* better detection for arrays containing objects

* include boolean also
2022-09-12 18:50:29 -07:00
Vadim Ogievetsky d978afc5b7
fix number of expected functions (#13050) 2022-09-09 13:42:01 -07:00
Vadim Ogievetsky 5cc5f7b60c
quote columns, datasources in auto complete if needed (#13060) 2022-09-09 11:22:40 -07:00
Kashif Faraz 7e20d70242
Fix web-console message in MSQ data loader (#12996)
* Fix typo in web-console message

* Prettify the changes
2022-09-07 13:34:10 -07:00
Vadim Ogievetsky 92789cfc4a
default to no compare (#13041) 2022-09-07 08:28:28 -07:00
Vadim Ogievetsky 2cf449386f
Web console: upgrade the console to use node 16 (#13017)
* upgrade the console to use node 16

* run npm audit fix
2022-09-06 11:15:23 -07:00
Vadim Ogievetsky 0ae515bd3c
Web console: don't crash if cookies are totally disabled (#13013)
* fix local storage detection

* fix numeric input dialog
2022-09-01 16:10:23 -07:00