Commit Graph

366 Commits

Author SHA1 Message Date
Vadim Ogievetsky 2a41b7bffa
Web console: correctly cancel JSON shaped SQL queries (#12134)
* misc fixes

* type typo
2022-01-10 14:24:05 -08:00
Vadim Ogievetsky 476d0bf4be
Web console: remove console.log (#12094)
* rm console.log

* force path-parse to 1.0.7
2021-12-22 19:31:23 -08:00
Vadim Ogievetsky 37112d24e2
Web console: new Ace, diff view, and cleanup. Decorating the console for the holidays 🎁 (#12085)
* Add diff view and upgrade AceEditor

* fix test

* function doc parsing fixes

* escape args

* allowKeys

* everyone gets a diff

* update snapshot
2021-12-22 16:31:17 -08:00
Vadim Ogievetsky 0cc998d8a1
improve spec upgrading (#12072) 2021-12-15 10:28:21 -08:00
Vadim Ogievetsky 6ac4e2dbb8
Web console: use query actions in query view (#12037)
* use query actions

* feedback fixes
2021-12-08 13:01:29 -08:00
Lucas Capistrant 150902b95c
clean up the balancing code around the batched vs deprecated way of sampling segments to balance (#11960)
* clean up the balancing code around the batched vs deprecated way of sampling segments to balance

* fix docs, clarify comments, add deprecated annotations to legacy code

* remove unused variable

* update dynamic config dialog in console to state percentOfSegmentsToConsiderPerMove deprecated

* fix dynamic config text for percentOfSegmentsToConsiderPerMove

* run prettier to cleanup coordinator-dynamic-config.tsx changes

* update jest snapshot

* update documentation per review feedback
2021-12-07 14:47:46 -08:00
Vadim Ogievetsky 1d3c8c187b
Web console: query view improvements and other fixes (#12031)
* don't copy commas

* use numeric type information

* add VALUES keyword

* propogate rollup config into spec

* fix

* cleanup

* understand range partitioning

* update snapshots

* better comp apis

* fix segment pages

* update snapshots
2021-12-07 10:16:16 -08:00
Vadim Ogievetsky 1f95a42bb8
Web console: updated the explain dialog to use new explain output (#12009)
This is the UI followup to the work done in #11908

Updated the Explain dialog to use the new output format.
2021-12-02 00:18:11 +05:30
Abhishek Agarwal 652e1491e0
Update default values for tuning parameters in kinesis data loader (#11867) 2021-11-02 23:51:28 +05:30
andreacyc 88bbc8e9e1
Add info for compation config dialog (#11847)
* add-info-for-compation-config-dialog

* correct the info

* remove space typo

* Revert "remove space typo"

This reverts commit 28b28733ae.

* remove typo space

* update snapshots for jest-test
2021-11-02 10:03:29 -07:00
Vadim Ogievetsky 8ea9309168
Web console: update typescript 4.4 for faster build speeds (#11725)
* update typescript

* do not show pagination when there is only one page

* update snapshots

* fix pagination
2021-10-25 21:53:38 -07:00
Vadim Ogievetsky f2106d7621
Web console: Add segment size in bytes column and hide it by default (#11797)
* add segment size column

* allow hidden default column

* fix tests

* update e2e tests
2021-10-25 13:24:44 -07:00
Vadim Ogievetsky 4354e43983
Use existing queryId if it exists (#11834) 2021-10-23 19:02:39 -07:00
andreacyc adb2237628
Fix CVE-2021-3749 reported in security vulnerabilities job (#11786)
* Fix CVE-2021-3749 reported in security vulnerabilities job

* test why test fail

* update axios

* remove console log for testing
2021-10-08 23:02:58 -07:00
Vadim Ogievetsky 635490d568
don't throw local storage errors (#11752) 2021-10-05 18:49:16 -07:00
Vadim Ogievetsky c1e0e6825f
auto refresh in foreground only (#11750) 2021-10-05 18:48:23 -07:00
andreacyc f82baf174e
Support real query cancelling for web console (#11738)
* Support real query cancelling for web console

* use uuid for queryId, create isSql reuse variable, and add catch for rejectionhandled promise

* remove delete api promise.then() response

* slove conflicts

* update read me with debug

* add degub code to test why CI failed

* included a druid extension called druid-testing-tools and it is not build nor loaded by default

* remove unuse variable

* remove debug log
2021-10-05 10:28:49 -07:00
Vadim Ogievetsky b54d989cda
Web console: Account for segment granularity all (#11697)
* accont for segment granularity all

* reverse granularity list to align with ranking
2021-09-13 20:46:11 -07:00
Vadim Ogievetsky 779fe8e6ad
fix segment timeline in FireFox (#11701) 2021-09-13 20:45:18 -07:00
Clint Wylie fe1d8c206a
bump version to 0.23.0-SNAPSHOT (#11670) 2021-09-08 15:56:04 -07:00
Vadim Ogievetsky e4ec3527a4
Web console: Improve the lookup view UX (#11620)
* polish lookup view UX

* update snapshots

* add snapshot to git

* fixes

* update sanpshots

* restore column treatment

* update snapshot

* add gs
2021-08-30 14:36:23 -07:00
Yi Yuan ca5e55cc8c
Fix TRIM help broken in Console SQL Editor (#11628)
* delete escape

* bug fix

Co-authored-by: yuanyi <yuanyi@freewheel.tv>
2021-08-30 13:04:35 +08:00
Vadim Ogievetsky 257bc5c62f
Web console: fix count aggregation input in the data loader (#11485)
* add typeIs

* fix unused field in count metric

* better types

* typos

* work with readonly types

* factor out apply cancel buttons

* form editor

* selection type

* unsaved changes

* form editor spec

* tidy up sampler

* more menu controls

* update e2e test
2021-08-05 12:30:30 -07:00
Yuanli Han b83742179a
Reduce method invocation of reservoir sampling (#11257)
* reduce method invocation of reservoir sampling

* add a dynamic parameter and add benchmark

* rebase
2021-07-30 22:09:50 +08:00
John Gozde 280c08045f
Update awesome-code-style (#11503) 2021-07-28 09:25:18 -07:00
Vadim Ogievetsky aee2f2e24f
Web console: better handle BigInt math (#11450)
* better handle BigInt math

* correctly brace bigint

* feedback fixes and tests
2021-07-20 17:17:19 -07:00
Vadim Ogievetsky 377b5e708c
Web console: Data loading walkthrough fixes (#11416)
* fix quotes

* fix sql doc parsing

* prevent array-input from losing position while the user is typing

* make group filter click-to-filterable

* fix casing bug in exact table search

* do not sort columns in smaples

* can bypass transform step

* fixed string json parsing

* improve PartitionMessage

* better error messages

* feedback fixes

* tool to order dimensions in schema view
2021-07-10 07:56:50 -07:00
Vadim Ogievetsky 561cc71838
Web console: allow encoding of ASCII control chars (#10795)
* allow encoding of ascii control chars

* change to JSON

* make json escpaes work

* update snapshot

* break out component

* fix test

* update test script

* update formatter to be more chill
2021-06-26 18:54:41 -07:00
Vadim Ogievetsky f56a5b9ba2
Web console: Better hotkeys and library upgrades (#11365)
* improve hotkeys

* fix test name

* refactor explain dialog

* explain tests

* small fixes

* update to popover2

* use resize sensor 2

* misc cleanup
2021-06-17 18:24:29 -07:00
Vadim Ogievetsky 2e98d3c1ad
Web console: Make segment timeline work over all time intervals (#11359)
* tidy up

* add to segments view

* add unit tests for date

* better util export

* fix ds view

* fix tests

* fix test in time

* unset untermediate state
2021-06-17 16:10:06 -07:00
John Gozde ac9b1f81b4
Web console: Remove support for IE11 and other older browsers (#11357)
* Use common browserlist and update to drop IE11

* Change TypeScript target to ES2016

* Update browserslist for "supports es6" support

* Show a warning if accessed from an unsupported browser

* Inline browser-update styles; detect SyntaxErrors too

* Better wording

* Upgrade to the latest Blueprint

* Refactor RunButton to be FC, use useHotkeys

* Remove dead license

* Update snapshots

* Address feedback

* Wording

Co-authored-by: Vadim Ogievetsky <vadimon@gmail.com>

Co-authored-by: Vadim Ogievetsky <vadimon@gmail.com>
2021-06-10 19:05:40 -07:00
Vadim Ogievetsky 0c5d1c9725
Web console: add more query fixing auto suggestions (#11203)
* add more query fixing auto suggestions

* update query gen

* update toolkit

* update licenses

* fix funky quotes

* funky => fancy

* revert engine change

* separate web-console and website npm and node deps
2021-06-04 09:29:00 -07:00
Vadim Ogievetsky 31c811d894
Web console: Fix maxRowsPerSegment validation in hashed compaction spec (#11308)
* allow defining of maxRowsPerSegment for now

* use common util

* update snapshots

* fix test

* fix e2e test
2021-05-27 16:36:42 -07:00
Vadim Ogievetsky d85e8b708b
Web console: fix SegmentTimeline query (#11204)
* fix SegmentTimeline query

* test query
2021-05-14 14:48:23 -07:00
Vadim Ogievetsky d11be88c4b
Fix required field treatment (#11228) 2021-05-11 17:58:37 -07:00
Vadim Ogievetsky 15de29a2c4
Handle not having default rules better (#11214) 2021-05-10 15:13:15 -04:00
Lucas Capistrant bb3c810b36
Create dynamic config that can limit number of non-primary replicants loaded per coordination cycle (#11135)
* lay the groundwork for throttling replicant loads per RunRules execution

* Add dynamic coordinator config to control new replicant threshold.

* remove redundant line

* add some unit tests

* fix checkstyle error

* add documentation for new dynamic config

* improve docs and logs

* Alter how null is handled for new config. If null, manually set as default
2021-05-05 07:39:36 -05:00
Clint Wylie cd1ddcd36b
Web Console: allow pollPeriod to be defined in JDBC lookup config (#11193)
* allow pollPeriod to be defined in JDBC lookup config

* add test

* save the whales
2021-05-04 00:11:26 -07:00
Vadim Ogievetsky 1b07d554a8
Misc QueryView UX improvments (#11158) 2021-04-27 10:26:01 -07:00
John Gozde 9745d9e1c3
Web console: Switch to ESLint (#11142)
* Initial eslint config

* I guess eslint sorts underscores differently

* Trim curlies (in jsx)

* Re-organize rules

* Use consistent quote props

* Restructure eslint rules as additions/overrides to recommended configs

* Fix the 'recommended' stuff

* Add prefer-readonly

* Add prefer-object-spread

* Prettify

* Add eslint-plugin-react-hooks

* Switch to eslint-plugin-simple-sort-order

So much better

* Add no-extraneous-dependencies

* ban-tslint-comment for funzies

* If we enabled no-shadow, we'd probably want this option

* Not prefer-for-of

* no-confusing-void-expression, no-confusing-non-null-assertion

* Add some no-unnecessary-* rules

* non-nullable-type-assertion-style!

* prefer-includes

* Reorganize

* prefer-things

* switch-exhaustiveness-check

* We don't need the jsdoc plugin, prettier has our backs

* Remove a useless rule

* Drop TSLint and (temporarily) awesome-code-style

* Removing Object.assign revealed a type issue

* Bring back awesome-code-style for sasslint config

* Disable react/jsx-no-target-blank

* Add prettify-check script

* Add license to eslint config

* Format readme

* Update README for eslint, IDE settings

* Add 'autofix' script

* Switch to @awesome-code-style
2021-04-22 19:33:03 -07:00
Vadim Ogievetsky 4caa221d72
Web console: Better inline docs (#11128)
* better highlight

* better highlighting

* add spec
2021-04-19 14:36:53 -07:00
John Gozde fdc3c2f362
Web console: update dev dependencies (#11119)
* Update some dev dependencies, prettify, tslint-fix

* Sort tsconfig keys for easy comparison

* Set noImplicitThis

* Slightly more accurate types

* Bump Jest and related

* Bump react to latest on v16

* Bump node-sass, sass-loader for node14 support

* Remove node-sass-chokidar (unused)

* More unused dependencies

* Fix blueprint imports

* Webpack 5

* Update webpack config for 'process' usage

* Update playwright-chromium

* Emit esnext modules for tree shaking

* Enable source maps in development

* Dedupe

* Bump babel and things

* npm audit fix

* Add .editorconfig file to match prettier settings

* Update licenses (tslib is 0BSD as of 1.11.2)

https://github.com/microsoft/tslib/pull/96

* Require node >= 10

* Use Node 10 to run e2e tests

* Use 'ws' transport mode for dev server (will be default in next version)

* Remove an 'any'

* No sourcemaps in prod

* Exclude .editorconfig from license checks

* Try nvm for setting node version
2021-04-16 20:15:19 -07:00
Yi Yuan d0a94a8c14
add avro stream input format (#11040)
* add avro stream input format

* bug fixed

* add document

* doc fix

* change doc

* add integretion test

* bug fixed

* bug fixed

* add string as binary getter

Co-authored-by: yuanyi <yuanyi@freewheel.tv>
2021-04-12 21:53:41 -07:00
Vadim Ogievetsky 8432d82c48
Web console: Do not put __time in the dimensions list (#11085)
* Do not make time dimensions

* update e2e test
2021-04-12 09:48:10 -07:00
Atul Mohan e2c4466fbd
Add missing config (#11020) 2021-03-26 04:45:53 -07:00
Gian Merlino bf20f9e979
DruidInputSource: Fix issues in column projection, timestamp handling. (#10267)
* DruidInputSource: Fix issues in column projection, timestamp handling.

DruidInputSource, DruidSegmentReader changes:

1) Remove "dimensions" and "metrics". They are not necessary, because we
   can compute which columns we need to read based on what is going to
   be used by the timestamp, transform, dimensions, and metrics.
2) Start using ColumnsFilter (see below) to decide which columns we need
   to read.
3) Actually respect the "timestampSpec". Previously, it was ignored, and
   the timestamp of the returned InputRows was set to the `__time` column
   of the input datasource.

(1) and (2) together fix a bug in which the DruidInputSource would not
properly read columns that are used as inputs to a transformSpec.

(3) fixes a bug where the timestampSpec would be ignored if you attempted
to set the column to something other than `__time`.

(1) and (3) are breaking changes.

Web console changes:

1) Remove "Dimensions" and "Metrics" from the Druid input source.
2) Set timestampSpec to `{"column": "__time", "format": "millis"}` for
   compatibility with the new behavior.

Other changes:

1) Add ColumnsFilter, a new class that allows input readers to determine
   which columns they need to read. Currently, it's only used by the
   DruidInputSource, but it could be used by other columnar input sources
   in the future.
2) Add a ColumnsFilter to InputRowSchema.
3) Remove the metric names from InputRowSchema (they were unused).
4) Add InputRowSchemas.fromDataSchema method that computes the proper
   ColumnsFilter for given timestamp, dimensions, transform, and metrics.
5) Add "getRequiredColumns" method to TransformSpec to support the above.

* Various fixups.

* Uncomment incorrectly commented lines.

* Move TransformSpecTest to the proper module.

* Add druid.indexer.task.ignoreTimestampSpecForDruidInputSource setting.

* Fix.

* Fix build.

* Checkstyle.

* Misc fixes.

* Fix test.

* Move config.

* Fix imports.

* Fixup.

* Fix ShuffleResourceTest.

* Add import.

* Smarter exclusions.

* Fixes based on tests.

Also, add TIME_COLUMN constant in the web console.

* Adjustments for tests.

* Reorder test data.

* Update docs.

* Update docs to say Druid 0.22.0 instead of 0.21.0.

* Fix test.

* Fix ITAutoCompactionTest.

* Changes from review & from merging.
2021-03-25 10:32:21 -07:00
Vadim Ogievetsky 4897731e37
Make web console fast around sys.segments (#10909)
* do not load all the segments

* fix filtering

* update datasource view

* updated tests

* remove trimmedSegments

* Availability detail

* be smart about when showing smart modes

* fix tests

* add coordinator overlord mode
2021-03-10 19:59:50 -08:00
Vadim Ogievetsky c0fb326788
Web console: fix service view actions when grouping (#10898)
* fix service view actions when grouping

* fix test
2021-03-09 21:38:56 -08:00
frank chen 80ec28578a
show leader in Services Tab (#10951)
Signed-off-by: frank chen <frank.chen021@outlook.com>
2021-03-09 08:03:56 -08:00
Vadim Ogievetsky 1a4c43f9fd
Web console: remove namespace prop that does not exist from JDBC lookup (#10888)
* remove namespace prop that does not exist from JDBC lookup

* remove namespace from tests
2021-02-17 17:07:32 -08:00