358 Commits

Author SHA1 Message Date
Abhishek Agarwal
652e1491e0
Update default values for tuning parameters in kinesis data loader (#11867) 2021-11-02 23:51:28 +05:30
andreacyc
88bbc8e9e1
Add info for compation config dialog (#11847)
* add-info-for-compation-config-dialog

* correct the info

* remove space typo

* Revert "remove space typo"

This reverts commit 28b28733ae21dda809b03995e804a65555ff9092.

* remove typo space

* update snapshots for jest-test
2021-11-02 10:03:29 -07:00
Vadim Ogievetsky
8ea9309168
Web console: update typescript 4.4 for faster build speeds (#11725)
* update typescript

* do not show pagination when there is only one page

* update snapshots

* fix pagination
2021-10-25 21:53:38 -07:00
Vadim Ogievetsky
f2106d7621
Web console: Add segment size in bytes column and hide it by default (#11797)
* add segment size column

* allow hidden default column

* fix tests

* update e2e tests
2021-10-25 13:24:44 -07:00
Vadim Ogievetsky
4354e43983
Use existing queryId if it exists (#11834) 2021-10-23 19:02:39 -07:00
andreacyc
adb2237628
Fix CVE-2021-3749 reported in security vulnerabilities job (#11786)
* Fix CVE-2021-3749 reported in security vulnerabilities job

* test why test fail

* update axios

* remove console log for testing
2021-10-08 23:02:58 -07:00
Vadim Ogievetsky
635490d568
don't throw local storage errors (#11752) 2021-10-05 18:49:16 -07:00
Vadim Ogievetsky
c1e0e6825f
auto refresh in foreground only (#11750) 2021-10-05 18:48:23 -07:00
andreacyc
f82baf174e
Support real query cancelling for web console (#11738)
* Support real query cancelling for web console

* use uuid for queryId, create isSql reuse variable, and add catch for rejectionhandled promise

* remove delete api promise.then() response

* slove conflicts

* update read me with debug

* add degub code to test why CI failed

* included a druid extension called druid-testing-tools and it is not build nor loaded by default

* remove unuse variable

* remove debug log
2021-10-05 10:28:49 -07:00
Vadim Ogievetsky
b54d989cda
Web console: Account for segment granularity all (#11697)
* accont for segment granularity all

* reverse granularity list to align with ranking
2021-09-13 20:46:11 -07:00
Vadim Ogievetsky
779fe8e6ad
fix segment timeline in FireFox (#11701) 2021-09-13 20:45:18 -07:00
Clint Wylie
fe1d8c206a
bump version to 0.23.0-SNAPSHOT (#11670) 2021-09-08 15:56:04 -07:00
Vadim Ogievetsky
e4ec3527a4
Web console: Improve the lookup view UX (#11620)
* polish lookup view UX

* update snapshots

* add snapshot to git

* fixes

* update sanpshots

* restore column treatment

* update snapshot

* add gs
2021-08-30 14:36:23 -07:00
Yi Yuan
ca5e55cc8c
Fix TRIM help broken in Console SQL Editor (#11628)
* delete escape

* bug fix

Co-authored-by: yuanyi <yuanyi@freewheel.tv>
2021-08-30 13:04:35 +08:00
Vadim Ogievetsky
257bc5c62f
Web console: fix count aggregation input in the data loader (#11485)
* add typeIs

* fix unused field in count metric

* better types

* typos

* work with readonly types

* factor out apply cancel buttons

* form editor

* selection type

* unsaved changes

* form editor spec

* tidy up sampler

* more menu controls

* update e2e test
2021-08-05 12:30:30 -07:00
Yuanli Han
b83742179a
Reduce method invocation of reservoir sampling (#11257)
* reduce method invocation of reservoir sampling

* add a dynamic parameter and add benchmark

* rebase
2021-07-30 22:09:50 +08:00
John Gozde
280c08045f
Update awesome-code-style (#11503) 2021-07-28 09:25:18 -07:00
Vadim Ogievetsky
aee2f2e24f
Web console: better handle BigInt math (#11450)
* better handle BigInt math

* correctly brace bigint

* feedback fixes and tests
2021-07-20 17:17:19 -07:00
Vadim Ogievetsky
377b5e708c
Web console: Data loading walkthrough fixes (#11416)
* fix quotes

* fix sql doc parsing

* prevent array-input from losing position while the user is typing

* make group filter click-to-filterable

* fix casing bug in exact table search

* do not sort columns in smaples

* can bypass transform step

* fixed string json parsing

* improve PartitionMessage

* better error messages

* feedback fixes

* tool to order dimensions in schema view
2021-07-10 07:56:50 -07:00
Vadim Ogievetsky
561cc71838
Web console: allow encoding of ASCII control chars (#10795)
* allow encoding of ascii control chars

* change to JSON

* make json escpaes work

* update snapshot

* break out component

* fix test

* update test script

* update formatter to be more chill
2021-06-26 18:54:41 -07:00
Vadim Ogievetsky
f56a5b9ba2
Web console: Better hotkeys and library upgrades (#11365)
* improve hotkeys

* fix test name

* refactor explain dialog

* explain tests

* small fixes

* update to popover2

* use resize sensor 2

* misc cleanup
2021-06-17 18:24:29 -07:00
Vadim Ogievetsky
2e98d3c1ad
Web console: Make segment timeline work over all time intervals (#11359)
* tidy up

* add to segments view

* add unit tests for date

* better util export

* fix ds view

* fix tests

* fix test in time

* unset untermediate state
2021-06-17 16:10:06 -07:00
John Gozde
ac9b1f81b4
Web console: Remove support for IE11 and other older browsers (#11357)
* Use common browserlist and update to drop IE11

* Change TypeScript target to ES2016

* Update browserslist for "supports es6" support

* Show a warning if accessed from an unsupported browser

* Inline browser-update styles; detect SyntaxErrors too

* Better wording

* Upgrade to the latest Blueprint

* Refactor RunButton to be FC, use useHotkeys

* Remove dead license

* Update snapshots

* Address feedback

* Wording

Co-authored-by: Vadim Ogievetsky <vadimon@gmail.com>

Co-authored-by: Vadim Ogievetsky <vadimon@gmail.com>
2021-06-10 19:05:40 -07:00
Vadim Ogievetsky
0c5d1c9725
Web console: add more query fixing auto suggestions (#11203)
* add more query fixing auto suggestions

* update query gen

* update toolkit

* update licenses

* fix funky quotes

* funky => fancy

* revert engine change

* separate web-console and website npm and node deps
2021-06-04 09:29:00 -07:00
Vadim Ogievetsky
31c811d894
Web console: Fix maxRowsPerSegment validation in hashed compaction spec (#11308)
* allow defining of maxRowsPerSegment for now

* use common util

* update snapshots

* fix test

* fix e2e test
2021-05-27 16:36:42 -07:00
Vadim Ogievetsky
d85e8b708b
Web console: fix SegmentTimeline query (#11204)
* fix SegmentTimeline query

* test query
2021-05-14 14:48:23 -07:00
Vadim Ogievetsky
d11be88c4b
Fix required field treatment (#11228) 2021-05-11 17:58:37 -07:00
Vadim Ogievetsky
15de29a2c4
Handle not having default rules better (#11214) 2021-05-10 15:13:15 -04:00
Lucas Capistrant
bb3c810b36
Create dynamic config that can limit number of non-primary replicants loaded per coordination cycle (#11135)
* lay the groundwork for throttling replicant loads per RunRules execution

* Add dynamic coordinator config to control new replicant threshold.

* remove redundant line

* add some unit tests

* fix checkstyle error

* add documentation for new dynamic config

* improve docs and logs

* Alter how null is handled for new config. If null, manually set as default
2021-05-05 07:39:36 -05:00
Clint Wylie
cd1ddcd36b
Web Console: allow pollPeriod to be defined in JDBC lookup config (#11193)
* allow pollPeriod to be defined in JDBC lookup config

* add test

* save the whales
2021-05-04 00:11:26 -07:00
Vadim Ogievetsky
1b07d554a8
Misc QueryView UX improvments (#11158) 2021-04-27 10:26:01 -07:00
John Gozde
9745d9e1c3
Web console: Switch to ESLint (#11142)
* Initial eslint config

* I guess eslint sorts underscores differently

* Trim curlies (in jsx)

* Re-organize rules

* Use consistent quote props

* Restructure eslint rules as additions/overrides to recommended configs

* Fix the 'recommended' stuff

* Add prefer-readonly

* Add prefer-object-spread

* Prettify

* Add eslint-plugin-react-hooks

* Switch to eslint-plugin-simple-sort-order

So much better

* Add no-extraneous-dependencies

* ban-tslint-comment for funzies

* If we enabled no-shadow, we'd probably want this option

* Not prefer-for-of

* no-confusing-void-expression, no-confusing-non-null-assertion

* Add some no-unnecessary-* rules

* non-nullable-type-assertion-style!

* prefer-includes

* Reorganize

* prefer-things

* switch-exhaustiveness-check

* We don't need the jsdoc plugin, prettier has our backs

* Remove a useless rule

* Drop TSLint and (temporarily) awesome-code-style

* Removing Object.assign revealed a type issue

* Bring back awesome-code-style for sasslint config

* Disable react/jsx-no-target-blank

* Add prettify-check script

* Add license to eslint config

* Format readme

* Update README for eslint, IDE settings

* Add 'autofix' script

* Switch to @awesome-code-style
2021-04-22 19:33:03 -07:00
Vadim Ogievetsky
4caa221d72
Web console: Better inline docs (#11128)
* better highlight

* better highlighting

* add spec
2021-04-19 14:36:53 -07:00
John Gozde
fdc3c2f362
Web console: update dev dependencies (#11119)
* Update some dev dependencies, prettify, tslint-fix

* Sort tsconfig keys for easy comparison

* Set noImplicitThis

* Slightly more accurate types

* Bump Jest and related

* Bump react to latest on v16

* Bump node-sass, sass-loader for node14 support

* Remove node-sass-chokidar (unused)

* More unused dependencies

* Fix blueprint imports

* Webpack 5

* Update webpack config for 'process' usage

* Update playwright-chromium

* Emit esnext modules for tree shaking

* Enable source maps in development

* Dedupe

* Bump babel and things

* npm audit fix

* Add .editorconfig file to match prettier settings

* Update licenses (tslib is 0BSD as of 1.11.2)

https://github.com/microsoft/tslib/pull/96

* Require node >= 10

* Use Node 10 to run e2e tests

* Use 'ws' transport mode for dev server (will be default in next version)

* Remove an 'any'

* No sourcemaps in prod

* Exclude .editorconfig from license checks

* Try nvm for setting node version
2021-04-16 20:15:19 -07:00
Yi Yuan
d0a94a8c14
add avro stream input format (#11040)
* add avro stream input format

* bug fixed

* add document

* doc fix

* change doc

* add integretion test

* bug fixed

* bug fixed

* add string as binary getter

Co-authored-by: yuanyi <yuanyi@freewheel.tv>
2021-04-12 21:53:41 -07:00
Vadim Ogievetsky
8432d82c48
Web console: Do not put __time in the dimensions list (#11085)
* Do not make time dimensions

* update e2e test
2021-04-12 09:48:10 -07:00
Atul Mohan
e2c4466fbd
Add missing config (#11020) 2021-03-26 04:45:53 -07:00
Gian Merlino
bf20f9e979
DruidInputSource: Fix issues in column projection, timestamp handling. (#10267)
* DruidInputSource: Fix issues in column projection, timestamp handling.

DruidInputSource, DruidSegmentReader changes:

1) Remove "dimensions" and "metrics". They are not necessary, because we
   can compute which columns we need to read based on what is going to
   be used by the timestamp, transform, dimensions, and metrics.
2) Start using ColumnsFilter (see below) to decide which columns we need
   to read.
3) Actually respect the "timestampSpec". Previously, it was ignored, and
   the timestamp of the returned InputRows was set to the `__time` column
   of the input datasource.

(1) and (2) together fix a bug in which the DruidInputSource would not
properly read columns that are used as inputs to a transformSpec.

(3) fixes a bug where the timestampSpec would be ignored if you attempted
to set the column to something other than `__time`.

(1) and (3) are breaking changes.

Web console changes:

1) Remove "Dimensions" and "Metrics" from the Druid input source.
2) Set timestampSpec to `{"column": "__time", "format": "millis"}` for
   compatibility with the new behavior.

Other changes:

1) Add ColumnsFilter, a new class that allows input readers to determine
   which columns they need to read. Currently, it's only used by the
   DruidInputSource, but it could be used by other columnar input sources
   in the future.
2) Add a ColumnsFilter to InputRowSchema.
3) Remove the metric names from InputRowSchema (they were unused).
4) Add InputRowSchemas.fromDataSchema method that computes the proper
   ColumnsFilter for given timestamp, dimensions, transform, and metrics.
5) Add "getRequiredColumns" method to TransformSpec to support the above.

* Various fixups.

* Uncomment incorrectly commented lines.

* Move TransformSpecTest to the proper module.

* Add druid.indexer.task.ignoreTimestampSpecForDruidInputSource setting.

* Fix.

* Fix build.

* Checkstyle.

* Misc fixes.

* Fix test.

* Move config.

* Fix imports.

* Fixup.

* Fix ShuffleResourceTest.

* Add import.

* Smarter exclusions.

* Fixes based on tests.

Also, add TIME_COLUMN constant in the web console.

* Adjustments for tests.

* Reorder test data.

* Update docs.

* Update docs to say Druid 0.22.0 instead of 0.21.0.

* Fix test.

* Fix ITAutoCompactionTest.

* Changes from review & from merging.
2021-03-25 10:32:21 -07:00
Vadim Ogievetsky
4897731e37
Make web console fast around sys.segments (#10909)
* do not load all the segments

* fix filtering

* update datasource view

* updated tests

* remove trimmedSegments

* Availability detail

* be smart about when showing smart modes

* fix tests

* add coordinator overlord mode
2021-03-10 19:59:50 -08:00
Vadim Ogievetsky
c0fb326788
Web console: fix service view actions when grouping (#10898)
* fix service view actions when grouping

* fix test
2021-03-09 21:38:56 -08:00
frank chen
80ec28578a
show leader in Services Tab (#10951)
Signed-off-by: frank chen <frank.chen021@outlook.com>
2021-03-09 08:03:56 -08:00
Vadim Ogievetsky
1a4c43f9fd
Web console: remove namespace prop that does not exist from JDBC lookup (#10888)
* remove namespace prop that does not exist from JDBC lookup

* remove namespace from tests
2021-02-17 17:07:32 -08:00
Vadim Ogievetsky
2a1e47afc3
Web console: Remove first / last suggestions (#10794)
* Remove first / last suggestions

* remove commened out code
2021-01-28 13:37:10 -08:00
Suneet Saldanha
a7542652ff
Fix dependabot warnings (#10796)
* Bump http-proxy from 1.18.0 to 1.18.1 in /web-console (#7)

Bumps [http-proxy](https://github.com/http-party/node-http-proxy) from 1.18.0 to 1.18.1.
- [Release notes](https://github.com/http-party/node-http-proxy/releases)
- [Changelog](https://github.com/http-party/node-http-proxy/blob/master/CHANGELOG.md)
- [Commits](https://github.com/http-party/node-http-proxy/compare/1.18.0...1.18.1)

Signed-off-by: dependabot[bot] <support@github.com>

Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>

* Bump elliptic from 6.5.1 to 6.5.3 in /web-console (#6)

Bumps [elliptic](https://github.com/indutny/elliptic) from 6.5.1 to 6.5.3.
- [Release notes](https://github.com/indutny/elliptic/releases)
- [Commits](https://github.com/indutny/elliptic/compare/v6.5.1...v6.5.3)

Signed-off-by: dependabot[bot] <support@github.com>

Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>

* Bump dot-prop from 4.2.0 to 4.2.1 in /web-console (#5)

Bumps [dot-prop](https://github.com/sindresorhus/dot-prop) from 4.2.0 to 4.2.1.
- [Release notes](https://github.com/sindresorhus/dot-prop/releases)
- [Commits](https://github.com/sindresorhus/dot-prop/compare/v4.2.0...v4.2.1)

Signed-off-by: dependabot[bot] <support@github.com>

Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>

* Bump bl from 1.2.2 to 1.2.3 in /website (#4)

Bumps [bl](https://github.com/rvagg/bl) from 1.2.2 to 1.2.3.
- [Release notes](https://github.com/rvagg/bl/releases)
- [Commits](https://github.com/rvagg/bl/compare/v1.2.2...v1.2.3)

Signed-off-by: dependabot[bot] <support@github.com>

Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>

* Bump prismjs from 1.20.0 to 1.23.0 in /website (#3)

Bumps [prismjs](https://github.com/PrismJS/prism) from 1.20.0 to 1.23.0.
- [Release notes](https://github.com/PrismJS/prism/releases)
- [Changelog](https://github.com/PrismJS/prism/blob/master/CHANGELOG.md)
- [Commits](https://github.com/PrismJS/prism/compare/v1.20.0...v1.23.0)

Signed-off-by: dependabot[bot] <support@github.com>

Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>

Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2021-01-26 23:11:26 -08:00
Vadim Ogievetsky
8c227bc566
use new example manifest (#10787) 2021-01-24 12:38:13 -08:00
Jihoon Son
95065bdf1a
Bump dev version to 0.22.0-SNAPSHOT (#10759) 2021-01-15 13:16:23 -08:00
Vadim Ogievetsky
e52db19823
treat null as not defined (#10751) 2021-01-14 18:22:59 -08:00
Vadim Ogievetsky
2fc2938b01
Web console: fix bad results if there is not native bigint (#10741)
* fix bigint when it does not exist

* add test
2021-01-12 16:32:23 -08:00
Yi Yuan
3624acbcf8
fix web-console show json bug (#10710)
* fix web-console show json bug

* replace all JSON.stringify

Co-authored-by: yuanyi <yuanyi@freewheel.tv>
2021-01-08 14:55:55 -08:00
Jonathan Wei
c7f2d3fbb5
Update deps for CVE-2020-28168 and CVE-2020-28052 (#10733)
* Update deps for CVE-2020-28168 and CVE-2020-28052

* Make BC runtime scope
2021-01-07 20:31:44 -08:00