Commit Graph

409 Commits

Author SHA1 Message Date
Vadim Ogievetsky 1f95a42bb8
Web console: updated the explain dialog to use new explain output (#12009)
This is the UI followup to the work done in #11908

Updated the Explain dialog to use the new output format.
2021-12-02 00:18:11 +05:30
Abhishek Agarwal 652e1491e0
Update default values for tuning parameters in kinesis data loader (#11867) 2021-11-02 23:51:28 +05:30
andreacyc 88bbc8e9e1
Add info for compation config dialog (#11847)
* add-info-for-compation-config-dialog

* correct the info

* remove space typo

* Revert "remove space typo"

This reverts commit 28b28733ae.

* remove typo space

* update snapshots for jest-test
2021-11-02 10:03:29 -07:00
Vadim Ogievetsky 8ea9309168
Web console: update typescript 4.4 for faster build speeds (#11725)
* update typescript

* do not show pagination when there is only one page

* update snapshots

* fix pagination
2021-10-25 21:53:38 -07:00
Vadim Ogievetsky f2106d7621
Web console: Add segment size in bytes column and hide it by default (#11797)
* add segment size column

* allow hidden default column

* fix tests

* update e2e tests
2021-10-25 13:24:44 -07:00
Vadim Ogievetsky 4354e43983
Use existing queryId if it exists (#11834) 2021-10-23 19:02:39 -07:00
andreacyc adb2237628
Fix CVE-2021-3749 reported in security vulnerabilities job (#11786)
* Fix CVE-2021-3749 reported in security vulnerabilities job

* test why test fail

* update axios

* remove console log for testing
2021-10-08 23:02:58 -07:00
Vadim Ogievetsky 635490d568
don't throw local storage errors (#11752) 2021-10-05 18:49:16 -07:00
Vadim Ogievetsky c1e0e6825f
auto refresh in foreground only (#11750) 2021-10-05 18:48:23 -07:00
andreacyc f82baf174e
Support real query cancelling for web console (#11738)
* Support real query cancelling for web console

* use uuid for queryId, create isSql reuse variable, and add catch for rejectionhandled promise

* remove delete api promise.then() response

* slove conflicts

* update read me with debug

* add degub code to test why CI failed

* included a druid extension called druid-testing-tools and it is not build nor loaded by default

* remove unuse variable

* remove debug log
2021-10-05 10:28:49 -07:00
Vadim Ogievetsky b54d989cda
Web console: Account for segment granularity all (#11697)
* accont for segment granularity all

* reverse granularity list to align with ranking
2021-09-13 20:46:11 -07:00
Vadim Ogievetsky 779fe8e6ad
fix segment timeline in FireFox (#11701) 2021-09-13 20:45:18 -07:00
Clint Wylie fe1d8c206a
bump version to 0.23.0-SNAPSHOT (#11670) 2021-09-08 15:56:04 -07:00
Vadim Ogievetsky e4ec3527a4
Web console: Improve the lookup view UX (#11620)
* polish lookup view UX

* update snapshots

* add snapshot to git

* fixes

* update sanpshots

* restore column treatment

* update snapshot

* add gs
2021-08-30 14:36:23 -07:00
Yi Yuan ca5e55cc8c
Fix TRIM help broken in Console SQL Editor (#11628)
* delete escape

* bug fix

Co-authored-by: yuanyi <yuanyi@freewheel.tv>
2021-08-30 13:04:35 +08:00
Vadim Ogievetsky 257bc5c62f
Web console: fix count aggregation input in the data loader (#11485)
* add typeIs

* fix unused field in count metric

* better types

* typos

* work with readonly types

* factor out apply cancel buttons

* form editor

* selection type

* unsaved changes

* form editor spec

* tidy up sampler

* more menu controls

* update e2e test
2021-08-05 12:30:30 -07:00
Yuanli Han b83742179a
Reduce method invocation of reservoir sampling (#11257)
* reduce method invocation of reservoir sampling

* add a dynamic parameter and add benchmark

* rebase
2021-07-30 22:09:50 +08:00
John Gozde 280c08045f
Update awesome-code-style (#11503) 2021-07-28 09:25:18 -07:00
Vadim Ogievetsky aee2f2e24f
Web console: better handle BigInt math (#11450)
* better handle BigInt math

* correctly brace bigint

* feedback fixes and tests
2021-07-20 17:17:19 -07:00
Vadim Ogievetsky 377b5e708c
Web console: Data loading walkthrough fixes (#11416)
* fix quotes

* fix sql doc parsing

* prevent array-input from losing position while the user is typing

* make group filter click-to-filterable

* fix casing bug in exact table search

* do not sort columns in smaples

* can bypass transform step

* fixed string json parsing

* improve PartitionMessage

* better error messages

* feedback fixes

* tool to order dimensions in schema view
2021-07-10 07:56:50 -07:00
Vadim Ogievetsky 561cc71838
Web console: allow encoding of ASCII control chars (#10795)
* allow encoding of ascii control chars

* change to JSON

* make json escpaes work

* update snapshot

* break out component

* fix test

* update test script

* update formatter to be more chill
2021-06-26 18:54:41 -07:00
Vadim Ogievetsky f56a5b9ba2
Web console: Better hotkeys and library upgrades (#11365)
* improve hotkeys

* fix test name

* refactor explain dialog

* explain tests

* small fixes

* update to popover2

* use resize sensor 2

* misc cleanup
2021-06-17 18:24:29 -07:00
Vadim Ogievetsky 2e98d3c1ad
Web console: Make segment timeline work over all time intervals (#11359)
* tidy up

* add to segments view

* add unit tests for date

* better util export

* fix ds view

* fix tests

* fix test in time

* unset untermediate state
2021-06-17 16:10:06 -07:00
John Gozde ac9b1f81b4
Web console: Remove support for IE11 and other older browsers (#11357)
* Use common browserlist and update to drop IE11

* Change TypeScript target to ES2016

* Update browserslist for "supports es6" support

* Show a warning if accessed from an unsupported browser

* Inline browser-update styles; detect SyntaxErrors too

* Better wording

* Upgrade to the latest Blueprint

* Refactor RunButton to be FC, use useHotkeys

* Remove dead license

* Update snapshots

* Address feedback

* Wording

Co-authored-by: Vadim Ogievetsky <vadimon@gmail.com>

Co-authored-by: Vadim Ogievetsky <vadimon@gmail.com>
2021-06-10 19:05:40 -07:00
Vadim Ogievetsky 0c5d1c9725
Web console: add more query fixing auto suggestions (#11203)
* add more query fixing auto suggestions

* update query gen

* update toolkit

* update licenses

* fix funky quotes

* funky => fancy

* revert engine change

* separate web-console and website npm and node deps
2021-06-04 09:29:00 -07:00
Vadim Ogievetsky 31c811d894
Web console: Fix maxRowsPerSegment validation in hashed compaction spec (#11308)
* allow defining of maxRowsPerSegment for now

* use common util

* update snapshots

* fix test

* fix e2e test
2021-05-27 16:36:42 -07:00
Vadim Ogievetsky d85e8b708b
Web console: fix SegmentTimeline query (#11204)
* fix SegmentTimeline query

* test query
2021-05-14 14:48:23 -07:00
Vadim Ogievetsky d11be88c4b
Fix required field treatment (#11228) 2021-05-11 17:58:37 -07:00
Vadim Ogievetsky 15de29a2c4
Handle not having default rules better (#11214) 2021-05-10 15:13:15 -04:00
Lucas Capistrant bb3c810b36
Create dynamic config that can limit number of non-primary replicants loaded per coordination cycle (#11135)
* lay the groundwork for throttling replicant loads per RunRules execution

* Add dynamic coordinator config to control new replicant threshold.

* remove redundant line

* add some unit tests

* fix checkstyle error

* add documentation for new dynamic config

* improve docs and logs

* Alter how null is handled for new config. If null, manually set as default
2021-05-05 07:39:36 -05:00
Clint Wylie cd1ddcd36b
Web Console: allow pollPeriod to be defined in JDBC lookup config (#11193)
* allow pollPeriod to be defined in JDBC lookup config

* add test

* save the whales
2021-05-04 00:11:26 -07:00
Vadim Ogievetsky 1b07d554a8
Misc QueryView UX improvments (#11158) 2021-04-27 10:26:01 -07:00
John Gozde 9745d9e1c3
Web console: Switch to ESLint (#11142)
* Initial eslint config

* I guess eslint sorts underscores differently

* Trim curlies (in jsx)

* Re-organize rules

* Use consistent quote props

* Restructure eslint rules as additions/overrides to recommended configs

* Fix the 'recommended' stuff

* Add prefer-readonly

* Add prefer-object-spread

* Prettify

* Add eslint-plugin-react-hooks

* Switch to eslint-plugin-simple-sort-order

So much better

* Add no-extraneous-dependencies

* ban-tslint-comment for funzies

* If we enabled no-shadow, we'd probably want this option

* Not prefer-for-of

* no-confusing-void-expression, no-confusing-non-null-assertion

* Add some no-unnecessary-* rules

* non-nullable-type-assertion-style!

* prefer-includes

* Reorganize

* prefer-things

* switch-exhaustiveness-check

* We don't need the jsdoc plugin, prettier has our backs

* Remove a useless rule

* Drop TSLint and (temporarily) awesome-code-style

* Removing Object.assign revealed a type issue

* Bring back awesome-code-style for sasslint config

* Disable react/jsx-no-target-blank

* Add prettify-check script

* Add license to eslint config

* Format readme

* Update README for eslint, IDE settings

* Add 'autofix' script

* Switch to @awesome-code-style
2021-04-22 19:33:03 -07:00
Vadim Ogievetsky 4caa221d72
Web console: Better inline docs (#11128)
* better highlight

* better highlighting

* add spec
2021-04-19 14:36:53 -07:00
John Gozde fdc3c2f362
Web console: update dev dependencies (#11119)
* Update some dev dependencies, prettify, tslint-fix

* Sort tsconfig keys for easy comparison

* Set noImplicitThis

* Slightly more accurate types

* Bump Jest and related

* Bump react to latest on v16

* Bump node-sass, sass-loader for node14 support

* Remove node-sass-chokidar (unused)

* More unused dependencies

* Fix blueprint imports

* Webpack 5

* Update webpack config for 'process' usage

* Update playwright-chromium

* Emit esnext modules for tree shaking

* Enable source maps in development

* Dedupe

* Bump babel and things

* npm audit fix

* Add .editorconfig file to match prettier settings

* Update licenses (tslib is 0BSD as of 1.11.2)

https://github.com/microsoft/tslib/pull/96

* Require node >= 10

* Use Node 10 to run e2e tests

* Use 'ws' transport mode for dev server (will be default in next version)

* Remove an 'any'

* No sourcemaps in prod

* Exclude .editorconfig from license checks

* Try nvm for setting node version
2021-04-16 20:15:19 -07:00
Yi Yuan d0a94a8c14
add avro stream input format (#11040)
* add avro stream input format

* bug fixed

* add document

* doc fix

* change doc

* add integretion test

* bug fixed

* bug fixed

* add string as binary getter

Co-authored-by: yuanyi <yuanyi@freewheel.tv>
2021-04-12 21:53:41 -07:00
Vadim Ogievetsky 8432d82c48
Web console: Do not put __time in the dimensions list (#11085)
* Do not make time dimensions

* update e2e test
2021-04-12 09:48:10 -07:00
Atul Mohan e2c4466fbd
Add missing config (#11020) 2021-03-26 04:45:53 -07:00
Gian Merlino bf20f9e979
DruidInputSource: Fix issues in column projection, timestamp handling. (#10267)
* DruidInputSource: Fix issues in column projection, timestamp handling.

DruidInputSource, DruidSegmentReader changes:

1) Remove "dimensions" and "metrics". They are not necessary, because we
   can compute which columns we need to read based on what is going to
   be used by the timestamp, transform, dimensions, and metrics.
2) Start using ColumnsFilter (see below) to decide which columns we need
   to read.
3) Actually respect the "timestampSpec". Previously, it was ignored, and
   the timestamp of the returned InputRows was set to the `__time` column
   of the input datasource.

(1) and (2) together fix a bug in which the DruidInputSource would not
properly read columns that are used as inputs to a transformSpec.

(3) fixes a bug where the timestampSpec would be ignored if you attempted
to set the column to something other than `__time`.

(1) and (3) are breaking changes.

Web console changes:

1) Remove "Dimensions" and "Metrics" from the Druid input source.
2) Set timestampSpec to `{"column": "__time", "format": "millis"}` for
   compatibility with the new behavior.

Other changes:

1) Add ColumnsFilter, a new class that allows input readers to determine
   which columns they need to read. Currently, it's only used by the
   DruidInputSource, but it could be used by other columnar input sources
   in the future.
2) Add a ColumnsFilter to InputRowSchema.
3) Remove the metric names from InputRowSchema (they were unused).
4) Add InputRowSchemas.fromDataSchema method that computes the proper
   ColumnsFilter for given timestamp, dimensions, transform, and metrics.
5) Add "getRequiredColumns" method to TransformSpec to support the above.

* Various fixups.

* Uncomment incorrectly commented lines.

* Move TransformSpecTest to the proper module.

* Add druid.indexer.task.ignoreTimestampSpecForDruidInputSource setting.

* Fix.

* Fix build.

* Checkstyle.

* Misc fixes.

* Fix test.

* Move config.

* Fix imports.

* Fixup.

* Fix ShuffleResourceTest.

* Add import.

* Smarter exclusions.

* Fixes based on tests.

Also, add TIME_COLUMN constant in the web console.

* Adjustments for tests.

* Reorder test data.

* Update docs.

* Update docs to say Druid 0.22.0 instead of 0.21.0.

* Fix test.

* Fix ITAutoCompactionTest.

* Changes from review & from merging.
2021-03-25 10:32:21 -07:00
Vadim Ogievetsky 4897731e37
Make web console fast around sys.segments (#10909)
* do not load all the segments

* fix filtering

* update datasource view

* updated tests

* remove trimmedSegments

* Availability detail

* be smart about when showing smart modes

* fix tests

* add coordinator overlord mode
2021-03-10 19:59:50 -08:00
Vadim Ogievetsky c0fb326788
Web console: fix service view actions when grouping (#10898)
* fix service view actions when grouping

* fix test
2021-03-09 21:38:56 -08:00
frank chen 80ec28578a
show leader in Services Tab (#10951)
Signed-off-by: frank chen <frank.chen021@outlook.com>
2021-03-09 08:03:56 -08:00
Vadim Ogievetsky 1a4c43f9fd
Web console: remove namespace prop that does not exist from JDBC lookup (#10888)
* remove namespace prop that does not exist from JDBC lookup

* remove namespace from tests
2021-02-17 17:07:32 -08:00
Vadim Ogievetsky 2a1e47afc3
Web console: Remove first / last suggestions (#10794)
* Remove first / last suggestions

* remove commened out code
2021-01-28 13:37:10 -08:00
Suneet Saldanha a7542652ff
Fix dependabot warnings (#10796)
* Bump http-proxy from 1.18.0 to 1.18.1 in /web-console (#7)

Bumps [http-proxy](https://github.com/http-party/node-http-proxy) from 1.18.0 to 1.18.1.
- [Release notes](https://github.com/http-party/node-http-proxy/releases)
- [Changelog](https://github.com/http-party/node-http-proxy/blob/master/CHANGELOG.md)
- [Commits](https://github.com/http-party/node-http-proxy/compare/1.18.0...1.18.1)

Signed-off-by: dependabot[bot] <support@github.com>

Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>

* Bump elliptic from 6.5.1 to 6.5.3 in /web-console (#6)

Bumps [elliptic](https://github.com/indutny/elliptic) from 6.5.1 to 6.5.3.
- [Release notes](https://github.com/indutny/elliptic/releases)
- [Commits](https://github.com/indutny/elliptic/compare/v6.5.1...v6.5.3)

Signed-off-by: dependabot[bot] <support@github.com>

Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>

* Bump dot-prop from 4.2.0 to 4.2.1 in /web-console (#5)

Bumps [dot-prop](https://github.com/sindresorhus/dot-prop) from 4.2.0 to 4.2.1.
- [Release notes](https://github.com/sindresorhus/dot-prop/releases)
- [Commits](https://github.com/sindresorhus/dot-prop/compare/v4.2.0...v4.2.1)

Signed-off-by: dependabot[bot] <support@github.com>

Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>

* Bump bl from 1.2.2 to 1.2.3 in /website (#4)

Bumps [bl](https://github.com/rvagg/bl) from 1.2.2 to 1.2.3.
- [Release notes](https://github.com/rvagg/bl/releases)
- [Commits](https://github.com/rvagg/bl/compare/v1.2.2...v1.2.3)

Signed-off-by: dependabot[bot] <support@github.com>

Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>

* Bump prismjs from 1.20.0 to 1.23.0 in /website (#3)

Bumps [prismjs](https://github.com/PrismJS/prism) from 1.20.0 to 1.23.0.
- [Release notes](https://github.com/PrismJS/prism/releases)
- [Changelog](https://github.com/PrismJS/prism/blob/master/CHANGELOG.md)
- [Commits](https://github.com/PrismJS/prism/compare/v1.20.0...v1.23.0)

Signed-off-by: dependabot[bot] <support@github.com>

Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>

Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2021-01-26 23:11:26 -08:00
Vadim Ogievetsky 8c227bc566
use new example manifest (#10787) 2021-01-24 12:38:13 -08:00
Jihoon Son 95065bdf1a
Bump dev version to 0.22.0-SNAPSHOT (#10759) 2021-01-15 13:16:23 -08:00
Vadim Ogievetsky e52db19823
treat null as not defined (#10751) 2021-01-14 18:22:59 -08:00
Vadim Ogievetsky 2fc2938b01
Web console: fix bad results if there is not native bigint (#10741)
* fix bigint when it does not exist

* add test
2021-01-12 16:32:23 -08:00
Yi Yuan 3624acbcf8
fix web-console show json bug (#10710)
* fix web-console show json bug

* replace all JSON.stringify

Co-authored-by: yuanyi <yuanyi@freewheel.tv>
2021-01-08 14:55:55 -08:00
Jonathan Wei c7f2d3fbb5
Update deps for CVE-2020-28168 and CVE-2020-28052 (#10733)
* Update deps for CVE-2020-28168 and CVE-2020-28052

* Make BC runtime scope
2021-01-07 20:31:44 -08:00
Lucas Capistrant 58ce2e55d8
Add dynamic coordinator config that allows control over how many segments are considered when picking a segment to move. (#10284)
* dynamic coord config adding more balancing control

add new dynamic coordinator config, maxSegmentsToConsiderPerMove. This
config caps the number of segments that are iterated over when selecting
a segment to move. The default value combined with current balancing
strategies will still iterate over all provided segments. However,
setting this value to something > 0 will cap the number of segments
visited. This could make sense in cases where a cluster has a very large
number of segments and the admins prefer less iterations vs a thorough
consideration of all segments provided.

* fix checkstyle failure

* Make doc more detailed for admin to understand when/why to use new config

* refactor PR to use a % of segments instead of raw number

* update the docs

* remove bad doc line

* fix typo in name of new dynamic config

* update RservoirSegmentSampler to gracefully deal with values > 100%

* add handler for <= 0 in ReservoirSegmentSampler

* fixup CoordinatorDynamicConfigTest naming and argument ordering

* fix items in docs after spellcheck flags

* Fix lgtm flag on missing space in string literal

* improve documentation for new config

* Add default value to config docs and add advice in cluster tuning doc

* Add percentOfSegmentsToConsiderPerMove to web console coord config dialog

* update jest snapshot after console change

* fix spell checker errors

* Improve debug logging in getRandomSegmentBalancerHolder to cover all bad inputs for % of segments to consider

* add new config back to web console module after merge with master

* fix ReservoirSegmentSamplerTest

* fix line breaks in coordinator console dialog

* Add a test that helps ensure not regressions for percentOfSegmentsToConsiderPerMove

* Make improvements based off of feedback in review

* additional cleanup coming from review

* Add a warning log if limit on segments to consider for move can't be calcluated

* remove unused import

* fix tests for CoordinatorDynamicConfig

* remove precondition test that is redundant in CoordinatorDynamicConfig Builder class
2020-12-22 08:27:55 -08:00
Vadim Ogievetsky 55b8cc428a
remove extra word (#10682) 2020-12-15 23:02:33 -08:00
Vadim Ogievetsky 577cd66002
Web console: reflect the changes to interval requirement in the data loader flow (#10647)
* no need for intervals

* don't set redundant fields

* fix tests

* better filter control

* work with not

* wrap callout with form group

* update snapshot

* add split hint

* highlight issues with spec

* fixes

* fix default value

* move intervals back to partition step

* work with all sorts of chars

* fix enabled view
2020-12-09 10:18:42 -08:00
Vadim Ogievetsky e3f7217546
Web console: Improve the handling of extreme data (funky datasources, longs) (#10641)
* better API escape

* fix escaping issue, bigints

* update licenses

* fix align

* do not show Query with SQL if no SQL

* add prettify script

* update dev readme

* add ordering to the datasource list

* add ordering to supervisor table
2020-12-08 09:25:14 -08:00
Vadim Ogievetsky 5b06c7a3a9
Web console: improve how code is imported, use API instance (#10597)
* fix imports

* clean up imports

* update DQT to fix escaping
2020-12-01 13:16:14 -08:00
zhangyue19921010 abb03d38b0
double confirm before delete all the ununsed segmensts (#10581)
Co-authored-by: yuezhang <yuezhang@freewheel.tv>
2020-11-19 10:07:52 -06:00
Vadim Ogievetsky 9964dd4cb2
Web console: fix data loader schema table column ordering bug and other polish (#10588)
* remove unused fields

* keep tables live

* advanced

* fix schema view

* better indication

* tests pass

* Show more instead of show advanced

* fix tests

* extract dynamic configs

* update snapshots

* fix issues

* update snapshot

* reword without >
2020-11-17 13:25:03 -08:00
Lucas Capistrant 83667172ce
Add missing coordinator dynamic config to the web-console dialog for dynamic coordinator config (#10545)
* Add missing coordinator dynamic config to the web-console dialog for dynamic coordinator config

* update jest snapshots after making console change
2020-11-09 13:00:52 -06:00
Vadim Ogievetsky ee61a165e3
Web console: improve make type preservation in ingestion configs in the data loader (#10533)
* improve validation

* move to druid modals

* adjust specs

* oneOf

* move transform

* segment grans

* tidy up webpack

* add Partitioning

* break out druid models

* tidy up

* rename to Expression

* move druid time

* cleanup

* update format

* better detail

* fix e2e test

* remove forceGuaranteedRollup from e2e tests

* clean up imports

* shardSpec -> shard_spec

* fix css

* adjust snapshot

* add granularity to segments view

* rename to Time span

* use OFFSET in segments view query

* update doc links

* simplify require

* more oneOf
2020-10-30 19:02:44 -07:00
Vadim Ogievetsky f391e89653
Web console: refresh and tighten up the console styles 💅💫 (#10515)
* no hardcoded dark mode

* update blueprint

* style pass

* add open sans font

* fix public path

* update licenses

* add comment

* allow zero bsd

* add new lines

* update tests

* remove default mods

* use variables

* fix background

* fix query init style

* pad fracional part

* paginate braces

* update to latest blueprint, fix column tree

* Smart query limit -> Auto limit

* adjust gutter color for dark mode

* fix spelling

* update snapshot

Co-authored-by: asherbitter <ofir@imply.io>
2020-10-20 22:11:29 -07:00
Vadim Ogievetsky e8c5893c34
Web console: show segment sizes in rows not bytes (#10496)
* added query error suggestions

* simplify the SQLs

* change segment size display to rows

* suggestion tests

* update snapshot

* make error detection more robust

* remove errant console log

* fix imports

* put suggestion on top

* better error rendering

* format as millions

* add .druid.pid to gitignore

* rename segment_size to segment_rows, fix visability, fix divide by zero

* update snapshots
2020-10-13 13:19:39 -07:00
Suneet Saldanha b45a56f989
Web console: targetRowsPerSegment for hashed partionin (#10500)
* Web console: targetRowsPerSegment for hashed partionin

Added `targetRowsPerSegment` to the web console for hashed partition for both
the auto compaction view and as part of the ingestion workflow.

The help text was also updated to indicate when a user should care about
updating these fields

* code review

* update test snapshots

* oops
2020-10-11 16:55:28 -07:00
Ashish Kapoor 4c78b514c9
Update README.md (#10357)
Compile scss files before npm start.
2020-10-09 20:21:59 +05:30
Joseph Glanville 7ce9ac4548
Fix Avro support in Web Console (#10232)
* Fix Avro OCF detection prefix and run formation detection on raw input

* Support Avro Fixed and Enum types correctly

* Check Avro version byte in format detection

* Add test for AvroOCFReader.sample

Ensures that the Sampler doesn't receive raw input that it can't
serialize into JSON.

* Document Avro type handling

* Add TS unit tests for guessInputFormat
2020-10-07 21:08:22 -07:00
Vadim Ogievetsky 2e50ada407
Web console: fix compaction status when no compaction config, and small cleanup (#10483)
* move timed button to icons

* cleanup redundant logic

* fix compaction status text

* remove extra style
2020-10-07 14:54:08 -07:00
Gian Merlino d78fedd13c
Web console: Don't include realtime segments in size calculations. (#10482)
It's always zero, and so it messes up averages, mins, and counts.
2020-10-06 18:56:54 -07:00
Chi Cao Minh 1c77491da6
Test UI to trigger auto compaction (#10469)
In the web console E2E tests, Use the new UI to trigger auto compaction
instead of calling the REST API directly so that the UI is covered by
tests.
2020-10-04 00:06:07 -07:00
Jonathan Wei 65c0d64676
Update version to 0.21.0-SNAPSHOT (#10450)
* [maven-release-plugin] prepare release druid-0.21.0

* [maven-release-plugin] prepare for next development iteration

* Update web-console versions
2020-10-03 16:08:34 -07:00
Vadim Ogievetsky f77c16bc6c
Web console: fix lookup edit dialog version setting (#10461)
* fix lookup edit dialog

* update snapshots

* clean up test
2020-10-03 08:35:20 -07:00
Chi Cao Minh d11537b5f7
Improve UI E2E test usability (#10466)
- Update playwright to latest version
- Provide environment variable to disable/enable headless mode
- Allow running E2E tests against any druid cluster running on standard
  ports (tutorial-batch.spec.ts now uses an absolute instead of relative
  path for the input data)
- Provide environment variable to change target web console port
- Druid setup does not need to download zookeeper
2020-10-03 08:21:44 -07:00
Chi Cao Minh ede25f1b45
Fix UI datasources view edit action compaction (#10459)
Restore the web console's ability to view a datasource's compaction
configuration via the "action" menu. Refactoring done in
https://github.com/apache/druid/pull/10438 introduced a regression that
always caused the default compaction configuration to be shown via the
"action" menu instead.

Regression test is added in e2e-tests/auto-compaction.spec.ts.
2020-10-01 23:59:21 -07:00
Chi Cao Minh 7385af0272
Web console reindexing E2E test (#10453)
Add an E2E test for the web console workflow of reindexing a Druid
datasource to change the secondary partitioning type.  The new test
changes dynamic to single dim partitions since the autocompaction test
already does dynamic to hashed partitions.

Also, run the web console E2E tests in parallel to reduce CI time and
change naming convention for test datasources to make it easier to map
them to the corresponding test run.

Main changes:

1) web-consolee2e-tests/reindexing.spec.ts
   - new E2E test

2) web-console/e2e-tests/component/load-data/data-connector/reindex.ts
   - new data loader connector for druid input source

3) web-console/e2e-tests/component/load-data/config/partition.ts
   - move partition spec definitions from compaction.ts
   - add new single dim partition spec definition
2020-10-01 15:14:41 -07:00
Vadim Ogievetsky d09fd8b035
Web console: switch to switches instead of checkboxes (#10454)
* switch to switches

* add img alt

* add relative

* change icons

* update snapshot
2020-09-30 15:13:22 -07:00
Vadim Ogievetsky 729bcba7ac
Web console: Display compaction status (#10438)
* init compaction status

* % compacted

* final UI tweaks

* extracted utils, added tests

* add tests to general foramt functions
2020-09-28 22:19:28 -07:00
Chi Cao Minh cbe2b44e29
Compaction config UI optional numShards (#10446)
* Compaction config UI optional numShards

Specifying `numShards` for hashed partitions is no longer required after
https://github.com/apache/druid/pull/10419. Update the UI to make
`numShards` an optional field for hash partitions.

* Update snapshot
2020-09-28 17:15:48 -07:00
Chi Cao Minh d16c78ba98
Add intent for web console IntervalInput (#10447)
When using the web console to load data by reindexing from Druid, the
`Datasource` and `Interval` inputs are required during the `Connect`
step. Unlike the `Datasource` input, the `Interval` input did not have a
blue outline to indicate that it was required as the `IntervalInput`
component did not support an `intent` property.
2020-09-28 17:13:07 -07:00
Chi Cao Minh cbd9ac8592
Web console autocompaction E2E test (#10425)
Add an E2E test for the common case web console workflow of setting up
autocompaction that changes the partitions from dynamic to hashed.

Also fix an issue with the async test setup to properly wait for the web
console to be ready.
2020-09-25 18:28:25 -07:00
Vadim Ogievetsky 89160c2f9b
better query view initial state (#10431) 2020-09-24 09:49:58 -07:00
Vadim Ogievetsky a60d034d01
Web console: compaction dialog update (#10417)
* compaction dialog update

* fix test snapshot

* Update web-console/src/dialogs/compaction-dialog/compaction-dialog.tsx

Co-authored-by: Chi Cao Minh <chi.caominh@imply.io>

* Update web-console/src/dialogs/compaction-dialog/compaction-dialog.tsx

Co-authored-by: Chi Cao Minh <chi.caominh@imply.io>

* feedback changes

Co-authored-by: Chi Cao Minh <chi.caominh@imply.io>
2020-09-23 09:34:19 -07:00
Vadim Ogievetsky ac0a45471e
Web console: add sort to tiers list (#10416)
* add sort to tiers list

* update snapshot
2020-09-22 19:00:55 -07:00
Vadim Ogievetsky 7cc0a7be68
Web console: clean up styling imports (#10410)
* fix styling for importing

* fix quotes
2020-09-21 17:30:25 -07:00
Vadim Ogievetsky 6c5c86d800
Web console: fix lookup edit dialog, allow column renaming (#10406)
* column rename

* update licenses file

* remove empty file

* update license file

* move comment
2020-09-20 14:10:05 -07:00
Vadim Ogievetsky e465f05717
Web console: Improve number alignment in tables (#10389)
* Improve tables

* removed unused state interfaces

* better copy

* one more functional component

* updated e2e tests

* extract braced text correctly
2020-09-14 19:53:38 -07:00
Vadim Ogievetsky 3c8eacb2d4
Web console: improve query manager (convert to React hook) (#10360)
* Better query running

* update licenses

* update tests

* updated tests v2

* fade in cancel

* add exemplary tests

* update mkcomp

* fix inconsistent state update

* remove lastParsedQuery

* work if not a valid literal

* remove unused params

* fix licenses

* better state update

* get error message

* isEmpty tidy

* add tests around error message highlighting

* pull live query selector into a component

* add LiveQueryModeSelector tests

* update snapshots
2020-09-11 19:42:50 -07:00
Vadim Ogievetsky e81a9df507
Web console: add tile for Azure Event Hubs (via Kafka API) (#10317)
* Add Azure Event Hubs

* better note

* update icon
2020-08-31 20:58:52 -07:00
Vadim Ogievetsky 748a83cb78
Web console: fix json input (#10271)
* fix json input

* tidy up

* add error extraction test
2020-08-13 12:20:58 -07:00
Suneet Saldanha 6baea0b4d5
Fix broken sampler for re-indexing (#10196)
* Fix broken sampler for re-indexer

When re-indexing a Druid datasource, the web-console would generate an
invalid inputFormat since the type is not specified.

* code review
2020-08-11 19:22:26 -07:00
Vadim Ogievetsky 1fae0e15c9
improve JSON paste (#10256) 2020-08-07 17:01:06 -07:00
Vadim Ogievetsky 9a29496b6c
Web console: Improve retention rules dialog in all sorts of ways (#10226)
* improve ret rules

* tidy up tests
2020-08-04 16:43:31 -07:00
keefe roedersheimer 4bb198eb4c
change search filter to includes (#10141) 2020-08-04 16:37:07 -07:00
Vadim Ogievetsky 6d8799f2df
Update QueryView to use latest DruidQueryToolkit (#10201)
* Update to latest DruidQueryToolkit

* add THEN keyword

* do not crash on invalid JSON
2020-07-23 22:45:01 -07:00
Suneet Saldanha eeb9012743
Change color of Run button for native queries (#10170)
* Change color of Run button for native queries

When a user tries to run a native query, change the color of the button to
Druid's secondary color to indicate that the user is not running a SQL query.

Before this change, the web-console would indicate this by changing the text
of the button from Run (SQL queries) to Rune (native queries). Rune could be
confusing to users as this appears to be a typo.

* Update web-console/src/views/query-view/run-button/run-button.scss

* Update web-console/src/views/query-view/run-button/run-button.scss

* Update web-console/src/views/query-view/run-button/run-button.scss

* code review
2020-07-15 15:46:02 -07:00
Clint Wylie 0a8fd62f22
split web-console e2e-tests from unit tests (#10173)
* split web-console e2e-test from unit test

* fix stuff

* smaller change

* oops
2020-07-12 18:51:08 -07:00
Egor Riashin d54a5e009f
ui: fix missing columns during Transform step (#10086)
Co-authored-by: egor-ryashin <egor.ryashin@metamarkets.com>
2020-07-07 15:16:52 -07:00
Clint Wylie c86e7ce30b
bump version to 0.20.0-SNAPSHOT (#10124) 2020-07-06 15:08:32 -07:00
Clint Wylie 477335abb4
update links datasketches.github.io to datasketches.apache.org (#10107)
* update links datasketches.github.io to datasketches.apache.org

* now with more apache

* oops

* oops
2020-07-01 14:56:17 -07:00
Vadim Ogievetsky c01fd56182
Web console: allow link overrides for docs, and more (#10100)
* link overrides

* change doc version

* fix snapshots
2020-06-30 12:46:50 -07:00
Yuanli Han fc555980e8
Remove payload field from table sys.segment (#9883)
* remove payload field from table sys.segments

* update doc

* fix test

* fix CI failure

* add necessary fields

* fix doc

* fix comment
2020-06-29 22:20:23 -07:00
Suneet Saldanha 15a0b4ffe2
Filter http requests by http method (#10085)
* Filter http requests by http method

Add a config that allows a user which http methods to allow against their
Druid server.

Druid will only accept http requests with the method: GET, PUT, POST, DELETE
and OPTIONS.
If a Druid admin wants to allow other methods, they can do so by using the
ServerConfig#allowedHttpMethods config.

If a Druid user would like to disallow OPTIONS, this can be done by changing
the AuthConfig#allowUnauthenticatedHttpOptions config

* Exclude OPTIONS from always supported HTTP methods

Add HEAD as an allowed method for web console e2e tests

* fix docs

* fix security IT

* Actually fix the web console e2e tests

* Ignore icode coverage for nitialization classes

* code review
2020-06-29 16:59:31 -07:00