Commit Graph

55 Commits

Author SHA1 Message Date
Vadim Ogievetsky 04ee7abeff
Web console: Multi-stage query support (#12919)
* MSQ web console

* fix typo in comments

* remove useless conditional

* wrap SQL_DATA_TYPES

* fixes sus regex

* rewrite regex

* remove problematic regex

* fix UTs

* convert PARTITIONED / CLUSTERED BY to ORDER BY for preview

* fix log

* updated to use shuffle

* Web console: Use Ace.Completion directly (#1405)

* Use Ace.Completion directly

* Another Ace.Completion

* better comment

* fix column ordering in e2e test

* add nested data example also

Co-authored-by: John Gozde <john.gozde@imply.io>
2022-08-24 16:17:12 -07:00
Clint Wylie f8097ccfaa
basic docs for nested column query functions (#12922)
* basic docs for nested column query functions
2022-08-19 17:12:19 -07:00
Gian Merlino ef6811ef88
Improved Java 17 support and Java runtime docs. (#12839)
* Improved Java 17 support and Java runtime docs.

1) Add a "Java runtime" doc page with information about supported
   Java versions, garbage collection, and strong encapsulation..

2) Update asm and equalsverifier to versions that support Java 17.

3) Add additional "--add-opens" lines to surefire configuration, so
   tests can pass successfully under Java 17.

4) Switch openjdk15 tests to openjdk17.

5) Update FrameFile to specifically mention Java runtime incompatibility
   as the cause of not being able to use Memory.map.

6) Update SegmentLoadDropHandler to log an error for Errors too, not
   just Exceptions. This is important because an IllegalAccessError is
   encountered when the correct "--add-opens" line is not provided,
   which would otherwise be silently ignored.

7) Update example configs to use druid.indexer.runner.javaOptsArray
   instead of druid.indexer.runner.javaOpts. (The latter is deprecated.)

* Adjustments.

* Use run-java in more places.

* Add run-java.

* Update .gitignore.

* Exclude hadoop-client-api.

Brought in when building on Java 17.

* Swap one more usage of java.

* Fix the run-java script.

* Fix flag.

* Include link to Temurin.

* Spelling.

* Update examples/bin/run-java

Co-authored-by: Xavier Léauté <xl+github@xvrl.net>

Co-authored-by: Xavier Léauté <xl+github@xvrl.net>
2022-08-03 23:16:05 -07:00
Vadim Ogievetsky fb08bac01a
Web console: Misc table fixes (#12489)
* Misc table fixes

* extract default className

* table spacing updates

* fix e2e action selector

* try more times

* make the web console exist again
2022-05-03 12:08:08 -07:00
Vadim Ogievetsky a139cd22aa
Web console: Misc fixes and improvements (#12361)
* Misc fixes

* pad column numbers

* make shard_type filterable
2022-04-12 22:20:28 -07:00
John Gozde 90680543d0
Blueprint 4 (#12391)
* Update blueprint dependencies & LICENSES

* Switch to bp4 namespace; use bp-ns variable in overrides

* Add webpack alias for colors.scss

* Snapshots

* Update selectors in e2e tests
2022-04-04 10:34:22 -07:00
Victoria Lim c61b19d443
Refactor SQL docs (#12239)
* refactor and link fixes

* add sql docs to left nav

* code format for needle

* updated web console script

* link fixes

* update earliest/latest functions

* edits for grammar and style

* more link fixes

* another link

* update with #12226

* update .spelling file
2022-02-11 14:43:30 -08:00
Vadim Ogievetsky 37112d24e2
Web console: new Ace, diff view, and cleanup. Decorating the console for the holidays 🎁 (#12085)
* Add diff view and upgrade AceEditor

* fix test

* function doc parsing fixes

* escape args

* allowKeys

* everyone gets a diff

* update snapshot
2021-12-22 16:31:17 -08:00
Vadim Ogievetsky 6ac4e2dbb8
Web console: use query actions in query view (#12037)
* use query actions

* feedback fixes
2021-12-08 13:01:29 -08:00
andreacyc f82baf174e
Support real query cancelling for web console (#11738)
* Support real query cancelling for web console

* use uuid for queryId, create isSql reuse variable, and add catch for rejectionhandled promise

* remove delete api promise.then() response

* slove conflicts

* update read me with debug

* add degub code to test why CI failed

* included a druid extension called druid-testing-tools and it is not build nor loaded by default

* remove unuse variable

* remove debug log
2021-10-05 10:28:49 -07:00
Vadim Ogievetsky e4ec3527a4
Web console: Improve the lookup view UX (#11620)
* polish lookup view UX

* update snapshots

* add snapshot to git

* fixes

* update sanpshots

* restore column treatment

* update snapshot

* add gs
2021-08-30 14:36:23 -07:00
Yi Yuan ca5e55cc8c
Fix TRIM help broken in Console SQL Editor (#11628)
* delete escape

* bug fix

Co-authored-by: yuanyi <yuanyi@freewheel.tv>
2021-08-30 13:04:35 +08:00
Vadim Ogievetsky 377b5e708c
Web console: Data loading walkthrough fixes (#11416)
* fix quotes

* fix sql doc parsing

* prevent array-input from losing position while the user is typing

* make group filter click-to-filterable

* fix casing bug in exact table search

* do not sort columns in smaples

* can bypass transform step

* fixed string json parsing

* improve PartitionMessage

* better error messages

* feedback fixes

* tool to order dimensions in schema view
2021-07-10 07:56:50 -07:00
Vadim Ogievetsky 4caa221d72
Web console: Better inline docs (#11128)
* better highlight

* better highlighting

* add spec
2021-04-19 14:36:53 -07:00
John Gozde fdc3c2f362
Web console: update dev dependencies (#11119)
* Update some dev dependencies, prettify, tslint-fix

* Sort tsconfig keys for easy comparison

* Set noImplicitThis

* Slightly more accurate types

* Bump Jest and related

* Bump react to latest on v16

* Bump node-sass, sass-loader for node14 support

* Remove node-sass-chokidar (unused)

* More unused dependencies

* Fix blueprint imports

* Webpack 5

* Update webpack config for 'process' usage

* Update playwright-chromium

* Emit esnext modules for tree shaking

* Enable source maps in development

* Dedupe

* Bump babel and things

* npm audit fix

* Add .editorconfig file to match prettier settings

* Update licenses (tslib is 0BSD as of 1.11.2)

https://github.com/microsoft/tslib/pull/96

* Require node >= 10

* Use Node 10 to run e2e tests

* Use 'ws' transport mode for dev server (will be default in next version)

* Remove an 'any'

* No sourcemaps in prod

* Exclude .editorconfig from license checks

* Try nvm for setting node version
2021-04-16 20:15:19 -07:00
Vadim Ogievetsky 5b06c7a3a9
Web console: improve how code is imported, use API instance (#10597)
* fix imports

* clean up imports

* update DQT to fix escaping
2020-12-01 13:16:14 -08:00
Vadim Ogievetsky f391e89653
Web console: refresh and tighten up the console styles 💅💫 (#10515)
* no hardcoded dark mode

* update blueprint

* style pass

* add open sans font

* fix public path

* update licenses

* add comment

* allow zero bsd

* add new lines

* update tests

* remove default mods

* use variables

* fix background

* fix query init style

* pad fracional part

* paginate braces

* update to latest blueprint, fix column tree

* Smart query limit -> Auto limit

* adjust gutter color for dark mode

* fix spelling

* update snapshot

Co-authored-by: asherbitter <ofir@imply.io>
2020-10-20 22:11:29 -07:00
Chi Cao Minh d11537b5f7
Improve UI E2E test usability (#10466)
- Update playwright to latest version
- Provide environment variable to disable/enable headless mode
- Allow running E2E tests against any druid cluster running on standard
  ports (tutorial-batch.spec.ts now uses an absolute instead of relative
  path for the input data)
- Provide environment variable to change target web console port
- Druid setup does not need to download zookeeper
2020-10-03 08:21:44 -07:00
Vadim Ogievetsky 3c8eacb2d4
Web console: improve query manager (convert to React hook) (#10360)
* Better query running

* update licenses

* update tests

* updated tests v2

* fade in cancel

* add exemplary tests

* update mkcomp

* fix inconsistent state update

* remove lastParsedQuery

* work if not a valid literal

* remove unused params

* fix licenses

* better state update

* get error message

* isEmpty tidy

* add tests around error message highlighting

* pull live query selector into a component

* add LiveQueryModeSelector tests

* update snapshots
2020-09-11 19:42:50 -07:00
Vadim Ogievetsky 6d8799f2df
Update QueryView to use latest DruidQueryToolkit (#10201)
* Update to latest DruidQueryToolkit

* add THEN keyword

* do not crash on invalid JSON
2020-07-23 22:45:01 -07:00
Clint Wylie 0a8fd62f22
split web-console e2e-tests from unit tests (#10173)
* split web-console e2e-test from unit test

* fix stuff

* smaller change

* oops
2020-07-12 18:51:08 -07:00
Suneet Saldanha 15a0b4ffe2
Filter http requests by http method (#10085)
* Filter http requests by http method

Add a config that allows a user which http methods to allow against their
Druid server.

Druid will only accept http requests with the method: GET, PUT, POST, DELETE
and OPTIONS.
If a Druid admin wants to allow other methods, they can do so by using the
ServerConfig#allowedHttpMethods config.

If a Druid user would like to disallow OPTIONS, this can be done by changing
the AuthConfig#allowUnauthenticatedHttpOptions config

* Exclude OPTIONS from always supported HTTP methods

Add HEAD as an allowed method for web console e2e tests

* fix docs

* fix security IT

* Actually fix the web console e2e tests

* Ignore icode coverage for nitialization classes

* code review
2020-06-29 16:59:31 -07:00
Xavier Léauté 65280a6953
update kafka client version to 2.5.0 (#9902)
- remove dependency on deprecated internal Kafka classes
- keep LZ4 version in line with the version shipped with Kafka
2020-05-27 13:20:32 -07:00
Clint Wylie 28f56978ab
web-console clean coverage report on build clean (#9718) 2020-04-21 17:02:05 -07:00
Chi Cao Minh 2262e33316
Fix flaky web console E2E test (#9685)
web-console/e2e-tests/tutorial-batch.spec.ts would occasionally timeout
between the transition from the data loader "configure schema" and
"partition" steps due to missing waits when toggling the rollup setting.

Also, fix shellcheck warnings for script/druid.
2020-04-14 15:27:16 -07:00
Chi Cao Minh 84c1c2505d
Web console basic end-to-end-test (#9595)
Load data and query (i.e., automate
https://druid.apache.org/docs/latest/tutorials/tutorial-batch.html) to
have some basic checks ensuring the web console is wired up to druid
correctly.

The new end-to-end tests (tutorial-batch.spec.ts) are added to
`web-console/e2e-tests`. Within that directory:
- `components` represent the various tabs of the web console. Currently,
  abstractions for `load data`, `ingestion`, `datasources`, and `query`
  are implemented.
- `components/load-data/data-connector` contains abstractions for the
  different data source options available to the data loader's `Connect`
  step. Currently, only the `Local file` data source connector is
  implemented.
- `components/load-data/config` contains abstractions for the different
  configuration options available for each step of the data loader flow.
  Currently, the `Configure Schema`, `Partition`, and `Publish` steps
  have initial implementation of their configuration options.
- `util` contains various helper methods for the tests and does not
  contain abstractions of the web console.

Changes to add the new tests to CI:
- `.travis.yml`: New "web console end-to-end tests" job
- `web-console/jest.*.js`: Refactor jest configurations to have
  different flavors for unit tests and for end-to-end tests. In
  particular, the latter adds a jest setup configuration to wait for the
  web console to be ready (`web-console/e2e-tests/util/setup.ts`).
- `web-console/package.json`: Refactor run scripts to add new script for
  running end-to-end tests.
- `web-console/script/druid`: Utility scripts for building, starting,
  and stopping druid.

Other changes:
- `pom.xml`: Refactor various settings disable java static checks and to
  disable java tests into two new maven profiles. Since the same
  settings are used in several places (e.g., .travis.yml, Dockerfiles,
  etc.), having them in maven profiles makes it more maintainable.
- `web-console/src/console-application.tsx`: Fix typo ("the the").
2020-04-09 12:38:09 -07:00
Vadim Ogievetsky 17d773dca2 Web console: replace (and remove) old consoles (#8838)
* first steps

* clean licenses

* fix capabilities

* fix specs

* more tests

* new web console on coordinator and overlord, remove setup for old consoles, old configs

* better message

* update licenses

* sync license files

* more button

* fix tslint issue

* jetty-rewrite dependency to add redirects for old console paths

* put dependency in the right place

* fix overlord detection

* fix notices, dedupe licenses

* make segment timeline work in no SQL mode

* update license

* revert hard coded coordinator mode from testing

* update restricted mode copy
2019-11-15 19:45:14 -08:00
Vadim Ogievetsky ec8ce74f1c Web console: Better data loader flow (#8763)
* filter table

* go over the entire data loader flow
2019-10-28 08:08:46 -07:00
Vadim Ogievetsky c948422f32 Web console: Better build experience (#8389)
* rename sql docs

* rm old file

* add old files
2019-08-24 11:35:30 -07:00
Vadim Ogievetsky 20ea90a5a6 Web console: more explicit limit on run button (#8378)
* update sql doc parsing

* keyword fixes

* fix header default

* tidy

* fix tests
2019-08-23 13:40:48 -07:00
Vadim Ogievetsky aa51668ea7 Web console: LGTM alert squash + final fixes (#8351)
* mv old console

* update dep

* reduce card size

* fix tuning configs for forceGuaranteedRollup

* fix tests

* revert console mv

* fix some old console issues

* revert build change
2019-08-21 13:52:48 -07:00
Gian Merlino d007477742
Docusaurus build framework + ingestion doc refresh. (#8311)
* Docusaurus build framework + ingestion doc refresh.

* stick to npm instead of yarn

* fix typos

* restore some _bin

* Adjustments.

* detect and fix redirect anchors

* update anchor lint

* Web-console: remove specific column filters (#8343)

* add clear filter

* update tool kit

* remove usless check

* auto run

* add %

* Fix resource leak (#8337)

* Fix resource leak

* Patch comments

* Enable Spotbugs NP_NONNULL_RETURN_VIOLATION (#8234)

* Fixes from PR review.

* Fix more anchors.

* Preamble nix.

* Fix more anchors, headers

* clean up placeholder page

* add to website lint to travis config

* better broken link checking

* travis fix

* Fixed more broken links

* better redirects

* unfancy catch

* fix LGTM error

* link fixes

* fix md issues

* Addl fixes
2019-08-20 21:48:59 -07:00
mcbrewster e4aa7fb268 Web-console: fix alerts from lgtm (#8346)
* fix alerts from lgtm

* remove unordered imports

* fix introduced alert

* move getExpanded to renderSchemaSelector

* use getDerivedStateFromProps

* use prevState

* add semi colons, remove unused imports

* fixes
2019-08-20 20:40:47 -07:00
Vadim Ogievetsky b9c68a5b7b Web console: refactor home view, add tests (#8247)
* refactor home view

* updated mode button placement
2019-08-06 12:41:07 -07:00
Vadim Ogievetsky ddec5ea82e Web console: Improve data loader time column detection (#8199)
* better timestamp suggestion

* misc fixes

* update dependancies and licenses

* add indexer, update snapshots

* add explicit render types

* pass through auto refresh in lookup view
2019-07-30 17:54:39 -07:00
mcbrewster 01f3da6fda Web Console: add Group by interval to segments view (#7986)
* add group by

* update snapshot

* fix spacing

* change design pattern

* rename interface

* add static function

* save

* add whereClause"

* add default page size

* add where to inner query

* snapshots
2019-07-19 11:25:25 -07:00
Vadim Ogievetsky d3a6753df9 Web console: update dependencies (#8007)
* update web console deps

* update license.yaml

* add tiny-warning

* do not sort all licenses

* fix snapshots

* add final licenses

* explicit mode

* webpack tweaks

* remove apache 2s

* fixed missing license

* added comments in script

* put in markers

* simpler script generator

* add copyrights

* more log
2019-07-16 09:22:12 -07:00
Vadim Ogievetsky f16f13cf61 Web console: Improve data loader styling, enforce stricter TS types (#8001)
* add assets to auth exclude path

* add frame to tile page

* better empty filter state

* strict TS

* fix segments go to sql

* add unavailable segments

* factor out sugestable input

* fix tests

* update datasources sql

* no depricated extend

* add index spec to tuning configs

* fix scss lint
2019-06-30 19:33:16 -07:00
Vadim Ogievetsky d677c83ce4 Web console: Power up the data loader init step (#7947)
* Power up the data loader init step

* update snapshot

* normalize spec

* allow deselect

* added HDFS tile

* update border style

* text updates

* goodies

* new reset icon
2019-06-26 15:50:48 -07:00
Vadim Ogievetsky 6c8f9482c7 Web console: prettier code (#7939)
* prettier code

* update tests

* scss also

* more ,

* update pachage json prettier config
2019-06-21 16:52:33 -07:00
Vadim Ogievetsky 2bc8e7c0e8 more acurate keyword auto complete (#7934) 2019-06-20 21:14:54 -07:00
Vadim Ogievetsky c44620a2e3 clean auto generated doc file also (#7911) 2019-06-17 18:01:29 -07:00
Vadim Ogievetsky 05346a9e0c allow semicolon with comment (#7892) 2019-06-14 07:53:19 -07:00
Vadim Ogievetsky c3d230b354 Web console: make the data loader more responsive when sampling data with many columns (#7856)
* Don't rerender data loader tables

* removed unused imports

* removed extra console.log

* fix typo
2019-06-09 10:44:13 -07:00
Vadim Ogievetsky 9d3dfd168a Web console: Truncate and allow easy copy of large values in query table (#7816)
* table-column-selection -> table-column-selector

* null-table-cell -> table-cell

* make action icon

* made copyable

* updated snapshots

* more tests

* normalizing tests

* tslint fixes

* fix up snapshot desciptions

* nicer edit icons

* remove unused

* better language

* remove unused

* undo unwanted change

* normalize new lines

* truncate long arrays also

* removed double space
2019-06-01 17:37:54 -07:00
Vadim Ogievetsky 4ad058c429 Fix spaces and make linting more strict (#7812) 2019-06-01 10:14:13 -07:00
Vadim Ogievetsky baf54f373c Data loader (GUI component) (#7572)
* data loader init

* fix timecolumn text

* feedback changes

* fixing typos and improving error reporting

* added local firehose warning

* update warning copy

* refine copy

* better copy

* fix tests

* remove console log

* copy change

* add banner message
2019-05-03 17:14:57 -07:00
Qi Shu 98ee8637e6 Console SQL function doc generation (#7406)
* Change sql function doc generation script from Bash to Node.js

* Minor code change for better reading

* Disable tslint in file

* clean up code

* Use es6 multiline string

* Solved a bug which would generate invalid TS variable if no function sections are found; in this case an empty array should be created

* Edit build script
2019-04-08 17:38:26 -07:00
Vadim Ogievetsky 0fa122ecf2 Adding ability to pass configs in and fixing misc bugs (#7414)
* Adding ability to pass configs in and fixing misc bugs

* update lock file

* remove dead version param
2019-04-05 12:40:43 -07:00
Vadim Ogievetsky d2ff3eb199 Migrate the web console back to Blueprint v3 (#7398)
* migrate back to Blueprint v3

* rename filler file

* fix missing icon
2019-04-01 23:06:48 -07:00