Commit Graph

615 Commits

Author SHA1 Message Date
Suneet Saldanha a7542652ff
Fix dependabot warnings (#10796)
* Bump http-proxy from 1.18.0 to 1.18.1 in /web-console (#7)

Bumps [http-proxy](https://github.com/http-party/node-http-proxy) from 1.18.0 to 1.18.1.
- [Release notes](https://github.com/http-party/node-http-proxy/releases)
- [Changelog](https://github.com/http-party/node-http-proxy/blob/master/CHANGELOG.md)
- [Commits](https://github.com/http-party/node-http-proxy/compare/1.18.0...1.18.1)

Signed-off-by: dependabot[bot] <support@github.com>

Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>

* Bump elliptic from 6.5.1 to 6.5.3 in /web-console (#6)

Bumps [elliptic](https://github.com/indutny/elliptic) from 6.5.1 to 6.5.3.
- [Release notes](https://github.com/indutny/elliptic/releases)
- [Commits](https://github.com/indutny/elliptic/compare/v6.5.1...v6.5.3)

Signed-off-by: dependabot[bot] <support@github.com>

Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>

* Bump dot-prop from 4.2.0 to 4.2.1 in /web-console (#5)

Bumps [dot-prop](https://github.com/sindresorhus/dot-prop) from 4.2.0 to 4.2.1.
- [Release notes](https://github.com/sindresorhus/dot-prop/releases)
- [Commits](https://github.com/sindresorhus/dot-prop/compare/v4.2.0...v4.2.1)

Signed-off-by: dependabot[bot] <support@github.com>

Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>

* Bump bl from 1.2.2 to 1.2.3 in /website (#4)

Bumps [bl](https://github.com/rvagg/bl) from 1.2.2 to 1.2.3.
- [Release notes](https://github.com/rvagg/bl/releases)
- [Commits](https://github.com/rvagg/bl/compare/v1.2.2...v1.2.3)

Signed-off-by: dependabot[bot] <support@github.com>

Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>

* Bump prismjs from 1.20.0 to 1.23.0 in /website (#3)

Bumps [prismjs](https://github.com/PrismJS/prism) from 1.20.0 to 1.23.0.
- [Release notes](https://github.com/PrismJS/prism/releases)
- [Changelog](https://github.com/PrismJS/prism/blob/master/CHANGELOG.md)
- [Commits](https://github.com/PrismJS/prism/compare/v1.20.0...v1.23.0)

Signed-off-by: dependabot[bot] <support@github.com>

Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>

Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2021-01-26 23:11:26 -08:00
Vadim Ogievetsky 8c227bc566
use new example manifest (#10787) 2021-01-24 12:38:13 -08:00
Jihoon Son 95065bdf1a
Bump dev version to 0.22.0-SNAPSHOT (#10759) 2021-01-15 13:16:23 -08:00
Vadim Ogievetsky e52db19823
treat null as not defined (#10751) 2021-01-14 18:22:59 -08:00
Vadim Ogievetsky 2fc2938b01
Web console: fix bad results if there is not native bigint (#10741)
* fix bigint when it does not exist

* add test
2021-01-12 16:32:23 -08:00
Yi Yuan 3624acbcf8
fix web-console show json bug (#10710)
* fix web-console show json bug

* replace all JSON.stringify

Co-authored-by: yuanyi <yuanyi@freewheel.tv>
2021-01-08 14:55:55 -08:00
Jonathan Wei c7f2d3fbb5
Update deps for CVE-2020-28168 and CVE-2020-28052 (#10733)
* Update deps for CVE-2020-28168 and CVE-2020-28052

* Make BC runtime scope
2021-01-07 20:31:44 -08:00
Lucas Capistrant 58ce2e55d8
Add dynamic coordinator config that allows control over how many segments are considered when picking a segment to move. (#10284)
* dynamic coord config adding more balancing control

add new dynamic coordinator config, maxSegmentsToConsiderPerMove. This
config caps the number of segments that are iterated over when selecting
a segment to move. The default value combined with current balancing
strategies will still iterate over all provided segments. However,
setting this value to something > 0 will cap the number of segments
visited. This could make sense in cases where a cluster has a very large
number of segments and the admins prefer less iterations vs a thorough
consideration of all segments provided.

* fix checkstyle failure

* Make doc more detailed for admin to understand when/why to use new config

* refactor PR to use a % of segments instead of raw number

* update the docs

* remove bad doc line

* fix typo in name of new dynamic config

* update RservoirSegmentSampler to gracefully deal with values > 100%

* add handler for <= 0 in ReservoirSegmentSampler

* fixup CoordinatorDynamicConfigTest naming and argument ordering

* fix items in docs after spellcheck flags

* Fix lgtm flag on missing space in string literal

* improve documentation for new config

* Add default value to config docs and add advice in cluster tuning doc

* Add percentOfSegmentsToConsiderPerMove to web console coord config dialog

* update jest snapshot after console change

* fix spell checker errors

* Improve debug logging in getRandomSegmentBalancerHolder to cover all bad inputs for % of segments to consider

* add new config back to web console module after merge with master

* fix ReservoirSegmentSamplerTest

* fix line breaks in coordinator console dialog

* Add a test that helps ensure not regressions for percentOfSegmentsToConsiderPerMove

* Make improvements based off of feedback in review

* additional cleanup coming from review

* Add a warning log if limit on segments to consider for move can't be calcluated

* remove unused import

* fix tests for CoordinatorDynamicConfig

* remove precondition test that is redundant in CoordinatorDynamicConfig Builder class
2020-12-22 08:27:55 -08:00
Vadim Ogievetsky 55b8cc428a
remove extra word (#10682) 2020-12-15 23:02:33 -08:00
Vadim Ogievetsky 577cd66002
Web console: reflect the changes to interval requirement in the data loader flow (#10647)
* no need for intervals

* don't set redundant fields

* fix tests

* better filter control

* work with not

* wrap callout with form group

* update snapshot

* add split hint

* highlight issues with spec

* fixes

* fix default value

* move intervals back to partition step

* work with all sorts of chars

* fix enabled view
2020-12-09 10:18:42 -08:00
Vadim Ogievetsky e3f7217546
Web console: Improve the handling of extreme data (funky datasources, longs) (#10641)
* better API escape

* fix escaping issue, bigints

* update licenses

* fix align

* do not show Query with SQL if no SQL

* add prettify script

* update dev readme

* add ordering to the datasource list

* add ordering to supervisor table
2020-12-08 09:25:14 -08:00
Vadim Ogievetsky 5b06c7a3a9
Web console: improve how code is imported, use API instance (#10597)
* fix imports

* clean up imports

* update DQT to fix escaping
2020-12-01 13:16:14 -08:00
zhangyue19921010 abb03d38b0
double confirm before delete all the ununsed segmensts (#10581)
Co-authored-by: yuezhang <yuezhang@freewheel.tv>
2020-11-19 10:07:52 -06:00
Vadim Ogievetsky 9964dd4cb2
Web console: fix data loader schema table column ordering bug and other polish (#10588)
* remove unused fields

* keep tables live

* advanced

* fix schema view

* better indication

* tests pass

* Show more instead of show advanced

* fix tests

* extract dynamic configs

* update snapshots

* fix issues

* update snapshot

* reword without >
2020-11-17 13:25:03 -08:00
Lucas Capistrant 83667172ce
Add missing coordinator dynamic config to the web-console dialog for dynamic coordinator config (#10545)
* Add missing coordinator dynamic config to the web-console dialog for dynamic coordinator config

* update jest snapshots after making console change
2020-11-09 13:00:52 -06:00
Vadim Ogievetsky ee61a165e3
Web console: improve make type preservation in ingestion configs in the data loader (#10533)
* improve validation

* move to druid modals

* adjust specs

* oneOf

* move transform

* segment grans

* tidy up webpack

* add Partitioning

* break out druid models

* tidy up

* rename to Expression

* move druid time

* cleanup

* update format

* better detail

* fix e2e test

* remove forceGuaranteedRollup from e2e tests

* clean up imports

* shardSpec -> shard_spec

* fix css

* adjust snapshot

* add granularity to segments view

* rename to Time span

* use OFFSET in segments view query

* update doc links

* simplify require

* more oneOf
2020-10-30 19:02:44 -07:00
Vadim Ogievetsky f391e89653
Web console: refresh and tighten up the console styles 💅💫 (#10515)
* no hardcoded dark mode

* update blueprint

* style pass

* add open sans font

* fix public path

* update licenses

* add comment

* allow zero bsd

* add new lines

* update tests

* remove default mods

* use variables

* fix background

* fix query init style

* pad fracional part

* paginate braces

* update to latest blueprint, fix column tree

* Smart query limit -> Auto limit

* adjust gutter color for dark mode

* fix spelling

* update snapshot

Co-authored-by: asherbitter <ofir@imply.io>
2020-10-20 22:11:29 -07:00
Vadim Ogievetsky e8c5893c34
Web console: show segment sizes in rows not bytes (#10496)
* added query error suggestions

* simplify the SQLs

* change segment size display to rows

* suggestion tests

* update snapshot

* make error detection more robust

* remove errant console log

* fix imports

* put suggestion on top

* better error rendering

* format as millions

* add .druid.pid to gitignore

* rename segment_size to segment_rows, fix visability, fix divide by zero

* update snapshots
2020-10-13 13:19:39 -07:00
Suneet Saldanha b45a56f989
Web console: targetRowsPerSegment for hashed partionin (#10500)
* Web console: targetRowsPerSegment for hashed partionin

Added `targetRowsPerSegment` to the web console for hashed partition for both
the auto compaction view and as part of the ingestion workflow.

The help text was also updated to indicate when a user should care about
updating these fields

* code review

* update test snapshots

* oops
2020-10-11 16:55:28 -07:00
Ashish Kapoor 4c78b514c9
Update README.md (#10357)
Compile scss files before npm start.
2020-10-09 20:21:59 +05:30
Joseph Glanville 7ce9ac4548
Fix Avro support in Web Console (#10232)
* Fix Avro OCF detection prefix and run formation detection on raw input

* Support Avro Fixed and Enum types correctly

* Check Avro version byte in format detection

* Add test for AvroOCFReader.sample

Ensures that the Sampler doesn't receive raw input that it can't
serialize into JSON.

* Document Avro type handling

* Add TS unit tests for guessInputFormat
2020-10-07 21:08:22 -07:00
Vadim Ogievetsky 2e50ada407
Web console: fix compaction status when no compaction config, and small cleanup (#10483)
* move timed button to icons

* cleanup redundant logic

* fix compaction status text

* remove extra style
2020-10-07 14:54:08 -07:00
Gian Merlino d78fedd13c
Web console: Don't include realtime segments in size calculations. (#10482)
It's always zero, and so it messes up averages, mins, and counts.
2020-10-06 18:56:54 -07:00
Chi Cao Minh 1c77491da6
Test UI to trigger auto compaction (#10469)
In the web console E2E tests, Use the new UI to trigger auto compaction
instead of calling the REST API directly so that the UI is covered by
tests.
2020-10-04 00:06:07 -07:00
Jonathan Wei 65c0d64676
Update version to 0.21.0-SNAPSHOT (#10450)
* [maven-release-plugin] prepare release druid-0.21.0

* [maven-release-plugin] prepare for next development iteration

* Update web-console versions
2020-10-03 16:08:34 -07:00
Vadim Ogievetsky f77c16bc6c
Web console: fix lookup edit dialog version setting (#10461)
* fix lookup edit dialog

* update snapshots

* clean up test
2020-10-03 08:35:20 -07:00
Chi Cao Minh d11537b5f7
Improve UI E2E test usability (#10466)
- Update playwright to latest version
- Provide environment variable to disable/enable headless mode
- Allow running E2E tests against any druid cluster running on standard
  ports (tutorial-batch.spec.ts now uses an absolute instead of relative
  path for the input data)
- Provide environment variable to change target web console port
- Druid setup does not need to download zookeeper
2020-10-03 08:21:44 -07:00
Chi Cao Minh ede25f1b45
Fix UI datasources view edit action compaction (#10459)
Restore the web console's ability to view a datasource's compaction
configuration via the "action" menu. Refactoring done in
https://github.com/apache/druid/pull/10438 introduced a regression that
always caused the default compaction configuration to be shown via the
"action" menu instead.

Regression test is added in e2e-tests/auto-compaction.spec.ts.
2020-10-01 23:59:21 -07:00
Chi Cao Minh 7385af0272
Web console reindexing E2E test (#10453)
Add an E2E test for the web console workflow of reindexing a Druid
datasource to change the secondary partitioning type.  The new test
changes dynamic to single dim partitions since the autocompaction test
already does dynamic to hashed partitions.

Also, run the web console E2E tests in parallel to reduce CI time and
change naming convention for test datasources to make it easier to map
them to the corresponding test run.

Main changes:

1) web-consolee2e-tests/reindexing.spec.ts
   - new E2E test

2) web-console/e2e-tests/component/load-data/data-connector/reindex.ts
   - new data loader connector for druid input source

3) web-console/e2e-tests/component/load-data/config/partition.ts
   - move partition spec definitions from compaction.ts
   - add new single dim partition spec definition
2020-10-01 15:14:41 -07:00
Vadim Ogievetsky d09fd8b035
Web console: switch to switches instead of checkboxes (#10454)
* switch to switches

* add img alt

* add relative

* change icons

* update snapshot
2020-09-30 15:13:22 -07:00
Vadim Ogievetsky 729bcba7ac
Web console: Display compaction status (#10438)
* init compaction status

* % compacted

* final UI tweaks

* extracted utils, added tests

* add tests to general foramt functions
2020-09-28 22:19:28 -07:00
Chi Cao Minh cbe2b44e29
Compaction config UI optional numShards (#10446)
* Compaction config UI optional numShards

Specifying `numShards` for hashed partitions is no longer required after
https://github.com/apache/druid/pull/10419. Update the UI to make
`numShards` an optional field for hash partitions.

* Update snapshot
2020-09-28 17:15:48 -07:00
Chi Cao Minh d16c78ba98
Add intent for web console IntervalInput (#10447)
When using the web console to load data by reindexing from Druid, the
`Datasource` and `Interval` inputs are required during the `Connect`
step. Unlike the `Datasource` input, the `Interval` input did not have a
blue outline to indicate that it was required as the `IntervalInput`
component did not support an `intent` property.
2020-09-28 17:13:07 -07:00
Chi Cao Minh cbd9ac8592
Web console autocompaction E2E test (#10425)
Add an E2E test for the common case web console workflow of setting up
autocompaction that changes the partitions from dynamic to hashed.

Also fix an issue with the async test setup to properly wait for the web
console to be ready.
2020-09-25 18:28:25 -07:00
Vadim Ogievetsky 89160c2f9b
better query view initial state (#10431) 2020-09-24 09:49:58 -07:00
Vadim Ogievetsky a60d034d01
Web console: compaction dialog update (#10417)
* compaction dialog update

* fix test snapshot

* Update web-console/src/dialogs/compaction-dialog/compaction-dialog.tsx

Co-authored-by: Chi Cao Minh <chi.caominh@imply.io>

* Update web-console/src/dialogs/compaction-dialog/compaction-dialog.tsx

Co-authored-by: Chi Cao Minh <chi.caominh@imply.io>

* feedback changes

Co-authored-by: Chi Cao Minh <chi.caominh@imply.io>
2020-09-23 09:34:19 -07:00
Vadim Ogievetsky ac0a45471e
Web console: add sort to tiers list (#10416)
* add sort to tiers list

* update snapshot
2020-09-22 19:00:55 -07:00
Vadim Ogievetsky 7cc0a7be68
Web console: clean up styling imports (#10410)
* fix styling for importing

* fix quotes
2020-09-21 17:30:25 -07:00
Vadim Ogievetsky 6c5c86d800
Web console: fix lookup edit dialog, allow column renaming (#10406)
* column rename

* update licenses file

* remove empty file

* update license file

* move comment
2020-09-20 14:10:05 -07:00
Vadim Ogievetsky e465f05717
Web console: Improve number alignment in tables (#10389)
* Improve tables

* removed unused state interfaces

* better copy

* one more functional component

* updated e2e tests

* extract braced text correctly
2020-09-14 19:53:38 -07:00
Vadim Ogievetsky 3c8eacb2d4
Web console: improve query manager (convert to React hook) (#10360)
* Better query running

* update licenses

* update tests

* updated tests v2

* fade in cancel

* add exemplary tests

* update mkcomp

* fix inconsistent state update

* remove lastParsedQuery

* work if not a valid literal

* remove unused params

* fix licenses

* better state update

* get error message

* isEmpty tidy

* add tests around error message highlighting

* pull live query selector into a component

* add LiveQueryModeSelector tests

* update snapshots
2020-09-11 19:42:50 -07:00
Vadim Ogievetsky e81a9df507
Web console: add tile for Azure Event Hubs (via Kafka API) (#10317)
* Add Azure Event Hubs

* better note

* update icon
2020-08-31 20:58:52 -07:00
Vadim Ogievetsky 748a83cb78
Web console: fix json input (#10271)
* fix json input

* tidy up

* add error extraction test
2020-08-13 12:20:58 -07:00
Suneet Saldanha 6baea0b4d5
Fix broken sampler for re-indexing (#10196)
* Fix broken sampler for re-indexer

When re-indexing a Druid datasource, the web-console would generate an
invalid inputFormat since the type is not specified.

* code review
2020-08-11 19:22:26 -07:00
Vadim Ogievetsky 1fae0e15c9
improve JSON paste (#10256) 2020-08-07 17:01:06 -07:00
Vadim Ogievetsky 9a29496b6c
Web console: Improve retention rules dialog in all sorts of ways (#10226)
* improve ret rules

* tidy up tests
2020-08-04 16:43:31 -07:00
keefe roedersheimer 4bb198eb4c
change search filter to includes (#10141) 2020-08-04 16:37:07 -07:00
Vadim Ogievetsky 6d8799f2df
Update QueryView to use latest DruidQueryToolkit (#10201)
* Update to latest DruidQueryToolkit

* add THEN keyword

* do not crash on invalid JSON
2020-07-23 22:45:01 -07:00
Suneet Saldanha eeb9012743
Change color of Run button for native queries (#10170)
* Change color of Run button for native queries

When a user tries to run a native query, change the color of the button to
Druid's secondary color to indicate that the user is not running a SQL query.

Before this change, the web-console would indicate this by changing the text
of the button from Run (SQL queries) to Rune (native queries). Rune could be
confusing to users as this appears to be a typo.

* Update web-console/src/views/query-view/run-button/run-button.scss

* Update web-console/src/views/query-view/run-button/run-button.scss

* Update web-console/src/views/query-view/run-button/run-button.scss

* code review
2020-07-15 15:46:02 -07:00
Clint Wylie 0a8fd62f22
split web-console e2e-tests from unit tests (#10173)
* split web-console e2e-test from unit test

* fix stuff

* smaller change

* oops
2020-07-12 18:51:08 -07:00
Egor Riashin d54a5e009f
ui: fix missing columns during Transform step (#10086)
Co-authored-by: egor-ryashin <egor.ryashin@metamarkets.com>
2020-07-07 15:16:52 -07:00
Clint Wylie c86e7ce30b
bump version to 0.20.0-SNAPSHOT (#10124) 2020-07-06 15:08:32 -07:00
Clint Wylie 477335abb4
update links datasketches.github.io to datasketches.apache.org (#10107)
* update links datasketches.github.io to datasketches.apache.org

* now with more apache

* oops

* oops
2020-07-01 14:56:17 -07:00
Vadim Ogievetsky c01fd56182
Web console: allow link overrides for docs, and more (#10100)
* link overrides

* change doc version

* fix snapshots
2020-06-30 12:46:50 -07:00
Yuanli Han fc555980e8
Remove payload field from table sys.segment (#9883)
* remove payload field from table sys.segments

* update doc

* fix test

* fix CI failure

* add necessary fields

* fix doc

* fix comment
2020-06-29 22:20:23 -07:00
Suneet Saldanha 15a0b4ffe2
Filter http requests by http method (#10085)
* Filter http requests by http method

Add a config that allows a user which http methods to allow against their
Druid server.

Druid will only accept http requests with the method: GET, PUT, POST, DELETE
and OPTIONS.
If a Druid admin wants to allow other methods, they can do so by using the
ServerConfig#allowedHttpMethods config.

If a Druid user would like to disallow OPTIONS, this can be done by changing
the AuthConfig#allowUnauthenticatedHttpOptions config

* Exclude OPTIONS from always supported HTTP methods

Add HEAD as an allowed method for web console e2e tests

* fix docs

* fix security IT

* Actually fix the web console e2e tests

* Ignore icode coverage for nitialization classes

* code review
2020-06-29 16:59:31 -07:00
Will Xu 35c7c0ec25
Segment timeline doesn't show results older than 3 months (#9956)
* Segment timeline doesn't show results older than 3 months

* Adoption testing patch for web segment timeline view and also refactoring default time config
2020-06-28 01:45:05 -07:00
Maytas Monsereenusorn 5d35f3e080
Remove colocated datasources from web console for broadcast indexed tables (#10018) 2020-06-11 14:08:03 -10:00
Xavier Léauté 65280a6953
update kafka client version to 2.5.0 (#9902)
- remove dependency on deprecated internal Kafka classes
- keep LZ4 version in line with the version shipped with Kafka
2020-05-27 13:20:32 -07:00
Joseph Glanville 132a1c9fe7
Re-order and document format detection in web console (#9887)
Motivation for this change is to not inadvertently identify binary
formats that contain uncompressed string data as TSV or CSV.

Moving detection of magic byte headers before heuristics should be more
robust in general.
2020-05-21 16:29:39 -07:00
Vadim Ogievetsky 63baa29ad1
Fix web console query view crashing on simple query (#9897)
* only parse full queries

* upgraded sql parser
2020-05-21 12:57:07 -07:00
Maytas Monsereenusorn f470bcd11f
Fix deleting a data node tier causes load rules to display incorrectly (#9891)
* Fix Deleting a data node tier causes load rules to malfunction & display incorrectly

* add tests

* fix style
2020-05-20 16:49:28 -07:00
Joseph Glanville 793f386d6a
Add support for Avro OCF using InputFormat (#9671)
* Add AvroOCFInputFormat

* Support supplying a reader schema in AvroOCFInputFormat

* Add docs for Avro OCF input format

* Address review comments

* Address second round of review
2020-05-16 14:09:12 -07:00
Chi Cao Minh 41cf826928
Console E2E test docs (#9864) 2020-05-13 16:41:04 -07:00
mcbrewster 28be107a1c
add flag to flattenSpec to keep null columns (#9814)
* add flag to flattenSpec to keep null columns

* remove changes to inputFormat interface

* add comment

* change comment message

* update web console e2e test

* move keepNullColmns to JSONParseSpec

* fix merge conflicts

* fix tests

* set keepNullColumns to false by default

* fix lgtm

* change Boolean to boolean, add keepNullColumns to hash, add tests for keepKeepNullColumns false + true with no nuulul columns

* Add equals verifier tests
2020-05-08 21:53:39 -07:00
Jian Wang 85dfbb64cb
Update documention for metricCompression (#9811) 2020-05-03 12:56:48 -07:00
Clint Wylie 28f56978ab
web-console clean coverage report on build clean (#9718) 2020-04-21 17:02:05 -07:00
mcbrewster b7fdb29423
add joins to column tree menu (#9705)
* add joins to column tree menu

* fix capitalization

* add keyword, keep columns if replaced

* actually fix capitalization

* add keywords
2020-04-16 17:51:59 -07:00
Chi Cao Minh 2262e33316
Fix flaky web console E2E test (#9685)
web-console/e2e-tests/tutorial-batch.spec.ts would occasionally timeout
between the transition from the data loader "configure schema" and
"partition" steps due to missing waits when toggling the rollup setting.

Also, fix shellcheck warnings for script/druid.
2020-04-14 15:27:16 -07:00
Chi Cao Minh 84c1c2505d
Web console basic end-to-end-test (#9595)
Load data and query (i.e., automate
https://druid.apache.org/docs/latest/tutorials/tutorial-batch.html) to
have some basic checks ensuring the web console is wired up to druid
correctly.

The new end-to-end tests (tutorial-batch.spec.ts) are added to
`web-console/e2e-tests`. Within that directory:
- `components` represent the various tabs of the web console. Currently,
  abstractions for `load data`, `ingestion`, `datasources`, and `query`
  are implemented.
- `components/load-data/data-connector` contains abstractions for the
  different data source options available to the data loader's `Connect`
  step. Currently, only the `Local file` data source connector is
  implemented.
- `components/load-data/config` contains abstractions for the different
  configuration options available for each step of the data loader flow.
  Currently, the `Configure Schema`, `Partition`, and `Publish` steps
  have initial implementation of their configuration options.
- `util` contains various helper methods for the tests and does not
  contain abstractions of the web console.

Changes to add the new tests to CI:
- `.travis.yml`: New "web console end-to-end tests" job
- `web-console/jest.*.js`: Refactor jest configurations to have
  different flavors for unit tests and for end-to-end tests. In
  particular, the latter adds a jest setup configuration to wait for the
  web console to be ready (`web-console/e2e-tests/util/setup.ts`).
- `web-console/package.json`: Refactor run scripts to add new script for
  running end-to-end tests.
- `web-console/script/druid`: Utility scripts for building, starting,
  and stopping druid.

Other changes:
- `pom.xml`: Refactor various settings disable java static checks and to
  disable java tests into two new maven profiles. Since the same
  settings are used in several places (e.g., .travis.yml, Dockerfiles,
  etc.), having them in maven profiles makes it more maintainable.
- `web-console/src/console-application.tsx`: Fix typo ("the the").
2020-04-09 12:38:09 -07:00
mcbrewster 6f3d403491
Use auto-form for add an edit lookups (#9587)
* use auto form

* jest -u

* fix unreachable statment

* complete the owl

* jest -u

* remove changes to query-view

* fix permissions

* add test, fix info

* add cool highlights

* fix formatting

* fix capitalization

* add optional placeholder

* add space
2020-04-08 16:34:59 -07:00
mcbrewster 2b2b9efcd7
add new text to lookup action dialog (#9643) 2020-04-08 11:30:47 -07:00
mcbrewster 6e50d29b4e
fix global filter input (#9567)
* fix global filter input

* remove clear

* close global filters after clicking apply

* add restFilter
2020-04-07 13:31:19 -07:00
Jihoon Son 0da8ffc3ff
Bump up development version to 0.19.0-SNAPSHOT (#9586) 2020-03-30 16:24:04 -07:00
mcbrewster e1b201c279
Add view values to lookup actions menu (#9549)
* add test, add query

* jest -u

* add limit, explicitly get columns, remoove map
2020-03-24 09:57:33 -07:00
Clint Wylie bf85ea19b2
roaring bitmaps by default (#9548)
* it is finally time

* fix it

* more docs

* fix doc
2020-03-23 18:15:57 -07:00
Vadim Ogievetsky cdf4a26904
clean up spec before reopening in data loader (#9536) 2020-03-23 16:57:51 -07:00
Vadim Ogievetsky 3b536eea7f
Web console: expose props for S3 (#9432)
* expose props for S3

* added env inputs

* add scarry warning

* use .password

* put the warning front and center

* Update web-console/src/views/load-data-view/load-data-view.tsx

Co-Authored-By: Suneet Saldanha <44787917+suneet-s@users.noreply.github.com>

* let prettier rewrap the text

Co-authored-by: Suneet Saldanha <44787917+suneet-s@users.noreply.github.com>
2020-03-18 15:32:12 -07:00
Vadim Ogievetsky 7626be26ca
Web console: add config control for the query context (#9499)
* add default and mandetory query contexts

* added config docs
2020-03-16 14:34:19 -07:00
mcbrewster bcb9a632c7
Web console: update druid-query-toolkit to version 0.4.x (#9500)
* add support for new version of DQT

* update druid-query-toolkit

* fix direction css

* fix remove

* update package

* remove useless conditional

* bump package

* jest -u

Co-authored-by: Maggie Brewster <maggiebrewster@implydata20sMBP.attlocal.net>
2020-03-13 18:09:47 -07:00
Vadim Ogievetsky ddc6f87920
Web console: standardize the spec format (#9477)
* standerdize the spec format

* fix spec upgrade
2020-03-12 14:21:23 -07:00
mcbrewster 95406ca20a
[IMPLY-2285] fix maxRowsPerSegment tool tip (#9468) 2020-03-09 20:12:05 -07:00
mcbrewster 96ed7210d3
Fix history dialog overflow (#9471)
* [IMPLY-1661] fix history dialog overflow

* jest -u
2020-03-09 19:09:59 -07:00
mcbrewster a676d16226
[IMPLY-1767] fix popover direction (#9470) 2020-03-09 17:35:02 -07:00
mcbrewster da0ea627d0
Add disabled run button during loading state (#9474)
* [IMPLY-1782] add disabled run button during loading state

* jest -u
2020-03-09 17:10:35 -07:00
Clint Wylie 32cd47bc8e
Fix home view styling (#9444) 2020-03-04 19:39:36 -08:00
mcbrewster 99095c4ac5
Add Azure ingestion flow to web console (#9437)
* add support for azure

* change bucket to container

* add azure to input menu

* remove static-azure
2020-03-03 11:06:00 -08:00
Vadim Ogievetsky c294e0b7c6
Web console: Column counter (#9334)
* Column counter

* more general test
2020-02-27 12:04:27 -08:00
Aditya e506fc9fdf
fix cursor position after function autocomplete (#9396)
Closes #9395
2020-02-26 09:41:24 -08:00
Chi Cao Minh 4221573317
Update web-console dependencies (#9361)
Fixes security vulnerabilities as flagged by the "security
vulnerability" travis cron job.
2020-02-13 21:02:34 -08:00
Vadim Ogievetsky 7e53f23f07
Web console: make supervisor reset really scary in the UI (#9253)
* make supervisor reset really scary

* change warnings

* add text
2020-02-04 15:33:52 -08:00
Roman Leventov b9186f8f9f Reconcile terminology and method naming to 'used/unused segments'; Rename MetadataSegmentManager to MetadataSegmentsManager (#7306)
* Reconcile terminology and method naming to 'used/unused segments'; Don't use terms 'enable/disable data source'; Rename MetadataSegmentManager to MetadataSegments; Make REST API methods which mark segments as used/unused to return server error instead of an empty response in case of error

* Fix brace

* Import order

* Rename withKillDataSourceWhitelist to withSpecificDataSourcesToKill

* Fix tests

* Fix tests by adding proper methods without interval parameters to IndexerMetadataStorageCoordinator instead of hacking with Intervals.ETERNITY

* More aligned names of DruidCoordinatorHelpers, rename several CoordinatorDynamicConfig parameters

* Rename ClientCompactTaskQuery to ClientCompactionTaskQuery for consistency with CompactionTask; ClientCompactQueryTuningConfig to ClientCompactionTaskQueryTuningConfig

* More variable and method renames

* Rename MetadataSegments to SegmentsMetadata

* Javadoc update

* Simplify SegmentsMetadata.getUnusedSegmentIntervals(), more javadocs

* Update Javadoc of VersionedIntervalTimeline.iterateAllObjects()

* Reorder imports

* Rename SegmentsMetadata.tryMark... methods to mark... and make them to return boolean and the numbers of segments changed and relay exceptions to callers

* Complete merge

* Add CollectionUtils.newTreeSet(); Refactor DruidCoordinatorRuntimeParams creation in tests

* Remove MetadataSegmentManager

* Rename millisLagSinceCoordinatorBecomesLeaderBeforeCanMarkAsUnusedOvershadowedSegments to leadingTimeMillisBeforeCanMarkAsUnusedOvershadowedSegments

* Fix tests, refactor DruidCluster creation in tests into DruidClusterBuilder

* Fix inspections

* Fix SQLMetadataSegmentManagerEmptyTest and rename it to SqlSegmentsMetadataEmptyTest

* Rename SegmentsAndMetadata to SegmentsAndCommitMetadata to reduce the similarity with SegmentsMetadata; Rename some methods

* Rename DruidCoordinatorHelper to CoordinatorDuty, refactor DruidCoordinator

* Unused import

* Optimize imports

* Rename IndexerSQLMetadataStorageCoordinator.getDataSourceMetadata() to retrieveDataSourceMetadata()

* Unused import

* Update terminology in datasource-view.tsx

* Fix label in datasource-view.spec.tsx.snap

* Fix lint errors in datasource-view.tsx

* Doc improvements

* Another attempt to please TSLint

* Another attempt to please TSLint

* Style fixes

* Fix IndexerSQLMetadataStorageCoordinator.createUsedSegmentsSqlQueryForIntervals() (wrong merge)

* Try to fix docs build issue

* Javadoc and spelling fixes

* Rename SegmentsMetadata to SegmentsMetadataManager, address other comments

* Address more comments
2020-01-27 11:24:29 -08:00
Caroline1000 3daf0f8e12 Update ingestion-view.tsx (#9250)
Grammar and accuracy updates.
2020-01-24 01:21:55 -08:00
Vadim Ogievetsky ab2672514b allow empty values to be set in the auto form (#9198) 2020-01-16 21:06:51 -08:00
Vadim Ogievetsky 09efd20b42
fix refresh button (#9195) 2020-01-16 10:13:47 -08:00
Clint Wylie f245292e5d add middle manager and indexer worker category to tier column of services view (#9158) 2020-01-09 12:20:42 -08:00
Clint Wylie 7439f73c23 web console services tab treat indexer as a real service (#9139) 2020-01-07 18:14:04 -08:00
Clint Wylie 28edd3b44e data loader style fix for double typed columns (#9137) 2020-01-07 16:07:30 -08:00
Clint Wylie 14702429a0 fix web console data loader dimension types (#9135) 2020-01-06 20:56:58 -08:00
Jonathan Wei aa539177ec De-incubation cleanup in code, docs, packaging (#9108)
* De-incubation cleanup in code, docs, packaging

* remove unused docs script
2020-01-03 12:33:19 -05:00
Jonathan Wei 4e8368a5d9 Set version to 0.18.0-SNAPSHOT (#9109) 2020-01-02 17:55:10 -05:00
Vadim Ogievetsky 320c50d24a Web console: fix spec reset (#9081)
* extract spec type

* better text

* better copy

* de incubate the console

* fix status dialog scss
2019-12-23 18:23:14 -08:00
Vadim Ogievetsky a24e2f347f make supervisor statistics dialog more robust (#9089) 2019-12-23 17:43:08 -08:00
Vadim Ogievetsky e7b1653d88 add button to reapply retention rules (#9055) 2019-12-17 18:08:57 -08:00
Jonathan Wei 8af41d7cd0 Update version to 0.18.0-incubating-SNAPSHOT (#9009) 2019-12-11 14:04:03 -08:00
Vadim Ogievetsky a6dcc99962 better input format detection (#9007) 2019-12-09 22:31:28 -08:00
Clint Wylie cefcfe26dc update web-console data loader to support unified s3 and google input sources (#8994)
* update web-console data loader to support unified s3 and google input source

* fixes

* add placeholder for objects

* only show objects if it already exists
2019-12-06 07:25:26 -08:00
Vadim Ogievetsky 1cff73f3e0 Web console: support new ingest spec format (#8828)
* converter v1

* working v1

* update tests

* update tests

* upgrades

* adjust to new API

* remove hack

* fwd

* step

* neo cache

* fix time selection

* smart reset

* parquest autodetection

* add binaryAsString option

* partitionsSpec

* add ORC support

* ingestSegment -> druid

* remove index tasks

* better min

* load data works

* remove downgrade

* filter on group_id

* fix group_id in test

* update auto form for new props

* add dropBeforeByPeriod rule

* simplify

* prettify json
2019-12-04 20:21:07 -08:00
Clint Wylie d0a6fe7f12 fix bug with sqlOuterLimit, use sqlOuterLimit in web console (#8919)
* fix bug with sqlOuterLimit, use sqlOuterLimit instead of wrapping sql query for web console

* fixes, refactors, tests

* meh

* better name

* fix comment location

* fix copy and paste
2019-12-03 18:36:28 -08:00
jon-wei dfbc066163 Revert "[maven-release-plugin] prepare release druid-0.16.1-incubating-rc1"
This reverts commit a0f21d9b07.
2019-11-27 23:22:43 -08:00
jon-wei 0402ff85b8 Revert "[maven-release-plugin] prepare for next development iteration"
This reverts commit 8ffa71e7e6.
2019-11-27 23:22:32 -08:00
jon-wei 8ffa71e7e6 [maven-release-plugin] prepare for next development iteration 2019-11-27 23:18:48 -08:00
jon-wei a0f21d9b07 [maven-release-plugin] prepare release druid-0.16.1-incubating-rc1 2019-11-27 23:18:37 -08:00
Vadim Ogievetsky 282b838b3f fix home view tabs (#8938) 2019-11-26 12:21:32 +08:00
Vadim Ogievetsky ee8f048381 Web console: rename Tasks to Ingestion (#8896)
* rename Tasks to Ingestion

* rename local storage key also

* align ordering
2019-11-20 06:53:40 -08:00
Vadim Ogievetsky 80fc04be71 bump typescript (#8890) 2019-11-17 16:23:47 -08:00
Vadim Ogievetsky 17d773dca2 Web console: replace (and remove) old consoles (#8838)
* first steps

* clean licenses

* fix capabilities

* fix specs

* more tests

* new web console on coordinator and overlord, remove setup for old consoles, old configs

* better message

* update licenses

* sync license files

* more button

* fix tslint issue

* jetty-rewrite dependency to add redirects for old console paths

* put dependency in the right place

* fix overlord detection

* fix notices, dedupe licenses

* make segment timeline work in no SQL mode

* update license

* revert hard coded coordinator mode from testing

* update restricted mode copy
2019-11-15 19:45:14 -08:00
Evan Ren 8cb213aa9f Web console: Fix missing include future flag for byPeriod rules (#8859)
* Add missing button for include future, and handle logic for default true case

* Remove duplicate go to tasks button

* Fix lgtm issue

* Revert changes on old console

* Made changes based on PR comments
2019-11-12 20:34:30 -08:00
Vadim Ogievetsky df2f77c58d Web console: better json-input feedback (#8851)
* better json-input feedback

* seamless Hjson

* fix tests
2019-11-11 17:06:03 -08:00
Vadim Ogievetsky 6eacaf446f Use more efficient tasks API (#8844) 2019-11-08 08:21:53 -08:00
Evan Ren b03aa060bd Web console: Interval input component (#8777)
* Created temporary interval input component

* Make reusable interval component

* Fixed errors with typing invalid dates

* Fix interval input styling and place into autoform

* Fix styling of popover calendar that opens off the page

* Add snapshot test and change interval to required props

* Add functionality to enter hours minutes second

* Fix min date limit

* Remove console log

* Fix difference in timezone

* Update snapshot test

* Fixed snapshot test without changing min max date

* Made changes based on discussion before converting to hooks

* Rewrote using hooks and deleted duplicate states

* Remove unused states

* Change sql query view numbers to monospace

* Made changes based on discussion

* Removed duplicate state
2019-11-07 13:07:17 -08:00
Vadim Ogievetsky 7addfc27da
Web console: fine grained capabilities / graceful degradation (#8805)
* fine grained capabilities

* fix tests

* configure all cards

* better detection

* update tests

* rename server to service

* node -> service

* remove console log

* better loader in data loader
2019-11-05 23:39:14 -08:00
Vadim Ogievetsky 6f7fbeb63a Fix logo overflow (#8817) 2019-11-05 21:52:38 -08:00
Vadim Ogievetsky c2889ca4f4 show hollow circle when unavailable (#8819) 2019-11-05 21:50:10 -08:00
Vadim Ogievetsky 16aaf7227e Web console: work in IE11 (#8804)
* fix IE11

* also support flexbox
2019-10-31 21:03:05 -07:00
Vadim Ogievetsky f6028de7a8 Web console: use SQL for the supervisor view (#8796)
* use SQL for supervisor view

* home view sql also

* no proxy mode

* fix alert

* improve message
2019-10-31 20:59:36 -07:00
Vadim Ogievetsky ed6be81d12 Web console: fix error when querying with grand totals (#8795)
* fix error when querying with grand totals

* also support object

* improve tests
2019-10-30 19:37:53 -07:00
Vadim Ogievetsky 929a8b6337 Web console: Support all possible metric types in the data loader (#8785)
* Support all possible metric types in the data loader

* added more sketches
2019-10-30 09:34:13 -07:00
Vadim Ogievetsky a95e3d438e Web console: Data loader user feedback changes (#8770)
* init fixes

* cleaning styling issues

* more conversion types
2019-10-29 08:42:51 -07:00
Vadim Ogievetsky 11230dff52 Support HDFS firehose (#8752) 2019-10-28 08:22:20 -07:00
Vadim Ogievetsky ec8ce74f1c Web console: Better data loader flow (#8763)
* filter table

* go over the entire data loader flow
2019-10-28 08:08:46 -07:00
Vadim Ogievetsky 1b9d4ce811 Web console: Memoize all the functional components and improve transform step highlighting (#8757)
* Fix transform table

* memoize all components

* use named functions
2019-10-26 17:58:26 -07:00
Vadim Ogievetsky 80be9462a6 Web console: Update versions and dependencies (#8751)
* update console version and deps

* bump script version
2019-10-25 17:53:06 -07:00
Vadim Ogievetsky 27127345b7 Add delimiter option for TSV parser (#8741) 2019-10-25 09:17:59 -07:00
Vadim Ogievetsky efd669757e fix save button (#8732) 2019-10-24 15:25:28 -07:00
Evan Ren fdbc4ae147 Web console: Button to pretty print Druid JSON query (#8724)
* Add button and functionality to pretty format Rune JSON

* Removed console log

* Fix lgtm error about updating state

* Update test snapshot
2019-10-23 20:24:47 -07:00
Vadim Ogievetsky 137c2a6025 Web console: disable data loader Submit button when submitting so as not to submit multiple times (#8725)
* disable Submit button when submitting so as not to submit twice

* also check in fn
2019-10-23 18:42:44 -07:00
Vadim Ogievetsky d9c9aef3d1 Druid Doctor (#8672)
* adding Druid doctor

* better meesage

* feedback changes

* add icons

* feedback fixes

* spelling

* add file.encoding check

* feedback changes
2019-10-23 16:50:08 -07:00
Vadim Ogievetsky 3abd0c4a33 Web console: start transition to React hooks and functional components (#8722)
* do the easy stuff

* some more components
2019-10-23 09:43:02 -07:00
Jihoon Son 4046c86d62
Stateful auto compaction (#8573)
* Stateful auto compaction

* javaodc

* add removed test back

* fix test

* adding indexSpec to compactionState

* fix build

* add lastCompactionState

* address comments

* extract CompactionState

* fix doc

* fix build and test

* Add a task context to store compaction state; add javadoc

* fix it test
2019-10-15 22:57:42 -07:00
Vadim Ogievetsky 6c609293b2 show full value in SQL view (#8660) 2019-10-10 14:16:11 -07:00
Vadim Ogievetsky 4c215b417e Web console: misc bug fixes and tidy up (#8654)
* fix updates

* fix status dialog

* fix scan query deserialization

* extract error message

* update snapshot
2019-10-10 10:52:46 -07:00
Jihoon Son 96d8523ecb Use hash of Segment IDs instead of a list of explicit segments in auto compaction (#8571)
* IOConfig for compaction task

* add javadoc, doc, unit test

* fix webconsole test

* add spelling

* address comments

* fix build and test

* address comments
2019-10-09 11:12:00 -07:00
Evan Ren 17d9d7daed Increase column size for taskID and createdTime, and decrease Type and Duration (#8594) 2019-09-27 14:19:04 -07:00
Evan Ren 0467cce7a0 Web console: Add frontend buttons to remove group by (#8537)
* Add frontend buttons to remove group by

* Change icon for remove group by

* Update web console to use latest toolkit

* Add test cases to verify that remove group by buttons are rendered

* Correct mistake of using incorrect components

* Update tests for two cases

* Put remove button after group by
2019-09-24 16:32:02 -07:00
Vadim Ogievetsky 7c14fa08f8 Web console: Expand filter UI (#8579)
* add controls to the filter UI

* fix double base
2019-09-24 12:27:46 -07:00
Vadim Ogievetsky 2104cee79b Web console: prevent extra trim in auto complete (#8543)
* prevent extra trim in auto complete

* add unit test
2019-09-22 15:18:51 -07:00
Vadim Ogievetsky 868bb42301 clean up default values and add infos (#8567) 2019-09-21 21:00:21 -07:00
Vadim Ogievetsky 36a6365d9f Web console: polish the data loader (#8554)
* rearrange ioConfig prop ordering in Tune step

* make useEarliestOffset mandatory

* required intent

* make segmentGranularity requred for batch ingestion

* update tests
2019-09-18 20:53:18 -07:00
Evan Ren 8650ee9fd0 Web console: Druid status displayed in a table (#8484)
* Retrieved data from endpoint and displayed on table

* Added view raw button, removed json, and fixed formatting for table

* Remove error var

* Fixed snapshot for updated status dialog

* Made changes based on PR review

* Added version tag

* Updated snapshot to match changes

* Made more changes based on review

* Fix filter and formatting

* Fix filter and add unit test

* fix styling of dialog

* Fix footer height

* Fixed testing of filtering
2019-09-17 15:26:51 -06:00
Vadim Ogievetsky 6c7f36d364 fix spec dialog highlighting (#8533) 2019-09-13 15:24:08 -07:00
Vadim Ogievetsky ad72f36360 fix forceGuaranteedRollup tooltip (#8529) 2019-09-12 17:24:29 -07:00
Vadim Ogievetsky a6eca5e935 Web console: Force intervals config (#8514)
* make sure intervals are required

* all truncated values everywhere

* continue to spec when going from tasks table

* remove unused thigns

* fix alert
2019-09-12 00:35:04 -07:00
Vadim Ogievetsky 1498250204 Fix snitch dialog final step (#8506) 2019-09-11 12:44:16 -07:00
Vadim Ogievetsky 2b04c22bfd split segment view query into two (#8485) 2019-09-06 21:32:39 -07:00
Vadim Ogievetsky b8dc6a94b3 Web console: fix datasource name auto detection (#8479)
* fix datasource name guessing

* fix comment
2019-09-06 00:57:26 -07:00
Vadim Ogievetsky bffaca672a swap ordering clauses in the menu (#8473) 2019-09-05 14:59:28 -07:00
Vadim Ogievetsky 85119b8721 fix iPad width (#8471) 2019-09-05 13:56:23 -07:00
Vadim Ogievetsky 637a9e8614 Web console: Fix segment re-ingest (#8454)
* fixing ingest spec

* rm console.log

* ingest segment spec

* make sure step query always being run

* better example interface

* add keywords

* placeholders

* do not overwrite datasource name in data loader spec

* fix comment typo
2019-09-03 12:13:57 -07:00
Vadim Ogievetsky f55e1be80b Web console: fix task log tailing (#8434)
* fix log tailing

* update snapshots
2019-08-29 14:15:06 -07:00
mcbrewster beed07022a Web-Console: Increase snapshot coverage (#8408)
* increase test-coverage

* fix ordered inputs

* fixes

* prettier fix

* fixes
2019-08-29 08:41:34 -07:00
Chi Cao Minh 31e6280b75 Use Codecov (#8388)
* Use Codecov

Upload coverage reports to Codecov. For now, having Codecov comment on
PRs or enforcing a minimum coverage threshold are both disabled until
the Codecov coverage reports look reliable:
https://codecov.io/gh/apache/incubator-druid

* Split bash and curl into separate lines
2019-08-28 08:49:30 -07:00
Clint Wylie c73a489335
bump master version to 0.17.0-incubating-SNAPSHOT (#8421) 2019-08-28 01:58:36 -07:00
Clint Wylie 7afe473fd3 How to asf release (#8370)
* add ASF release manager guide

* fix broken link

* fix bold

* fix order

* clean up

* oops

* pom

* more

* fix

* fixes

* fix

* fix
2019-08-27 18:36:13 -07:00
Vadim Ogievetsky 894b4efb36 Web console: Fix segment timeline styling (#8402)
* Fix segment timeline styling

* update snapshot
2019-08-25 21:31:20 -07:00
Vadim Ogievetsky 9254dc8b80 Web console: Save query context also (#8395)
* tidy up menus

* fix query output API

* save context also

* pull out auto run into a switch

* better copy

* add group_id

* support FLOAT also

* use built in time logic

* fix trunc direction

* add skipCache

* add manifest url

* remove depricated props
2019-08-25 16:56:27 -07:00
Vadim Ogievetsky c948422f32 Web console: Better build experience (#8389)
* rename sql docs

* rm old file

* add old files
2019-08-24 11:35:30 -07:00
Vadim Ogievetsky efbf1a5822 Restore old console (#8387) 2019-08-23 17:48:11 -07:00
Vadim Ogievetsky 20ea90a5a6 Web console: more explicit limit on run button (#8378)
* update sql doc parsing

* keyword fixes

* fix header default

* tidy

* fix tests
2019-08-23 13:40:48 -07:00
mcbrewster bfbae76031 Web-console: small fixes (#8376)
* fix column tree

* move remove filter

* fix lgtm

* remove from state

* stop mutating state diirectly

* fix column-tree

* remove component did mount

* remove expanded node
2019-08-22 21:54:11 -07:00
mcbrewster 5274c5ab73 Web-Console: Replace from clause (#8371)
* add replace from

* add failture state

* fix uninential downgrade

* use uppercase for SQL keywords
2019-08-22 09:09:37 -07:00
Vadim Ogievetsky aa51668ea7 Web console: LGTM alert squash + final fixes (#8351)
* mv old console

* update dep

* reduce card size

* fix tuning configs for forceGuaranteedRollup

* fix tests

* revert console mv

* fix some old console issues

* revert build change
2019-08-21 13:52:48 -07:00
Benedict Jin e1f94a5e26 Reduce the size of images with lossless compression (#8358) 2019-08-21 13:29:30 -07:00
Gian Merlino d007477742
Docusaurus build framework + ingestion doc refresh. (#8311)
* Docusaurus build framework + ingestion doc refresh.

* stick to npm instead of yarn

* fix typos

* restore some _bin

* Adjustments.

* detect and fix redirect anchors

* update anchor lint

* Web-console: remove specific column filters (#8343)

* add clear filter

* update tool kit

* remove usless check

* auto run

* add %

* Fix resource leak (#8337)

* Fix resource leak

* Patch comments

* Enable Spotbugs NP_NONNULL_RETURN_VIOLATION (#8234)

* Fixes from PR review.

* Fix more anchors.

* Preamble nix.

* Fix more anchors, headers

* clean up placeholder page

* add to website lint to travis config

* better broken link checking

* travis fix

* Fixed more broken links

* better redirects

* unfancy catch

* fix LGTM error

* link fixes

* fix md issues

* Addl fixes
2019-08-20 21:48:59 -07:00
mcbrewster e4aa7fb268 Web-console: fix alerts from lgtm (#8346)
* fix alerts from lgtm

* remove unordered imports

* fix introduced alert

* move getExpanded to renderSchemaSelector

* use getDerivedStateFromProps

* use prevState

* add semi colons, remove unused imports

* fixes
2019-08-20 20:40:47 -07:00
mcbrewster e64070b8bf Web-console: remove specific column filters (#8343)
* add clear filter

* update tool kit

* remove usless check

* auto run

* add %
2019-08-19 19:54:18 -07:00
Vadim Ogievetsky 56e440383f Web console: data loader native batch shuffle UI (#8229)
* data loader shuffle UI

* better support for inline firehose

* feedback changes

* make numShards mandatory sometimes

* edit context dialog fixes

* make auto form less agro

* download full log

* improve auto column detection

* add sample for example
2019-08-19 13:27:34 -07:00
mcbrewster cae1361c32 Web-console: gate auto complete on current table and schema (#8322)
* gate auto complete on current table and schema

* reset defaults

* add static functions

* move completions to state
2019-08-17 09:34:36 -07:00
Jihoon Son 31af4eb9ad
Rename maxNumSubTasks to maxNumConcurrentSubTasks for native parallel index task (#8324) 2019-08-16 15:57:13 -07:00
Vadim Ogievetsky 0d69438395 Web console: make console's Explain more honest (#8327)
* fix structure

* update snapshots
2019-08-16 15:51:49 -07:00
mcbrewster 8924d285dc Web-Console: add more side column functions to query view (#8283)
* small fixes

* run jest -u

* add trim

* add to groupby

* save

* add functionality to column tree menu

* remove history dialog

* rename file

* fixes

* add new values to time-menu

* add query dialog

* add more functions to collumn-tree menu

* add menu functions

* remove test

* update tests, fix extra 0s, clear in between time filters

* fixes

* add deffered

* save

* add alias's

* change menu style

* fix substring

* small fixes

* add auto run to run button

* move divider to a reasonable position

* fix auto expland

* run jest -u

* fix auto run

* prevent aggregate cooumn changes

* fix group by bug

* add quertAst to state

* ui fixes

* prevent filter on non existant ast

* add auto run to shows

* use handle run

* handle *

* add alias and tslint fix
2019-08-15 01:22:25 -07:00
mcbrewster d1f286b7f4 fix no from clause error (#8275) 2019-08-08 21:42:52 -07:00
mcbrewster 03a37936ec Web-Console: Column tree button fixes (#8253)
* small fixes

* run jest -u

* add trim

* remove trim, update package

* fix package-lock
2019-08-07 16:48:17 -07:00
Chi Cao Minh 05b44e3467 Speedup Travis CI jobs (#8240)
Reorganize Travis CI jobs into smaller faster (and more) jobs. Add
various maven options to skip unnecessary work and refactored Travis CI
job definitions to follow DRY.

Detailed changes:

.travis.yml
- Refactor build logic to get rid of copy-and-paste logic
- Skip static checks and enable parallelism for maven install
- Split static analysis into different jobs to ease triage
- Use "name" attribute instead of NAME environment variable
- Split "indexing" and "web console" out of "other modules test"
- Split 2 integration test jobs into multiple smaller jobs

build.sh
- Enable parallelism
- Disable more static checks

travis_script_integration.sh
travis_script_integration_part2.sh
integration-tests/README.md
- Use TestNG groups instead of shell scripts and move definition of jobs
  into Travis CI yaml

integration-tests/pom.xml
- Show elapsed time of individual tests to aid in future rebalancing of
  Travis CI integration test jobs run time

TestNGGroup.java
- Use TestNG groups to make it easy to have multiple Travis CI
  integration test jobs. TestNG groups also make it easier to have an
  "other" integration test group and make it less likely a test will
  accidentally not be included in a CI job.

IT*Test.java
AbstractITBatchIndexTest.java
AbstractKafkaIndexerTest.java
- Add TestNG group
- Fix various IntelliJ inspection warnings
- Reduce scope of helper methods since the TestNG group annotation on
  the class makes TestNG consider all public methods as test methods

pom.xml
- Allow enforce plugin to be run from command-line
- Bump resources plugin version so that "[debug] execute contextualize"
  output is correctly suppressed by "mvn -q"
- Bump exec plugin version so that skip property is renamed from "skip"
  to "exec.skip"

web-console/pom.xml
- Add property to allow disabling javascript-related work. This property
  is overridden in Travis CI to speed up the jobs.
2019-08-07 09:52:42 -07:00
mcbrewster e2d1d00fb8 Web-Console: Add show and copy actions to column tree (#8251)
* save

* add copy and show popup to column tree menu

* fixes

* nest css, uset functions

* fix copy state

* update package

* update package-lock
2019-08-06 23:58:00 -07:00
Vadim Ogievetsky b9c68a5b7b Web console: refactor home view, add tests (#8247)
* refactor home view

* updated mode button placement
2019-08-06 12:41:07 -07:00
mcbrewster 62da1efc0f Web-Console: add dimensions to datasources icon (#8232)
* add dimensions to datasources icon

* fix test

* update snapshots

* save

* use table

* update snapshots

* rename to columns

* run jest -u
2019-08-06 00:04:37 -07:00
Vadim Ogievetsky 0235b338fc Web console: celebrate array based groupBy by supporting resultAsArray in the console (#8231)
* teach table about resultAsArray

* use query result decoder

* fix snapshot
2019-08-05 18:54:39 -07:00
mcbrewster 5d0805dd48 Web-Console: change go to sql button to more button (#8227)
* change go to sql button

* rename to See in SQL view

* update snapshots
2019-08-05 14:12:16 -07:00
mcbrewster ac856fe4c1 Web-Console: Adds edit-context-dialog (#8228)
* add edit query context

* remove console log

* add icon

* change to submit button

* fix c

* add stricter checking

* update snapshots
2019-08-04 07:18:08 -07:00
mcbrewster 5528dec4a4 Web-Console: stop log from making request after dialog close, implement query manager (#8219)
* stop logs from runnign after component unmount, use query managers

* address comments

* check if defined
2019-08-01 18:17:09 -07:00
mcbrewster 02c3a785a2 Web-Console: add go to task for datasources (#8210)
* add go to task for datasources

* general fixes

* use find
2019-08-01 18:15:22 -07:00
Vadim Ogievetsky a0f88868e3 add timeout to capabilities check (#8213) 2019-08-01 08:37:02 -07:00
Vadim Ogievetsky 15dd3d7256 Web console: better action linking between views (#8203)
* better actions

* better action linking
2019-07-31 07:13:25 -07:00
Vadim Ogievetsky 645fca53d8 Web console: segment timeline (#8202)
* Add segment timeline chart

* fix file

* Fix bugs: no data handling & scaling problems

* resolve conflict

* changed package-lock

* do not show by default

* trust the interop

* stricter type fixes

* fix sasslint
2019-07-30 22:35:30 -07:00
Vadim Ogievetsky ddec5ea82e Web console: Improve data loader time column detection (#8199)
* better timestamp suggestion

* misc fixes

* update dependancies and licenses

* add indexer, update snapshots

* add explicit render types

* pass through auto refresh in lookup view
2019-07-30 17:54:39 -07:00
mcbrewster 4e60afc86e Web-console: add Supervisor statistics table and show json query manager (#8176)
* add statistics table

* change error message

* add query managers

* update snapshots

* fixes

* add stricter types

* fix error type

* prettier fic

* fix undefined

* add typing

* fixe query manager
2019-07-30 12:53:46 -07:00
mcbrewster f7abc300a8 Web-console: add loader to show-json, add status-dialog (#8172)
* add loader to show json, add status-dialog

* fix ordered import

* fix changes

* order inputs

* update snapshots

* fix bad merge conflicts
2019-07-30 08:08:36 -07:00
mcbrewster a0270512a2 Web-console: add history menu to supervisors modal (#8164)
* Add history menu to supervisors dialog

* add scrolling

* change how tabs render

* remove copy button

* user query manager
2019-07-30 08:07:58 -07:00
Vadim Ogievetsky 771c04841e remove husky hooks (#8193) 2019-07-29 19:32:16 -07:00
Vadim Ogievetsky cc4450db12 Web console: add reindex (ingestSegment firehose) to the data loader (#8181)
* tidy up nulls

* standardize more on undefined

* updated licenses

* do not do heavy handed rendering

* reindex from druid

* tidy up

* add inline firehose

* add husky

* sass lint

* better suggestion

* fix script typo

* adjust time formats

* add missing time formats

* use term 'reindex'

* fix lodash.compact
2019-07-29 14:41:27 -07:00
Vadim Ogievetsky 0b3ede193e Web console: adding Slack channel integration (#8182)
* adding slack channel integration

* update snapshots
2019-07-28 16:17:54 -07:00
Vadim Ogievetsky 8bd0f8c2ac Web console: code quality improvements (null tidy up) (#8162)
* tidy up nulls

* standardize more on undefined

* updated licenses

* update snapshot

* do not do heavy handed rendering

* add placeholder to SQL view

* remove pointelss fragment
2019-07-27 01:46:37 -07:00
mcbrewster ef9230f916 Web-console: Add continue to spec view (#8144)
* Add continue to spec view

* update spec when type selected

* remove redundancy

* fix mistakes
2019-07-25 14:48:26 -07:00
mcbrewster 8ba1f06632 Web-console: add lookups tile to home view (#8124)
* add lookups-tile

* add uninitailized state

* remove error:
2019-07-25 13:08:58 -07:00
Vadim Ogievetsky 2b2fcc0371 Web console: cleanup build, check licenses in test (#8113)
* cleanup build, check licenses

* cleanup filterMap type

* don't run license check twice
2019-07-19 22:59:07 -07:00
mcbrewster 01f3da6fda Web Console: add Group by interval to segments view (#7986)
* add group by

* update snapshot

* fix spacing

* change design pattern

* rename interface

* add static function

* save

* add whereClause"

* add default page size

* add where to inner query

* snapshots
2019-07-19 11:25:25 -07:00
Roman Leventov ceb969903f
Refactor SQLMetadataSegmentManager; Change contract of REST met… (#7653)
* Refactor SQLMetadataSegmentManager; Change contract of REST methods in DataSourcesResource

* Style fixes

* Unused imports

* Fix tests

* Fix style

* Comments

* Comment fix

* Remove unresolvable Javadoc references; address comments

* Add comments to ImmutableDruidDataSource

* Merge with master

* Fix bad web-console merge

* Fixes in api-reference.md

* Rename in DruidCoordinatorRuntimeParams

* Fix compilation

* Residual changes
2019-07-17 17:18:48 +03:00
Vadim Ogievetsky b80f20f769 add missing dynamic coordinator configs (#8090) 2019-07-16 17:40:41 -07:00
Vadim Ogievetsky d3a6753df9 Web console: update dependencies (#8007)
* update web console deps

* update license.yaml

* add tiny-warning

* do not sort all licenses

* fix snapshots

* add final licenses

* explicit mode

* webpack tweaks

* remove apache 2s

* fixed missing license

* added comments in script

* put in markers

* simpler script generator

* add copyrights

* more log
2019-07-16 09:22:12 -07:00
Vadim Ogievetsky 1712158734 added replicated size (#8043) 2019-07-10 08:29:05 -07:00
Vadim Ogievetsky 8493bd5c1c Web console: added bulk supervisor actions (#8028)
* added bulk supervisor actions to the web console

* improve async action dialog

* fix tests
2019-07-09 17:31:23 -07:00
Vadim Ogievetsky 4f4cda3ec9 fix query input undo (#8033) 2019-07-06 09:21:53 -07:00
Vadim Ogievetsky 2de6cc3b30 fix segment table filter (#8034) 2019-07-06 09:21:28 -07:00
Vadim Ogievetsky f16f13cf61 Web console: Improve data loader styling, enforce stricter TS types (#8001)
* add assets to auth exclude path

* add frame to tile page

* better empty filter state

* strict TS

* fix segments go to sql

* add unavailable segments

* factor out sugestable input

* fix tests

* update datasources sql

* no depricated extend

* add index spec to tuning configs

* fix scss lint
2019-06-30 19:33:16 -07:00
mcbrewster f1270c14f8 Web-Console: add clipboard modal (#7964)
* add clipboard modal

* rename button

* remove console.log

* fix off by one

* update tests

* update snapshot

* fix casing

* update snapshot
2019-06-28 15:50:18 -07:00
Vadim Ogievetsky d677c83ce4 Web console: Power up the data loader init step (#7947)
* Power up the data loader init step

* update snapshot

* normalize spec

* allow deselect

* added HDFS tile

* update border style

* text updates

* goodies

* new reset icon
2019-06-26 15:50:48 -07:00
mcbrewster a171b4a399 Web-console: Add action column to segments view (#7954)
* add actions column to segments view

* add sements action column
2019-06-25 20:14:06 -07:00
Benedict Jin 16aafd5788 [ImgBot] Optimize images (#7873)
*Total -- 10,997.25kb -> 7,160.16kb (34.89%)

/publications/radstack/figures/precompute.png -- 54.20kb -> 16.97kb (68.69%)
/web-console/favicon.png -- 4.41kb -> 1.61kb (63.58%)
/docs/img/indexing_service.png -- 47.37kb -> 21.96kb (53.64%)
/docs/img/segmentPropagation.png -- 62.94kb -> 29.85kb (52.57%)
/docs/content/tutorials/img/tutorial-quickstart-01.png -- 55.62kb -> 29.13kb (47.62%)
/docs/content/tutorials/img/tutorial-deletion-02.png -- 791.43kb -> 429.30kb (45.76%)
/docs/content/tutorials/img/tutorial-deletion-03.png -- 786.79kb -> 427.05kb (45.72%)
/docs/content/tutorials/img/tutorial-retention-00.png -- 135.06kb -> 75.88kb (43.82%)
/docs/content/tutorials/img/tutorial-batch-data-loader-10.png -- 77.23kb -> 43.47kb (43.71%)
/docs/content/tutorials/img/tutorial-batch-data-loader-01.png -- 97.03kb -> 55.16kb (43.15%)
/docs/content/tutorials/img/tutorial-batch-data-loader-07.png -- 79.49kb -> 45.44kb (42.84%)
/docs/content/tutorials/img/tutorial-retention-02.png -- 401.30kb -> 234.68kb (41.52%)
/docs/content/tutorials/img/tutorial-compaction-06.png -- 343.27kb -> 201.87kb (41.19%)
/docs/content/tutorials/img/tutorial-batch-data-loader-09.png -- 105.14kb -> 61.86kb (41.16%)
/docs/content/tutorials/img/tutorial-retention-06.png -- 227.57kb -> 134.35kb (40.97%)
/docs/content/tutorials/img/tutorial-compaction-04.png -- 304.83kb -> 180.04kb (40.94%)
/docs/content/tutorials/img/tutorial-compaction-02.png -- 273.18kb -> 162.67kb (40.45%)
/docs/content/tutorials/img/tutorial-query-05.png -- 85.03kb -> 50.64kb (40.44%)
/publications/radstack/figures/druid_vs_bigquery.png -- 155.44kb -> 92.85kb (40.27%)
/docs/content/tutorials/img/tutorial-kafka-02.png -- 122.51kb -> 73.93kb (39.65%)
/docs/content/tutorials/img/tutorial-deletion-01.png -- 70.37kb -> 42.56kb (39.52%)
/docs/content/tutorials/img/tutorial-batch-data-loader-06.png -- 103.50kb -> 62.79kb (39.33%)
/docs/content/tutorials/img/tutorial-batch-submit-task-01.png -- 111.25kb -> 67.73kb (39.12%)
/docs/content/tutorials/img/tutorial-query-03.png -- 103.60kb -> 63.51kb (38.69%)
/docs/content/tutorials/img/tutorial-query-04.png -- 105.79kb -> 64.87kb (38.69%)
/docs/content/tutorials/img/tutorial-batch-data-loader-11.png -- 130.20kb -> 81.34kb (37.53%)
/docs/content/tutorials/img/tutorial-query-07.png -- 122.52kb -> 76.79kb (37.32%)
/docs/content/tutorials/img/tutorial-kafka-01.png -- 133.12kb -> 83.47kb (37.3%)
/docs/content/tutorials/img/tutorial-query-06.png -- 127.55kb -> 80.28kb (37.06%)
/docs/content/tutorials/img/tutorial-batch-submit-task-02.png -- 133.07kb -> 84.06kb (36.83%)
/docs/content/tutorials/img/tutorial-retention-05.png -- 60.19kb -> 38.08kb (36.74%)
/docs/content/tutorials/img/tutorial-batch-data-loader-03.png -- 211.92kb -> 134.22kb (36.66%)
/docs/content/tutorials/img/tutorial-batch-data-loader-05.png -- 250.36kb -> 158.68kb (36.62%)
/publications/radstack/figures/radstack.png -- 16.80kb -> 10.67kb (36.48%)
/docs/content/tutorials/img/tutorial-batch-data-loader-08.png -- 158.59kb -> 101.49kb (36%)
/docs/content/tutorials/img/tutorial-batch-data-loader-04.png -- 255.10kb -> 163.33kb (35.97%)
/docs/content/tutorials/img/tutorial-query-02.png -- 126.92kb -> 81.42kb (35.85%)
/docs/content/tutorials/img/tutorial-compaction-01.png -- 53.86kb -> 34.87kb (35.25%)
/docs/img/druid-architecture.png -- 202.23kb -> 130.97kb (35.24%)
/docs/content/tutorials/img/tutorial-retention-01.png -- 52.69kb -> 34.35kb (34.81%)
/docs/img/druid-timeline.png -- 35.87kb -> 23.59kb (34.22%)
/docs/content/tutorials/img/tutorial-query-01.png -- 149.53kb -> 98.56kb (34.08%)
/docs/content/tutorials/img/tutorial-retention-04.png -- 65.91kb -> 43.57kb (33.89%)
/docs/content/tutorials/img/tutorial-compaction-08.png -- 42.24kb -> 28.08kb (33.53%)
/docs/content/tutorials/img/tutorial-compaction-07.png -- 39.17kb -> 26.06kb (33.47%)
/docs/content/tutorials/img/tutorial-compaction-03.png -- 39.17kb -> 26.13kb (33.3%)
/docs/content/tutorials/img/tutorial-compaction-05.png -- 38.85kb -> 25.96kb (33.17%)
/publications/demo/figures/throughput_vs_cardinality.png -- 73.49kb -> 49.31kb (32.9%)
/publications/radstack/figures/throughput_vs_cardinality.png -- 73.49kb -> 49.31kb (32.9%)
/publications/whitepaper/figures/throughput_vs_cardinality.png -- 73.49kb -> 49.31kb (32.9%)
/docs/content/tutorials/img/tutorial-retention-03.png -- 43.11kb -> 29.33kb (31.97%)
/publications/radstack/figures/throughput_vs_num_dims.png -- 72.86kb -> 49.72kb (31.76%)
/publications/whitepaper/figures/throughput_vs_num_dims.png -- 72.86kb -> 49.72kb (31.76%)
/publications/demo/figures/throughput_vs_num_dims.png -- 72.86kb -> 49.72kb (31.76%)
/publications/radstack/figures/joined.png -- 164.14kb -> 113.47kb (30.87%)
/docs/content/tutorials/img/tutorial-batch-data-loader-02.png -- 508.93kb -> 351.85kb (30.87%)
/publications/radstack/figures/imps_clicks.png -- 190.95kb -> 132.70kb (30.51%)
/publications/radstack/figures/shuffled.png -- 180.46kb -> 128.21kb (28.95%)
/publications/radstack/figures/pipeline.png -- 392.54kb -> 281.93kb (28.18%)
/docs/img/druid-manage-1.png -- 108.94kb -> 78.53kb (27.92%)
/publications/radstack/figures/throughput_vs_num_metrics.png -- 85.25kb -> 61.80kb (27.51%)
/publications/demo/figures/throughput_vs_num_metrics.png -- 85.25kb -> 61.80kb (27.51%)
/publications/whitepaper/figures/throughput_vs_num_metrics.png -- 85.25kb -> 61.80kb (27.51%)
/docs/img/druid-production.png -- 50.00kb -> 39.18kb (21.63%)
/docs/img/druid-dataflow-3.png -- 88.25kb -> 69.75kb (20.96%)
/publications/demo/figures/realtime_flow.png -- 51.12kb -> 40.61kb (20.56%)
/publications/demo/figures/realtime_timeline.png -- 36.15kb -> 29.24kb (19.12%)
/publications/demo/figures/tpch_scaling.png -- 43.21kb -> 34.97kb (19.08%)
/publications/demo/figures/caching.png -- 35.26kb -> 29.09kb (17.49%)
/dev/intellij-sdk-config.jpg -- 1,019.35kb -> 864.37kb (15.2%)
/docs/img/druid-column-types.png -- 101.53kb -> 91.17kb (10.2%)
/docs/img/druid-dataflow-2x.png -- 138.30kb -> 127.11kb (8.09%)
2019-06-24 21:27:48 -07:00
mcbrewster cd01a48eab Web-console: add timed button (#7912)
* add refresh button

* update snapshots

* fix spacing

* add supervisors button

* modify query manager

* add icon

* update snapshots

* fix space

* stop refreshing when option selected

* small fixes

* add default value

* fix spelling mistake

* fix query-input

* rename class
2019-06-24 21:18:19 -07:00
Vadim Ogievetsky 6c8f9482c7 Web console: prettier code (#7939)
* prettier code

* update tests

* scss also

* more ,

* update pachage json prettier config
2019-06-21 16:52:33 -07:00
Vadim Ogievetsky 2bc8e7c0e8 more acurate keyword auto complete (#7934) 2019-06-20 21:14:54 -07:00
Vadim Ogievetsky 28eaa620a9 Web console: Data loader respects parse spec columns for data preview (#7922)
* small fixes in the data loader

* respect columns

* fix test
2019-06-19 12:48:31 -07:00
Clint Wylie 71997c16a2 switch links from druid.io to druid.apache.org (#7914)
* switch links from druid.io to druid.apache.org

* fix it
2019-06-18 09:06:27 -07:00
Vadim Ogievetsky c44620a2e3 clean auto generated doc file also (#7911) 2019-06-17 18:01:29 -07:00
Vadim Ogievetsky df9cdcf13b Web console: added Query View metadata pane, more query details (#7905)
* refactor

* adding metadata hookup

* add scroll

* standerdize icons

* better loading state

* update snapshots

* fix lint

* do not mix up queryId and sqlQueryId
2019-06-17 13:14:42 -07:00
Jenny Zhu f603498e11 Web console: Add download path to SQL Query (#7898)
* adding download path to Query

* add more checker

* updated snap tests

* change after Vad's comments
2019-06-15 13:51:22 -07:00
mcbrewster 5e6d833fd9 Web-Console: tail log based on task status (#7888)
* clean up

* fix declarations

* spacing
2019-06-14 14:37:21 -07:00
Vadim Ogievetsky 05346a9e0c allow semicolon with comment (#7892) 2019-06-14 07:53:19 -07:00
mcbrewster 0763585208 Web-console: update supervisors table (#7799)
* add new columns

* fix syling

* fix spaces

* update snapshots

* fix Spelling

* fix capitalization

* reorder action dialog

* set color using state

* fix snapshots

* fix array

* update snapshots

* remove extra columns

* update snapshots

* update snapshots

* fixes

* update snapshots

* use cell

* fix spacing

* update snapshot
2019-06-11 13:06:35 -07:00
Vadim Ogievetsky c612ddc0f4 Support more JODA time formats (#7857) 2019-06-09 20:08:29 -07:00
Vadim Ogievetsky c3d230b354 Web console: make the data loader more responsive when sampling data with many columns (#7856)
* Don't rerender data loader tables

* removed unused imports

* removed extra console.log

* fix typo
2019-06-09 10:44:13 -07:00
Vadim Ogievetsky 53b9fc4eb0 switch to use exInterlop (#7853) 2019-06-07 22:27:19 -07:00
Vadim Ogievetsky 248e075e24 Web console: Simplify task and supervisor detail displays (#7848)
* simplify detail display

* update tests
2019-06-07 11:02:45 -07:00
Vadim Ogievetsky 061b465d34 Web console: Normalize ingestion spec type (#7847)
* Normalize ingestion spec types

* also normalize on direct spec change
2019-06-07 08:32:40 -07:00
Vadim Ogievetsky df01335789 Web console: prevent the time parse view from going into a bad state (#7846)
* prevent the time parse view from going into a bad state

* add test
2019-06-07 08:32:10 -07:00
Vadim Ogievetsky ca5afd29b8 switch the web console to use react pure components (#7845) 2019-06-07 08:31:46 -07:00
Vadim Ogievetsky 0ca867efbf less aggressive default value (#7844) 2019-06-07 08:31:21 -07:00
Vadim Ogievetsky 0de7983db5 Add minor CSS to make long value copy even easier (#7829) 2019-06-03 19:28:57 -07:00
Jihoon Son 61ec521135
Remove keepSegmentGranularity option for compaction (#7747)
* Remove keepSegmentGranularity option from compaction

* fix it test

* clean up

* remove from web console

* fix test
2019-06-03 12:59:15 -07:00
Vadim Ogievetsky 9d3dfd168a Web console: Truncate and allow easy copy of large values in query table (#7816)
* table-column-selection -> table-column-selector

* null-table-cell -> table-cell

* make action icon

* made copyable

* updated snapshots

* more tests

* normalizing tests

* tslint fixes

* fix up snapshot desciptions

* nicer edit icons

* remove unused

* better language

* remove unused

* undo unwanted change

* normalize new lines

* truncate long arrays also

* removed double space
2019-06-01 17:37:54 -07:00
Vadim Ogievetsky 4ad058c429 Fix spaces and make linting more strict (#7812) 2019-06-01 10:14:13 -07:00
mcbrewster 21d3d0cde8 Web-console: prevent unnecessary loading of disabled data sources (#7804)
* prevent loading disabled datasources

* prevent double loading

* remove complexity

* rename function

* remove space
2019-06-01 02:08:34 -07:00
Clint Wylie 94b3591171 friendly printed arrays for json responses (#7805) 2019-05-31 17:34:12 +08:00
Jihoon Son 7abfbb066a Bump up snapshot version to 0.16.0 (#7802) 2019-05-30 17:17:33 -07:00
mcbrewster 245eded350 Web-console: Add run keyboard short cut to sql-control (#7787)
* add run keyboard shortcut

* add allow input
2019-05-29 16:11:59 -07:00
Vadim Ogievetsky b051d6688d bump deps (#7788) 2019-05-29 08:50:06 -07:00
Vadim Ogievetsky 96b9b3073c Web console: Adding a server view that can display all servers (#7770)
* adding a server view that can see all servers

* fixing action-cell snapshot test

* dynamic resize on sql view

* remove extra word

* update home view server tile also

* offer more context options

* pass context to explain also
2019-05-28 17:12:50 -07:00
Vadim Ogievetsky 6fdc78abbc fix supervisor resume route (#7782) 2019-05-28 13:22:42 -07:00
mcbrewster 7eac685e47 Web-console: add hostname to tasks view (#7758)
* add host name

* change to location

* remove space
2019-05-24 23:18:18 -07:00
mcbrewster 3a77a3e112 Web-Console: add actions icon to lookups (#7725)
* add lookup-table-action-dialog

* reorder arguments

* remove lookup-table-action-dialog

* fix supervisors table action dialog

* fix imports

* Add host column to task view

* reorder buttons

* remove log

* update snapshots

* remove host

* update snapshots

* remove console log

* remove fragment

* update snapshots and remove space
2019-05-24 22:53:12 -07:00
mcbrewster 5f50f357a4 Web-Console: add go to editor button to tasks and supervisors view (#7705)
* add go to editor button to tasks and supervisors

* fix package.json

* remove intent

* quuick fixes

* fixes

* add getsupervisorjson and gettaskjson

* remove space

* remove gotoloaddata

* fixes

* add error handling

* save

* add loader

* remove initspec

* fixup! add loader

* update snapshots

* remove gotoloaddataview form headerbar.spec
2019-05-24 22:52:26 -07:00
mcbrewster 2b34e1b710 Web console: add log tailing to task log view (#7703)
* add tail dialog checkbox to tasks show-log-component for webconsole

* add check box button css

* add 2 second delay

* left align

* change tail formatting

* use async/await syntax

* add space

* add public prefix

* update jest to reflect changes in show-log

* use sentence case

* update snapshots
2019-05-24 15:34:29 -07:00
mcbrewster d7e3fa510e Web-console: add resizable split screen layout to tasks and servers views (#7723)
* add split scroll layout

* fix dev dependancies

* clean up

* adjust margin

* update jest to reflect ui changes

* fix css

* fix css issues, update npm

* remove scrollable
2019-05-24 15:01:45 -07:00
Clint Wylie db3792727e use unminified jquery to be more friendly for source releases, fix license stuff (#7751)
* use unminified jquery to be more friendly for source releases, fix license stuff

* other license file

* rats
2019-05-24 11:53:25 -07:00
Bartosz Ługowski cbdac49ab3 Web console - add enable/disable actions for middle manager workers (#7642)
* Overlord console - add enable/disable button for remote workers.

* Overlord console - add proxy for remote workers API.

* WorkerResourceTest - revert newline change.

* Remote worker proxy tests - remove empty line.

* Refactor remote worker proxy for readability and security

* Rename method in remote task runner tests for readability

* Remove enable/disable button for remote workers from old web console

* Add enable/disable actions for middle manager worker in new web console

* Fix variable type

* Add worker task runner query adapter

* Fix web console tests: segments-view, servers-view

* Fix overlord resource tests
2019-05-23 16:47:23 -07:00
mcbrewster 1b284ca847 add tests to dialogs, compnents and views. Add index files to components and dialogs. add nested file structure (#7669) 2019-05-22 20:36:51 -07:00
Vadim Ogievetsky 169d2493bc Web console: fix missing value input in timestampSpec step (#7698) 2019-05-21 03:59:48 -07:00
Vadim Ogievetsky be16e4a4ae Web console, adding Apache Kafka and AWS Kinesis to the data loader (#7643)
* adding kafka and kinesis to the data loader

* feature detect

* copy fixes

* wording fixes

* added missing spec type

* increase timeout

* Call it Google Cloud Storage
2019-05-17 14:01:27 -07:00
Jenny Zhu ec0b7787cf Web console: fixed issue when grouping tasks by different attributes (#7657)
* deleted created_time in "status"

* added toString on the status object
2019-05-16 16:46:04 -07:00
Clint Wylie b87c8f0314 fix lookup editor to use lookup tiers instead of historical tiers (#7647)
* fix lookup editor to use lookup tiers instead of historical tiers

* use default tier if empty response, fix if configured lookups is null

* fixes

* fix typo
2019-05-14 13:30:51 -07:00
Jihoon Son b32c9abdb9
Run npm ci instead of npm install (#7613) 2019-05-09 15:42:16 -07:00
Vadim Ogievetsky 9b197b436b added scrolling to data loader panes (#7605) 2019-05-07 08:45:20 -07:00
Roman Leventov a00d3e1931
Fix old-console/cluster.html; Remove redundant throws in ITBasicAuthConfigurationTest (#7589)
* Fix old-console/cluster.html; Remove redundant throws in ITBasicAuthConfigurationTest

* Remove comment in cluster.html
2019-05-06 18:08:49 +02:00
Qi Shu 83b38c0911 Improved UI for actions in task/supervisor table (#7528)
* Grouped actions in task/supervisor table to a dialog

* Design change

* fix divider

* Removed await for onclick event on buttons

* fix package.json

* Better error handling

* remove log

* name change; better code organization

* Code refactor; extrac table action dialog

* minor change

* tidy up actions

* drop reload actions
2019-05-06 03:20:12 -07:00
Vadim Ogievetsky baf54f373c Data loader (GUI component) (#7572)
* data loader init

* fix timecolumn text

* feedback changes

* fixing typos and improving error reporting

* added local firehose warning

* update warning copy

* refine copy

* better copy

* fix tests

* remove console log

* copy change

* add banner message
2019-05-03 17:14:57 -07:00
Qi Shu ed2beb6ba5 Add is_overshadowed column to segment table (#7555) 2019-05-01 12:28:53 -07:00
Vadim Ogievetsky d97c0d19a0 add url baser (#7585) 2019-05-01 01:53:03 -07:00
Qi Shu 20755f4ca0 Add supervisor card in home view (#7551)
* Add supervisor card in home view

* hide the text under conditions
2019-04-29 10:15:10 -07:00
Qi Shu 3ec9fbaa47 Fix history dialog styling (#7536)
* Fix history dialog

* more styling
2019-04-25 07:59:34 -07:00
Qi Shu 11a7e91a73 No SQL mode in web console (#7493)
* Added no sql mode

* Use status code

* Add no sql mode to server view

* add sql broker check to decide if no sql mode should be enabled

* Fix historicals in home view

* Name change

* Add types for query result; improved functions

* Fixed a conflict/bug

* Fixed a bug

* multiple fix

* removed unused imports

* terminate query manager

* fix wording
2019-04-23 16:15:02 -07:00
Qi Shu 9929f8b022 Fixed filter for status in task table (#7507)
* Fixed filter for Task view status

* Refactored code

* Fixed a bug for SQL filter by not converting input to lower case since the comparison is done through SQL
2019-04-19 15:51:45 -07:00
Qi Shu 99ddce1277 Capitalize keywords in SQL auto complete (#7462)
* Split commit; keep auto complete only

* Add comment
2019-04-17 23:12:55 -07:00
Qi Shu f06f0c3ad8 Add created_time as tie breaker when sorting by status in task table (#7460)
* Add created_time as tie breaker when sorting by status

* Fixed bug

* Fixed created_time tie breaker

* Use locale compare
2019-04-17 23:11:45 -07:00
Qi Shu c8bffd9351 Add exact match filtering to console table (#7448)
* Add exact filtering

* Bug fix

* Extract filter as a function

* change code position
2019-04-17 23:05:44 -07:00
Qi Shu 9b5c216684 Fix datasource availability sorting (#7451)
* Fix availability sorting

* Style change
2019-04-14 10:09:57 -07:00
Qi Shu 233d485ad0 Display null in SQL table cell (#7463)
* Display null if cell in table is empty

* Add null

* re-indent
2019-04-13 12:05:14 -07:00
Vadim Ogievetsky 60dd75d3d9 Wrap query with limit within the web console (#7449)
* wrap with limit

* make actual menu checkbox component
2019-04-12 08:51:37 -07:00
Vadim Ogievetsky 3aae4aaf8b Ensuring that all the controls are aligned correctly (#7457)
* making sure all the controls are inline

* update deps
2019-04-12 08:51:03 -07:00
Vadim Ogievetsky 07eb2effe9 customHeader (#7456) 2019-04-11 21:58:09 -07:00
Qi Shu 7382dac44e Fixed grouping and sorting by status in task table (#7440)
* Fixed group by status

* Shorten code
2019-04-11 16:10:37 -07:00
Vadim Ogievetsky c5c2bf77f0 adding host to the error display (#7453) 2019-04-11 14:39:57 -07:00
Qi Shu 4ea37e2614 Add time taken by last query in SQL view (#7421)
* Add time taken by query

* Fix time to 2 dp; set state in on state change; hide time taken at first

* Refactored code: defined query result interface; more concise shorthand expression

* Use single quote
2019-04-10 23:35:47 -07:00
Qi Shu bbb620125d Table pagination style change (#7435)
* Style pagination

* Removed use of important

* Removed space

* Align items

* Switched back to react table 6.8.6

* Component name change
2019-04-10 23:34:03 -07:00
Qi Shu 98ee8637e6 Console SQL function doc generation (#7406)
* Change sql function doc generation script from Bash to Node.js

* Minor code change for better reading

* Disable tslint in file

* clean up code

* Use es6 multiline string

* Solved a bug which would generate invalid TS variable if no function sections are found; in this case an empty array should be created

* Edit build script
2019-04-08 17:38:26 -07:00
Vadim Ogievetsky c69c37e99d Update console packages and enforce tslint quotes (#7419)
* update packages and enforce tslint quotes

* remove direct dep on tslint-react
2019-04-07 20:19:08 -07:00
Qi Shu a66291a9cd SQL explain in web console (#7402)
* Add explain for SQL query

* Terminate explain query manager

* Add signature for semi-joined queries

* Extract components out

* Hide more button in rune mode

* Add types for query explanation parser

* Add type for query-plan-dialog
2019-04-05 12:44:51 -07:00
Vadim Ogievetsky 0fa122ecf2 Adding ability to pass configs in and fixing misc bugs (#7414)
* Adding ability to pass configs in and fixing misc bugs

* update lock file

* remove dead version param
2019-04-05 12:40:43 -07:00
Vadim Ogievetsky d2ff3eb199 Migrate the web console back to Blueprint v3 (#7398)
* migrate back to Blueprint v3

* rename filler file

* fix missing icon
2019-04-01 23:06:48 -07:00
Vadim Ogievetsky 1137bb2586 Popovers are no longer inline (#7389) 2019-04-01 13:10:48 -07:00
Qi Shu 2ad9544242 Support segment metadata in rune (#7371) 2019-03-29 15:46:09 -07:00
Qi Shu a09aa13ead Put all local storage keys and functions into one file in util (#7314)
* Add all local storage keys and functions into one file in util

* Use LocalStorageKey type instead of string

* Remove druid author key
2019-03-28 17:48:37 -07:00
Qi Shu be1d7f885b Add scss linter in web console (#7365)
* Add SCSS linter

* Fixed SCSS styles
2019-03-28 17:32:01 -07:00
Qi Shu 18e5167245 Add history dialog in snitch dialog to allow viewing the editing history (#7321)
* Add history dialog in snitch dialog to allow viewing the editing history

* Improved CSS; better animation

* Use position: absolute instead of float: right to position element

* Removed author for history changes
2019-03-27 17:52:44 -07:00
Vadim Ogievetsky 375668ef45 fix Supervisor terminate call (#7315)
* fix Supervisor terminate call

* fix refresh after submitting supervisor or task
2019-03-21 13:54:43 -07:00
Qi Shu 494c1a2ef8 Add overlord edit dialog to allow user to change overlord dynamic config on the fly (#7308)
* Add overlord edit dialog to allow user to edit overlord dynmamic config on the fly

* Set config to {} if druid return nothing for overlord config
2019-03-21 03:38:35 -07:00
Qi Shu b0271b4f4d Saving author to local storage in snitch dialog (#7307)
* Saving author to local storage

* Capitalized const
2019-03-21 03:01:10 -07:00
Qi Shu 30e646308a Add table column selection in druid console to allow hiding/showing of columns (#7292)
* Add table column selections to all tables to allow user to hide/show columns

* Small change for re-rendering

* Use column selection handler class to process all column hiding/showing

* dereference table handler function at the start; use more specific file name for table.tsx
2019-03-20 13:11:00 -07:00
Qi Shu e58f541047 Small tweak to override Blueprint's CSS rule for lookup edit dialog (#7283)
* Small tweak to override Blueprint's CSS rule

* Follow the same CSS style by using &
2019-03-18 14:24:50 -07:00
Qi Shu 1f4ad518d8 Add compaction dialog in druid console which allows users to add/edit data source compaction configuration (#7242)
* Add compaction dialog in druid console which allows users to add/edit data source compaction configuration

* Addressed naming issues; changed json input validating process
2019-03-17 19:21:23 -07:00
Vadim Ogievetsky b6b1e6160c Added tslint to web console (#7280)
* added tslint to web console

* added react linting and made rules stricter

* order imports

* update package-lock
2019-03-17 09:23:17 -07:00
Qi Shu 1b6b40e511 Add lookups view to allow adding/editing/deleting of druid lookups (#7259)
* Add lookups view to allow adding/editing/deleting of druid lookups

* Remove unused bp3 class

* Make lookup editor dialog wider
2019-03-16 01:46:43 -07:00
Qi Shu 5406aaa49d Add SQL auto complete in druid console (#7244)
* Add SQL auto complete in druid console

* Add comment in sql.md to alert user to change create-sql-function-doc if sql.md format gets changed
2019-03-16 01:45:53 -07:00
Qi Shu fdbbf82cda Fixed bugs in retention cell and retention dialog (#7276)
* Fixed a bug when there are so many retention rules that it becomes unclickable; fixed a bug when retention dialog is at bottom and becomes unreachable

* Using class for CSS instead of inline style
2019-03-15 17:30:47 -07:00
Qi Shu a337f6d4cc Fixed height issue in dynamic config dialog where user cannot scroll (#7268) 2019-03-15 15:15:12 -07:00
Vadim Ogievetsky e11364883c allow killing waiting and pending tasks (#7247) 2019-03-14 13:17:19 -07:00
Clint Wylie 050728b115 add license checker to web-console (#7028)
* add license checker to web-console to ensure npm dependencies are apache license compatible

* add generate licenses file

* update check to remove excludes due to blueprintjs downgrade
2019-03-02 12:22:54 -08:00
Vadim Ogievetsky 66e8d35ddf downgrade react-table (#7170) 2019-02-28 18:09:38 -08:00
Vadim Ogievetsky b8f762037a Downgrade blueprintjs version in the web console to one with a vanilla Apache 2.0 license (#7139)
* revert bp

* fix tests

* move @types/hjson to dev dep

* removed all the package upgrades
2019-02-25 20:54:56 -08:00
Jonathan Wei 61272d6daa Update handlebars dep to patch vulnerability (#7083) 2019-02-18 18:06:47 +08:00
Jonathan Wei fafbc4a80e
Set version to 0.15.0-incubating-SNAPSHOT (#7014) 2019-02-07 14:02:52 -08:00
Jonathan Wei 8bc5eaa908
Set version to 0.14.0-incubating-SNAPSHOT (#7003) 2019-02-04 19:36:20 -08:00
Vadim Ogievetsky 06d56c2abd Fixing little issues and CSS bugs in the new web console. (#6995) 2019-02-04 09:04:58 -08:00
Jihoon Son 7d4cc28730 Fix node path for building the unified console (#6981) 2019-02-01 13:57:17 -08:00
Vadim Ogievetsky 7f1b19bfb1 Adding a Unified web console. (#6923)
* Adding new web console.

* fixed css

* fix form height

* fix typo

* do import custom react-table css

* added repo field so npm does not complain

* ask travis for node 10

* move indexing-service/src/main/resources/indexer_static into web-console

* fix resource names and paths

* add licenses

* fix exclude file

* add licenses to misc files and tidy up

* remove rebase marker

* fix link

* updated env variable name

* tidy up licenses and surface errors

* cleanup

* remove unused code, fix missing await

* TeamCity does not like the name aux

* add more links to tasks view

* rm pages

* update gitignore

* update readme to be accurate

* make clean script

* removed old console dependancy

* update Jetty routes

* add a comment for welcome files for coordinator

* do not show inital notifaction for now

* renamed overlord console back to console.html

* fix coordinator console

* rename coordinator-console.html to index.html
2019-01-31 17:26:41 -08:00