Apache
/
druid
mirror of https://github.com/apache/druid.git
1
0
Fork 0
Code Issues Packages Projects Releases Wiki Activity
druid/web-console
History
Chi Cao Minh 84c1c2505d
Web console basic end-to-end-test (#9595) 
Load data and query (i.e., automate
https://druid.apache.org/docs/latest/tutorials/tutorial-batch.html) to
have some basic checks ensuring the web console is wired up to druid
correctly.

The new end-to-end tests (tutorial-batch.spec.ts) are added to
`web-console/e2e-tests`. Within that directory:
- `components` represent the various tabs of the web console. Currently,
  abstractions for `load data`, `ingestion`, `datasources`, and `query`
  are implemented.
- `components/load-data/data-connector` contains abstractions for the
  different data source options available to the data loader's `Connect`
  step. Currently, only the `Local file` data source connector is
  implemented.
- `components/load-data/config` contains abstractions for the different
  configuration options available for each step of the data loader flow.
  Currently, the `Configure Schema`, `Partition`, and `Publish` steps
  have initial implementation of their configuration options.
- `util` contains various helper methods for the tests and does not
  contain abstractions of the web console.

Changes to add the new tests to CI:
- `.travis.yml`: New "web console end-to-end tests" job
- `web-console/jest.*.js`: Refactor jest configurations to have
  different flavors for unit tests and for end-to-end tests. In
  particular, the latter adds a jest setup configuration to wait for the
  web console to be ready (`web-console/e2e-tests/util/setup.ts`).
- `web-console/package.json`: Refactor run scripts to add new script for
  running end-to-end tests.
- `web-console/script/druid`: Utility scripts for building, starting,
  and stopping druid.

Other changes:
- `pom.xml`: Refactor various settings disable java static checks and to
  disable java tests into two new maven profiles. Since the same
  settings are used in several places (e.g., .travis.yml, Dockerfiles,
  etc.), having them in maven profiles makes it more maintainable.
- `web-console/src/console-application.tsx`: Fix typo ("the the").
 		2020-04-09 12:38:09 -07:00
..
assets Add Azure ingestion flow to web console (#9437) 2020-03-03 11:06:00 -08:00
e2e-tests Web console basic end-to-end-test (#9595) 2020-04-09 12:38:09 -07:00
lib Web console: Fix segment re-ingest (#8454) 2019-09-03 12:13:57 -07:00
script Web console basic end-to-end-test (#9595) 2020-04-09 12:38:09 -07:00
src Web console basic end-to-end-test (#9595) 2020-04-09 12:38:09 -07:00
.gitignore Web console: Better build experience (#8389) 2019-08-24 11:35:30 -07:00
.stylelintrc.json Add scss linter in web console (#7365) 2019-03-28 17:32:01 -07:00
README.md Web console: replace (and remove) old consoles (#8838) 2019-11-15 19:45:14 -08:00
babel.config.js Web console: work in IE11 (#8804) 2019-10-31 21:03:05 -07:00
console-config.js Web console: Save query context also (#8395) 2019-08-25 16:56:27 -07:00
favicon.png [ImgBot] Optimize images (#7873) 2019-06-24 21:27:48 -07:00
jest.common.config.js Web console basic end-to-end-test (#9595) 2020-04-09 12:38:09 -07:00
jest.e2e.config.js Web console basic end-to-end-test (#9595) 2020-04-09 12:38:09 -07:00
jest.unit.config.js Web console basic end-to-end-test (#9595) 2020-04-09 12:38:09 -07:00
package-lock.json Web console basic end-to-end-test (#9595) 2020-04-09 12:38:09 -07:00
package.json Web console basic end-to-end-test (#9595) 2020-04-09 12:38:09 -07:00
pom.xml Bump up development version to 0.19.0-SNAPSHOT (#9586) 2020-03-30 16:24:04 -07:00
sasslint.json Web-console: add Supervisor statistics table and show json query manager (#8176) 2019-07-30 12:53:46 -07:00
tsconfig.json Web console basic end-to-end-test (#9595) 2020-04-09 12:38:09 -07:00
tslint.json Web console: Improve data loader styling, enforce stricter TS types (#8001) 2019-06-30 19:33:16 -07:00
unified-console.html Web console: Update versions and dependencies (#8751) 2019-10-25 17:53:06 -07:00
webpack.config.js Web console: work in IE11 (#8804) 2019-10-31 21:03:05 -07:00

README.md

Apache Druid web console

This is the unified Druid web console that servers as a data management layer for Druid.

How to watch and run for development

  1. You need to be withing the web-console directory
  2. Install the modules with npm install
  3. Run npm start will start in development mode and will proxy druid requests to localhost:8888

Note: you can provide an environment variable to proxy to a different Druid host like so: druid_host=1.2.3.4:8888 npm start Note: you can provide an environment variable use webpack-bundle-analyzer as a plugin in the build script or like so: BUNDLE_ANALYZER_PLUGIN='TRUE' npm start

To try the console in (say) coordinator mode you could run it as such:

druid_host=localhost:8081 npm start

Description of the directory structure

A lot of the directory structure was created to preserve the existing console structure as much as possible.

As part of this repo:

  • console.html - Entry file for the overlord console
  • lib/ - A place where some overrides to the react-table stylus files live, this is outside of the normal SCSS build system.
  • old-console/ - Files for the overlord console
  • public/ - The compiled destination of the file powering this console
  • assets/ - The images (and other assets) used within the console
  • script/ - Some helper bash scripts for running this console
  • src/ - This directory (together with lib) constitutes all the source code for this console

Generated/copied dynamically

  • index.html - Entry file for the coordinator console
  • pages/ - The files for the older coordinator console
  • coordinator-console/ - Files for the coordinator console

List of non SQL data reading APIs used

GET /status
GET /druid/indexer/v1/supervisor?full
POST /druid/indexer/v1/worker
GET /druid/indexer/v1/workers
GET /druid/indexer/v1/tasks
GET /druid/coordinator/v1/loadqueue?simple
GET /druid/coordinator/v1/config
GET /druid/coordinator/v1/metadata/datasources?includeUnused
GET /druid/coordinator/v1/rules
GET /druid/coordinator/v1/config/compaction
GET /druid/coordinator/v1/tiers

Updating the list of license files

From the web-console directory run script/licenses