Load data and query (i.e., automate
https://druid.apache.org/docs/latest/tutorials/tutorial-batch.html) to
have some basic checks ensuring the web console is wired up to druid
correctly.
The new end-to-end tests (tutorial-batch.spec.ts) are added to
`web-console/e2e-tests`. Within that directory:
- `components` represent the various tabs of the web console. Currently,
abstractions for `load data`, `ingestion`, `datasources`, and `query`
are implemented.
- `components/load-data/data-connector` contains abstractions for the
different data source options available to the data loader's `Connect`
step. Currently, only the `Local file` data source connector is
implemented.
- `components/load-data/config` contains abstractions for the different
configuration options available for each step of the data loader flow.
Currently, the `Configure Schema`, `Partition`, and `Publish` steps
have initial implementation of their configuration options.
- `util` contains various helper methods for the tests and does not
contain abstractions of the web console.
Changes to add the new tests to CI:
- `.travis.yml`: New "web console end-to-end tests" job
- `web-console/jest.*.js`: Refactor jest configurations to have
different flavors for unit tests and for end-to-end tests. In
particular, the latter adds a jest setup configuration to wait for the
web console to be ready (`web-console/e2e-tests/util/setup.ts`).
- `web-console/package.json`: Refactor run scripts to add new script for
running end-to-end tests.
- `web-console/script/druid`: Utility scripts for building, starting,
and stopping druid.
Other changes:
- `pom.xml`: Refactor various settings disable java static checks and to
disable java tests into two new maven profiles. Since the same
settings are used in several places (e.g., .travis.yml, Dockerfiles,
etc.), having them in maven profiles makes it more maintainable.
- `web-console/src/console-application.tsx`: Fix typo ("the the").
It is currently not possible to unset the druid.host property in the docker image to let Druid default to the canonical hostname. It always gets set to the container's IP address. Passing the override environment variable druid_host= unfortunately does not solve the problem, as this gets interpreted as empty string and does not let the default kick in.
This change adds the option to pass DRUID_SET_HOST=0 as environment variable to disable the default behavior, and allows passing a common runtime.properties file without druid.host.
* switch builder image to smaller "slim" debian base
* use built-in pyyaml
* skip various build checks to speed up compilation
* create user/group in target image directly
* Upgrade various build and doc links to https.
Where it wasn't possible to upgrade build-time dependencies to https,
I kept http in place but used hardcoded checksums or GPG keys to ensure
that artifacts fetched over http are verified properly.
* Switch to https://apache.org.
* Remove mysql database connector from container image
See discussion on `dev` mailing list from 2019-03
around [docker
build](http://mail-archives.apache.org/mod_mbox/druid-dev/201903.mbox/browser)
Remove the mysql database connector from the container
image in favour of solely the postgresql connector.
Instructions are present in distribution/docker/README.md
for someone to re-add if they need.
Signed-off-by: Don Bowman <don@agilicus.com>
* Add sample Dockerfile for mysql using pre-built upstream
* Add docker container for druid
This container is an 'omnibus' (since there is such a high
overlap with the various services). It includes all contrib
extension as well as the msql connector.
It is intended to be run as `docker run NAME SERVICE`
(e.g. docker run druid:latest broker)
* Add Apache license header
* Resolve issues from Pull Request review
* Add comments at top of script per PR comments
* Revert BUILDKIT. Not available everywhere.
* Don't set hostname, allow default (IP)
Some environments (e.g. Kubernetes Deployments) don't resolve
hostname to IP.
* Switch to amd64 glibc-based busybox from 32-bit uclibc
* Override service-specific configuration
* Replace MAINTAINER w/ LABEL
* Add mysql connector to application classpath
This works around issue #3770https://github.com/apache/incubator-druid/issues/3770
* Add docker-compose and sample environment
Signed-off-by: Don Bowman <don@agilicus.com>
Torsten Hain
Chi Cao Minh
Clint Wylie
Yamada Koji
Jonathan Wei
Xavier Léauté
ilhanadiyaman
Gian Merlino
Don Bowman
Clint Wylie