Apache
/
druid
mirror of https://github.com/apache/druid.git
1
0
Fork 0
Code Issues Packages Projects Releases Wiki Activity
druid/distribution/docker
History
Dr. Sizzles e5ad24ff9f
Support for middle manager less druid, tasks launch as k8s jobs (#13156) 
* Support for middle manager less druid, tasks launch as k8s jobs

* Fixing forking task runner test

* Test cleanup, dependency cleanup, intellij inspections cleanup

* Changes per PR review

Add configuration option to disable http/https proxy for the k8s client
Update the docs to provide more detail about sidecar support

* Removing un-needed log lines

* Small changes per PR review

* Upon task completion we callback to the overlord to update the status / locaiton, for slower k8s clusters, this reduces locking time significantly

* Merge conflict fix

* Fixing tests and docs

* update tiny-cluster.yaml 

changed `enableTaskLevelLogPush` to `encapsulatedTask`

* Apply suggestions from code review

Co-authored-by: Abhishek Agarwal <1477457+abhishekagarwal87@users.noreply.github.com>

* Minor changes per PR request

* Cleanup, adding test to AbstractTask

* Add comment in peon.sh

* Bumping code coverage

* More tests to make code coverage happy

* Doh a duplicate dependnecy

* Integration test setup is weird for k8s, will do this in a different PR

* Reverting back all integration test changes, will do in anotbher PR

* use StringUtils.base64 instead of Base64

* Jdk is nasty, if i compress in jdk 11 in jdk 17 the decompressed result is different

Co-authored-by: Rahul Gidwani <r_gidwani@apple.com>
Co-authored-by: Abhishek Agarwal <1477457+abhishekagarwal87@users.noreply.github.com>
 		2022-11-02 19:44:47 -07:00
..
Dockerfile Support for middle manager less druid, tasks launch as k8s jobs (#13156) 2022-11-02 19:44:47 -07:00
Dockerfile.mariadb support using mariadb connector with mysql extensions (#11402) 2021-07-08 12:25:37 -07:00
Dockerfile.mysql deps: upgrade mysql-connector-java to v5.1.49 (#12704) 2022-06-29 23:15:46 +08:00
DockerfileBuildTarAdvanced Consolidate the number of Dockerfiles (#11187) 2021-05-07 10:41:34 -07:00
README.md Some improvements about Docker (#13059) 2022-09-16 09:25:52 +08:00
docker-compose.yml Fix version in master (#12644) 2022-06-14 11:32:46 +05:30
druid.sh Some improvements about Docker (#13059) 2022-09-16 09:25:52 +08:00
environment Enable msq for docker by default (#13069) 2022-09-11 21:00:32 +05:30
peon.sh Support for middle manager less druid, tasks launch as k8s jobs (#13156) 2022-11-02 19:44:47 -07:00

README.md

Build

From the root of the repo, run following command:

DOCKER_BUILDKIT=1 docker build -t apache/druid:tag -f distribution/docker/Dockerfile .

Building images on Apple M1/M2

To build images on Apple M1/M2, you need to follow the instructions in this section.

  1. build Druid distribution from the root of the repo 
    mvn clean package -DskipTests -Pdist
  2. build target image 
    DOCKER_BUILDKIT=1 docker build -t apache/druid:tag -f distribution/docker/Dockerfile --build-arg BUILD_FROM_SOURCE=false .

Run

  1. Edit distribution/docker/docker-compose.yml file to change the tag of Druid's images to the tag that's used in the 'Build' phase above.
  2. Edit environment file to suite if necessary.
  3. Run: 
    docker-compose -f distribution/docker/docker-compose.yml up

MySQL Database Connector

This image contains solely the postgres metadata storage connector. If you need the mysql metadata storage connector, you can use Dockerfile.mysql to add it to the base image above.

docker build -t apache/druid:tag-mysql --build-arg DRUID_RELEASE=apache/druid:tag -f distribution/docker/Dockerfile.mysql .

where druid:tag is the version of Druid image to use as the base.