Commit Graph

32 Commits

Author SHA1 Message Date
Maytas Monsereenusorn dd7a32ad48
Fix ITSqlInputSourceTest (#10194)
* Fix ITSqlInputSourceTest.java

* Fix ITSqlInputSourceTest.java

* Fix ITSqlInputSourceTest.java

* fix

* fix

* fix

* fix

* fix

* fix

* fix

* fix
2020-07-21 09:52:13 -07:00
Maytas Monsereenusorn 4e8570b71b
Add integration tests for all InputFormat (#10088)
* Add integration tests for Avro OCF InputFormat

* Add integration tests for Avro OCF InputFormat

* add tests

* fix bug

* fix bug

* fix failing tests

* add comments

* address comments

* address comments

* address comments

* fix test data

* reduce resource needed for IT

* remove bug fix

* fix checkstyle

* add bug fix
2020-07-08 12:50:29 -07:00
Maytas Monsereenusorn 859ff6e9c0
Reduce memory footprint of integration test by not starting unneeded containers (#10150)
* Reduce memory footprint of integration test

* fix README

* fix README

* fix error in script

* fix security IT
2020-07-08 09:46:18 -07:00
agricenko cad9eea15d
Integration test docker compose readme (#10016)
* Integration Tests. Docker-compose readme part

* Readme updates. PR fixes

Co-authored-by: agritsenko <agritsenko@provectus.com>
2020-06-15 14:48:34 -10:00
agricenko 56a9cad532
Integration Tests. (#9854)
* Integration Tests.
Added docker-compose with druid-cluster configuration.
Refactored shell scripts. split code in a few files

* Integration Tests.
Added environment variable: DRUID_INTEGRATION_TEST_GROUP

* Integration Tests. Removed nit

* Integration Tests. Updated if block in docker_run_cluster.sh.

* Integration Tests. Readme. Added Docker-compose section.

* Integration Tests. removed yml files for s3, gcs, azure.
Renamed variables for skip start/stop/build docker.
Updated readme.
Rollback maven profile: int-tests-config-file

* Integration Tests. Removed docker-compose.test-env.yml file.
Added DRUID_INTEGRATION_TEST_GROUP variable to docker-compose.yml

* Integration Tests. Readme. Added details about docker-compose

* Integration Tests. cleanup shell scripts

Co-authored-by: agritsenko <agritsenko@provectus.com>
2020-06-02 09:38:53 -07:00
Jonathan Wei 61295bd002
More Hadoop integration tests (#9714)
* More Hadoop integration tests

* Add missing s3 instructions

* Address PR comments

* Address PR comments

* PR comments

* Fix typo
2020-04-30 14:33:01 -07:00
Jihoon Son 39722bd064
Integration tests for stream ingestion with various data formats (#9783)
* Integration tests for stream ingestion with various data formats

* fix npe

* better logging; fix tsv

* fix tsv

* exclude kinesis from travis

* some readme
2020-04-29 13:18:01 -07:00
Maytas Monsereenusorn 16f5ae4405
Add integration tests for kafka ingestion (#9724)
* add kafka admin and kafka writer

* refactor kinesis IT

* fix typo refactor

* parallel

* parallel

* parallel

* parallel works now

* add kafka it

* add doc to readme

* fix tests

* fix failing test

* test

* test

* test

* test

* address comments

* addressed comments
2020-04-22 10:43:34 -07:00
Maytas Monsereenusorn d930f04e6a
Test file format extensions for inputSource (orc, parquet) (#9632)
* Test file format extensions for inputSource (orc, parquet)

* Test file format extensions for inputSource (orc, parquet)

* fix path

* resolve merge conflict

* fix typo
2020-04-13 13:03:56 -07:00
Aleksei Chumagin 79522f3e25
Integration-tests: typo (#9624)
* QA-57: change $ to # as comment

* QA-57: fix haddop to hadoop
2020-04-06 17:40:05 -07:00
Maytas Monsereenusorn 1852bf33ea
Add Integration Test for functionality of kinesis ingestion (#9576)
* kinesis IT

* Kinesis IT

* Kinesis IT

* Kinesis IT

* Kinesis IT

* Kinesis IT

* Kinesis IT

* Kinesis IT

* Kinesis IT

* Kinesis IT

* Kinesis IT

* Kinesis IT

* Kinesis IT

* Kinesis IT

* Kinesis IT

* fix kinesis timeout

* Kinesis IT

* Kinesis IT

* fix checkstyle

* Kinesis IT

* address comments

* fix checkstyle
2020-04-03 09:45:22 -07:00
Suneet Saldanha e6e2836b0e
Instructions to run integration tests against quickstart (#9560)
* Instructions to run integration tests against quickstart

* Address review comments

* actually exclude the test group

* Revert "actually exclude the test group"

This reverts commit 66f366409e.

* update comment
2020-03-26 13:22:53 -07:00
Suneet Saldanha 55c08e0746
DruidSegmentReader should work if timestamp is specified as a dimension (#9530)
* DruidSegmentReader should work if timestamp is specified as a dimension

* Add integration tests

Tests for compaction and re-indexing a datasource with the timestamp column

* Instructions to run integration tests against quickstart

* address pr
2020-03-25 13:47:34 -07:00
Maytas Monsereenusorn 5f127a1829
Add integration tests for HDFS (#9542)
* HDFS IT

* HDFS IT

* HDFS IT

* fix checkstyle
2020-03-20 15:46:08 -07:00
Maytas Monsereenusorn 4c620b8f1c
Adding s3, gcs, azure integration tests (#9501)
* exclude pulling s3 segments for tests that doesnt need it

* fix script

* fix script

* fix script

* add s3 test

* refactor sample data script

* add tests

* add tests

* add license header

* fix failing tests

* change bucket and path to config

* update integration test readme

* fix typo
2020-03-17 03:08:44 -07:00
sthetland 6d52edddab
Remove references to Docker Machine (#9366)
* Remove references to Docker Machine

Removing a broken link to an obsolete repo. 

While at it, removing references to Docker Machine, which was obsolete as of Docker v1.12 (avail. 2016). This version introduced Docker as native MacOS and Windows apps.

* Update README.md

Wording nit.
2020-02-15 03:08:43 -08:00
Suneet Saldanha 176bc8fd97 Remove resolve-ip dependency for integration-tests (#9065)
* Remove resolve-ip dependency for integration-tests

* use host hostname and fallback to dscacheutil

* better shell script comparisons
2019-12-19 14:53:36 -08:00
Chi Cao Minh 05b44e3467 Speedup Travis CI jobs (#8240)
Reorganize Travis CI jobs into smaller faster (and more) jobs. Add
various maven options to skip unnecessary work and refactored Travis CI
job definitions to follow DRY.

Detailed changes:

.travis.yml
- Refactor build logic to get rid of copy-and-paste logic
- Skip static checks and enable parallelism for maven install
- Split static analysis into different jobs to ease triage
- Use "name" attribute instead of NAME environment variable
- Split "indexing" and "web console" out of "other modules test"
- Split 2 integration test jobs into multiple smaller jobs

build.sh
- Enable parallelism
- Disable more static checks

travis_script_integration.sh
travis_script_integration_part2.sh
integration-tests/README.md
- Use TestNG groups instead of shell scripts and move definition of jobs
  into Travis CI yaml

integration-tests/pom.xml
- Show elapsed time of individual tests to aid in future rebalancing of
  Travis CI integration test jobs run time

TestNGGroup.java
- Use TestNG groups to make it easy to have multiple Travis CI
  integration test jobs. TestNG groups also make it easier to have an
  "other" integration test group and make it less likely a test will
  accidentally not be included in a CI job.

IT*Test.java
AbstractITBatchIndexTest.java
AbstractKafkaIndexerTest.java
- Add TestNG group
- Fix various IntelliJ inspection warnings
- Reduce scope of helper methods since the TestNG group annotation on
  the class makes TestNG consider all public methods as test methods

pom.xml
- Allow enforce plugin to be run from command-line
- Bump resources plugin version so that "[debug] execute contextualize"
  output is correctly suppressed by "mvn -q"
- Bump exec plugin version so that skip property is renamed from "skip"
  to "exec.skip"

web-console/pom.xml
- Add property to allow disabling javascript-related work. This property
  is overridden in Travis CI to speed up the jobs.
2019-08-07 09:52:42 -07:00
Roman Leventov f2e4e654b2 Add instruction about skipping up-to-date checks when running integration tests (#7843) 2019-07-08 13:44:32 +05:30
Surekha 6991735f73 Fix and add sys IT tests to travis script (#7208)
* Add sys IT tests to travis script

* minor fixes

* Modify the test queries

* modify query
2019-03-08 16:40:59 -08:00
David Lim afb239b17a add missing license headers, in particular to MD files; clean up RAT … (#6563)
* add missing license headers, in particular to MD files; clean up RAT exclusions

* revert inadvertent doc changes

* docs

* cr changes

* fix modified druid-production.svg
2018-11-13 09:38:37 -08:00
Jonathan Wei 6a909c85d0
Fix TLS IT for docker-machine (#6379) 2018-09-26 12:59:05 -07:00
Gian Merlino 431d3d8497
Rename io.druid to org.apache.druid. (#6266)
* Rename io.druid to org.apache.druid.

* Fix META-INF files and remove some benchmark results.

* MonitorsConfig update for metrics package migration.

* Reorder some dimensions in inner queries for some reason.

* Fix protobuf tests.
2018-08-30 09:56:26 -07:00
Kirill Kozlov 23b0a091b6 Local docker for running integration tests (#5314) 2018-02-07 13:06:06 -08:00
Kirill Kozlov 2c9a881dec Remove TODO implemented in pull/5165 (#5262) 2018-01-18 00:05:53 +05:30
Nishant 8ea5f9324d Integration Tests - fix middlemanager property name in doc (#3586) 2016-10-18 08:23:34 -05:00
Robin e7a7ecd65d add test for batch indexing from hadoop 2016-03-14 20:02:30 -05:00
Robin 19774b5e9a integration-tests/README: fix link to docker-install 2016-02-17 11:21:14 -06:00
Xavier Léauté bd7d2c9b3a update integration-test docker instructions for docker-machine 2015-11-12 16:44:28 -08:00
sahner df5bc8974f add ability to get configuration information from a file 2015-07-17 10:19:42 -05:00
Charles Allen 80a18243dd Minor change to integration-tests README formatting 2015-07-14 17:01:06 -07:00
nishantmonu51 f69e0591b0 working integration tests 2015-01-07 14:23:38 -08:00