fd0931d35e
Azure data lake input source ( #11153 )
...
* Mention Azure Data Lake
* Make consistent with other entries
Co-authored-by: Charles Smith <38529548+techdocsmith@users.noreply.github.com>
2021-06-25 15:54:34 -07:00
ed0a57e106
Update kafka-ingestion.md to clarify PasswordProvider support limitation ( #11374 )
...
Co-authored-by: Clint Wylie <cjwylie@gmail.com>
Co-authored-by: Clint Wylie <cjwylie@gmail.com>
2021-06-24 21:54:48 -07:00
de8daf8139
Delete buildV9Directly in Kafka and Kinesis Indexing Service ( #11351 )
...
* delete_buildV9Directly_in_kafka_and_kinesis_indexing_service
* delete
* delete them from server
* delete buildV9Directly from hadoop indexing
* bug fixed
Co-authored-by: yuanyi <yuanyi@freewheel.tv>
2021-06-23 16:36:46 -07:00
911a0c6c8c
Fix bug in auto compaction needsCompaction method that can skip segments incorrectly ( #11366 )
...
* fix bug in needsCompaction
* fix bug in needsCompaction
2021-06-23 13:03:41 -07:00
267c298293
define superuser permissions set in druid-server instead of druid-basic-auth extension ( #11376 )
2021-06-22 16:03:58 -07:00
c8b3f8cc00
avoid logging pause message multiple times ( #11375 )
...
In some instances the ingestion thread could be woken up spuriously,
resulting in the "Received pause command..." log message getting logged
multiple times. This change ensures we only log it once the first time
the pause is requested.
2021-06-22 09:30:38 -07:00
f0b105ec63
Temporarily skip compaction for locked intervals ( #11190 )
...
* Add overlord API /lockedIntervals. Skip compaction for locked intervals
* Revert formatting changes
* Add license info
* Fix checkstyle
* Remove invalid API invocation
* Fix checkstyle
* Add DatasourceIntervalsTest
* Fix checkstyle
* Remove LockedIntervalsResponse
* Add integration tests for lockedIntervals
* Add ITAutoCompactionLockContentionTest
* Add config druid.coordinator.compaction.skipLockedIntervals
* Add test for TaskQueue
2021-06-20 17:21:59 -07:00
f56a5b9ba2
Web console: Better hotkeys and library upgrades ( #11365 )
...
* improve hotkeys
* fix test name
* refactor explain dialog
* explain tests
* small fixes
* update to popover2
* use resize sensor 2
* misc cleanup
2021-06-17 18:24:29 -07:00
2e98d3c1ad
Web console: Make segment timeline work over all time intervals ( #11359 )
...
* tidy up
* add to segments view
* add unit tests for date
* better util export
* fix ds view
* fix tests
* fix test in time
* unset untermediate state
2021-06-17 16:10:06 -07:00
a1c20d7457
update jackson dependencies to use bom ( #11353 )
...
Switching to the bom dependency declaration simplifies managing jackson
dependencies. It also removes the need to override individual library
versions for CVE fixes, since the bom takes care of that internally.
This change aligns our jackson dependency versions on 2.10.5(.x):
- updates jackson libraries from 2.10.2 to 2.10.5
- jackson-databind remains at 2.10.5.1 as defined in the bom
Release notes: https://github.com/FasterXML/jackson/wiki/Jackson-Release-2.10
2021-06-16 18:37:30 -07:00
712f2a5d00
upgrade error-prone to 2.7.1 and support checks with Java 11+ ( #11363 )
...
* upgrade error-prone to 2.7.1 and support checks with Java 11+
- upgrade error-prone to 2.7.1
- support running error-prone with Java 11 and above using -Xplugin
instead of custom compiler
- add compiler arguments to ignore warnings/errors in Java 15/16
- introduce strictCompile property to enable strict profiles since we
now need multiple strict profiles for Java 8
- properly exclude all generated source files from error-prone
- fix druid-processing overriding annotation processors from parent pom
- fix druid-core disabling most non-default checks
- align plugin and annotation errorprone versions
- fix / suppress additional issues found by error-prone:
* fix bug in SeekableStreamSupervisor initializing ArrayList size with
the taskGroupdId
* fix missing @Override annotations
- remove outdated compiler plugin in benchmarks
- remove deleted ParameterPackage error-prone rule
- re-enable checks on benchmark module as well
* fix IntelliJ inspections
* disable LongFloatConversion due to bug in error-prone with JDK 8
* add comment about InsecureCrypto
2021-06-16 12:55:34 -07:00
bfbd7ec432
fix a bugs related to SQL type inference return type nullability ( #11327 )
...
* fix a bunch of type inference nullability bugs
* fixes
* style
* fix test
* fix concat
2021-06-15 12:26:59 -07:00
920aa414ca
enrich expression cache key information to support expressions which depend on external state ( #11358 )
...
* enrich expression cache key information to support expressions which depend on external state such as lookups
* cache rules everything around me
* low carb
* rename
2021-06-14 17:26:43 -07:00
6edc017fad
update jacoco for Java 15/16 support ( #11361 )
2021-06-12 13:57:41 +08:00
50327b8f63
ignore bySegment query context for SQL queries ( #11352 )
...
* ignore bySegment query context for SQL queries
* revert unintended change
2021-06-11 13:49:03 -07:00
a1ed3a407d
clarify bySegment is native only ( #11331 )
2021-06-11 13:48:17 -07:00
ac9b1f81b4
Web console: Remove support for IE11 and other older browsers ( #11357 )
...
* Use common browserlist and update to drop IE11
* Change TypeScript target to ES2016
* Update browserslist for "supports es6" support
* Show a warning if accessed from an unsupported browser
* Inline browser-update styles; detect SyntaxErrors too
* Better wording
* Upgrade to the latest Blueprint
* Refactor RunButton to be FC, use useHotkeys
* Remove dead license
* Update snapshots
* Address feedback
* Wording
Co-authored-by: Vadim Ogievetsky <vadimon@gmail.com>
Co-authored-by: Vadim Ogievetsky <vadimon@gmail.com>
2021-06-10 19:05:40 -07:00
6b272c857f
adjust topn heap algorithm to only use known cardinality path when dictionary is unique ( #11186 )
...
* adjust topn heap algorithm to only use known cardinality path when dictionary is unique
* better check and add comment
* adjust comment more
2021-06-10 18:32:22 -05:00
51f983101e
Fix wrong encoding in PredicateFilteredDimensionSelector.getRow ( #11339 )
2021-06-10 09:14:34 -07:00
167044f715
Bump fastutil from 8.2.3 to 8.5.4 ( #11347 )
...
* Bump fastutil from 8.2.3 to 8.5.4
Bumps [fastutil](https://github.com/vigna/fastutil ) from 8.2.3 to 8.5.4.
- [Release notes](https://github.com/vigna/fastutil/releases )
- [Changelog](https://github.com/vigna/fastutil/blob/master/CHANGES )
- [Commits](https://github.com/vigna/fastutil/compare/8.2.3...8.5.4 )
---
updated-dependencies:
- dependency-name: it.unimi.dsi:fastutil
dependency-type: direct:production
update-type: version-update:semver-minor
...
Signed-off-by: dependabot[bot] <support@github.com>
* update licenses.yaml
* update maven dependency list for -core and -extra libraries to pass maven dependency checks
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: Xavier Léauté <xvrl@apache.org>
2021-06-10 07:43:18 -07:00
db5ee83ca6
pin ZooKeeper dependencies to 3.5.x ( #11354 )
2021-06-10 07:41:53 -07:00
8de0d36c52
Allow query through router when load moving average extension ( #11276 )
...
* init commit
* change NoopQuerySegmentWalker name
* change doc
* move NoopQuerySegmentWalker and add document
* fix doc
Co-authored-by: yuanyi <yuanyi@freewheel.tv>
2021-06-10 18:46:53 +08:00
9047fa3d9c
S3 ingestion can assume role ( #10995 )
...
* feature s3 assume role
* feature s3 assume role
* feature s3 assume role
* feature s3 assume role
* feature s3 assume role
* feature s3 assume role
* tests fix
* spelling fix
* sts fix
Co-authored-by: egor-ryashin <egor.ryashin@rilldata.com>
2021-06-09 16:02:35 +05:30
651810f9e5
remove DEPRECATION part ( #11326 )
2021-06-09 15:52:43 +08:00
1a27ed0b86
Bump junit from 4.12 to 4.13.2 ( #11344 )
...
Bumps [junit](https://github.com/junit-team/junit4 ) from 4.12 to 4.13.2.
- [Release notes](https://github.com/junit-team/junit4/releases )
- [Changelog](https://github.com/junit-team/junit4/blob/main/doc/ReleaseNotes4.12.md )
- [Commits](https://github.com/junit-team/junit4/compare/r4.12...r4.13.2 )
---
updated-dependencies:
- dependency-name: junit:junit
dependency-type: direct:production
update-type: version-update:semver-minor
...
Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2021-06-08 18:18:39 -07:00
8f4662d05a
restrict travis builds to main and release branches ( #11349 )
2021-06-08 16:34:17 -07:00
be10a236d5
Bump commons-io from 2.6 to 2.9.0 ( #11338 )
...
* Bump commons-io from 2.6 to 2.9.0
Bumps commons-io from 2.6 to 2.9.0.
---
updated-dependencies:
- dependency-name: commons-io:commons-io
dependency-type: direct:production
update-type: version-update:semver-minor
...
Signed-off-by: dependabot[bot] <support@github.com>
* update licenses.yaml to reflect version bumps
* fix tests relying on specific log messages
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: Xavier Léauté <xvrl@apache.org>
2021-06-08 10:02:59 -07:00
145cf9e5c3
fix document about input format ( #11342 )
...
Co-authored-by: yuanyi <yuanyi@freewheel.tv>
2021-06-08 23:44:54 +08:00
c82ede9afc
Bump maven-antrun-plugin from 1.8 to 3.0.0 ( #11336 )
...
Bumps [maven-antrun-plugin](https://github.com/apache/maven-antrun-plugin ) from 1.8 to 3.0.0.
- [Release notes](https://github.com/apache/maven-antrun-plugin/releases )
- [Commits](https://github.com/apache/maven-antrun-plugin/compare/maven-antrun-plugin-1.8...maven-antrun-plugin-3.0.0 )
---
updated-dependencies:
- dependency-name: org.apache.maven.plugins:maven-antrun-plugin
dependency-type: direct:production
update-type: version-update:semver-major
...
Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2021-06-08 08:28:11 -07:00
2ee7e31e5b
Fix syntax error ( #11332 )
2021-06-07 22:35:02 -07:00
1e8b5360b3
Bump docker-java-transport-netty from 3.2.0 to 3.2.8 ( #11337 )
...
Bumps [docker-java-transport-netty](https://github.com/docker-java/docker-java ) from 3.2.0 to 3.2.8.
- [Release notes](https://github.com/docker-java/docker-java/releases )
- [Changelog](https://github.com/docker-java/docker-java/blob/master/CHANGELOG.md )
- [Commits](https://github.com/docker-java/docker-java/compare/3.2.0...3.2.8 )
---
updated-dependencies:
- dependency-name: com.github.docker-java:docker-java-transport-netty
dependency-type: direct:production
update-type: version-update:semver-patch
...
Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2021-06-07 18:58:38 -07:00
dbec8da7ea
enable dependabot for dependency updates ( #11079 )
...
* enable dependabot for dependency updates
* exclude dependabot config from license checks
2021-06-07 13:56:32 -07:00
7ffe402b87
Fix priority parsing in HiLoQueryLaningStrategy ( #11302 )
2021-06-07 15:27:27 +05:30
44d629319d
handle timestamps of complex types when parsing protobuf messages ( #11293 )
...
* handle timestamps correctly when parsing protobuf
* Add timestamp handling to ProtobufReader
* disable checkstyle for generated sourcecode
* Fix test
* try this
* refactor tests
2021-06-07 15:19:39 +05:30
0c5d1c9725
Web console: add more query fixing auto suggestions ( #11203 )
...
* add more query fixing auto suggestions
* update query gen
* update toolkit
* update licenses
* fix funky quotes
* funky => fancy
* revert engine change
* separate web-console and website npm and node deps
2021-06-04 09:29:00 -07:00
d5139c9543
Fix permission problems in docker ( #11299 )
...
* Create /opt/data to fix permission problem
* eliminate symlink to avoid compatibility problem on AWS Fargate
* Add a workaround section
* Update instruction for named volume
* Use named volume in docker-compose
* Revert some doc change
* Resolve review comments
2021-06-01 17:33:27 -07:00
27f1b6cbf3
Fix Index hadoop failing with index.zip is not a valid DFS filename ( #11316 )
...
* * Fix bug
* * simplify class loading
* * fix example configs for integration tests
* Small classloader cleanup
Co-authored-by: jon-wei <jon.wei@imply.io>
2021-06-01 19:14:50 -04:00
a24817d20a
Migrate helm chart from helm v2 to v3 ( #11290 )
...
* move all files from helm to helm/druid
* migrate druid helm chart from helm v2 to v3
* add license headers
* bump version to 0.3.0
2021-05-28 13:46:02 +08:00
e664bfd433
Improve doc of movingAverage ( #11262 )
...
* Make doc more directive
Signed-off-by: frank chen <frank.chen021@outlook.com>
* Add limitation
Signed-off-by: frank chen <frank.chen021@outlook.com>
* Suppress spelling check error
2021-05-28 13:10:55 +08:00
04fefb0ca3
Fix ClassCastException ( #11266 )
...
Signed-off-by: frank chen <frank.chen021@outlook.com>
2021-05-27 21:25:51 -07:00
31c811d894
Web console: Fix maxRowsPerSegment validation in hashed compaction spec ( #11308 )
...
* allow defining of maxRowsPerSegment for now
* use common util
* update snapshots
* fix test
* fix e2e test
2021-05-27 16:36:42 -07:00
e5633d7842
Fix bug: 502 bad gateway thrown when we edit/delete any auto compaction config created 0.21.0 or before ( #11311 )
...
* fix bug
* add test
* fix IT
* fix checkstyle
* address comments
2021-05-27 16:34:32 -07:00
60843bd11f
Add configuration suggestion to `druid.indexer.storage.type` ( #11304 )
2021-05-27 06:44:47 -07:00
7a2bcb3ca1
chill, travis ( #11283 )
...
* maybe make ci more selective in what it do
* maybe this
* skip self and travis config
* can it be in before_script? lets find out
* nope
* fixes
* oops
* only skip stuff for PR builds, add some tests
* revert unintended change
* more comments, more tests, more better
2021-05-27 05:40:40 -07:00
6c7177714f
Add test for join on __time column ( #11289 )
2021-05-26 22:20:39 -07:00
b517c3339b
remove ZooKeeper 3.4 support + pass tests with Java 15 ( #11073 )
...
With this change, Druid will only support ZooKeeper 3.5.x and later.
In order to support Java 15 we need to switch to ZK 3.5.x client libraries and drop support for ZK 3.4.x
(see #10780 for the detailed reasons)
* remove ZooKeeper 3.4.x compatibility
* exclude additional ZK 3.5.x netty dependencies to ensure we use our version
* keep ZooKeeper version used for integration tests in sync with client library version
* remove the need to specify ZK version at runtime for docker
* add support to run integration tests with JDK 15
* build and run unit tests with Java 15 in travis
2021-05-25 12:49:49 -07:00
82380b67e0
Improve IT job 79 ITNestedQueryPushDownTest integration test ( #11268 )
...
* improve occasional failure caused by resource competition
* adjust more configs in tiny-cluters.yaml
Co-authored-by: yfhanfei <yfhanfei@ZBMac-C02DW5SMMD6P.local>
2021-05-24 10:12:34 +08:00
2bfcee5824
Fix issue with empty array converting to string expression instead of string array ( #11270 )
2021-05-22 09:31:28 +08:00
4ba5738ffb
Add an issues section to deal with common issues when building druid ( #11271 )
2021-05-21 09:04:51 -07:00
403dcf5cfb
fixes some typos, edits for style ( #11258 )
2021-05-21 08:58:39 -07:00
sthetland
Hoseung Lee
Yi Yuan
Maytas Monsereenusorn
Clint Wylie
Xavier Léauté
Kashif Faraz
Vadim Ogievetsky
Charles Smith
John Gozde
Jihoon Son
dependabot[bot]
Egor Riashin
Oops
frank chen
Abhishek Agarwal
zachjsh
Hellis
Rohan Garg
fhan
Agustin Gonzalez