7e9522870f
introduce DynamicConfigProvider interface and make kafka consumer props extensible ( #10309 )
...
* introduce DynamicConfigProvider interface and make kafka consumer props extensible
* fix intellij inspection error
* make DynamicConfigProvider generic
Change-Id: I2e3e89f8617b6fe7fc96859deca4011f609dc5a3
* deprecate PasswordProvider
2020-12-02 16:38:27 -08:00
f965464f36
Fix empty directory handling ( #10319 )
...
Co-authored-by: Atul Mohan <atulmohan@yahoo-inc.com>
2020-12-02 10:37:08 -08:00
2e02eebd9d
Add context dimension to DefaultQueryMetrics ( #10578 )
...
* Add context dimension to DefaultQueryMetrics
* remove redundant addition of context dimension from DruidMetrics now that QueryMetrics adds it by default
* update SearchQueryMetrics to reflect the same pattern as other default dimensions in QueryMetrics
* add PublicApi annotation for context in QueryMetrics Interface
2020-12-01 18:34:03 -08:00
e7e07eab11
[Improve Doc] : Modify the disadvantages of the lazyLoadOnStart feature. ( #10608 )
...
* modify docs
* modify docs
Co-authored-by: yuezhang <yuezhang@freewheel.tv>
2020-12-01 18:33:22 -08:00
24f1e35b5d
fix desc of 'required' for granularity property ( #10616 )
2020-12-01 18:29:51 -08:00
5b06c7a3a9
Web console: improve how code is imported, use API instance ( #10597 )
...
* fix imports
* clean up imports
* update DQT to fix escaping
2020-12-01 13:16:14 -08:00
d47d6cf081
Add time-to-first-result benchmark for groupBy ( #10612 )
2020-12-01 10:32:37 -08:00
2560bf0a19
Add new coordinator metrics for coordinator duty runtimes ( #10603 )
...
* Add new coordinator metrics for duty runtimes
* fix spelling for a constant variable value
* add comment clarifying why the global runtime metric is emitted where it is
* Remove duty alias in lieu of using the class name for metrics
* fix docs
* CoordinatorStats tests + add duty stats to accumulate() logic
2020-11-29 14:47:35 -08:00
30bcb0fd74
DataSourcesSnapshotBenchmark to measure iterateAllUsedSegmentsInSnapshot perf ( #10604 )
2020-11-29 14:42:14 -08:00
7462b0b953
Allow missing intervals for Parallel task with hash/range partitioning ( #10592 )
...
* Allow missing intervals for Parallel task
* fix row filter
* fix tests
* fix log
2020-11-25 14:50:22 -08:00
d0c2ede50c
Added CronScheduler support as a proof to clock drift while emitting metrics ( #10448 )
...
Co-authored-by: Ayush Kulshrestha <ayush.kulshrestha@miqdigital.com>
2020-11-25 12:31:38 +01:00
fe693a4f01
Improve doc and exception message for invalid user configurations ( #10598 )
...
* improve doc and exception message
* add spelling check rules and remove unused import
* add a test to improve test coverage
2020-11-23 15:03:13 -08:00
31740b3b29
Fix : Druid throws java.util.concurrent.RejectedExecutionException when ingest task is stopping. ( #10555 )
...
* check exec status before return Signal
* add more log
* change log level to debug and add UT
* change log leverl to warn and merge master
Co-authored-by: yuezhang <yuezhang@freewheel.tv>
2020-11-23 14:52:03 -08:00
4537016cad
Update google client libraries ( #10536 )
...
modify license.yaml
Update google oauth client version
2020-11-20 15:23:30 -08:00
111b431c07
Introduce query/timeout/count metric ( #10567 )
...
* Add timeout metric
* Add tests
2020-11-20 15:17:26 -08:00
3cafd531de
fix issue causing incorrect config in Docker ( #10595 )
...
Previously, when the Docker entrypoint script generated the config
files, it would append the configuration without including a newline.
This could result in incorrect configuration. This has been fixed by
always appending a newline before any configuration.
Co-authored-by: Bryson Chen <brysonjackychen@gmail.com>
Co-authored-by: Bryson Chen <brysonjackychen@gmail.com>
2020-11-20 14:52:38 -08:00
2201ffa2f0
druid-docker-image: add DRUID_DIRS_TO_CREATE variable to customize directories created on startup ( #10591 )
...
* druid-docker-image: add DRUID_DIRS_TO_CREATE variable to customize directories created on startup
* address review comment
* remove unintentional change
2020-11-20 14:46:19 -08:00
ba915b7f56
Security overview documentation ( #10339 )
...
* initial file
* initial file
* security overview added
* ldap added
* spacing adjustments
* nits
* security graphics and doc review
* Update docs/operations/security-overview.md
Co-authored-by: Jonathan Wei <jon-wei@users.noreply.github.com>
* Update docs/operations/security-user-auth.md
Co-authored-by: Jonathan Wei <jon-wei@users.noreply.github.com>
* Update docs/operations/security-overview.md
Co-authored-by: Jonathan Wei <jon-wei@users.noreply.github.com>
* Update docs/operations/security-overview.md
Co-authored-by: Jonathan Wei <jon-wei@users.noreply.github.com>
* updates frm review
* review comments
* finish up review and light edits
* broken links
* spell check
Co-authored-by: Jonathan Wei <jon-wei@users.noreply.github.com>
2020-11-19 15:24:58 -08:00
2f4d6da33f
Updates segment metadata query documentation ( #10589 )
...
* updates segment metadata query documentation to be clearer about cardinality estimation
* typo in documentation
2020-11-20 00:08:27 +05:30
abb03d38b0
double confirm before delete all the ununsed segmensts ( #10581 )
...
Co-authored-by: yuezhang <yuezhang@freewheel.tv>
2020-11-19 10:07:52 -06:00
d7d2c804ad
Add zero period support to TIMESTAMPADD ( #10550 )
...
* Allow zero period for TIMESTAMPADD
* update test cases
* add empty zone test case
* add unit test cases for TimestampShiftMacro
2020-11-18 18:26:53 -08:00
1272fb17e5
modify druid.historical.cache.maxEntrySize property in Unified format ( #10590 )
...
Co-authored-by: yuezhang <yuezhang@freewheel.tv>
2020-11-17 16:36:50 -06:00
9964dd4cb2
Web console: fix data loader schema table column ordering bug and other polish ( #10588 )
...
* remove unused fields
* keep tables live
* advanced
* fix schema view
* better indication
* tests pass
* Show more instead of show advanced
* fix tests
* extract dynamic configs
* update snapshots
* fix issues
* update snapshot
* reword without >
2020-11-17 13:25:03 -08:00
3447934a75
Ensure Krb auth before killing YARN apps in graceful shutdown ( #9785 )
2020-11-16 09:59:14 -08:00
9e6978609a
Add TravisCI job that builds and tests on ARM64 CPU architecture ( #10562 )
2020-11-16 21:08:43 +05:30
e83d5cb59e
Fix ingestion failure of pretty-formatted JSON message ( #10383 )
...
* support multi-line text
* add test cases
* split json text into lines case by case
* improve exception handle
* fix CI
* use IntermediateRowParsingReader as base of JsonReader
* update doc
* ignore the non-immutable field in test case
* add more test cases
* mark `lineSplittable` as final
* fix testcases
* fix doc
* add a test case for SqlReader
* return all raw columns when exception occurs
* fix CI
* fix test cases
* resolve review comments
* handle ParseException returned by index.add
* apply Iterables.getOnlyElement
* fix CI
* fix test cases
* improve code in more graceful way
* fix test cases
* fix test cases
* add a test case to check multiple json string in one text block
* fix inspection check
2020-11-13 13:59:23 -08:00
21e3c4b39c
Add missing docs for timeout exceptions ( #10554 )
...
* Add missing docs for timeout exceptions
* Add info on auth failures
2020-11-13 08:45:40 -06:00
c3cad461bc
Correct getRandomBalancerSegmentHolderTest ( #10569 )
2020-11-12 23:21:05 +05:30
6563599de4
modify access to protected SQLMetadataConnector methods to allow extensions to create SQL metadata tables using implementation specific constructs (payload type, serial type, etc) ( #10573 )
2020-11-12 23:20:01 +05:30
f1b5745eb6
Add helm shield ( #10526 )
2020-11-10 16:18:36 -06:00
20e7e53ede
Use FileUtils.createTempDir in IndexGeneratorJob. ( #10568 )
...
Simplifies code by using a method designed for this purpose.
2020-11-09 22:40:28 -08:00
cd231d8511
Run integration test queries once ( #10564 )
...
* Run integration test queries once
* missed a few
2020-11-09 17:34:27 -08:00
83667172ce
Add missing coordinator dynamic config to the web-console dialog for dynamic coordinator config ( #10545 )
...
* Add missing coordinator dynamic config to the web-console dialog for dynamic coordinator config
* update jest snapshots after making console change
2020-11-09 13:00:52 -06:00
6c8a77b7a9
Bump jetty to latest version ( #10563 )
...
This addresses CVE-2020-27216 which was flagged by the security vulnerability
job.
2020-11-09 08:51:36 -08:00
3436297354
Clarify how ORDER BY works with UNION ALL ( #10561 )
...
Hopefully a bit clearer.
2020-11-05 20:12:03 -08:00
6ccddedb7a
Improved exception handling in case of query timeouts ( #10464 )
...
* Separate timeout exceptions
* Add more tests
Co-authored-by: Atul Mohan <atulmohan@yahoo-inc.com>
2020-11-03 09:00:33 -06:00
d8e5a159e8
Update index.md ( #10549 )
...
Removing the extra `_` in the default for middlemanager category
2020-11-03 13:44:47 +05:30
9286153145
doc wrong description of configuration ( #10546 )
2020-11-02 17:57:16 -08:00
6b14bdb3a5
Add support for Blacklisting some domains for HTTPInputSource ( #10535 )
...
fix inspections
refactor class name
change name
add allowList as well
distinguish between empty and null list
Fix CI
2020-11-02 21:47:25 +05:30
ee61a165e3
Web console: improve make type preservation in ingestion configs in the data loader ( #10533 )
...
* improve validation
* move to druid modals
* adjust specs
* oneOf
* move transform
* segment grans
* tidy up webpack
* add Partitioning
* break out druid models
* tidy up
* rename to Expression
* move druid time
* cleanup
* update format
* better detail
* fix e2e test
* remove forceGuaranteedRollup from e2e tests
* clean up imports
* shardSpec -> shard_spec
* fix css
* adjust snapshot
* add granularity to segments view
* rename to Time span
* use OFFSET in segments view query
* update doc links
* simplify require
* more oneOf
2020-10-30 19:02:44 -07:00
835b328851
docs/: use tuningConfig ( #10540 )
2020-10-30 09:39:21 -05:00
9c51047cc8
Document correlation between credential iterations and query latency ( #10532 )
...
use link / heading instead of footnote
2020-10-29 12:47:24 -07:00
65a42f9eb1
Update overlord api docs ( #10539 )
2020-10-29 11:19:12 -05:00
4de4d4d111
remove ServerDiscoverySelector from DruidLeaderClient ( #10537 )
2020-10-28 10:55:11 -07:00
aa9c0ec650
update quickstart docker-compose example, add to release instructions ( #10527 )
...
* update quickstart docker-compose example, add to release instructions
* adjust
* spelling
2020-10-26 23:14:07 -07:00
ee136303bb
optionally disable all of hardcoded zookeeper use ( #9507 )
...
* optionally disable all of hardcoded zookeeper use
* fix DruidCoordinatorTest compilation
* fix test in DruidCoordinatorTest
* fix strict compilation
Co-authored-by: Himanshu Gupta <fill email>
2020-10-26 22:35:59 -07:00
d0821de854
support for vectorizing expressions with non-existent inputs, more consistent type handling for non-vectorized expressions ( #10499 )
...
* support for vectorizing expressions with non-existent inputs, more consistent type handling for non-vectorized expressions
* inspector
* changes
* more test
* clean
2020-10-26 19:55:24 -07:00
a966de5319
Add https to druid-influxdb-emitter extension ( #9938 )
...
* Add https to druid-influxdb-emitter extension
* address CI failures
* increase test coverage
* tests for being unable to load trustStore
* fix EqualsVerifier test
* fix intellij inspection error
* use try-with-resources when loading trustStore
2020-10-26 19:49:26 -07:00
f3a2903218
Configurable Index Type ( #10335 )
...
* Introduce a Configurable Index Type
* Change to @UnstableApi
* Add AppendableIndexSpecTest
* Update doc
* Add spelling exception
* Add tests coverage
* Revert some of the changes to reduce diff
* Minor fixes
* Update getMaxBytesInMemoryOrDefault() comment
* Fix typo, remove redundant interface
* Remove off-heap spec (postponed to a later PR)
* Add javadocs to AppendableIndexSpec
* Describe testCreateTask()
* Add tests for AppendableIndexSpec within TuningConfig
* Modify hashCode() to conform with equals()
* Add comment where building incremental-index
* Add "EqualsVerifier" tests
* Revert some of the API back to AppenderatorConfig
* Don't use multi-line comments
* Remove knob documentation (deferred)
2020-10-23 18:34:26 -07:00
1b9a8c4687
Fix compaction integration test CI timeout ( #10517 )
...
* fix flaky IT Compaction test
* fix flaky IT Compaction test
* test
* test
* test
* test
* Fix compaction integration test CI timeout
* address comments
* test
* test
* Add print logs
* add error msg
* add taskId to logging
2020-10-21 22:38:11 -07:00
Himanshu
Atul Mohan
Lucas Capistrant
zhangyue19921010
frank chen
Vadim Ogievetsky
Jihoon Son
Ayush Kulshrestha
Nishant Bangarwa
David Palmer
sthetland
michaelschiff
Martin Grigorov
Clint Wylie
Benedict Jin
Gian Merlino
Suneet Saldanha
Mainak Ghosh
Husky Zeng
Pierre Carrier
Charles Smith
awelsh93
Liran Funaro
Maytas Monsereenusorn