1670 Commits

Author SHA1 Message Date
Surekha Saharan
ba7afe9dd3 Merge branch 'master' of github.com:druid-io/druid into system-table 2018-10-01 22:20:37 -07:00
Gian Merlino
244046fda5 SQL: Fix too-long headers in http responses. (#6411)
Fixes #6409 by moving column name info from HTTP headers into the
result body.
2018-10-01 18:13:08 -07:00
Surekha Saharan
b1a219ab87 Merge branch 'master' of github.com:druid-io/druid into system-table 2018-10-01 14:43:02 -07:00
Jihoon Son
cb14a43038 Remove ConvertSegmentTask, HadoopConverterTask, and ConvertSegmentBackwardsCompatibleTask (#6393)
* Remove ConvertSegmentTask, HadoopConverterTask, and ConvertSegmentBackwardsCompatibleTask

* update doc and remove auto conversion

* remove remaining doc

* fix teamcity
2018-10-01 12:03:35 -07:00
Surekha Saharan
b605ab9afa Merge branch 'master' of github.com:druid-io/druid into system-table 2018-09-30 22:09:41 -07:00
Surekha Saharan
95b5bc8de8 Address comments
* Add auth check
* Update docs
* Refactoring
2018-09-30 22:03:45 -07:00
Shiv Toolsidass
a56ffe6ab2 Added backpressure metric to docs and defaultMetricDimensions (#6405)
* Added backpressure metric to docs and defaultMetricDimensions.json

* Reworded description for backpressure metric in docs
2018-09-29 17:57:29 -07:00
adursun
6f44e568db Add missing comma (#6399) 2018-09-28 09:02:36 -07:00
Surekha Saharan
9efbe9609e Merge branch 'master' of github.com:druid-io/druid into system-table 2018-09-27 10:50:12 -07:00
QiuMM
47a6cca013 Add TimestampSpec format for microsecond (#6395) 2018-09-27 09:38:44 -07:00
Jihoon Son
6fb503c073 Deprecate task audit logging (#6368)
* Deprecate task audit logging

* fix test

* fix it test
2018-09-26 16:28:02 -07:00
Nishant Bangarwa
c9d281a2e9 Add ability to pass in Bloom filter from Hive Queries (#6222)
* Bloom filter initial implementation

fix checkstyle

review comments

Fix wierd failure

review comments

Revert "Fix wierd failure"

This reverts commit a13a83ad7887e679f6d539191b52aeaaea85b613.

* fix test

* review comment
2018-09-26 16:04:26 -07:00
Caroline1000
034d006d24 add to docs on including Drop rule with Load rule (#6378) 2018-09-25 20:13:52 -07:00
Surekha Saharan
e92237fc8c Merge branch 'master' of github.com:druid-io/druid into system-table 2018-09-25 12:21:42 -07:00
Surekha Saharan
b66a81b8cc Update tests 2018-09-25 12:21:25 -07:00
Benedict Jin
e5d9fcfe8f Add maven.exec.xxx.skip option for exec-maven-plugin (#6162)
* Fix conflicts

* Modify io.druid into org.apache.druid
2018-09-25 10:05:26 -07:00
Jihoon Son
99428e20d2 Deprecate dimensions / metrics APIs on brokers (#6361)
* Deprecate dimensions / metrics APIs on brokers

* add segmentMetadataQuery link

* add more doc
2018-09-24 17:56:38 -07:00
Jonathan Wei
ee7b565469 Docs for ingestion stat reports and new parse exception handling (#6373) 2018-09-24 17:45:05 -07:00
Surekha Saharan
8f7b0b6657 Merge branch 'master' of github.com:druid-io/druid into system-table 2018-09-24 15:46:10 -07:00
Surekha Saharan
1f44382053 Doc updates, other PR comments 2018-09-24 15:45:46 -07:00
Alexander Saydakov
93345064b5 HllSketch module (#5712)
* HllSketch module

* updated license and imports

* updated package name

* implemented makeAggregateCombiner()

* removed json marks

* style fix

* added module

* removed unnecessary import, side effect of package renaming

* use TreadLocalRandom

* addressing code review points, mostly formatting and comments

* javadoc

* natural order with nulls

* typo

* factored out raw input value extraction

* singleton

* style fix

* style fix

* use Collections.singletonList instead of Arrays.asList

* suppress warning
2018-09-24 08:41:56 -07:00
Surekha Saharan
14064c5a61 Merge branch 'master' of github.com:druid-io/druid into system-table 2018-09-21 18:40:52 -07:00
Surekha Saharan
bab61c6145 Fix some tests and address comments 2018-09-21 18:40:18 -07:00
Jonathan Wei
f12ffd19a8
Add Kafka reset instructions for tutorial (#6362) 2018-09-21 14:18:31 -07:00
Jonathan Wei
8972244c68 Mutual TLS support (#6076)
* Mutual TLS support

* Kafka test fixes

* TeamCity fix

* Split integration tests

* Use localhost DOCKER_IP

* Increase server thread count

* Increase SSL handshake timeouts

* Add broken pipe retries, use injected client config params

* PR comments, Rat license check exclusion
2018-09-19 09:56:15 -07:00
Dayue Gao
edf0c13807 add a sql option to force user to specify time condition (#6246)
* add a sql option to force user to specify time condition

* rename forceTimeCondition to requireTimeCondition, refine error message
2018-09-17 13:52:24 -07:00
QiuMM
288aa4d504 Add missing metadata table information in docs (#6309)
* Add missing metadata table information in doc file

* address review comment
2018-09-14 12:17:05 -07:00
QiuMM
85391e9fb3 fix opentsdb emitter always be running and fail sending tags whose value contains colon (#6251)
* fix opentsdb emitter always be running

* check if emitter started

* add more details about consumeDelay in doc

* fix possible thread unsafe

* fix fail sending tags whose value contain colon
2018-09-14 12:14:15 -07:00
QiuMM
87ccee05f7 Add ability to specify list of task ports and port range (#6263)
* support specify list of task ports

* fix typos

* address comments

* remove druid.indexer.runner.separateIngestionEndpoint config

* tweak doc

* fix doc

* code cleanup

* keep some useful comments
2018-09-13 19:36:04 -07:00
Jonathan Wei
fd6786ac6c Fix endpoint permissions section in basic-security docs (#6331) 2018-09-13 15:23:41 -07:00
Clint Wylie
91a37c692d 'suspend' and 'resume' support for supervisors (kafka indexing service, materialized views) (#6234)
* 'suspend' and 'resume' support for kafka indexing service
changes:
* introduces `SuspendableSupervisorSpec` interface to describe supervisors which support suspend/resume functionality controlled through the `SupervisorManager`, which will gracefully shutdown the supervisor and it's tasks, update it's `SupervisorSpec` with either a suspended or running state, and update with the toggled spec. Spec updates are provided by `SuspendableSupervisorSpec.createSuspendedSpec` and `SuspendableSupervisorSpec.createRunningSpec` respectively.
* `KafkaSupervisorSpec` extends `SuspendableSupervisorSpec` and now supports suspend/resume functionality. The difference in behavior between 'running' and 'suspended' state is whether the supervisor will attempt to ensure that indexing tasks are or are not running respectively. Behavior is identical otherwise.
* `SupervisorResource` now provides `/druid/indexer/v1/supervisor/{id}/suspend` and `/druid/indexer/v1/supervisor/{id}/resume` which are used to suspend/resume suspendable supervisors
* Deprecated `/druid/indexer/v1/supervisor/{id}/shutdown` and moved it's functionality to `/druid/indexer/v1/supervisor/{id}/terminate` since 'shutdown' is ambiguous verbage for something that effectively stops a supervisor forever
* Added ability to get all supervisor specs from `/druid/indexer/v1/supervisor` by supplying the 'full' query parameter `/druid/indexer/v1/supervisor?full` which will return a list of json objects of the form `{"id":<id>, "spec":<SupervisorSpec>}`
* Updated overlord console ui to enable suspend/resume, and changed 'shutdown' to 'terminate'

* move overlord console status to own column in supervisor table so does not look like garbage

* spacing

* padding

* other kind of spacing

* fix rebase fail

* fix more better

* all supervisors now suspendable, updated materialized view supervisor to support suspend, more tests

* fix log
2018-09-13 14:42:18 -07:00
Surekha Saharan
0239f94947 Merge branch 'master' of github.com:druid-io/druid into system-table 2018-09-10 17:33:58 -07:00
Surekha Saharan
68d45a0ce5 Fix docs, make num_rows column nullable, some unit test changes 2018-09-10 17:22:47 -07:00
Gian Merlino
d6cbdf86c2
Broker backpressure. (#6313)
* Broker backpressure.

Adds a new property "druid.broker.http.maxQueuedBytes" and a new context
parameter "maxQueuedBytes". Both represent a maximum number of bytes queued
per query before exerting backpressure on the channel to the data server.

Fixes #4933.

* Fix query context doc.
2018-09-10 09:33:29 -07:00
Gian Merlino
4669f0878f SQL: UNION ALL operator. (#6314)
* SQL: UNION ALL operator.

* Remove unused import.
2018-09-09 22:32:56 -07:00
Surekha Saharan
cf1895902b Major change is to make tasks and segment queries streaming
* Made tasks/segments stream to calcite instead of storing it in memory
* Add num_rows to segments table
* Refactor JsonParserIterator
* Replace with closeable iterator
2018-09-07 16:33:02 -07:00
Surekha Saharan
3cb0f52db9 Merge branch 'master' of github.com:druid-io/druid into system-table 2018-09-07 16:20:59 -07:00
Clint Wylie
e6e068ce60 Add support for 'maxTotalRows' to incremental publishing kafka indexing task and appenderator based realtime task (#6129)
* resolves #5898 by adding maxTotalRows to incremental publishing kafka index task and appenderator based realtime indexing task, as available in IndexTask

* address review comments

* changes due to review

* merge fail
2018-09-07 13:17:49 -07:00
Jonathan Wei
60cbc64472
Use PasswordProvider, fix info on initial passwords in basic security extension docs (#6303)
* Fix info on initial passwords in basic security extension docs

* Use PasswordProvider

* Compile fix
2018-09-05 17:07:16 -07:00
Jonathan Wei
4caa61d8fa Fix tutorial sample data filename, fix logger classname in metrics docs (#6299) 2018-09-04 21:47:12 -07:00
Eyal Yurman
10ca290d64 Correct file name typo in Quickstart tutorial (#6297)
Correct name wikipedia-2015-09-12-sampled.json.gz to wikiticker-2015-09-12-sampled.json.gz
2018-09-04 14:20:17 -07:00
Jonathan Wei
180e3ccfad
Docs consistency cleanup (#6259) 2018-09-04 12:54:41 -07:00
Surekha Saharan
54dd64c8de Merge branch 'master' of github.com:druid-io/druid into system-table 2018-08-30 11:30:44 -07:00
Surekha Saharan
cec87379ef Merge branch 'master' of github.com:druid-io/druid into system-table 2018-08-30 11:24:41 -07:00
QiuMM
9b04846e6b correct metric name in doc file (#6271) 2018-08-30 10:57:35 -07:00
Gian Merlino
431d3d8497
Rename io.druid to org.apache.druid. (#6266)
* Rename io.druid to org.apache.druid.

* Fix META-INF files and remove some benchmark results.

* MonitorsConfig update for metrics package migration.

* Reorder some dimensions in inner queries for some reason.

* Fix protobuf tests.
2018-08-30 09:56:26 -07:00
Himanshu
1fae6513e1 add "subtotalsSpec" attribute to groupBy query (#5280)
* add subtotalsSpec attribute to groupBy query

* dont sent subtotalsSpec to downstream nodes from broker and other updates

* address review comment

* fix checkstyle issues after merge to master

* add docs for subtotalsSpec feature

* address doc review comments
2018-08-28 17:46:38 -07:00
Jim Slattery
d957295b98 spelling: storage (#6248) 2018-08-27 16:35:31 -07:00
Surekha Saharan
05fd4ce66b Merge branch 'master' of github.com:druid-io/druid into system-table 2018-08-27 10:18:44 -07:00
Gian Merlino
0172326c62 SQL: Support more result formats, add columns header. (#6191)
* SQL: Support more result formats, add columns header.

- Add result formats for line-based JSON and CSV.
- Add X-Druid-Sql-Columns header with a list of all columns that
the response will contain.
- Add more comprehensive documentation on what callers should expect
when making Druid SQL queries.

* Fix some tests.

* Adjust tests.

* Adjust trailer, add types header.

* Fix trailers.
2018-08-26 23:00:14 -06:00