Apache/druid
1
0
Fork 0
mirror of https://github.com/apache/druid.git synced 2025-02-07 02:28:19 +00:00
Code Issues Packages Projects Releases Wiki Activity
9,196 Commits 115 Branches 507 Tags
Commit Graph

9196 Commits

This Branch
This Branch
All Branches
Author SHA1 Message Date
Vadim Ogievetsky
3aae4aaf8b Ensuring that all the controls are aligned correctly (#7457) 
* making sure all the controls are inline

* update deps
 2019-04-12 08:51:03 -07:00
Vadim Ogievetsky
07eb2effe9 customHeader (#7456) 2019-04-11 21:58:09 -07:00
Gian Merlino
3854cfd15e SQLMetadataSegmentManager: Comments, formatting adjustments (#7452) 
Follow up to #7447.
 2019-04-11 21:57:50 -07:00
Gian Merlino
2470b3279f SQL: Fix docs for STRING_FORMAT. (#7455) 2019-04-11 21:57:28 -07:00
Qi Shu
7382dac44e Fixed grouping and sorting by status in task table (#7440) 
* Fixed group by status

* Shorten code
 2019-04-11 16:10:37 -07:00
Vadim Ogievetsky
c5c2bf77f0 adding host to the error display (#7453) 2019-04-11 14:39:57 -07:00
Don Bowman
08d0d6d20d chore: pin upstream java container to jdk 8 (#7313) 2019-04-11 13:18:29 -07:00
Gian Merlino
a517f8ce49 Coordinator: Allow dropping all segments. (#7447) 
Removes the coordinator sanity check that prevents it from dropping all
segments. It's useful to get rid of this, since the behavior is
unintuitive for dev/testing clusters where users might regularly want
to drop all their data to get back to a clean slate.

But the sanity check was there for a reason: to prevent a race condition
where the coordinator might drop all segments if it ran before the
first metadata store poll finished. This patch addresses that concern
differently, by allowing methods in MetadataSegmentManager to return
null if a poll has not happened yet, and canceling coordinator runs
in that case.

This patch also makes the "dataSources" reference in
SQLMetadataSegmentManager volatile. I'm not sure why it wasn't volatile
before, but it seems necessary to me: it's not final, and it's dereferenced
from multiple threads without synchronization.
 2019-04-11 08:45:38 -07:00
Qi Shu
4ea37e2614 Add time taken by last query in SQL view (#7421) 
* Add time taken by query

* Fix time to 2 dp; set state in on state change; hide time taken at first

* Refactored code: defined query result interface; more concise shorthand expression

* Use single quote
 2019-04-10 23:35:47 -07:00
Qi Shu
bbb620125d Table pagination style change (#7435) 
* Style pagination

* Removed use of important

* Removed space

* Align items

* Switched back to react table 6.8.6

* Component name change
 2019-04-10 23:34:03 -07:00
Justin Borromeo
408e3e1b2a Remove select execution code from SQL planner (#7416) 
* Removed select execution code from SQL planner

* Update doc
 2019-04-10 22:32:57 -07:00
Benjamin Hopp
78e6f6fb38 Updated Javascript Affinity config docs (#7441) 
Updated with hostname:port rather than IP Address.
 2019-04-10 21:44:50 -07:00
Clint Wylie
15c3124e7f add auth check exclusion for console-config.js added in #7414 (#7442) 2019-04-10 21:44:26 -07:00
Justin Borromeo
2771ed50b0 Support Kafka supervisor adopting running tasks between versions (#7212) 
* Recompute hash in isTaskCurrent() and added tests

* Fixed checkstyle stuff

* Fixed failing tests

* Make TestableKafkaSupervisorWithCustomIsTaskCurrent static

* Add doc

* baseSequenceName change

* Added comment

* WIP

* Fixed imports

* Undid lambda change for diff sake

* Cleanup

* Added comment

* Reinsert Kafka tests

* Readded kinesis test

* Readd bad partition assignment in kinesis supervisor test

* Nit

* Misnamed var
 2019-04-10 18:16:38 -07:00
Benedict Jin
2f64414ade Add "REVERSE" / "REPEAT" / "RIGHT" / "LEFT" functions (#7334) 
* Add "REVERSE" / "REPEAT" / "RIGHT" / "LEFT" functions

* Fix ImportOrder

* Use RuntimeException instead of OutOfMemoryError according to "Effective Java"

* Simplify

* Patch suggestions
 2019-04-10 11:46:29 +08:00
Clint Wylie
89bb43f382 'core' ORC extension (#7138) 
* orc extension reworked to use apache orc map-reduce lib, moved to core extensions, support for flattenSpec, tests, docs

* change binary handling to be compatible with avro and parquet, Rows.objectToStrings now converts byte[] to base64, change date handling

* better docs and tests

* fix it

* formatting

* doc fix

* fix it

* exclude redundant dependencies

* use latest orc-mapreduce, add hadoop jobProperties recommendations to docs

* doc fix

* review stuff and fix binaryAsString

* cache for root level fields

* more better
 2019-04-09 09:03:26 -07:00
Jonathan Wei
09abc4a856 Fix comments in KinesisIndexTaskIOConfig (#7430) 2019-04-09 08:40:08 -07:00
Jihoon Son
e87d6e32b3 Support kinesis compatibility (#7351) 2019-04-08 19:19:34 -07:00
Gian Merlino
7cd5477658 DoublesSketchComplexMetricSerde: Handle empty strings. (#7429) 2019-04-08 18:01:31 -07:00
Qi Shu
98ee8637e6 Console SQL function doc generation (#7406) 
* Change sql function doc generation script from Bash to Node.js

* Minor code change for better reading

* Disable tslint in file

* clean up code

* Use es6 multiline string

* Solved a bug which would generate invalid TS variable if no function sections are found; in this case an empty array should be created

* Edit build script
 2019-04-08 17:38:26 -07:00
Vadim Ogievetsky
c69c37e99d Update console packages and enforce tslint quotes (#7419) 
* update packages and enforce tslint quotes

* remove direct dep on tslint-react
 2019-04-07 20:19:08 -07:00
Justin Borromeo
799c66d9ac Allow max rows and max segments for time-ordered scans to be overridden using the scan query JSON spec (#7413) 
* Initial changes

* Fixed NPEs

* Fixed failing spec test

* Fixed failing Calcite test

* Move configs to context

* Validated and added docs

* fixed weird indentation

* Update default context vals in doc

* Fixed allowable values
 2019-04-07 20:12:52 -07:00
Clint Wylie
7778f29781 add travis check to make sure -Pdist works (#7418) 2019-04-07 20:12:23 -07:00
Clint Wylie
e28a15f9f5 fix expressions docs operator table (#7420) 
* fix expressions docs operator table

* Update math-expr.md
 2019-04-07 20:12:00 -07:00
Justin Borromeo
e23fd41fa7 Update SQL doc for planning change (#7415) 2019-04-05 15:14:07 -07:00
Clint Wylie
76b4a5c62e refactor lookups to be more chill to router (#7222) 
* refactor lookups to be more chill to router

* remove accidental change

* fix and combine LookupIntrospectionResourceTest

* fix inspection

* rename RouterLookupModule to LookupSerdeModule and RouterLookupExtractorFactoryContainerProvider to NoopLookupExtractorFactoryContainerProvider

* make comment generic

* use ConfigResourceFilter instead of StateResourceFilter

* fix indentation

* unused import

* another unused import

* refactor some stuff into processing module, split up LookupModule.java classes into their own files
 2019-04-05 14:49:41 -07:00
Qi Shu
a66291a9cd SQL explain in web console (#7402) 
* Add explain for SQL query

* Terminate explain query manager

* Add signature for semi-joined queries

* Extract components out

* Hide more button in rune mode

* Add types for query explanation parser

* Add type for query-plan-dialog
 2019-04-05 12:44:51 -07:00
Vadim Ogievetsky
0fa122ecf2 Adding ability to pass configs in and fixing misc bugs (#7414) 
* Adding ability to pass configs in and fixing misc bugs

* update lock file

* remove dead version param
 2019-04-05 12:40:43 -07:00
Clint Wylie
e2aa6e47f8 change Dockerfile maintainers email to dev mailing list (#7417) 2019-04-04 19:34:48 -07:00
Gian Merlino
78745fea84 Fix two issues with Coordinator -> Overlord communication. (#7412) 
* Fix two issues with Coordinator -> Overlord communication.

1) ClientCompactQuery needs to recognize the potential for 'intervals'
to be set instead of 'segments'. The lack of this led to a
NullPointerException on DruidCoordinatorSegmentCompactor.java:102.

2) In two locations (DruidCoordinatorSegmentCompactor,
DruidCoordinatorCleanupPendingSegments) tasks were being retrieved
using waiting/pending/running tasks in the wrong order: by checking
'running' first and then 'pending', tasks could be missed if they
moved from 'pending' to 'running' in between the two calls. Replaced
these methods with calls to 'getActiveTasks', a new method that does
the calls in the right order.

* Remove unused import.
 2019-04-04 10:25:18 -07:00
Richard Startin
d29a32062f upgrade to RoaringBitmap 0.8.0 and serialise directly to ByteBuffer (#7408) 2019-04-04 13:22:50 -04:00
Jonathan Wei
0f6cb1e7e0 Update theta/hll sketch doc comparison (#7407) 2019-04-03 15:21:33 -07:00
Gian Merlino
8c104a115c
SQL: Add STRING_FORMAT function. (#7327) 2019-04-03 17:09:54 -04:00
Alexander Saydakov
28b4e8586d use latest sketches-core-0.13.1 (#7320) 
* use latest sketches-core-0.13.0

* latest release
 2019-04-03 17:06:02 -04:00
lxqfy
6b741f0800 Overwrite index task maxTotalRows with computed maxRowsPerSegments (#7338) 
fixed identation

set maxTotalRows to higher value in case mutiple segments in appendrator
with comments
 2019-04-03 11:47:59 -07:00
Atul Mohan
c883c52cb1 Fix tests (#7401) 2019-04-02 16:49:21 -07:00
Justin Borromeo
4584b5e139 SQL support for time-ordered scan (#7373) 
* Squashed commit of the following:

commit 287a367f4170e7d0b3010d57788ea993688b9335
Author: Justin Borromeo <jborrome@edu.uwaterloo.ca>
Date:   Wed Mar 27 20:03:41 2019 -0700

    Implemented Clint's recommendations

commit 07503ea5c00892bf904c0e16e7062fadabcb7830
Author: Justin Borromeo <jborrome@edu.uwaterloo.ca>
Date:   Wed Mar 27 17:49:09 2019 -0700

    doc fix

commit 231a72e7d9c0f4bb2b3272134cf53fc8db8f0e73
Author: Justin Borromeo <jborrome@edu.uwaterloo.ca>
Date:   Wed Mar 27 17:38:20 2019 -0700

    Modified sequence limit to accept longs and added test for long limits

commit 1df50de32137961d949c1aaa4e4791f6edfb3d77
Merge: 480e932fd c7fea6ac8
Author: Justin Borromeo <jborrome@edu.uwaterloo.ca>
Date:   Tue Mar 26 15:23:01 2019 -0700

    Merge branch 'master' into 6088-Time-Ordering-On-Scans-N-Way-Merge

commit 480e932fdf02ef85ba81181deb865d9977dfed24
Author: Justin Borromeo <jborrome@edu.uwaterloo.ca>
Date:   Tue Mar 26 14:58:04 2019 -0700

    Checkstyle and doc update

commit 487f31fcf63a5e1fa9e802212b62206aec47fe25
Author: Justin Borromeo <jborrome@edu.uwaterloo.ca>
Date:   Tue Mar 26 14:39:25 2019 -0700

    Refixed regression

commit fb858efbb75218bb80b8c77effb2456554aa57b2
Author: Justin Borromeo <jborrome@edu.uwaterloo.ca>
Date:   Tue Mar 26 13:14:48 2019 -0700

    Added test for n-way merge

commit 376e8bf90610d43d2c7b278bf64525cab80267c5
Author: Justin Borromeo <jborrome@edu.uwaterloo.ca>
Date:   Tue Mar 26 11:42:54 2019 -0700

    Refactor n-way merge

commit 8a6bb1127c1814470424da2e9d6bfdd55e726199
Author: Justin Borromeo <jborrome@edu.uwaterloo.ca>
Date:   Mon Mar 25 17:17:41 2019 -0700

    Fix docs and flipped boolean in ScanQueryLimitRowIterator

commit 35692680fc7aba21c498a92307ef082a581cb23a
Author: Justin Borromeo <jborrome@edu.uwaterloo.ca>
Date:   Mon Mar 25 16:15:49 2019 -0700

    Fix bug messing up count of rows

commit 219af478c8ec243700973e616c5be556a83422e2
Author: Justin Borromeo <jborrome@edu.uwaterloo.ca>
Date:   Mon Mar 25 15:57:55 2019 -0700

    Fix bug in numRowsScanned

commit da4fc664031debae1dc3b4a0190125e979564aac
Author: Justin Borromeo <jborrome@edu.uwaterloo.ca>
Date:   Mon Mar 25 15:19:45 2019 -0700

    Check type of segment spec before using for time ordering

commit b822fc73dfba7f69c7e960bb95b31cab8d27ef25
Author: Justin Borromeo <jborrome@edu.uwaterloo.ca>
Date:   Mon Mar 25 13:19:02 2019 -0700

    Revert "Merge branch '6088-Time-Ordering-On-Scans-N-Way-Merge' of github.com:justinborromeo/incubator-druid into 6088-Time-Ordering-On-Scans-N-Way-Merge"

    This reverts commit 57033f36df6e3ef887e5f0399ad74bb091306de8, reversing
    changes made to 8f01d8dd16f40d10c60519ca0ec0d2e6b2dde941.

commit 57033f36df6e3ef887e5f0399ad74bb091306de8
Merge: 8f01d8dd1 86d9730fc
Author: Justin Borromeo <jborrome@edu.uwaterloo.ca>
Date:   Mon Mar 25 13:13:52 2019 -0700

    Merge branch '6088-Time-Ordering-On-Scans-N-Way-Merge' of github.com:justinborromeo/incubator-druid into 6088-Time-Ordering-On-Scans-N-Way-Merge

commit 8f01d8dd16f40d10c60519ca0ec0d2e6b2dde941
Author: Justin Borromeo <jborrome@edu.uwaterloo.ca>
Date:   Mon Mar 25 13:13:32 2019 -0700

    Revert "Fixed failing tests -> allow usage of all types of segment spec"

    This reverts commit ec470288c7b725f5310bcf69d1db9f85ff509c8d.

commit ec470288c7b725f5310bcf69d1db9f85ff509c8d
Author: Justin Borromeo <jborrome@edu.uwaterloo.ca>
Date:   Mon Mar 25 11:01:35 2019 -0700

    Fixed failing tests -> allow usage of all types of segment spec

commit 86d9730fc9f241b3010b123a45b1fc38a206a9af
Author: Justin Borromeo <jborrome@edu.uwaterloo.ca>
Date:   Mon Mar 25 11:01:35 2019 -0700

    Fixed failing tests -> allow usage of all types of segment spec

commit 8b3b6b51ed0d3bc3c937620d5b92096998e32080
Author: Justin Borromeo <jborrome@edu.uwaterloo.ca>
Date:   Fri Mar 22 16:01:56 2019 -0700

    Nit comment

commit a87d02127c72aa5e307af94b12b6be25150349be
Author: Justin Borromeo <jborrome@edu.uwaterloo.ca>
Date:   Fri Mar 22 15:54:42 2019 -0700

    Fix checkstyle and test

commit 62dcedacdeeed570134e8b5185633b207e91a547
Author: Justin Borromeo <jborrome@edu.uwaterloo.ca>
Date:   Fri Mar 22 15:30:41 2019 -0700

    More comments

commit 1b46b58aeccf13adc516a1d94054a98efc32184c
Author: Justin Borromeo <jborrome@edu.uwaterloo.ca>
Date:   Fri Mar 22 15:19:52 2019 -0700

    Added a bit of docs

commit 49472162b7fc0879159866c3736e192fc88837a4
Author: Justin Borromeo <jborrome@edu.uwaterloo.ca>
Date:   Fri Mar 22 10:27:41 2019 -0700

    Rename segment limit -> segment partitions limit

commit 43d490cc3ae697d0a61159ed6ae06906006cdf31
Author: Justin Borromeo <jborrome@edu.uwaterloo.ca>
Date:   Thu Mar 21 13:16:58 2019 -0700

    Optimized n-way merge strategy

commit 42f5246b8d0c1879c2dc45334966bf52f543ea74
Author: Justin Borromeo <jborrome@edu.uwaterloo.ca>
Date:   Wed Mar 20 17:40:19 2019 -0700

    Smarter limiting for pQueue method

commit 4823dab895770a87356fe2ae4e9858bb4ba03fc3
Author: Justin Borromeo <jborrome@edu.uwaterloo.ca>
Date:   Wed Mar 20 16:05:53 2019 -0700

    Finish rename

commit 2528a5614267c48714abdb30fd7a2ccdb61b802d
Author: Justin Borromeo <jborrome@edu.uwaterloo.ca>
Date:   Mon Mar 18 14:00:50 2019 -0700

    Renaming

commit 7bfa77d3c177be42d0db0b0bc3c19f9ef536ffeb
Merge: a032c46ee 7e49d4739
Author: Justin Borromeo <jborrome@edu.uwaterloo.ca>
Date:   Tue Mar 12 16:57:45 2019 -0700

    Merge branch 'Update-Query-Interrupted-Exception' into 6088-Time-Ordering-On-Scans-N-Way-Merge

commit 7e49d47391d17b411b0620794e503592d8f37481
Author: Justin Borromeo <jborrome@edu.uwaterloo.ca>
Date:   Tue Mar 12 16:51:25 2019 -0700

    Added error message for UOE

commit a032c46ee09cd80b78f25d0da51f5179774aa75f
Author: Justin Borromeo <jborrome@edu.uwaterloo.ca>
Date:   Tue Mar 12 16:47:17 2019 -0700

    Updated error message

commit 57b568265488066c046f225c1982dba85e8a64ba
Author: Justin Borromeo <jborrome@edu.uwaterloo.ca>
Date:   Tue Mar 12 12:44:02 2019 -0700

    Fixed tests

commit 45e95bb1f40d50ab3a0a745d2b5fca34c3f53a82
Author: Justin Borromeo <jborrome@edu.uwaterloo.ca>
Date:   Tue Mar 12 11:09:08 2019 -0700

    Optimization

commit cce917ab846198706ec8177a91869f9aa43e0525
Author: Justin Borromeo <jborrome@edu.uwaterloo.ca>
Date:   Fri Mar 8 14:11:07 2019 -0800

    Checkstyle fix

commit 73f4038068f2e30cb3487cc175730f3b97c5c8d2
Author: Justin Borromeo <jborrome@edu.uwaterloo.ca>
Date:   Thu Mar 7 18:40:00 2019 -0800

    Applied Jon's recommended changes

commit fb966def8335e6808f0fe5d2d6a122dcd28f2355
Author: Justin Borromeo <jborrome@edu.uwaterloo.ca>
Date:   Thu Mar 7 11:03:01 2019 -0800

    Sorry, checkstyle

commit 6dc53b311c568e29a6937fdd5f17a5623d14533f
Author: Justin Borromeo <jborrome@edu.uwaterloo.ca>
Date:   Wed Mar 6 10:34:13 2019 -0800

    Improved test and appeased TeamCity

commit 35c96d355726cf5d238435655fecbfe19ea8ddb6
Author: Justin Borromeo <jborrome@edu.uwaterloo.ca>
Date:   Mon Mar 4 16:00:44 2019 -0800

    Checkstyle fix

commit 2d1978d5713187561a534c08eba51e383df66ce7
Merge: 83ec3fe1f 3398d3982
Author: Justin Borromeo <jborrome@edu.uwaterloo.ca>
Date:   Mon Mar 4 15:24:49 2019 -0800

    Merge branch 'master' into 6088-Time-Ordering-On-Scans-N-Way-Merge

commit 83ec3fe1f13c384aca52ceef0ba03b300b03d8d9
Author: Justin Borromeo <jborrome@edu.uwaterloo.ca>
Date:   Fri Mar 1 13:40:22 2019 -0800

    Nit-change on javadoc

commit 47c970b5f476e5bfe5e03aa798f314f59aeb67db
Author: Justin Borromeo <jborrome@edu.uwaterloo.ca>
Date:   Fri Mar 1 13:38:29 2019 -0800

    Wrote tests and added Javadoc

commit 5ff59f5ca6c8058c04e500662b3691a4910aa842
Author: Justin Borromeo <jborrome@edu.uwaterloo.ca>
Date:   Thu Feb 28 15:58:20 2019 -0800

    Reset config

commit 806166f9777cccae5e10eabbb256c5e33b0e13f7
Author: Justin Borromeo <jborrome@edu.uwaterloo.ca>
Date:   Thu Feb 28 15:49:07 2019 -0800

    Fixed failing tests

commit de83b11a1bb24a0ae964240d9cb1ed17ea4a6c26
Author: Justin Borromeo <jborrome@edu.uwaterloo.ca>
Date:   Tue Feb 26 16:40:48 2019 -0800

    Fixed mistakes in merge

commit 5bd0e1a32cec1a0e4dadd74dd530d13341ab7349
Merge: 18cce9a64 9fa649b3b
Author: Justin Borromeo <jborrome@edu.uwaterloo.ca>
Date:   Tue Feb 26 16:39:16 2019 -0800

    Merge branch 'master' into 6088-Time-Ordering-On-Scans-N-Way-Merge

commit 18cce9a646139a57004ef4eccef8077c9775e992
Author: Justin Borromeo <jborrome@edu.uwaterloo.ca>
Date:   Tue Feb 26 13:16:44 2019 -0800

    Change so batching only occurs on broker for time-ordered scans

    Restricted batching to broker for time-ordered queries and adjusted
    tests

    Formatting

    Cleanup

commit 451e2b43652020d6acb8b8db113fb34db0f50517
Author: Justin Borromeo <jborrome@edu.uwaterloo.ca>
Date:   Tue Feb 26 11:14:27 2019 -0800

    WIP

commit 69b24bd851d721592324bcbeec1c4229ad9ff462
Merge: 763c43df7 417b9f2fe
Author: Justin Borromeo <jborrome@edu.uwaterloo.ca>
Date:   Fri Feb 22 18:13:26 2019 -0800

    Merge branch 'master' into 6088-Time-Ordering-On-Scans-N-Way-Merge

commit 763c43df7e99d4ab000f038a7c1b9ef98b479138
Author: Justin Borromeo <jborrome@edu.uwaterloo.ca>
Date:   Fri Feb 22 18:07:06 2019 -0800

    Multi-historical setup works

commit 06a5218917bca0716b98c32c07415a7271711431
Author: Justin Borromeo <jborrome@edu.uwaterloo.ca>
Date:   Fri Feb 22 16:59:57 2019 -0800

    Wrote docs

commit 3b923dac9cc82475795ee2f7691e6f96249560aa
Author: Justin Borromeo <jborrome@edu.uwaterloo.ca>
Date:   Fri Feb 22 14:03:22 2019 -0800

    Fixed bug introduced by replacing deque with list

commit 023538d83117086647c69d5030f2e8cb3e039558
Author: Justin Borromeo <jborrome@edu.uwaterloo.ca>
Date:   Fri Feb 22 13:30:08 2019 -0800

    Sequence stuff is so dirty :(

commit e1fc2955d361676eb6721ad31defd96d47fab999
Author: Justin Borromeo <jborrome@edu.uwaterloo.ca>
Date:   Fri Feb 22 10:39:59 2019 -0800

    WIP

commit f57ff253fa659cbb5aa09b7c9bf03d8e7670b865
Author: Justin Borromeo <jborrome@edu.uwaterloo.ca>
Date:   Thu Feb 21 18:22:06 2019 -0800

    Ordering is correct on n-way merge -> still need to batch events into
    ScanResultValues

commit 1813a5472c791509ba903f734b40b6102079876a
Author: Justin Borromeo <jborrome@edu.uwaterloo.ca>
Date:   Thu Feb 21 17:06:18 2019 -0800

    Cleanup

commit f83e99655d11247f44018e0e5d36bd6eac1fb2a6
Author: Justin Borromeo <jborrome@edu.uwaterloo.ca>
Date:   Thu Feb 21 16:56:36 2019 -0800

    Refactor and pQueue works

commit b13ff624a92a7e740eb1f74aa40c7a72165b9708
Author: Justin Borromeo <jborrome@edu.uwaterloo.ca>
Date:   Thu Feb 21 15:13:33 2019 -0800

    Set up time ordering strategy decision tree

commit fba6b022f0395cc297e3b3726f817c986f97010b
Author: Justin Borromeo <jborrome@edu.uwaterloo.ca>
Date:   Thu Feb 21 15:08:27 2019 -0800

    Added config and get # of segments

commit c9142e721c7ed824a54de7a160230cd959bb906d
Merge: cd489a020 554b0142c
Author: Justin Borromeo <jborrome@edu.uwaterloo.ca>
Date:   Wed Feb 20 10:12:50 2019 -0800

    Merge branch 'master' into 6088-Time-Ordering-On-Scans-V2

commit cd489a0208b0cfc475a34440caa1c1e99d22a281
Author: Justin Borromeo <jborrome@edu.uwaterloo.ca>
Date:   Wed Feb 20 00:16:48 2019 -0800

    Fixed failing test due to null resultFormat

commit 7baeade8328776244e72a3cb5f2efb59111cf58b
Author: Justin Borromeo <jborrome@edu.uwaterloo.ca>
Date:   Tue Feb 19 17:52:06 2019 -0800

    Changes based on Gian's comments

commit 35150fe1a63c5143f564c4435461929e619a0de2
Author: Justin Borromeo <jborrome@edu.uwaterloo.ca>
Date:   Fri Feb 15 15:57:53 2019 -0800

    Small changes

commit 4e69276d57de4a9042b927efa5a864411aedacb4
Author: Justin Borromeo <jborrome@edu.uwaterloo.ca>
Date:   Mon Feb 11 12:09:54 2019 -0800

    Removed unused import to satisfy PMD check

commit ecb0f483a9525ffc2844cb01a0daafe6bc4d2161
Author: Justin Borromeo <jborrome@edu.uwaterloo.ca>
Date:   Mon Feb 11 10:37:11 2019 -0800

    improved doc

commit f0eddee66598095a767a1570516c5af59e58e2f6
Author: Justin Borromeo <jborrome@edu.uwaterloo.ca>
Date:   Mon Feb 11 10:18:45 2019 -0800

    Added more javadoc

commit 5f92dd7325aeff0b2e3f87003263e083ba2b427d
Author: Justin Borromeo <jborrome@edu.uwaterloo.ca>
Date:   Mon Feb 11 10:05:58 2019 -0800

    Unused import

commit 93e1636287f45d38c80f275e4644c0b3222c65e7
Author: Justin Borromeo <jborrome@edu.uwaterloo.ca>
Date:   Mon Feb 11 10:03:14 2019 -0800

    Added javadoc on ScanResultValueTimestampComparator

commit 134041c47965a8a199862ca33ef2119e29f67287
Author: Justin Borromeo <jborrome@edu.uwaterloo.ca>
Date:   Fri Feb 8 13:13:54 2019 -0800

    Renamed sort function

commit 2e3577cd3d7b43e140d36aad944536f49287fbfa
Author: Justin Borromeo <jborrome@edu.uwaterloo.ca>
Date:   Thu Feb 7 13:01:25 2019 -0800

    Fixed benchmark queries

commit d3b335af42602a771063bd8a63c89acf5c715938
Author: Justin Borromeo <jborrome@edu.uwaterloo.ca>
Date:   Thu Feb 7 11:08:07 2019 -0800

    added all query types to scan benchmark

commit ab00eade9f0b8e8642da40905214653c04cba4d4
Author: Justin Borromeo <jborrome@edu.uwaterloo.ca>
Date:   Thu Feb 7 09:42:48 2019 -0800

    Kicking travis with change to benchmark param

commit b432beaf84de5b363454fd8058ff653a097c713d
Author: Justin Borromeo <jborrome@edu.uwaterloo.ca>
Date:   Wed Feb 6 17:45:59 2019 -0800

    Fixed failing calcite tests

commit b2c8c77ad4ee5a9a273ee7a7870fb2d9b0ec9dd4
Author: Justin Borromeo <jborrome@edu.uwaterloo.ca>
Date:   Wed Feb 6 17:39:48 2019 -0800

    Fixing tests WIP

commit 85e72a614ef49736d1142ce82d26b533b609c911
Author: Justin Borromeo <jborrome@edu.uwaterloo.ca>
Date:   Wed Feb 6 15:42:02 2019 -0800

    Set to spaces over tabs

commit 7e872a8ebcea0d3a141addd122dd9f8b6629ead6
Author: Justin Borromeo <jborrome@edu.uwaterloo.ca>
Date:   Wed Feb 6 15:36:24 2019 -0800

    Created an error message for when someone tries to time order a result
    set > threshold limit

commit e8a4b490443b1efe6c70f964b1757bf17a64e9f6
Merge: 305876a43 8e3a58f72
Author: Justin Borromeo <jborrome@edu.uwaterloo.ca>
Date:   Wed Feb 6 15:05:11 2019 -0800

    Merge branch 'master' into 6088-Time-Ordering-On-Scans-V2

commit 305876a4346c292296db623c1fcea688a29c0bb8
Author: Justin Borromeo <jborrome@edu.uwaterloo.ca>
Date:   Wed Feb 6 15:02:02 2019 -0800

    nit

commit 8212a21cafc2ed4002607362f0661f4b5f6bef9d
Author: Justin Borromeo <jborrome@edu.uwaterloo.ca>
Date:   Wed Feb 6 14:40:35 2019 -0800

    Improved conciseness

commit 10b5e0ca93a529d1b0e018c11fafc9c63071b8cd
Author: Justin Borromeo <jborrome@edu.uwaterloo.ca>
Date:   Wed Feb 6 13:42:12 2019 -0800

    .

commit dfe4aa9681d04b8a31dcc1486e0447f29f6eb7bd
Author: Justin Borromeo <jborrome@edu.uwaterloo.ca>
Date:   Wed Feb 6 13:41:18 2019 -0800

    Fixed codestyle and forbidden API errors

commit 148939e88bfff021356bb532e4246e4c8e8ac333
Merge: 4f51024b3 5edbe2ae1
Author: Justin Borromeo <jborrome@edu.uwaterloo.ca>
Date:   Wed Feb 6 13:26:17 2019 -0800

    Merge branch '6088-Create-Scan-Benchmark' into 6088-Time-Ordering-On-Scans-V2

commit 5edbe2ae12648b527e3e97b516127bf4b65196a3
Merge: 60b7684db 315ccb76b
Author: Justin Borromeo <jborrome@edu.uwaterloo.ca>
Date:   Wed Feb 6 13:18:55 2019 -0800

    Merge github.com:apache/incubator-druid into 6088-Create-Scan-Benchmark

commit 60b7684db725387b4d843385d9c61d50f2ed6744
Author: Justin Borromeo <jborrome@edu.uwaterloo.ca>
Date:   Wed Feb 6 13:02:13 2019 -0800

    Committing a param change to kick teamcity

commit 4f51024b318bf744eddb9d2f9638f7590872cf14
Author: Justin Borromeo <jborrome@edu.uwaterloo.ca>
Date:   Wed Feb 6 12:08:12 2019 -0800

    Wrote more tests for scan result value sort

commit 8b7d5f50818b00730965a55b1bf8ed27860bd6a4
Author: Justin Borromeo <jborrome@edu.uwaterloo.ca>
Date:   Wed Feb 6 11:55:09 2019 -0800

    Wrote tests for heapsort scan result values and fixed bug where iterator
    wasn't returning elements in correct order

commit b6d4df3864e3910fa406dcc83f6644f45f496c5f
Author: Justin Borromeo <jborrome@edu.uwaterloo.ca>
Date:   Tue Feb 5 16:45:20 2019 -0800

    Decrease segment size for less memory usage

commit d1a1793f36d4c9c910f84318f2bbbd355533c977
Author: Justin Borromeo <jborrome@edu.uwaterloo.ca>
Date:   Tue Feb 5 12:40:26 2019 -0800

    nit

commit 7deb06f6df47c55469a77a92622509ce88150ad5
Merge: b7d3a4900 86c5eee13
Author: Justin Borromeo <jborrome@edu.uwaterloo.ca>
Date:   Tue Feb 5 10:53:38 2019 -0800

    Merge branch '6088-Create-Scan-Benchmark' into 6088-Time-Ordering-On-Scans-V2

commit 86c5eee13b6ce18b33c723cd0c4e464eaf41f010
Author: Justin Borromeo <jborrome@edu.uwaterloo.ca>
Date:   Tue Feb 5 10:31:27 2019 -0800

    Broke some long lines into two lines

commit b7d3a4900afb2b56b5e2667c2d37fa4872c67219
Merge: 796083f2b 8bc5eaa90
Author: Justin Borromeo <jborrome@edu.uwaterloo.ca>
Date:   Tue Feb 5 10:23:32 2019 -0800

    Merge branch 'master' into 6088-Time-Ordering-On-Scans-V2

commit 737a83321d74cd0b1f7b4ca800509c36056d08ff
Author: Justin Borromeo <jborrome@edu.uwaterloo.ca>
Date:   Tue Feb 5 10:15:32 2019 -0800

    Made Jon's changes and removed TODOs

commit 796083f2bb188421f68858111bb39c988cb2f71c
Author: Justin Borromeo <jborrome@edu.uwaterloo.ca>
Date:   Mon Feb 4 15:37:42 2019 -0800

    Benchmark param change

commit 20c36644dbbf46df1a9209a635e661c01aeec627
Author: Justin Borromeo <jborrome@edu.uwaterloo.ca>
Date:   Mon Feb 4 15:36:35 2019 -0800

    More param changes

commit 9e6e71616bdcd9a7eea56e4bc1ef869c08bcf83c
Author: Justin Borromeo <jborrome@edu.uwaterloo.ca>
Date:   Mon Feb 4 15:31:21 2019 -0800

    Changed benchmark params

commit 01b25ed11293f472dac78d4f793f2941c3b22a18
Author: Justin Borromeo <jborrome@edu.uwaterloo.ca>
Date:   Mon Feb 4 14:36:18 2019 -0800

    Added time ordering to the scan benchmark

commit 432acaf08575c451ea02e8ec8d6318678dcf20cb
Author: Justin Borromeo <jborrome@edu.uwaterloo.ca>
Date:   Mon Feb 4 12:03:14 2019 -0800

    Change number of benchmark iterations

commit 12e51a272124c7a75628fe5b2f65ddc00e34ba27
Author: Justin Borromeo <jborrome@edu.uwaterloo.ca>
Date:   Mon Feb 4 12:02:13 2019 -0800

    Added TimestampComparator tests

commit e66339cd76cdb7f08a291e8488e3415518f3df63
Author: Justin Borromeo <jborrome@edu.uwaterloo.ca>
Date:   Mon Feb 4 10:56:41 2019 -0800

    Remove todos

commit ad731a362b465e9b4ca0c9ad7050fc6555606d52
Author: Justin Borromeo <jborrome@edu.uwaterloo.ca>
Date:   Mon Feb 4 10:55:56 2019 -0800

    Change benchmark

commit 989bd2d50e2419a715426b8fe903398d20429ff5
Merge: 7b5847139 26930f8d2
Author: Justin Borromeo <jborrome@edu.uwaterloo.ca>
Date:   Mon Feb 4 10:46:38 2019 -0800

    Merge branch '6088-Create-Scan-Benchmark' into 6088-Time-Ordering-On-Scans-V2

commit 7b584713946b538d15da591e306ca4c0a7a378e3
Author: Justin Borromeo <jborrome@edu.uwaterloo.ca>
Date:   Sat Feb 2 03:48:18 2019 -0800

    Licensing stuff

commit 79e8319383eddbb49ecb4c1785dcd3eed14a0634
Author: Justin Borromeo <jborrome@edu.uwaterloo.ca>
Date:   Fri Feb 1 18:22:58 2019 -0800

    Move ScanResultValue timestamp comparator to a separate class for testing

commit 7a6080f636ab2ead5ff85a29ea6b9cc04d93b353
Author: Justin Borromeo <jborrome@edu.uwaterloo.ca>
Date:   Fri Feb 1 18:00:58 2019 -0800

    Stuff for time-ordered scan query

commit 26930f8d2021d1d62322c54e0ec35e260137ab1d
Author: Justin Borromeo <jborrome@edu.uwaterloo.ca>
Date:   Fri Feb 1 16:38:49 2019 -0800

    It runs.

commit dd4ec1ac9c1194144e3ec98b811adc59598c8d8c
Author: Justin Borromeo <jborrome@edu.uwaterloo.ca>
Date:   Fri Feb 1 15:12:17 2019 -0800

    Need to form queries

commit dba6e492a067b9bb4f77f3db4b19c340f85ef54f
Merge: 10e57d5f9 7d4cc2873
Author: Justin Borromeo <jborrome@edu.uwaterloo.ca>
Date:   Fri Feb 1 14:13:39 2019 -0800

    Merge branch 'master' into 6088-Create-Scan-Benchmark

commit 10e57d5f9ed003e032c82240045125002903a5bb
Author: Justin Borromeo <jborrome@edu.uwaterloo.ca>
Date:   Fri Feb 1 14:04:13 2019 -0800

    Moved Scan Builder to Druids class and started on Scan Benchmark setup

* Changed SQL planning to use scan over select

* Fixed some bugs

* Removed unused imports

* Updated calcite query test and test segment walker

* Fixed formatting recommendations
 2019-04-02 15:46:01 -07:00
Clint Wylie
a99f0ff450 prefix no-op aggs with "Noop" (#6960) 2019-04-02 15:05:07 -07:00
David Glasser
4e23c11345 Make IngestSegmentFirehoseFactory splittable for parallel ingestion (#7048) 
* Make IngestSegmentFirehoseFactory splittable for parallel ingestion

* Code review feedback

- Get rid of WindowedSegment
- Don't document 'segments' parameter or support splitting firehoses that use it
- Require 'intervals' in WindowedSegmentId (since it won't be written by hand)

* Add missing @JsonProperty

* Integration test passes

* Add unit test

* Remove two FIXME comments from CompactionTask

I'd like to leave this PR in a potentially mergeable state, but I still would
appreciate reviewer eyes on the questions I'm removing here.

* Updates from code review
 2019-04-02 14:59:17 -07:00
Xue Yu
78fd5aff21 support radians and degrees in sql (#7336) 
* support radians and degrees in sql

* update test case
 2019-04-02 12:47:49 -07:00
Qi Shu
134f71d1b4 Add documentation for Druid native query in SQL view of web console (#7381) 
* Add docmentation for Druid native query in SQL view of web console

* Edit sentence
 2019-04-02 12:20:51 -07:00
Vadim Ogievetsky
d2ff3eb199 Migrate the web console back to Blueprint v3 (#7398) 
* migrate back to Blueprint v3

* rename filler file

* fix missing icon
 2019-04-01 23:06:48 -07:00
Vadim Ogievetsky
1137bb2586 Popovers are no longer inline (#7389) 2019-04-01 13:10:48 -07:00
Jonathan Wei
2bf6fc353a Update scan benchmark for time ordering (#7385) 2019-03-30 11:36:25 -07:00
Qi Shu
2ad9544242 Support segment metadata in rune (#7371) 2019-03-29 15:46:09 -07:00
Michael Trelinski
347779b17a Zookeeper loss (#6740) 
* Update init

Fix bin/init to source from proper directory.

* Fix for Proposal #6518: Shutdown druid processes upon complete loss of ZK connectivity

* Zookeeper Loss:

- Add feature documentation
- Cosmetic refactors
- Variable extractions
- Remove getter

* - Change config key name and reword documentation
- Switch from Function<Void,Void> to Runnable/Lambda
- try { … } finally { … }

* Fix line length too long

* - change to formatted string for logging
- use System.err.println after lifecycle stops

* commenting on makeEnsembleProvider()-created Zookeeper termination

* Add javadoc

* added java doc reference back to apache discussion thread.

* move comment to other class

* favor two-slash comments instead of multiline comments
 2019-03-29 15:10:42 -07:00
Jihoon Son
62c3e89266 maxTotalRows should be checked in DataSourceCompactionConfig before setting targetCompactionSizeBytes (#7368) 
* maxTotalRows should be checked in DataSourceCompactionConfig before setting targetCompactionSizeBytes

* remove unnecessary default values

* remove flacky test

* fix build

* Add comments
 2019-03-28 20:25:10 -07:00
Qi Shu
a09aa13ead Put all local storage keys and functions into one file in util (#7314) 
* Add all local storage keys and functions into one file in util

* Use LocalStorageKey type instead of string

* Remove druid author key
 2019-03-28 17:48:37 -07:00
Qi Shu
be1d7f885b Add scss linter in web console (#7365) 
* Add SCSS linter

* Fixed SCSS styles
 2019-03-28 17:32:01 -07:00
Justin Borromeo
ad7862c58a Time Ordering On Scans (#7133) 
* Moved Scan Builder to Druids class and started on Scan Benchmark setup

* Need to form queries

* It runs.

* Stuff for time-ordered scan query

* Move ScanResultValue timestamp comparator to a separate class for testing

* Licensing stuff

* Change benchmark

* Remove todos

* Added TimestampComparator tests

* Change number of benchmark iterations

* Added time ordering to the scan benchmark

* Changed benchmark params

* More param changes

* Benchmark param change

* Made Jon's changes and removed TODOs

* Broke some long lines into two lines

* nit

* Decrease segment size for less memory usage

* Wrote tests for heapsort scan result values and fixed bug where iterator
wasn't returning elements in correct order

* Wrote more tests for scan result value sort

* Committing a param change to kick teamcity

* Fixed codestyle and forbidden API errors

* .

* Improved conciseness

* nit

* Created an error message for when someone tries to time order a result
set > threshold limit

* Set to spaces over tabs

* Fixing tests WIP

* Fixed failing calcite tests

* Kicking travis with change to benchmark param

* added all query types to scan benchmark

* Fixed benchmark queries

* Renamed sort function

* Added javadoc on ScanResultValueTimestampComparator

* Unused import

* Added more javadoc

* improved doc

* Removed unused import to satisfy PMD check

* Small changes

* Changes based on Gian's comments

* Fixed failing test due to null resultFormat

* Added config and get # of segments

* Set up time ordering strategy decision tree

* Refactor and pQueue works

* Cleanup

* Ordering is correct on n-way merge -> still need to batch events into
ScanResultValues

* WIP

* Sequence stuff is so dirty :(

* Fixed bug introduced by replacing deque with list

* Wrote docs

* Multi-historical setup works

* WIP

* Change so batching only occurs on broker for time-ordered scans

Restricted batching to broker for time-ordered queries and adjusted
tests

Formatting

Cleanup

* Fixed mistakes in merge

* Fixed failing tests

* Reset config

* Wrote tests and added Javadoc

* Nit-change on javadoc

* Checkstyle fix

* Improved test and appeased TeamCity

* Sorry, checkstyle

* Applied Jon's recommended changes

* Checkstyle fix

* Optimization

* Fixed tests

* Updated error message

* Added error message for UOE

* Renaming

* Finish rename

* Smarter limiting for pQueue method

* Optimized n-way merge strategy

* Rename segment limit -> segment partitions limit

* Added a bit of docs

* More comments

* Fix checkstyle and test

* Nit comment

* Fixed failing tests -> allow usage of all types of segment spec

* Fixed failing tests -> allow usage of all types of segment spec

* Revert "Fixed failing tests -> allow usage of all types of segment spec"

This reverts commit ec470288c7b725f5310bcf69d1db9f85ff509c8d.

* Revert "Merge branch '6088-Time-Ordering-On-Scans-N-Way-Merge' of github.com:justinborromeo/incubator-druid into 6088-Time-Ordering-On-Scans-N-Way-Merge"

This reverts commit 57033f36df6e3ef887e5f0399ad74bb091306de8, reversing
changes made to 8f01d8dd16f40d10c60519ca0ec0d2e6b2dde941.

* Check type of segment spec before using for time ordering

* Fix bug in numRowsScanned

* Fix bug messing up count of rows

* Fix docs and flipped boolean in ScanQueryLimitRowIterator

* Refactor n-way merge

* Added test for n-way merge

* Refixed regression

* Checkstyle and doc update

* Modified sequence limit to accept longs and added test for long limits

* doc fix

* Implemented Clint's recommendations
 2019-03-28 14:37:09 -07:00