Commit Graph

10065 Commits

Author SHA1 Message Date
Jonathan Wei 0f6cb1e7e0 Update theta/hll sketch doc comparison (#7407) 2019-04-03 15:21:33 -07:00
Gian Merlino 8c104a115c
SQL: Add STRING_FORMAT function. (#7327) 2019-04-03 17:09:54 -04:00
Alexander Saydakov 28b4e8586d use latest sketches-core-0.13.1 (#7320)
* use latest sketches-core-0.13.0

* latest release
2019-04-03 17:06:02 -04:00
lxqfy 6b741f0800 Overwrite index task maxTotalRows with computed maxRowsPerSegments (#7338)
fixed identation

set maxTotalRows to higher value in case mutiple segments in appendrator
with comments
2019-04-03 11:47:59 -07:00
Atul Mohan c883c52cb1 Fix tests (#7401) 2019-04-02 16:49:21 -07:00
Justin Borromeo 4584b5e139 SQL support for time-ordered scan (#7373)
* Squashed commit of the following:

commit 287a367f41
Author: Justin Borromeo <jborrome@edu.uwaterloo.ca>
Date:   Wed Mar 27 20:03:41 2019 -0700

    Implemented Clint's recommendations

commit 07503ea5c0
Author: Justin Borromeo <jborrome@edu.uwaterloo.ca>
Date:   Wed Mar 27 17:49:09 2019 -0700

    doc fix

commit 231a72e7d9
Author: Justin Borromeo <jborrome@edu.uwaterloo.ca>
Date:   Wed Mar 27 17:38:20 2019 -0700

    Modified sequence limit to accept longs and added test for long limits

commit 1df50de321
Merge: 480e932fd c7fea6ac8
Author: Justin Borromeo <jborrome@edu.uwaterloo.ca>
Date:   Tue Mar 26 15:23:01 2019 -0700

    Merge branch 'master' into 6088-Time-Ordering-On-Scans-N-Way-Merge

commit 480e932fdf
Author: Justin Borromeo <jborrome@edu.uwaterloo.ca>
Date:   Tue Mar 26 14:58:04 2019 -0700

    Checkstyle and doc update

commit 487f31fcf6
Author: Justin Borromeo <jborrome@edu.uwaterloo.ca>
Date:   Tue Mar 26 14:39:25 2019 -0700

    Refixed regression

commit fb858efbb7
Author: Justin Borromeo <jborrome@edu.uwaterloo.ca>
Date:   Tue Mar 26 13:14:48 2019 -0700

    Added test for n-way merge

commit 376e8bf906
Author: Justin Borromeo <jborrome@edu.uwaterloo.ca>
Date:   Tue Mar 26 11:42:54 2019 -0700

    Refactor n-way merge

commit 8a6bb1127c
Author: Justin Borromeo <jborrome@edu.uwaterloo.ca>
Date:   Mon Mar 25 17:17:41 2019 -0700

    Fix docs and flipped boolean in ScanQueryLimitRowIterator

commit 35692680fc
Author: Justin Borromeo <jborrome@edu.uwaterloo.ca>
Date:   Mon Mar 25 16:15:49 2019 -0700

    Fix bug messing up count of rows

commit 219af478c8
Author: Justin Borromeo <jborrome@edu.uwaterloo.ca>
Date:   Mon Mar 25 15:57:55 2019 -0700

    Fix bug in numRowsScanned

commit da4fc66403
Author: Justin Borromeo <jborrome@edu.uwaterloo.ca>
Date:   Mon Mar 25 15:19:45 2019 -0700

    Check type of segment spec before using for time ordering

commit b822fc73df
Author: Justin Borromeo <jborrome@edu.uwaterloo.ca>
Date:   Mon Mar 25 13:19:02 2019 -0700

    Revert "Merge branch '6088-Time-Ordering-On-Scans-N-Way-Merge' of github.com:justinborromeo/incubator-druid into 6088-Time-Ordering-On-Scans-N-Way-Merge"

    This reverts commit 57033f36df, reversing
    changes made to 8f01d8dd16.

commit 57033f36df
Merge: 8f01d8dd1 86d9730fc
Author: Justin Borromeo <jborrome@edu.uwaterloo.ca>
Date:   Mon Mar 25 13:13:52 2019 -0700

    Merge branch '6088-Time-Ordering-On-Scans-N-Way-Merge' of github.com:justinborromeo/incubator-druid into 6088-Time-Ordering-On-Scans-N-Way-Merge

commit 8f01d8dd16
Author: Justin Borromeo <jborrome@edu.uwaterloo.ca>
Date:   Mon Mar 25 13:13:32 2019 -0700

    Revert "Fixed failing tests -> allow usage of all types of segment spec"

    This reverts commit ec470288c7.

commit ec470288c7
Author: Justin Borromeo <jborrome@edu.uwaterloo.ca>
Date:   Mon Mar 25 11:01:35 2019 -0700

    Fixed failing tests -> allow usage of all types of segment spec

commit 86d9730fc9
Author: Justin Borromeo <jborrome@edu.uwaterloo.ca>
Date:   Mon Mar 25 11:01:35 2019 -0700

    Fixed failing tests -> allow usage of all types of segment spec

commit 8b3b6b51ed
Author: Justin Borromeo <jborrome@edu.uwaterloo.ca>
Date:   Fri Mar 22 16:01:56 2019 -0700

    Nit comment

commit a87d02127c
Author: Justin Borromeo <jborrome@edu.uwaterloo.ca>
Date:   Fri Mar 22 15:54:42 2019 -0700

    Fix checkstyle and test

commit 62dcedacde
Author: Justin Borromeo <jborrome@edu.uwaterloo.ca>
Date:   Fri Mar 22 15:30:41 2019 -0700

    More comments

commit 1b46b58aec
Author: Justin Borromeo <jborrome@edu.uwaterloo.ca>
Date:   Fri Mar 22 15:19:52 2019 -0700

    Added a bit of docs

commit 49472162b7
Author: Justin Borromeo <jborrome@edu.uwaterloo.ca>
Date:   Fri Mar 22 10:27:41 2019 -0700

    Rename segment limit -> segment partitions limit

commit 43d490cc3a
Author: Justin Borromeo <jborrome@edu.uwaterloo.ca>
Date:   Thu Mar 21 13:16:58 2019 -0700

    Optimized n-way merge strategy

commit 42f5246b8d
Author: Justin Borromeo <jborrome@edu.uwaterloo.ca>
Date:   Wed Mar 20 17:40:19 2019 -0700

    Smarter limiting for pQueue method

commit 4823dab895
Author: Justin Borromeo <jborrome@edu.uwaterloo.ca>
Date:   Wed Mar 20 16:05:53 2019 -0700

    Finish rename

commit 2528a56142
Author: Justin Borromeo <jborrome@edu.uwaterloo.ca>
Date:   Mon Mar 18 14:00:50 2019 -0700

    Renaming

commit 7bfa77d3c1
Merge: a032c46ee 7e49d4739
Author: Justin Borromeo <jborrome@edu.uwaterloo.ca>
Date:   Tue Mar 12 16:57:45 2019 -0700

    Merge branch 'Update-Query-Interrupted-Exception' into 6088-Time-Ordering-On-Scans-N-Way-Merge

commit 7e49d47391
Author: Justin Borromeo <jborrome@edu.uwaterloo.ca>
Date:   Tue Mar 12 16:51:25 2019 -0700

    Added error message for UOE

commit a032c46ee0
Author: Justin Borromeo <jborrome@edu.uwaterloo.ca>
Date:   Tue Mar 12 16:47:17 2019 -0700

    Updated error message

commit 57b5682654
Author: Justin Borromeo <jborrome@edu.uwaterloo.ca>
Date:   Tue Mar 12 12:44:02 2019 -0700

    Fixed tests

commit 45e95bb1f4
Author: Justin Borromeo <jborrome@edu.uwaterloo.ca>
Date:   Tue Mar 12 11:09:08 2019 -0700

    Optimization

commit cce917ab84
Author: Justin Borromeo <jborrome@edu.uwaterloo.ca>
Date:   Fri Mar 8 14:11:07 2019 -0800

    Checkstyle fix

commit 73f4038068
Author: Justin Borromeo <jborrome@edu.uwaterloo.ca>
Date:   Thu Mar 7 18:40:00 2019 -0800

    Applied Jon's recommended changes

commit fb966def83
Author: Justin Borromeo <jborrome@edu.uwaterloo.ca>
Date:   Thu Mar 7 11:03:01 2019 -0800

    Sorry, checkstyle

commit 6dc53b311c
Author: Justin Borromeo <jborrome@edu.uwaterloo.ca>
Date:   Wed Mar 6 10:34:13 2019 -0800

    Improved test and appeased TeamCity

commit 35c96d3557
Author: Justin Borromeo <jborrome@edu.uwaterloo.ca>
Date:   Mon Mar 4 16:00:44 2019 -0800

    Checkstyle fix

commit 2d1978d571
Merge: 83ec3fe1f 3398d3982
Author: Justin Borromeo <jborrome@edu.uwaterloo.ca>
Date:   Mon Mar 4 15:24:49 2019 -0800

    Merge branch 'master' into 6088-Time-Ordering-On-Scans-N-Way-Merge

commit 83ec3fe1f1
Author: Justin Borromeo <jborrome@edu.uwaterloo.ca>
Date:   Fri Mar 1 13:40:22 2019 -0800

    Nit-change on javadoc

commit 47c970b5f4
Author: Justin Borromeo <jborrome@edu.uwaterloo.ca>
Date:   Fri Mar 1 13:38:29 2019 -0800

    Wrote tests and added Javadoc

commit 5ff59f5ca6
Author: Justin Borromeo <jborrome@edu.uwaterloo.ca>
Date:   Thu Feb 28 15:58:20 2019 -0800

    Reset config

commit 806166f977
Author: Justin Borromeo <jborrome@edu.uwaterloo.ca>
Date:   Thu Feb 28 15:49:07 2019 -0800

    Fixed failing tests

commit de83b11a1b
Author: Justin Borromeo <jborrome@edu.uwaterloo.ca>
Date:   Tue Feb 26 16:40:48 2019 -0800

    Fixed mistakes in merge

commit 5bd0e1a32c
Merge: 18cce9a64 9fa649b3b
Author: Justin Borromeo <jborrome@edu.uwaterloo.ca>
Date:   Tue Feb 26 16:39:16 2019 -0800

    Merge branch 'master' into 6088-Time-Ordering-On-Scans-N-Way-Merge

commit 18cce9a646
Author: Justin Borromeo <jborrome@edu.uwaterloo.ca>
Date:   Tue Feb 26 13:16:44 2019 -0800

    Change so batching only occurs on broker for time-ordered scans

    Restricted batching to broker for time-ordered queries and adjusted
    tests

    Formatting

    Cleanup

commit 451e2b4365
Author: Justin Borromeo <jborrome@edu.uwaterloo.ca>
Date:   Tue Feb 26 11:14:27 2019 -0800

    WIP

commit 69b24bd851
Merge: 763c43df7 417b9f2fe
Author: Justin Borromeo <jborrome@edu.uwaterloo.ca>
Date:   Fri Feb 22 18:13:26 2019 -0800

    Merge branch 'master' into 6088-Time-Ordering-On-Scans-N-Way-Merge

commit 763c43df7e
Author: Justin Borromeo <jborrome@edu.uwaterloo.ca>
Date:   Fri Feb 22 18:07:06 2019 -0800

    Multi-historical setup works

commit 06a5218917
Author: Justin Borromeo <jborrome@edu.uwaterloo.ca>
Date:   Fri Feb 22 16:59:57 2019 -0800

    Wrote docs

commit 3b923dac9c
Author: Justin Borromeo <jborrome@edu.uwaterloo.ca>
Date:   Fri Feb 22 14:03:22 2019 -0800

    Fixed bug introduced by replacing deque with list

commit 023538d831
Author: Justin Borromeo <jborrome@edu.uwaterloo.ca>
Date:   Fri Feb 22 13:30:08 2019 -0800

    Sequence stuff is so dirty :(

commit e1fc2955d3
Author: Justin Borromeo <jborrome@edu.uwaterloo.ca>
Date:   Fri Feb 22 10:39:59 2019 -0800

    WIP

commit f57ff253fa
Author: Justin Borromeo <jborrome@edu.uwaterloo.ca>
Date:   Thu Feb 21 18:22:06 2019 -0800

    Ordering is correct on n-way merge -> still need to batch events into
    ScanResultValues

commit 1813a5472c
Author: Justin Borromeo <jborrome@edu.uwaterloo.ca>
Date:   Thu Feb 21 17:06:18 2019 -0800

    Cleanup

commit f83e99655d
Author: Justin Borromeo <jborrome@edu.uwaterloo.ca>
Date:   Thu Feb 21 16:56:36 2019 -0800

    Refactor and pQueue works

commit b13ff624a9
Author: Justin Borromeo <jborrome@edu.uwaterloo.ca>
Date:   Thu Feb 21 15:13:33 2019 -0800

    Set up time ordering strategy decision tree

commit fba6b022f0
Author: Justin Borromeo <jborrome@edu.uwaterloo.ca>
Date:   Thu Feb 21 15:08:27 2019 -0800

    Added config and get # of segments

commit c9142e721c
Merge: cd489a020 554b0142c
Author: Justin Borromeo <jborrome@edu.uwaterloo.ca>
Date:   Wed Feb 20 10:12:50 2019 -0800

    Merge branch 'master' into 6088-Time-Ordering-On-Scans-V2

commit cd489a0208
Author: Justin Borromeo <jborrome@edu.uwaterloo.ca>
Date:   Wed Feb 20 00:16:48 2019 -0800

    Fixed failing test due to null resultFormat

commit 7baeade832
Author: Justin Borromeo <jborrome@edu.uwaterloo.ca>
Date:   Tue Feb 19 17:52:06 2019 -0800

    Changes based on Gian's comments

commit 35150fe1a6
Author: Justin Borromeo <jborrome@edu.uwaterloo.ca>
Date:   Fri Feb 15 15:57:53 2019 -0800

    Small changes

commit 4e69276d57
Author: Justin Borromeo <jborrome@edu.uwaterloo.ca>
Date:   Mon Feb 11 12:09:54 2019 -0800

    Removed unused import to satisfy PMD check

commit ecb0f483a9
Author: Justin Borromeo <jborrome@edu.uwaterloo.ca>
Date:   Mon Feb 11 10:37:11 2019 -0800

    improved doc

commit f0eddee665
Author: Justin Borromeo <jborrome@edu.uwaterloo.ca>
Date:   Mon Feb 11 10:18:45 2019 -0800

    Added more javadoc

commit 5f92dd7325
Author: Justin Borromeo <jborrome@edu.uwaterloo.ca>
Date:   Mon Feb 11 10:05:58 2019 -0800

    Unused import

commit 93e1636287
Author: Justin Borromeo <jborrome@edu.uwaterloo.ca>
Date:   Mon Feb 11 10:03:14 2019 -0800

    Added javadoc on ScanResultValueTimestampComparator

commit 134041c479
Author: Justin Borromeo <jborrome@edu.uwaterloo.ca>
Date:   Fri Feb 8 13:13:54 2019 -0800

    Renamed sort function

commit 2e3577cd3d
Author: Justin Borromeo <jborrome@edu.uwaterloo.ca>
Date:   Thu Feb 7 13:01:25 2019 -0800

    Fixed benchmark queries

commit d3b335af42
Author: Justin Borromeo <jborrome@edu.uwaterloo.ca>
Date:   Thu Feb 7 11:08:07 2019 -0800

    added all query types to scan benchmark

commit ab00eade9f
Author: Justin Borromeo <jborrome@edu.uwaterloo.ca>
Date:   Thu Feb 7 09:42:48 2019 -0800

    Kicking travis with change to benchmark param

commit b432beaf84
Author: Justin Borromeo <jborrome@edu.uwaterloo.ca>
Date:   Wed Feb 6 17:45:59 2019 -0800

    Fixed failing calcite tests

commit b2c8c77ad4
Author: Justin Borromeo <jborrome@edu.uwaterloo.ca>
Date:   Wed Feb 6 17:39:48 2019 -0800

    Fixing tests WIP

commit 85e72a614e
Author: Justin Borromeo <jborrome@edu.uwaterloo.ca>
Date:   Wed Feb 6 15:42:02 2019 -0800

    Set to spaces over tabs

commit 7e872a8ebc
Author: Justin Borromeo <jborrome@edu.uwaterloo.ca>
Date:   Wed Feb 6 15:36:24 2019 -0800

    Created an error message for when someone tries to time order a result
    set > threshold limit

commit e8a4b49044
Merge: 305876a43 8e3a58f72
Author: Justin Borromeo <jborrome@edu.uwaterloo.ca>
Date:   Wed Feb 6 15:05:11 2019 -0800

    Merge branch 'master' into 6088-Time-Ordering-On-Scans-V2

commit 305876a434
Author: Justin Borromeo <jborrome@edu.uwaterloo.ca>
Date:   Wed Feb 6 15:02:02 2019 -0800

    nit

commit 8212a21caf
Author: Justin Borromeo <jborrome@edu.uwaterloo.ca>
Date:   Wed Feb 6 14:40:35 2019 -0800

    Improved conciseness

commit 10b5e0ca93
Author: Justin Borromeo <jborrome@edu.uwaterloo.ca>
Date:   Wed Feb 6 13:42:12 2019 -0800

    .

commit dfe4aa9681
Author: Justin Borromeo <jborrome@edu.uwaterloo.ca>
Date:   Wed Feb 6 13:41:18 2019 -0800

    Fixed codestyle and forbidden API errors

commit 148939e88b
Merge: 4f51024b3 5edbe2ae1
Author: Justin Borromeo <jborrome@edu.uwaterloo.ca>
Date:   Wed Feb 6 13:26:17 2019 -0800

    Merge branch '6088-Create-Scan-Benchmark' into 6088-Time-Ordering-On-Scans-V2

commit 5edbe2ae12
Merge: 60b7684db 315ccb76b
Author: Justin Borromeo <jborrome@edu.uwaterloo.ca>
Date:   Wed Feb 6 13:18:55 2019 -0800

    Merge github.com:apache/incubator-druid into 6088-Create-Scan-Benchmark

commit 60b7684db7
Author: Justin Borromeo <jborrome@edu.uwaterloo.ca>
Date:   Wed Feb 6 13:02:13 2019 -0800

    Committing a param change to kick teamcity

commit 4f51024b31
Author: Justin Borromeo <jborrome@edu.uwaterloo.ca>
Date:   Wed Feb 6 12:08:12 2019 -0800

    Wrote more tests for scan result value sort

commit 8b7d5f5081
Author: Justin Borromeo <jborrome@edu.uwaterloo.ca>
Date:   Wed Feb 6 11:55:09 2019 -0800

    Wrote tests for heapsort scan result values and fixed bug where iterator
    wasn't returning elements in correct order

commit b6d4df3864
Author: Justin Borromeo <jborrome@edu.uwaterloo.ca>
Date:   Tue Feb 5 16:45:20 2019 -0800

    Decrease segment size for less memory usage

commit d1a1793f36
Author: Justin Borromeo <jborrome@edu.uwaterloo.ca>
Date:   Tue Feb 5 12:40:26 2019 -0800

    nit

commit 7deb06f6df
Merge: b7d3a4900 86c5eee13
Author: Justin Borromeo <jborrome@edu.uwaterloo.ca>
Date:   Tue Feb 5 10:53:38 2019 -0800

    Merge branch '6088-Create-Scan-Benchmark' into 6088-Time-Ordering-On-Scans-V2

commit 86c5eee13b
Author: Justin Borromeo <jborrome@edu.uwaterloo.ca>
Date:   Tue Feb 5 10:31:27 2019 -0800

    Broke some long lines into two lines

commit b7d3a4900a
Merge: 796083f2b 8bc5eaa90
Author: Justin Borromeo <jborrome@edu.uwaterloo.ca>
Date:   Tue Feb 5 10:23:32 2019 -0800

    Merge branch 'master' into 6088-Time-Ordering-On-Scans-V2

commit 737a83321d
Author: Justin Borromeo <jborrome@edu.uwaterloo.ca>
Date:   Tue Feb 5 10:15:32 2019 -0800

    Made Jon's changes and removed TODOs

commit 796083f2bb
Author: Justin Borromeo <jborrome@edu.uwaterloo.ca>
Date:   Mon Feb 4 15:37:42 2019 -0800

    Benchmark param change

commit 20c36644db
Author: Justin Borromeo <jborrome@edu.uwaterloo.ca>
Date:   Mon Feb 4 15:36:35 2019 -0800

    More param changes

commit 9e6e71616b
Author: Justin Borromeo <jborrome@edu.uwaterloo.ca>
Date:   Mon Feb 4 15:31:21 2019 -0800

    Changed benchmark params

commit 01b25ed112
Author: Justin Borromeo <jborrome@edu.uwaterloo.ca>
Date:   Mon Feb 4 14:36:18 2019 -0800

    Added time ordering to the scan benchmark

commit 432acaf085
Author: Justin Borromeo <jborrome@edu.uwaterloo.ca>
Date:   Mon Feb 4 12:03:14 2019 -0800

    Change number of benchmark iterations

commit 12e51a2721
Author: Justin Borromeo <jborrome@edu.uwaterloo.ca>
Date:   Mon Feb 4 12:02:13 2019 -0800

    Added TimestampComparator tests

commit e66339cd76
Author: Justin Borromeo <jborrome@edu.uwaterloo.ca>
Date:   Mon Feb 4 10:56:41 2019 -0800

    Remove todos

commit ad731a362b
Author: Justin Borromeo <jborrome@edu.uwaterloo.ca>
Date:   Mon Feb 4 10:55:56 2019 -0800

    Change benchmark

commit 989bd2d50e
Merge: 7b5847139 26930f8d2
Author: Justin Borromeo <jborrome@edu.uwaterloo.ca>
Date:   Mon Feb 4 10:46:38 2019 -0800

    Merge branch '6088-Create-Scan-Benchmark' into 6088-Time-Ordering-On-Scans-V2

commit 7b58471394
Author: Justin Borromeo <jborrome@edu.uwaterloo.ca>
Date:   Sat Feb 2 03:48:18 2019 -0800

    Licensing stuff

commit 79e8319383
Author: Justin Borromeo <jborrome@edu.uwaterloo.ca>
Date:   Fri Feb 1 18:22:58 2019 -0800

    Move ScanResultValue timestamp comparator to a separate class for testing

commit 7a6080f636
Author: Justin Borromeo <jborrome@edu.uwaterloo.ca>
Date:   Fri Feb 1 18:00:58 2019 -0800

    Stuff for time-ordered scan query

commit 26930f8d20
Author: Justin Borromeo <jborrome@edu.uwaterloo.ca>
Date:   Fri Feb 1 16:38:49 2019 -0800

    It runs.

commit dd4ec1ac9c
Author: Justin Borromeo <jborrome@edu.uwaterloo.ca>
Date:   Fri Feb 1 15:12:17 2019 -0800

    Need to form queries

commit dba6e492a0
Merge: 10e57d5f9 7d4cc2873
Author: Justin Borromeo <jborrome@edu.uwaterloo.ca>
Date:   Fri Feb 1 14:13:39 2019 -0800

    Merge branch 'master' into 6088-Create-Scan-Benchmark

commit 10e57d5f9e
Author: Justin Borromeo <jborrome@edu.uwaterloo.ca>
Date:   Fri Feb 1 14:04:13 2019 -0800

    Moved Scan Builder to Druids class and started on Scan Benchmark setup

* Changed SQL planning to use scan over select

* Fixed some bugs

* Removed unused imports

* Updated calcite query test and test segment walker

* Fixed formatting recommendations
2019-04-02 15:46:01 -07:00
Clint Wylie a99f0ff450 prefix no-op aggs with "Noop" (#6960) 2019-04-02 15:05:07 -07:00
David Glasser 4e23c11345 Make IngestSegmentFirehoseFactory splittable for parallel ingestion (#7048)
* Make IngestSegmentFirehoseFactory splittable for parallel ingestion

* Code review feedback

- Get rid of WindowedSegment
- Don't document 'segments' parameter or support splitting firehoses that use it
- Require 'intervals' in WindowedSegmentId (since it won't be written by hand)

* Add missing @JsonProperty

* Integration test passes

* Add unit test

* Remove two FIXME comments from CompactionTask

I'd like to leave this PR in a potentially mergeable state, but I still would
appreciate reviewer eyes on the questions I'm removing here.

* Updates from code review
2019-04-02 14:59:17 -07:00
Xue Yu 78fd5aff21 support radians and degrees in sql (#7336)
* support radians and degrees in sql

* update test case
2019-04-02 12:47:49 -07:00
Qi Shu 134f71d1b4 Add documentation for Druid native query in SQL view of web console (#7381)
* Add docmentation for Druid native query in SQL view of web console

* Edit sentence
2019-04-02 12:20:51 -07:00
Vadim Ogievetsky d2ff3eb199 Migrate the web console back to Blueprint v3 (#7398)
* migrate back to Blueprint v3

* rename filler file

* fix missing icon
2019-04-01 23:06:48 -07:00
Vadim Ogievetsky 1137bb2586 Popovers are no longer inline (#7389) 2019-04-01 13:10:48 -07:00
Jonathan Wei 2bf6fc353a Update scan benchmark for time ordering (#7385) 2019-03-30 11:36:25 -07:00
Qi Shu 2ad9544242 Support segment metadata in rune (#7371) 2019-03-29 15:46:09 -07:00
Michael Trelinski 347779b17a Zookeeper loss (#6740)
* Update init

Fix bin/init to source from proper directory.

* Fix for Proposal #6518: Shutdown druid processes upon complete loss of ZK connectivity

* Zookeeper Loss:

- Add feature documentation
- Cosmetic refactors
- Variable extractions
- Remove getter

* - Change config key name and reword documentation
- Switch from Function<Void,Void> to Runnable/Lambda
- try { … } finally { … }

* Fix line length too long

* - change to formatted string for logging
- use System.err.println after lifecycle stops

* commenting on makeEnsembleProvider()-created Zookeeper termination

* Add javadoc

* added java doc reference back to apache discussion thread.

* move comment to other class

* favor two-slash comments instead of multiline comments
2019-03-29 15:10:42 -07:00
Jihoon Son 62c3e89266 maxTotalRows should be checked in DataSourceCompactionConfig before setting targetCompactionSizeBytes (#7368)
* maxTotalRows should be checked in DataSourceCompactionConfig before setting targetCompactionSizeBytes

* remove unnecessary default values

* remove flacky test

* fix build

* Add comments
2019-03-28 20:25:10 -07:00
Qi Shu a09aa13ead Put all local storage keys and functions into one file in util (#7314)
* Add all local storage keys and functions into one file in util

* Use LocalStorageKey type instead of string

* Remove druid author key
2019-03-28 17:48:37 -07:00
Qi Shu be1d7f885b Add scss linter in web console (#7365)
* Add SCSS linter

* Fixed SCSS styles
2019-03-28 17:32:01 -07:00
Justin Borromeo ad7862c58a Time Ordering On Scans (#7133)
* Moved Scan Builder to Druids class and started on Scan Benchmark setup

* Need to form queries

* It runs.

* Stuff for time-ordered scan query

* Move ScanResultValue timestamp comparator to a separate class for testing

* Licensing stuff

* Change benchmark

* Remove todos

* Added TimestampComparator tests

* Change number of benchmark iterations

* Added time ordering to the scan benchmark

* Changed benchmark params

* More param changes

* Benchmark param change

* Made Jon's changes and removed TODOs

* Broke some long lines into two lines

* nit

* Decrease segment size for less memory usage

* Wrote tests for heapsort scan result values and fixed bug where iterator
wasn't returning elements in correct order

* Wrote more tests for scan result value sort

* Committing a param change to kick teamcity

* Fixed codestyle and forbidden API errors

* .

* Improved conciseness

* nit

* Created an error message for when someone tries to time order a result
set > threshold limit

* Set to spaces over tabs

* Fixing tests WIP

* Fixed failing calcite tests

* Kicking travis with change to benchmark param

* added all query types to scan benchmark

* Fixed benchmark queries

* Renamed sort function

* Added javadoc on ScanResultValueTimestampComparator

* Unused import

* Added more javadoc

* improved doc

* Removed unused import to satisfy PMD check

* Small changes

* Changes based on Gian's comments

* Fixed failing test due to null resultFormat

* Added config and get # of segments

* Set up time ordering strategy decision tree

* Refactor and pQueue works

* Cleanup

* Ordering is correct on n-way merge -> still need to batch events into
ScanResultValues

* WIP

* Sequence stuff is so dirty :(

* Fixed bug introduced by replacing deque with list

* Wrote docs

* Multi-historical setup works

* WIP

* Change so batching only occurs on broker for time-ordered scans

Restricted batching to broker for time-ordered queries and adjusted
tests

Formatting

Cleanup

* Fixed mistakes in merge

* Fixed failing tests

* Reset config

* Wrote tests and added Javadoc

* Nit-change on javadoc

* Checkstyle fix

* Improved test and appeased TeamCity

* Sorry, checkstyle

* Applied Jon's recommended changes

* Checkstyle fix

* Optimization

* Fixed tests

* Updated error message

* Added error message for UOE

* Renaming

* Finish rename

* Smarter limiting for pQueue method

* Optimized n-way merge strategy

* Rename segment limit -> segment partitions limit

* Added a bit of docs

* More comments

* Fix checkstyle and test

* Nit comment

* Fixed failing tests -> allow usage of all types of segment spec

* Fixed failing tests -> allow usage of all types of segment spec

* Revert "Fixed failing tests -> allow usage of all types of segment spec"

This reverts commit ec470288c7.

* Revert "Merge branch '6088-Time-Ordering-On-Scans-N-Way-Merge' of github.com:justinborromeo/incubator-druid into 6088-Time-Ordering-On-Scans-N-Way-Merge"

This reverts commit 57033f36df, reversing
changes made to 8f01d8dd16.

* Check type of segment spec before using for time ordering

* Fix bug in numRowsScanned

* Fix bug messing up count of rows

* Fix docs and flipped boolean in ScanQueryLimitRowIterator

* Refactor n-way merge

* Added test for n-way merge

* Refixed regression

* Checkstyle and doc update

* Modified sequence limit to accept longs and added test for long limits

* doc fix

* Implemented Clint's recommendations
2019-03-28 14:37:09 -07:00
Roman Leventov ffa95859c2 Remove [PROPOSAL] from Proposal issue template; Add 'Other' template for convenience (#7364)
* Remove [PROPOSAL] from Proposal issue template; Add 'Regular' template for convenience

* Regular -> Other
2019-03-28 18:40:43 +08:00
Surekha be318f4de3 Add column type to sys table docs (#7359)
* Add column type

* oops should be used=1
2019-03-27 20:21:57 -07:00
Clint Wylie db0125b709 this will fix it (#7358) 2019-03-27 19:42:58 -07:00
Qi Shu 18e5167245 Add history dialog in snitch dialog to allow viewing the editing history (#7321)
* Add history dialog in snitch dialog to allow viewing the editing history

* Improved CSS; better animation

* Use position: absolute instead of float: right to position element

* Removed author for history changes
2019-03-27 17:52:44 -07:00
Charles Allen eeb3dbe79d Move GCP to a core extension (#6953)
* Move GCP to a core extension

* Don't provide druid-core >.<

* Keep AWS and GCP modules separate

* Move AWSModule to its own module

* Add aws ec2 extension and more modules in more places

* Fix bad imports

* Fix test jackson module

* Include AWS and GCP core in server

* Add simple empty method comment

* Update version to 15

* One more 0.13.0-->0.15.0 change

* Fix multi-binding problem

* Grep for s3-extensions and update docs

* Update extensions.md
2019-03-27 09:00:43 -07:00
Jihoon Son 543324f8a9 Fix logging in IndexerSQLMetadataStorageCoordinator (#7349) 2019-03-26 20:36:19 -07:00
Jihoon Son 4d37edac1e Suppress stack trace in warning (#7348) 2019-03-26 17:27:29 -07:00
Roman Leventov b9354fbddb
Make 'field can be final' inspection a warning in IntelliJ config (#7301)
A small step towards #7227.
2019-03-26 20:45:03 -03:00
Justin Borromeo c7fea6ac8f Added better QueryInterruptedException error message for UnsupportedOperationException (#7248)
* Added error message for UOE

* Updated docs

* Doc change

* Doc change
2019-03-26 15:20:24 -07:00
Jihoon Son 5294277cb4
Fix exclusive start partitions for sequenceMetadata (#7339)
* Fix exclusvie start partitions for sequenceMetadata

* add empty check
2019-03-26 14:39:07 -07:00
Puneet Jaiswal 105b4fa237 Fixing a typo in logger initialization (#7341) 2019-03-27 00:38:28 +08:00
Roman Leventov bca40dcdaf
Fix some IntelliJ inspections (#7273)
Prepare TeamCity for IntelliJ 2018.3.1 upgrade. Mostly removed redundant exceptions declarations in `throws` clauses.
2019-03-25 21:11:01 -03:00
Jonathan Wei 8ca7cb4886 Fix rat check for source assembly after build (#7333) 2019-03-22 22:48:35 -07:00
Jihoon Son f410c28af6
Always convert start metadata to start (#7332) 2019-03-22 21:12:15 -07:00
Gian Merlino 4ca5fe0f60 SQL: Add PARSE_LONG function. (#7326)
* SQL: Add PARSE_LONG function.

* Fix test.
2019-03-22 15:40:10 -07:00
Surekha 01c021e6da Add Kafka integration test for transactional topics (#7295)
* Add integration test for transactional kafka

* Add true for transactions enabled for transactional test

* Add new test to travis_script_integration.sh, use version 0.2 of druid docker image

* Use different datasource name for ITKafkaIndexingServiceTest and ITKafkaIndexingServiceTransactionalTest

* use KafkaConsumerConfigs to get common consumer properties

* Remove double line breaks

* remove extra space
2019-03-22 13:39:05 -07:00
Jonathan Wei 7a57bc0dc3
Exclude git.version from rat check (#7322) 2019-03-21 20:54:27 -07:00
Vadim Ogievetsky 375668ef45 fix Supervisor terminate call (#7315)
* fix Supervisor terminate call

* fix refresh after submitting supervisor or task
2019-03-21 13:54:43 -07:00
Roman Leventov a4270da5f3
More TeamCity and Structural Search inspection instructions (#7275)
* More TeamCity and Structural Search inspection instructions

* the -> a

* an e-mail
2019-03-21 17:24:21 -03:00
Jihoon Son 0c5dcf5586 Fix exclusivity for start offset in kinesis indexing service & check exclusivity properly in IndexerSQLMetadataStorageCoordinator (#7291)
* Fix exclusivity for start offset in kinesis indexing service

* some adjustment

* Fix SeekableStreamDataSourceMetadata

* Add missing javadocs

* Add missing comments and unit test

* fix SeekableStreamStartSequenceNumbers.plus and add comments

* remove extra exclusivePartitions in KafkaIOConfig and fix downgrade issue

* Add javadocs

* fix compilation

* fix test

* remove unused variable
2019-03-21 13:12:22 -07:00
Surekha e170203876 Consolidate kafka consumer configs (#7249)
* Consolidate kafka consumer configs

* change the order of adding properties

* Add consumer properties to fix test

it seems kafka consumer does not reveive any message without these configs

* Use KafkaConsumerConfigs in integration test

* Update zookeeper and kafka versions in the setup.sh for the base druid image

*  use version 0.2 of base druid image

* Try to fix tests in KafkaRecordSupplierTest

* unused import

* Fix tests in KafkaSupervisorTest
2019-03-21 11:19:49 -07:00
Qi Shu 494c1a2ef8 Add overlord edit dialog to allow user to change overlord dynamic config on the fly (#7308)
* Add overlord edit dialog to allow user to edit overlord dynmamic config on the fly

* Set config to {} if druid return nothing for overlord config
2019-03-21 03:38:35 -07:00
Qi Shu b0271b4f4d Saving author to local storage in snitch dialog (#7307)
* Saving author to local storage

* Capitalized const
2019-03-21 03:01:10 -07:00
Vadim Ogievetsky e4f2dcacf2 Druid console docs (#7300)
* console docs

* fix typo
2019-03-21 00:37:33 -07:00
Qi Shu 30e646308a Add table column selection in druid console to allow hiding/showing of columns (#7292)
* Add table column selections to all tables to allow user to hide/show columns

* Small change for re-rendering

* Use column selection handler class to process all column hiding/showing

* dereference table handler function at the start; use more specific file name for table.tsx
2019-03-20 13:11:00 -07:00
Don Bowman 2814729d32 Remove mysql database connector from container image (#7296)
* Remove mysql database connector from container image

See discussion on `dev` mailing list from 2019-03
around [docker
build](http://mail-archives.apache.org/mod_mbox/druid-dev/201903.mbox/browser)

Remove the mysql database connector from the container
image in favour of solely the postgresql connector.

Instructions are present in distribution/docker/README.md
for someone to re-add if they need.

Signed-off-by: Don Bowman <don@agilicus.com>

* Add sample Dockerfile for mysql using pre-built upstream
2019-03-20 08:33:28 -07:00
Justin Borromeo ff94bd16e6 Fix conflicting information in configuration doc (#7299)
* Doc fix

* Fix typo
2019-03-19 14:55:58 -07:00
Roman Leventov dfd27e00c0
Avoid many unnecessary materializations of collections of 'all segments in cluster' cardinality (#7185)
* Avoid many  unnecessary materializations of collections of 'all segments in cluster' cardinality

* Fix DruidCoordinatorTest; Renamed DruidCoordinator.getReplicationStatus() to computeUnderReplicationCountsPerDataSourcePerTier()

* More Javadocs, typos, refactor DruidCoordinatorRuntimeParams.createAvailableSegmentsSet()

* Style

* typo

* Disable StaticPseudoFunctionalStyleMethod inspection because of too much false positives

* Fixes
2019-03-19 18:22:56 -03:00
Qi Shu e58f541047 Small tweak to override Blueprint's CSS rule for lookup edit dialog (#7283)
* Small tweak to override Blueprint's CSS rule

* Follow the same CSS style by using &
2019-03-18 14:24:50 -07:00
Christian Senet b2a940addd Fix flattening Avro Maps with Utf8 keys (#7258)
* #5884 Avro Maps can't be flattened in Kafka Indexing on master

* correct typo and reduce code size
2019-03-18 14:23:53 -07:00
Jonathan Wei c17f4bbad8 Fix 0.14.0 rc1 voting issues (#7284) 2019-03-18 14:16:21 -07:00