Commit Graph

9485 Commits

Author SHA1 Message Date
Benjamin Hopp 78e6f6fb38 Updated Javascript Affinity config docs (#7441)
Updated with hostname:port rather than IP Address.
2019-04-10 21:44:50 -07:00
Clint Wylie 15c3124e7f add auth check exclusion for console-config.js added in #7414 (#7442) 2019-04-10 21:44:26 -07:00
Justin Borromeo 2771ed50b0 Support Kafka supervisor adopting running tasks between versions (#7212)
* Recompute hash in isTaskCurrent() and added tests

* Fixed checkstyle stuff

* Fixed failing tests

* Make TestableKafkaSupervisorWithCustomIsTaskCurrent static

* Add doc

* baseSequenceName change

* Added comment

* WIP

* Fixed imports

* Undid lambda change for diff sake

* Cleanup

* Added comment

* Reinsert Kafka tests

* Readded kinesis test

* Readd bad partition assignment in kinesis supervisor test

* Nit

* Misnamed var
2019-04-10 18:16:38 -07:00
Benedict Jin 2f64414ade Add "REVERSE" / "REPEAT" / "RIGHT" / "LEFT" functions (#7334)
* Add "REVERSE" / "REPEAT" / "RIGHT" / "LEFT" functions

* Fix ImportOrder

* Use RuntimeException instead of OutOfMemoryError according to "Effective Java"

* Simplify

* Patch suggestions
2019-04-10 11:46:29 +08:00
Clint Wylie 89bb43f382 'core' ORC extension (#7138)
* orc extension reworked to use apache orc map-reduce lib, moved to core extensions, support for flattenSpec, tests, docs

* change binary handling to be compatible with avro and parquet, Rows.objectToStrings now converts byte[] to base64, change date handling

* better docs and tests

* fix it

* formatting

* doc fix

* fix it

* exclude redundant dependencies

* use latest orc-mapreduce, add hadoop jobProperties recommendations to docs

* doc fix

* review stuff and fix binaryAsString

* cache for root level fields

* more better
2019-04-09 09:03:26 -07:00
Jonathan Wei 09abc4a856 Fix comments in KinesisIndexTaskIOConfig (#7430) 2019-04-09 08:40:08 -07:00
Jihoon Son e87d6e32b3 Support kinesis compatibility (#7351) 2019-04-08 19:19:34 -07:00
Gian Merlino 7cd5477658 DoublesSketchComplexMetricSerde: Handle empty strings. (#7429) 2019-04-08 18:01:31 -07:00
Qi Shu 98ee8637e6 Console SQL function doc generation (#7406)
* Change sql function doc generation script from Bash to Node.js

* Minor code change for better reading

* Disable tslint in file

* clean up code

* Use es6 multiline string

* Solved a bug which would generate invalid TS variable if no function sections are found; in this case an empty array should be created

* Edit build script
2019-04-08 17:38:26 -07:00
Vadim Ogievetsky c69c37e99d Update console packages and enforce tslint quotes (#7419)
* update packages and enforce tslint quotes

* remove direct dep on tslint-react
2019-04-07 20:19:08 -07:00
Justin Borromeo 799c66d9ac Allow max rows and max segments for time-ordered scans to be overridden using the scan query JSON spec (#7413)
* Initial changes

* Fixed NPEs

* Fixed failing spec test

* Fixed failing Calcite test

* Move configs to context

* Validated and added docs

* fixed weird indentation

* Update default context vals in doc

* Fixed allowable values
2019-04-07 20:12:52 -07:00
Clint Wylie 7778f29781 add travis check to make sure -Pdist works (#7418) 2019-04-07 20:12:23 -07:00
Clint Wylie e28a15f9f5 fix expressions docs operator table (#7420)
* fix expressions docs operator table

* Update math-expr.md
2019-04-07 20:12:00 -07:00
Justin Borromeo e23fd41fa7 Update SQL doc for planning change (#7415) 2019-04-05 15:14:07 -07:00
Clint Wylie 76b4a5c62e refactor lookups to be more chill to router (#7222)
* refactor lookups to be more chill to router

* remove accidental change

* fix and combine LookupIntrospectionResourceTest

* fix inspection

* rename RouterLookupModule to LookupSerdeModule and RouterLookupExtractorFactoryContainerProvider to NoopLookupExtractorFactoryContainerProvider

* make comment generic

* use ConfigResourceFilter instead of StateResourceFilter

* fix indentation

* unused import

* another unused import

* refactor some stuff into processing module, split up LookupModule.java classes into their own files
2019-04-05 14:49:41 -07:00
Qi Shu a66291a9cd SQL explain in web console (#7402)
* Add explain for SQL query

* Terminate explain query manager

* Add signature for semi-joined queries

* Extract components out

* Hide more button in rune mode

* Add types for query explanation parser

* Add type for query-plan-dialog
2019-04-05 12:44:51 -07:00
Vadim Ogievetsky 0fa122ecf2 Adding ability to pass configs in and fixing misc bugs (#7414)
* Adding ability to pass configs in and fixing misc bugs

* update lock file

* remove dead version param
2019-04-05 12:40:43 -07:00
Clint Wylie e2aa6e47f8 change Dockerfile maintainers email to dev mailing list (#7417) 2019-04-04 19:34:48 -07:00
Gian Merlino 78745fea84 Fix two issues with Coordinator -> Overlord communication. (#7412)
* Fix two issues with Coordinator -> Overlord communication.

1) ClientCompactQuery needs to recognize the potential for 'intervals'
to be set instead of 'segments'. The lack of this led to a
NullPointerException on DruidCoordinatorSegmentCompactor.java:102.

2) In two locations (DruidCoordinatorSegmentCompactor,
DruidCoordinatorCleanupPendingSegments) tasks were being retrieved
using waiting/pending/running tasks in the wrong order: by checking
'running' first and then 'pending', tasks could be missed if they
moved from 'pending' to 'running' in between the two calls. Replaced
these methods with calls to 'getActiveTasks', a new method that does
the calls in the right order.

* Remove unused import.
2019-04-04 10:25:18 -07:00
Richard Startin d29a32062f upgrade to RoaringBitmap 0.8.0 and serialise directly to ByteBuffer (#7408) 2019-04-04 13:22:50 -04:00
Jonathan Wei 0f6cb1e7e0 Update theta/hll sketch doc comparison (#7407) 2019-04-03 15:21:33 -07:00
Gian Merlino 8c104a115c
SQL: Add STRING_FORMAT function. (#7327) 2019-04-03 17:09:54 -04:00
Alexander Saydakov 28b4e8586d use latest sketches-core-0.13.1 (#7320)
* use latest sketches-core-0.13.0

* latest release
2019-04-03 17:06:02 -04:00
lxqfy 6b741f0800 Overwrite index task maxTotalRows with computed maxRowsPerSegments (#7338)
fixed identation

set maxTotalRows to higher value in case mutiple segments in appendrator
with comments
2019-04-03 11:47:59 -07:00
Atul Mohan c883c52cb1 Fix tests (#7401) 2019-04-02 16:49:21 -07:00
Justin Borromeo 4584b5e139 SQL support for time-ordered scan (#7373)
* Squashed commit of the following:

commit 287a367f41
Author: Justin Borromeo <jborrome@edu.uwaterloo.ca>
Date:   Wed Mar 27 20:03:41 2019 -0700

    Implemented Clint's recommendations

commit 07503ea5c0
Author: Justin Borromeo <jborrome@edu.uwaterloo.ca>
Date:   Wed Mar 27 17:49:09 2019 -0700

    doc fix

commit 231a72e7d9
Author: Justin Borromeo <jborrome@edu.uwaterloo.ca>
Date:   Wed Mar 27 17:38:20 2019 -0700

    Modified sequence limit to accept longs and added test for long limits

commit 1df50de321
Merge: 480e932fd c7fea6ac8
Author: Justin Borromeo <jborrome@edu.uwaterloo.ca>
Date:   Tue Mar 26 15:23:01 2019 -0700

    Merge branch 'master' into 6088-Time-Ordering-On-Scans-N-Way-Merge

commit 480e932fdf
Author: Justin Borromeo <jborrome@edu.uwaterloo.ca>
Date:   Tue Mar 26 14:58:04 2019 -0700

    Checkstyle and doc update

commit 487f31fcf6
Author: Justin Borromeo <jborrome@edu.uwaterloo.ca>
Date:   Tue Mar 26 14:39:25 2019 -0700

    Refixed regression

commit fb858efbb7
Author: Justin Borromeo <jborrome@edu.uwaterloo.ca>
Date:   Tue Mar 26 13:14:48 2019 -0700

    Added test for n-way merge

commit 376e8bf906
Author: Justin Borromeo <jborrome@edu.uwaterloo.ca>
Date:   Tue Mar 26 11:42:54 2019 -0700

    Refactor n-way merge

commit 8a6bb1127c
Author: Justin Borromeo <jborrome@edu.uwaterloo.ca>
Date:   Mon Mar 25 17:17:41 2019 -0700

    Fix docs and flipped boolean in ScanQueryLimitRowIterator

commit 35692680fc
Author: Justin Borromeo <jborrome@edu.uwaterloo.ca>
Date:   Mon Mar 25 16:15:49 2019 -0700

    Fix bug messing up count of rows

commit 219af478c8
Author: Justin Borromeo <jborrome@edu.uwaterloo.ca>
Date:   Mon Mar 25 15:57:55 2019 -0700

    Fix bug in numRowsScanned

commit da4fc66403
Author: Justin Borromeo <jborrome@edu.uwaterloo.ca>
Date:   Mon Mar 25 15:19:45 2019 -0700

    Check type of segment spec before using for time ordering

commit b822fc73df
Author: Justin Borromeo <jborrome@edu.uwaterloo.ca>
Date:   Mon Mar 25 13:19:02 2019 -0700

    Revert "Merge branch '6088-Time-Ordering-On-Scans-N-Way-Merge' of github.com:justinborromeo/incubator-druid into 6088-Time-Ordering-On-Scans-N-Way-Merge"

    This reverts commit 57033f36df, reversing
    changes made to 8f01d8dd16.

commit 57033f36df
Merge: 8f01d8dd1 86d9730fc
Author: Justin Borromeo <jborrome@edu.uwaterloo.ca>
Date:   Mon Mar 25 13:13:52 2019 -0700

    Merge branch '6088-Time-Ordering-On-Scans-N-Way-Merge' of github.com:justinborromeo/incubator-druid into 6088-Time-Ordering-On-Scans-N-Way-Merge

commit 8f01d8dd16
Author: Justin Borromeo <jborrome@edu.uwaterloo.ca>
Date:   Mon Mar 25 13:13:32 2019 -0700

    Revert "Fixed failing tests -> allow usage of all types of segment spec"

    This reverts commit ec470288c7.

commit ec470288c7
Author: Justin Borromeo <jborrome@edu.uwaterloo.ca>
Date:   Mon Mar 25 11:01:35 2019 -0700

    Fixed failing tests -> allow usage of all types of segment spec

commit 86d9730fc9
Author: Justin Borromeo <jborrome@edu.uwaterloo.ca>
Date:   Mon Mar 25 11:01:35 2019 -0700

    Fixed failing tests -> allow usage of all types of segment spec

commit 8b3b6b51ed
Author: Justin Borromeo <jborrome@edu.uwaterloo.ca>
Date:   Fri Mar 22 16:01:56 2019 -0700

    Nit comment

commit a87d02127c
Author: Justin Borromeo <jborrome@edu.uwaterloo.ca>
Date:   Fri Mar 22 15:54:42 2019 -0700

    Fix checkstyle and test

commit 62dcedacde
Author: Justin Borromeo <jborrome@edu.uwaterloo.ca>
Date:   Fri Mar 22 15:30:41 2019 -0700

    More comments

commit 1b46b58aec
Author: Justin Borromeo <jborrome@edu.uwaterloo.ca>
Date:   Fri Mar 22 15:19:52 2019 -0700

    Added a bit of docs

commit 49472162b7
Author: Justin Borromeo <jborrome@edu.uwaterloo.ca>
Date:   Fri Mar 22 10:27:41 2019 -0700

    Rename segment limit -> segment partitions limit

commit 43d490cc3a
Author: Justin Borromeo <jborrome@edu.uwaterloo.ca>
Date:   Thu Mar 21 13:16:58 2019 -0700

    Optimized n-way merge strategy

commit 42f5246b8d
Author: Justin Borromeo <jborrome@edu.uwaterloo.ca>
Date:   Wed Mar 20 17:40:19 2019 -0700

    Smarter limiting for pQueue method

commit 4823dab895
Author: Justin Borromeo <jborrome@edu.uwaterloo.ca>
Date:   Wed Mar 20 16:05:53 2019 -0700

    Finish rename

commit 2528a56142
Author: Justin Borromeo <jborrome@edu.uwaterloo.ca>
Date:   Mon Mar 18 14:00:50 2019 -0700

    Renaming

commit 7bfa77d3c1
Merge: a032c46ee 7e49d4739
Author: Justin Borromeo <jborrome@edu.uwaterloo.ca>
Date:   Tue Mar 12 16:57:45 2019 -0700

    Merge branch 'Update-Query-Interrupted-Exception' into 6088-Time-Ordering-On-Scans-N-Way-Merge

commit 7e49d47391
Author: Justin Borromeo <jborrome@edu.uwaterloo.ca>
Date:   Tue Mar 12 16:51:25 2019 -0700

    Added error message for UOE

commit a032c46ee0
Author: Justin Borromeo <jborrome@edu.uwaterloo.ca>
Date:   Tue Mar 12 16:47:17 2019 -0700

    Updated error message

commit 57b5682654
Author: Justin Borromeo <jborrome@edu.uwaterloo.ca>
Date:   Tue Mar 12 12:44:02 2019 -0700

    Fixed tests

commit 45e95bb1f4
Author: Justin Borromeo <jborrome@edu.uwaterloo.ca>
Date:   Tue Mar 12 11:09:08 2019 -0700

    Optimization

commit cce917ab84
Author: Justin Borromeo <jborrome@edu.uwaterloo.ca>
Date:   Fri Mar 8 14:11:07 2019 -0800

    Checkstyle fix

commit 73f4038068
Author: Justin Borromeo <jborrome@edu.uwaterloo.ca>
Date:   Thu Mar 7 18:40:00 2019 -0800

    Applied Jon's recommended changes

commit fb966def83
Author: Justin Borromeo <jborrome@edu.uwaterloo.ca>
Date:   Thu Mar 7 11:03:01 2019 -0800

    Sorry, checkstyle

commit 6dc53b311c
Author: Justin Borromeo <jborrome@edu.uwaterloo.ca>
Date:   Wed Mar 6 10:34:13 2019 -0800

    Improved test and appeased TeamCity

commit 35c96d3557
Author: Justin Borromeo <jborrome@edu.uwaterloo.ca>
Date:   Mon Mar 4 16:00:44 2019 -0800

    Checkstyle fix

commit 2d1978d571
Merge: 83ec3fe1f 3398d3982
Author: Justin Borromeo <jborrome@edu.uwaterloo.ca>
Date:   Mon Mar 4 15:24:49 2019 -0800

    Merge branch 'master' into 6088-Time-Ordering-On-Scans-N-Way-Merge

commit 83ec3fe1f1
Author: Justin Borromeo <jborrome@edu.uwaterloo.ca>
Date:   Fri Mar 1 13:40:22 2019 -0800

    Nit-change on javadoc

commit 47c970b5f4
Author: Justin Borromeo <jborrome@edu.uwaterloo.ca>
Date:   Fri Mar 1 13:38:29 2019 -0800

    Wrote tests and added Javadoc

commit 5ff59f5ca6
Author: Justin Borromeo <jborrome@edu.uwaterloo.ca>
Date:   Thu Feb 28 15:58:20 2019 -0800

    Reset config

commit 806166f977
Author: Justin Borromeo <jborrome@edu.uwaterloo.ca>
Date:   Thu Feb 28 15:49:07 2019 -0800

    Fixed failing tests

commit de83b11a1b
Author: Justin Borromeo <jborrome@edu.uwaterloo.ca>
Date:   Tue Feb 26 16:40:48 2019 -0800

    Fixed mistakes in merge

commit 5bd0e1a32c
Merge: 18cce9a64 9fa649b3b
Author: Justin Borromeo <jborrome@edu.uwaterloo.ca>
Date:   Tue Feb 26 16:39:16 2019 -0800

    Merge branch 'master' into 6088-Time-Ordering-On-Scans-N-Way-Merge

commit 18cce9a646
Author: Justin Borromeo <jborrome@edu.uwaterloo.ca>
Date:   Tue Feb 26 13:16:44 2019 -0800

    Change so batching only occurs on broker for time-ordered scans

    Restricted batching to broker for time-ordered queries and adjusted
    tests

    Formatting

    Cleanup

commit 451e2b4365
Author: Justin Borromeo <jborrome@edu.uwaterloo.ca>
Date:   Tue Feb 26 11:14:27 2019 -0800

    WIP

commit 69b24bd851
Merge: 763c43df7 417b9f2fe
Author: Justin Borromeo <jborrome@edu.uwaterloo.ca>
Date:   Fri Feb 22 18:13:26 2019 -0800

    Merge branch 'master' into 6088-Time-Ordering-On-Scans-N-Way-Merge

commit 763c43df7e
Author: Justin Borromeo <jborrome@edu.uwaterloo.ca>
Date:   Fri Feb 22 18:07:06 2019 -0800

    Multi-historical setup works

commit 06a5218917
Author: Justin Borromeo <jborrome@edu.uwaterloo.ca>
Date:   Fri Feb 22 16:59:57 2019 -0800

    Wrote docs

commit 3b923dac9c
Author: Justin Borromeo <jborrome@edu.uwaterloo.ca>
Date:   Fri Feb 22 14:03:22 2019 -0800

    Fixed bug introduced by replacing deque with list

commit 023538d831
Author: Justin Borromeo <jborrome@edu.uwaterloo.ca>
Date:   Fri Feb 22 13:30:08 2019 -0800

    Sequence stuff is so dirty :(

commit e1fc2955d3
Author: Justin Borromeo <jborrome@edu.uwaterloo.ca>
Date:   Fri Feb 22 10:39:59 2019 -0800

    WIP

commit f57ff253fa
Author: Justin Borromeo <jborrome@edu.uwaterloo.ca>
Date:   Thu Feb 21 18:22:06 2019 -0800

    Ordering is correct on n-way merge -> still need to batch events into
    ScanResultValues

commit 1813a5472c
Author: Justin Borromeo <jborrome@edu.uwaterloo.ca>
Date:   Thu Feb 21 17:06:18 2019 -0800

    Cleanup

commit f83e99655d
Author: Justin Borromeo <jborrome@edu.uwaterloo.ca>
Date:   Thu Feb 21 16:56:36 2019 -0800

    Refactor and pQueue works

commit b13ff624a9
Author: Justin Borromeo <jborrome@edu.uwaterloo.ca>
Date:   Thu Feb 21 15:13:33 2019 -0800

    Set up time ordering strategy decision tree

commit fba6b022f0
Author: Justin Borromeo <jborrome@edu.uwaterloo.ca>
Date:   Thu Feb 21 15:08:27 2019 -0800

    Added config and get # of segments

commit c9142e721c
Merge: cd489a020 554b0142c
Author: Justin Borromeo <jborrome@edu.uwaterloo.ca>
Date:   Wed Feb 20 10:12:50 2019 -0800

    Merge branch 'master' into 6088-Time-Ordering-On-Scans-V2

commit cd489a0208
Author: Justin Borromeo <jborrome@edu.uwaterloo.ca>
Date:   Wed Feb 20 00:16:48 2019 -0800

    Fixed failing test due to null resultFormat

commit 7baeade832
Author: Justin Borromeo <jborrome@edu.uwaterloo.ca>
Date:   Tue Feb 19 17:52:06 2019 -0800

    Changes based on Gian's comments

commit 35150fe1a6
Author: Justin Borromeo <jborrome@edu.uwaterloo.ca>
Date:   Fri Feb 15 15:57:53 2019 -0800

    Small changes

commit 4e69276d57
Author: Justin Borromeo <jborrome@edu.uwaterloo.ca>
Date:   Mon Feb 11 12:09:54 2019 -0800

    Removed unused import to satisfy PMD check

commit ecb0f483a9
Author: Justin Borromeo <jborrome@edu.uwaterloo.ca>
Date:   Mon Feb 11 10:37:11 2019 -0800

    improved doc

commit f0eddee665
Author: Justin Borromeo <jborrome@edu.uwaterloo.ca>
Date:   Mon Feb 11 10:18:45 2019 -0800

    Added more javadoc

commit 5f92dd7325
Author: Justin Borromeo <jborrome@edu.uwaterloo.ca>
Date:   Mon Feb 11 10:05:58 2019 -0800

    Unused import

commit 93e1636287
Author: Justin Borromeo <jborrome@edu.uwaterloo.ca>
Date:   Mon Feb 11 10:03:14 2019 -0800

    Added javadoc on ScanResultValueTimestampComparator

commit 134041c479
Author: Justin Borromeo <jborrome@edu.uwaterloo.ca>
Date:   Fri Feb 8 13:13:54 2019 -0800

    Renamed sort function

commit 2e3577cd3d
Author: Justin Borromeo <jborrome@edu.uwaterloo.ca>
Date:   Thu Feb 7 13:01:25 2019 -0800

    Fixed benchmark queries

commit d3b335af42
Author: Justin Borromeo <jborrome@edu.uwaterloo.ca>
Date:   Thu Feb 7 11:08:07 2019 -0800

    added all query types to scan benchmark

commit ab00eade9f
Author: Justin Borromeo <jborrome@edu.uwaterloo.ca>
Date:   Thu Feb 7 09:42:48 2019 -0800

    Kicking travis with change to benchmark param

commit b432beaf84
Author: Justin Borromeo <jborrome@edu.uwaterloo.ca>
Date:   Wed Feb 6 17:45:59 2019 -0800

    Fixed failing calcite tests

commit b2c8c77ad4
Author: Justin Borromeo <jborrome@edu.uwaterloo.ca>
Date:   Wed Feb 6 17:39:48 2019 -0800

    Fixing tests WIP

commit 85e72a614e
Author: Justin Borromeo <jborrome@edu.uwaterloo.ca>
Date:   Wed Feb 6 15:42:02 2019 -0800

    Set to spaces over tabs

commit 7e872a8ebc
Author: Justin Borromeo <jborrome@edu.uwaterloo.ca>
Date:   Wed Feb 6 15:36:24 2019 -0800

    Created an error message for when someone tries to time order a result
    set > threshold limit

commit e8a4b49044
Merge: 305876a43 8e3a58f72
Author: Justin Borromeo <jborrome@edu.uwaterloo.ca>
Date:   Wed Feb 6 15:05:11 2019 -0800

    Merge branch 'master' into 6088-Time-Ordering-On-Scans-V2

commit 305876a434
Author: Justin Borromeo <jborrome@edu.uwaterloo.ca>
Date:   Wed Feb 6 15:02:02 2019 -0800

    nit

commit 8212a21caf
Author: Justin Borromeo <jborrome@edu.uwaterloo.ca>
Date:   Wed Feb 6 14:40:35 2019 -0800

    Improved conciseness

commit 10b5e0ca93
Author: Justin Borromeo <jborrome@edu.uwaterloo.ca>
Date:   Wed Feb 6 13:42:12 2019 -0800

    .

commit dfe4aa9681
Author: Justin Borromeo <jborrome@edu.uwaterloo.ca>
Date:   Wed Feb 6 13:41:18 2019 -0800

    Fixed codestyle and forbidden API errors

commit 148939e88b
Merge: 4f51024b3 5edbe2ae1
Author: Justin Borromeo <jborrome@edu.uwaterloo.ca>
Date:   Wed Feb 6 13:26:17 2019 -0800

    Merge branch '6088-Create-Scan-Benchmark' into 6088-Time-Ordering-On-Scans-V2

commit 5edbe2ae12
Merge: 60b7684db 315ccb76b
Author: Justin Borromeo <jborrome@edu.uwaterloo.ca>
Date:   Wed Feb 6 13:18:55 2019 -0800

    Merge github.com:apache/incubator-druid into 6088-Create-Scan-Benchmark

commit 60b7684db7
Author: Justin Borromeo <jborrome@edu.uwaterloo.ca>
Date:   Wed Feb 6 13:02:13 2019 -0800

    Committing a param change to kick teamcity

commit 4f51024b31
Author: Justin Borromeo <jborrome@edu.uwaterloo.ca>
Date:   Wed Feb 6 12:08:12 2019 -0800

    Wrote more tests for scan result value sort

commit 8b7d5f5081
Author: Justin Borromeo <jborrome@edu.uwaterloo.ca>
Date:   Wed Feb 6 11:55:09 2019 -0800

    Wrote tests for heapsort scan result values and fixed bug where iterator
    wasn't returning elements in correct order

commit b6d4df3864
Author: Justin Borromeo <jborrome@edu.uwaterloo.ca>
Date:   Tue Feb 5 16:45:20 2019 -0800

    Decrease segment size for less memory usage

commit d1a1793f36
Author: Justin Borromeo <jborrome@edu.uwaterloo.ca>
Date:   Tue Feb 5 12:40:26 2019 -0800

    nit

commit 7deb06f6df
Merge: b7d3a4900 86c5eee13
Author: Justin Borromeo <jborrome@edu.uwaterloo.ca>
Date:   Tue Feb 5 10:53:38 2019 -0800

    Merge branch '6088-Create-Scan-Benchmark' into 6088-Time-Ordering-On-Scans-V2

commit 86c5eee13b
Author: Justin Borromeo <jborrome@edu.uwaterloo.ca>
Date:   Tue Feb 5 10:31:27 2019 -0800

    Broke some long lines into two lines

commit b7d3a4900a
Merge: 796083f2b 8bc5eaa90
Author: Justin Borromeo <jborrome@edu.uwaterloo.ca>
Date:   Tue Feb 5 10:23:32 2019 -0800

    Merge branch 'master' into 6088-Time-Ordering-On-Scans-V2

commit 737a83321d
Author: Justin Borromeo <jborrome@edu.uwaterloo.ca>
Date:   Tue Feb 5 10:15:32 2019 -0800

    Made Jon's changes and removed TODOs

commit 796083f2bb
Author: Justin Borromeo <jborrome@edu.uwaterloo.ca>
Date:   Mon Feb 4 15:37:42 2019 -0800

    Benchmark param change

commit 20c36644db
Author: Justin Borromeo <jborrome@edu.uwaterloo.ca>
Date:   Mon Feb 4 15:36:35 2019 -0800

    More param changes

commit 9e6e71616b
Author: Justin Borromeo <jborrome@edu.uwaterloo.ca>
Date:   Mon Feb 4 15:31:21 2019 -0800

    Changed benchmark params

commit 01b25ed112
Author: Justin Borromeo <jborrome@edu.uwaterloo.ca>
Date:   Mon Feb 4 14:36:18 2019 -0800

    Added time ordering to the scan benchmark

commit 432acaf085
Author: Justin Borromeo <jborrome@edu.uwaterloo.ca>
Date:   Mon Feb 4 12:03:14 2019 -0800

    Change number of benchmark iterations

commit 12e51a2721
Author: Justin Borromeo <jborrome@edu.uwaterloo.ca>
Date:   Mon Feb 4 12:02:13 2019 -0800

    Added TimestampComparator tests

commit e66339cd76
Author: Justin Borromeo <jborrome@edu.uwaterloo.ca>
Date:   Mon Feb 4 10:56:41 2019 -0800

    Remove todos

commit ad731a362b
Author: Justin Borromeo <jborrome@edu.uwaterloo.ca>
Date:   Mon Feb 4 10:55:56 2019 -0800

    Change benchmark

commit 989bd2d50e
Merge: 7b5847139 26930f8d2
Author: Justin Borromeo <jborrome@edu.uwaterloo.ca>
Date:   Mon Feb 4 10:46:38 2019 -0800

    Merge branch '6088-Create-Scan-Benchmark' into 6088-Time-Ordering-On-Scans-V2

commit 7b58471394
Author: Justin Borromeo <jborrome@edu.uwaterloo.ca>
Date:   Sat Feb 2 03:48:18 2019 -0800

    Licensing stuff

commit 79e8319383
Author: Justin Borromeo <jborrome@edu.uwaterloo.ca>
Date:   Fri Feb 1 18:22:58 2019 -0800

    Move ScanResultValue timestamp comparator to a separate class for testing

commit 7a6080f636
Author: Justin Borromeo <jborrome@edu.uwaterloo.ca>
Date:   Fri Feb 1 18:00:58 2019 -0800

    Stuff for time-ordered scan query

commit 26930f8d20
Author: Justin Borromeo <jborrome@edu.uwaterloo.ca>
Date:   Fri Feb 1 16:38:49 2019 -0800

    It runs.

commit dd4ec1ac9c
Author: Justin Borromeo <jborrome@edu.uwaterloo.ca>
Date:   Fri Feb 1 15:12:17 2019 -0800

    Need to form queries

commit dba6e492a0
Merge: 10e57d5f9 7d4cc2873
Author: Justin Borromeo <jborrome@edu.uwaterloo.ca>
Date:   Fri Feb 1 14:13:39 2019 -0800

    Merge branch 'master' into 6088-Create-Scan-Benchmark

commit 10e57d5f9e
Author: Justin Borromeo <jborrome@edu.uwaterloo.ca>
Date:   Fri Feb 1 14:04:13 2019 -0800

    Moved Scan Builder to Druids class and started on Scan Benchmark setup

* Changed SQL planning to use scan over select

* Fixed some bugs

* Removed unused imports

* Updated calcite query test and test segment walker

* Fixed formatting recommendations
2019-04-02 15:46:01 -07:00
Clint Wylie a99f0ff450 prefix no-op aggs with "Noop" (#6960) 2019-04-02 15:05:07 -07:00
David Glasser 4e23c11345 Make IngestSegmentFirehoseFactory splittable for parallel ingestion (#7048)
* Make IngestSegmentFirehoseFactory splittable for parallel ingestion

* Code review feedback

- Get rid of WindowedSegment
- Don't document 'segments' parameter or support splitting firehoses that use it
- Require 'intervals' in WindowedSegmentId (since it won't be written by hand)

* Add missing @JsonProperty

* Integration test passes

* Add unit test

* Remove two FIXME comments from CompactionTask

I'd like to leave this PR in a potentially mergeable state, but I still would
appreciate reviewer eyes on the questions I'm removing here.

* Updates from code review
2019-04-02 14:59:17 -07:00
Xue Yu 78fd5aff21 support radians and degrees in sql (#7336)
* support radians and degrees in sql

* update test case
2019-04-02 12:47:49 -07:00
Qi Shu 134f71d1b4 Add documentation for Druid native query in SQL view of web console (#7381)
* Add docmentation for Druid native query in SQL view of web console

* Edit sentence
2019-04-02 12:20:51 -07:00
Vadim Ogievetsky d2ff3eb199 Migrate the web console back to Blueprint v3 (#7398)
* migrate back to Blueprint v3

* rename filler file

* fix missing icon
2019-04-01 23:06:48 -07:00
Vadim Ogievetsky 1137bb2586 Popovers are no longer inline (#7389) 2019-04-01 13:10:48 -07:00
Jonathan Wei 2bf6fc353a Update scan benchmark for time ordering (#7385) 2019-03-30 11:36:25 -07:00
Qi Shu 2ad9544242 Support segment metadata in rune (#7371) 2019-03-29 15:46:09 -07:00
Michael Trelinski 347779b17a Zookeeper loss (#6740)
* Update init

Fix bin/init to source from proper directory.

* Fix for Proposal #6518: Shutdown druid processes upon complete loss of ZK connectivity

* Zookeeper Loss:

- Add feature documentation
- Cosmetic refactors
- Variable extractions
- Remove getter

* - Change config key name and reword documentation
- Switch from Function<Void,Void> to Runnable/Lambda
- try { … } finally { … }

* Fix line length too long

* - change to formatted string for logging
- use System.err.println after lifecycle stops

* commenting on makeEnsembleProvider()-created Zookeeper termination

* Add javadoc

* added java doc reference back to apache discussion thread.

* move comment to other class

* favor two-slash comments instead of multiline comments
2019-03-29 15:10:42 -07:00
Jihoon Son 62c3e89266 maxTotalRows should be checked in DataSourceCompactionConfig before setting targetCompactionSizeBytes (#7368)
* maxTotalRows should be checked in DataSourceCompactionConfig before setting targetCompactionSizeBytes

* remove unnecessary default values

* remove flacky test

* fix build

* Add comments
2019-03-28 20:25:10 -07:00
Qi Shu a09aa13ead Put all local storage keys and functions into one file in util (#7314)
* Add all local storage keys and functions into one file in util

* Use LocalStorageKey type instead of string

* Remove druid author key
2019-03-28 17:48:37 -07:00
Qi Shu be1d7f885b Add scss linter in web console (#7365)
* Add SCSS linter

* Fixed SCSS styles
2019-03-28 17:32:01 -07:00
Justin Borromeo ad7862c58a Time Ordering On Scans (#7133)
* Moved Scan Builder to Druids class and started on Scan Benchmark setup

* Need to form queries

* It runs.

* Stuff for time-ordered scan query

* Move ScanResultValue timestamp comparator to a separate class for testing

* Licensing stuff

* Change benchmark

* Remove todos

* Added TimestampComparator tests

* Change number of benchmark iterations

* Added time ordering to the scan benchmark

* Changed benchmark params

* More param changes

* Benchmark param change

* Made Jon's changes and removed TODOs

* Broke some long lines into two lines

* nit

* Decrease segment size for less memory usage

* Wrote tests for heapsort scan result values and fixed bug where iterator
wasn't returning elements in correct order

* Wrote more tests for scan result value sort

* Committing a param change to kick teamcity

* Fixed codestyle and forbidden API errors

* .

* Improved conciseness

* nit

* Created an error message for when someone tries to time order a result
set > threshold limit

* Set to spaces over tabs

* Fixing tests WIP

* Fixed failing calcite tests

* Kicking travis with change to benchmark param

* added all query types to scan benchmark

* Fixed benchmark queries

* Renamed sort function

* Added javadoc on ScanResultValueTimestampComparator

* Unused import

* Added more javadoc

* improved doc

* Removed unused import to satisfy PMD check

* Small changes

* Changes based on Gian's comments

* Fixed failing test due to null resultFormat

* Added config and get # of segments

* Set up time ordering strategy decision tree

* Refactor and pQueue works

* Cleanup

* Ordering is correct on n-way merge -> still need to batch events into
ScanResultValues

* WIP

* Sequence stuff is so dirty :(

* Fixed bug introduced by replacing deque with list

* Wrote docs

* Multi-historical setup works

* WIP

* Change so batching only occurs on broker for time-ordered scans

Restricted batching to broker for time-ordered queries and adjusted
tests

Formatting

Cleanup

* Fixed mistakes in merge

* Fixed failing tests

* Reset config

* Wrote tests and added Javadoc

* Nit-change on javadoc

* Checkstyle fix

* Improved test and appeased TeamCity

* Sorry, checkstyle

* Applied Jon's recommended changes

* Checkstyle fix

* Optimization

* Fixed tests

* Updated error message

* Added error message for UOE

* Renaming

* Finish rename

* Smarter limiting for pQueue method

* Optimized n-way merge strategy

* Rename segment limit -> segment partitions limit

* Added a bit of docs

* More comments

* Fix checkstyle and test

* Nit comment

* Fixed failing tests -> allow usage of all types of segment spec

* Fixed failing tests -> allow usage of all types of segment spec

* Revert "Fixed failing tests -> allow usage of all types of segment spec"

This reverts commit ec470288c7.

* Revert "Merge branch '6088-Time-Ordering-On-Scans-N-Way-Merge' of github.com:justinborromeo/incubator-druid into 6088-Time-Ordering-On-Scans-N-Way-Merge"

This reverts commit 57033f36df, reversing
changes made to 8f01d8dd16.

* Check type of segment spec before using for time ordering

* Fix bug in numRowsScanned

* Fix bug messing up count of rows

* Fix docs and flipped boolean in ScanQueryLimitRowIterator

* Refactor n-way merge

* Added test for n-way merge

* Refixed regression

* Checkstyle and doc update

* Modified sequence limit to accept longs and added test for long limits

* doc fix

* Implemented Clint's recommendations
2019-03-28 14:37:09 -07:00
Roman Leventov ffa95859c2 Remove [PROPOSAL] from Proposal issue template; Add 'Other' template for convenience (#7364)
* Remove [PROPOSAL] from Proposal issue template; Add 'Regular' template for convenience

* Regular -> Other
2019-03-28 18:40:43 +08:00
Surekha be318f4de3 Add column type to sys table docs (#7359)
* Add column type

* oops should be used=1
2019-03-27 20:21:57 -07:00
Clint Wylie db0125b709 this will fix it (#7358) 2019-03-27 19:42:58 -07:00
Qi Shu 18e5167245 Add history dialog in snitch dialog to allow viewing the editing history (#7321)
* Add history dialog in snitch dialog to allow viewing the editing history

* Improved CSS; better animation

* Use position: absolute instead of float: right to position element

* Removed author for history changes
2019-03-27 17:52:44 -07:00
Charles Allen eeb3dbe79d Move GCP to a core extension (#6953)
* Move GCP to a core extension

* Don't provide druid-core >.<

* Keep AWS and GCP modules separate

* Move AWSModule to its own module

* Add aws ec2 extension and more modules in more places

* Fix bad imports

* Fix test jackson module

* Include AWS and GCP core in server

* Add simple empty method comment

* Update version to 15

* One more 0.13.0-->0.15.0 change

* Fix multi-binding problem

* Grep for s3-extensions and update docs

* Update extensions.md
2019-03-27 09:00:43 -07:00
Jihoon Son 543324f8a9 Fix logging in IndexerSQLMetadataStorageCoordinator (#7349) 2019-03-26 20:36:19 -07:00
Jihoon Son 4d37edac1e Suppress stack trace in warning (#7348) 2019-03-26 17:27:29 -07:00
Roman Leventov b9354fbddb
Make 'field can be final' inspection a warning in IntelliJ config (#7301)
A small step towards #7227.
2019-03-26 20:45:03 -03:00
Justin Borromeo c7fea6ac8f Added better QueryInterruptedException error message for UnsupportedOperationException (#7248)
* Added error message for UOE

* Updated docs

* Doc change

* Doc change
2019-03-26 15:20:24 -07:00
Jihoon Son 5294277cb4
Fix exclusive start partitions for sequenceMetadata (#7339)
* Fix exclusvie start partitions for sequenceMetadata

* add empty check
2019-03-26 14:39:07 -07:00
Puneet Jaiswal 105b4fa237 Fixing a typo in logger initialization (#7341) 2019-03-27 00:38:28 +08:00