druid/extensions-core
Justin Borromeo ad7862c58a Time Ordering On Scans (#7133)
* Moved Scan Builder to Druids class and started on Scan Benchmark setup

* Need to form queries

* It runs.

* Stuff for time-ordered scan query

* Move ScanResultValue timestamp comparator to a separate class for testing

* Licensing stuff

* Change benchmark

* Remove todos

* Added TimestampComparator tests

* Change number of benchmark iterations

* Added time ordering to the scan benchmark

* Changed benchmark params

* More param changes

* Benchmark param change

* Made Jon's changes and removed TODOs

* Broke some long lines into two lines

* nit

* Decrease segment size for less memory usage

* Wrote tests for heapsort scan result values and fixed bug where iterator
wasn't returning elements in correct order

* Wrote more tests for scan result value sort

* Committing a param change to kick teamcity

* Fixed codestyle and forbidden API errors

* .

* Improved conciseness

* nit

* Created an error message for when someone tries to time order a result
set > threshold limit

* Set to spaces over tabs

* Fixing tests WIP

* Fixed failing calcite tests

* Kicking travis with change to benchmark param

* added all query types to scan benchmark

* Fixed benchmark queries

* Renamed sort function

* Added javadoc on ScanResultValueTimestampComparator

* Unused import

* Added more javadoc

* improved doc

* Removed unused import to satisfy PMD check

* Small changes

* Changes based on Gian's comments

* Fixed failing test due to null resultFormat

* Added config and get # of segments

* Set up time ordering strategy decision tree

* Refactor and pQueue works

* Cleanup

* Ordering is correct on n-way merge -> still need to batch events into
ScanResultValues

* WIP

* Sequence stuff is so dirty :(

* Fixed bug introduced by replacing deque with list

* Wrote docs

* Multi-historical setup works

* WIP

* Change so batching only occurs on broker for time-ordered scans

Restricted batching to broker for time-ordered queries and adjusted
tests

Formatting

Cleanup

* Fixed mistakes in merge

* Fixed failing tests

* Reset config

* Wrote tests and added Javadoc

* Nit-change on javadoc

* Checkstyle fix

* Improved test and appeased TeamCity

* Sorry, checkstyle

* Applied Jon's recommended changes

* Checkstyle fix

* Optimization

* Fixed tests

* Updated error message

* Added error message for UOE

* Renaming

* Finish rename

* Smarter limiting for pQueue method

* Optimized n-way merge strategy

* Rename segment limit -> segment partitions limit

* Added a bit of docs

* More comments

* Fix checkstyle and test

* Nit comment

* Fixed failing tests -> allow usage of all types of segment spec

* Fixed failing tests -> allow usage of all types of segment spec

* Revert "Fixed failing tests -> allow usage of all types of segment spec"

This reverts commit ec470288c7.

* Revert "Merge branch '6088-Time-Ordering-On-Scans-N-Way-Merge' of github.com:justinborromeo/incubator-druid into 6088-Time-Ordering-On-Scans-N-Way-Merge"

This reverts commit 57033f36df, reversing
changes made to 8f01d8dd16.

* Check type of segment spec before using for time ordering

* Fix bug in numRowsScanned

* Fix bug messing up count of rows

* Fix docs and flipped boolean in ScanQueryLimitRowIterator

* Refactor n-way merge

* Added test for n-way merge

* Refixed regression

* Checkstyle and doc update

* Modified sequence limit to accept longs and added test for long limits

* doc fix

* Implemented Clint's recommendations
2019-03-28 14:37:09 -07:00
..
avro-extensions Fix flattening Avro Maps with Utf8 keys (#7258) 2019-03-18 14:23:53 -07:00
datasketches sql, filters, and virtual columns (#6902) 2019-03-11 11:37:58 -07:00
druid-basic-security Improve error message for unavailable auth cache in BasicHTTPAuthenticator (#7263) 2019-03-14 14:39:43 +08:00
druid-bloom-filter sql, filters, and virtual columns (#6902) 2019-03-11 11:37:58 -07:00
druid-kerberos Prohibit Throwables.propagate() (#7121) 2019-03-14 18:28:33 -03:00
ec2-extensions Move GCP to a core extension (#6953) 2019-03-27 09:00:43 -07:00
google-extensions this will fix it (#7358) 2019-03-27 19:42:58 -07:00
hdfs-storage Prohibit Throwables.propagate() (#7121) 2019-03-14 18:28:33 -03:00
histogram Fix some IntelliJ inspections (#7273) 2019-03-25 21:11:01 -03:00
kafka-eight Support kafka transactional topics (#5404) (#6496) 2019-02-18 11:50:08 -08:00
kafka-extraction-namespace Prohibit Throwables.propagate() (#7121) 2019-03-14 18:28:33 -03:00
kafka-indexing-service Time Ordering On Scans (#7133) 2019-03-28 14:37:09 -07:00
kinesis-indexing-service Move GCP to a core extension (#6953) 2019-03-27 09:00:43 -07:00
lookups-cached-global Prohibit Throwables.propagate() (#7121) 2019-03-14 18:28:33 -03:00
lookups-cached-single Set version to 0.15.0-incubating-SNAPSHOT (#7014) 2019-02-07 14:02:52 -08:00
mysql-metadata-storage Set version to 0.15.0-incubating-SNAPSHOT (#7014) 2019-02-07 14:02:52 -08:00
parquet-extensions Set version to 0.15.0-incubating-SNAPSHOT (#7014) 2019-02-07 14:02:52 -08:00
postgresql-metadata-storage Add Postgresql SqlFirehose (#6813) 2019-02-14 22:52:03 -08:00
protobuf-extensions Set version to 0.15.0-incubating-SNAPSHOT (#7014) 2019-02-07 14:02:52 -08:00
s3-extensions Move GCP to a core extension (#6953) 2019-03-27 09:00:43 -07:00
simple-client-sslcontext Set version to 0.15.0-incubating-SNAPSHOT (#7014) 2019-02-07 14:02:52 -08:00
stats Set version to 0.15.0-incubating-SNAPSHOT (#7014) 2019-02-07 14:02:52 -08:00