Commit Graph

9047 Commits

Author SHA1 Message Date
Justin Borromeo b2c8c77ad4 Fixing tests WIP 2019-02-06 17:39:48 -08:00
Justin Borromeo 85e72a614e Set to spaces over tabs 2019-02-06 15:42:02 -08:00
Justin Borromeo 7e872a8ebc Created an error message for when someone tries to time order a result
set > threshold limit
2019-02-06 15:36:24 -08:00
Justin Borromeo e8a4b49044 Merge branch 'master' into 6088-Time-Ordering-On-Scans-V2 2019-02-06 15:05:11 -08:00
Justin Borromeo 305876a434 nit 2019-02-06 15:02:02 -08:00
Jihoon Son 8e3a58f723
Improve druid.storage.sse.kms.keyId and druid.s3.protocol (#7012)
* Improve druid.storage.sse.kms.keyId and druid.s3.protocol

* fix article
2019-02-06 15:00:51 -08:00
Justin Borromeo 6723243ed2 Create Scan Benchmark (#6986)
* Moved Scan Builder to Druids class and started on Scan Benchmark setup

* Need to form queries

* It runs.

* Remove todos

* Change number of benchmark iterations

* Changed benchmark params

* More param changes

* Made Jon's changes and removed TODOs

* Broke some long lines into two lines

* Decrease segment size for less memory usage

* Committing a param change to kick teamcity
2019-02-06 14:45:01 -08:00
Justin Borromeo 8212a21caf Improved conciseness 2019-02-06 14:40:35 -08:00
Furkan KAMACI 58f9507ccf Improper equals override is fixed to prevent NullPointerException (#6938)
* Improper equals override is fixed to prevent NullPointerException

* Fixed curly brace indentation.

* Test method is added for equals method of TaskLockPosse class.
2019-02-06 14:08:50 -08:00
Justin Borromeo 10b5e0ca93 . 2019-02-06 13:42:12 -08:00
Justin Borromeo dfe4aa9681 Fixed codestyle and forbidden API errors 2019-02-06 13:41:18 -08:00
Justin Borromeo 148939e88b Merge branch '6088-Create-Scan-Benchmark' into 6088-Time-Ordering-On-Scans-V2 2019-02-06 13:26:17 -08:00
Justin Borromeo 5edbe2ae12 Merge github.com:apache/incubator-druid into 6088-Create-Scan-Benchmark 2019-02-06 13:18:55 -08:00
Justin Borromeo 60b7684db7 Committing a param change to kick teamcity 2019-02-06 13:02:13 -08:00
Justin Borromeo 4f51024b31 Wrote more tests for scan result value sort 2019-02-06 12:08:12 -08:00
Justin Borromeo 8b7d5f5081 Wrote tests for heapsort scan result values and fixed bug where iterator
wasn't returning elements in correct order
2019-02-06 11:55:09 -08:00
Justin Borromeo b6d4df3864 Decrease segment size for less memory usage 2019-02-05 16:45:20 -08:00
anantmf 315ccb76b8 Fix for getSingleObjectSummary, replacing keyCount with objectSummaries().size (#7000)
* Instead of using keyCount, changing it to check the size of objectSummaries.

For issue:
https://github.com/apache/incubator-druid/issues/6980
https://github.com/apache/incubator-druid/issues/6980#issuecomment-460006580

* Changing another usage of keyCount with size of objectSummaries.

* Adding some comments to explain why using keyCount is not working as expected.
2019-02-05 15:45:44 -08:00
Surekha ef451d3603 Add null checks in DruidSchema (#6830)
* Add null checks in DruidSchema

* Add unit tests

* Add VisibleForTesting annotation

* PR comments

* unused import
2019-02-05 13:42:20 -08:00
Justin Borromeo d1a1793f36 nit 2019-02-05 12:40:26 -08:00
Jihoon Son 75c70c2ccc Add doc for S3 permissions settings (#7011)
* Add doc for S3 permissions settings

* add a comment about additional settings
2019-02-05 11:52:09 -08:00
Justin Borromeo 7deb06f6df Merge branch '6088-Create-Scan-Benchmark' into 6088-Time-Ordering-On-Scans-V2 2019-02-05 10:53:38 -08:00
Justin Borromeo 86c5eee13b Broke some long lines into two lines 2019-02-05 10:31:27 -08:00
Justin Borromeo b7d3a4900a Merge branch 'master' into 6088-Time-Ordering-On-Scans-V2 2019-02-05 10:23:32 -08:00
Justin Borromeo 737a83321d Made Jon's changes and removed TODOs 2019-02-05 10:15:32 -08:00
Jonathan Wei 8bc5eaa908
Set version to 0.14.0-incubating-SNAPSHOT (#7003) 2019-02-04 19:36:20 -08:00
Egor Riashin 97b6407983 maintenance mode for Historical (#6349)
* maintenance mode for Historical

forbidden api fix, config deserialization fix

logging fix, unit tests

* addressed comments

* addressed comments

* a style fix

* addressed comments

* a unit-test fix due to recent code-refactoring

* docs & refactoring

* addressed comments

* addressed a LoadRule drop flaw

* post merge cleaning up
2019-02-04 18:11:00 -08:00
Justin Borromeo 796083f2bb Benchmark param change 2019-02-04 15:37:42 -08:00
Justin Borromeo 20c36644db More param changes 2019-02-04 15:36:35 -08:00
Justin Borromeo 9e6e71616b Changed benchmark params 2019-02-04 15:31:21 -08:00
Justin Borromeo 01b25ed112 Added time ordering to the scan benchmark 2019-02-04 14:36:18 -08:00
Justin Borromeo 432acaf085 Change number of benchmark iterations 2019-02-04 12:03:14 -08:00
Justin Borromeo 12e51a2721 Added TimestampComparator tests 2019-02-04 12:02:13 -08:00
David Glasser 7e48593b57 ParallelIndexSupervisorTask: don't warn about a default value (#6987)
Native batch indexing doesn't yet support the maxParseExceptions,
maxSavedParseExceptions, and logParseExceptions tuning config options, so
ParallelIndexSupervisorTask logs if these are set. But the default value for
maxParseExceptions is Integer.MAX_VALUE, which means that you'll get the
maxParseExceptions flavor of this warning even if you don't configure
maxParseExceptions.

This PR changes all three warnings to occur if you change the settings from the
default; this mostly affects the maxParseExceptions warning.
2019-02-04 12:00:26 -08:00
Justin Borromeo e66339cd76 Remove todos 2019-02-04 10:56:41 -08:00
Justin Borromeo ad731a362b Change benchmark 2019-02-04 10:55:56 -08:00
Justin Borromeo 989bd2d50e Merge branch '6088-Create-Scan-Benchmark' into 6088-Time-Ordering-On-Scans-V2 2019-02-04 10:46:38 -08:00
Roman Leventov 0e926e8652 Prohibit assigning concurrent maps into Map-typed variables and fields and fix a race condition in CoordinatorRuleManager (#6898)
* Prohibit assigning concurrent maps into Map-types variables and fields; Fix a race condition in CoordinatorRuleManager; improve logic in DirectDruidClient and ResourcePool

* Enforce that if compute(), computeIfAbsent(), computeIfPresent() or merge() is called on a ConcurrentHashMap, it's stored in a ConcurrentHashMap-typed variable, not ConcurrentMap; add comments explaining get()-before-computeIfAbsent() optimization; refactor Counters; fix a race condition in Intialization.java

* Remove unnecessary comment

* Checkstyle

* Fix getFromExtensions()

* Add a reference to the comment about guarded computeIfAbsent() optimization; IdentityHashMap optimization

* Fix UriCacheGeneratorTest

* Workaround issue with MaterializedViewQueryQueryToolChest

* Strengthen Appenderator's contract regarding concurrency
2019-02-04 09:18:12 -08:00
Vadim Ogievetsky 06d56c2abd Fixing little issues and CSS bugs in the new web console. (#6995) 2019-02-04 09:04:58 -08:00
Clint Wylie 6207b66e20 fix build (#6994) 2019-02-03 09:38:51 -08:00
Jonathan Wei 953b96d0a4 Add more sketch aggregator support in Druid SQL (#6951)
* Add more sketch aggregator support in Druid SQL

* Add docs

* Tweak module serde register

* Fix tests

* Checkstyle

* Test fix

* PR comment

* PR comment

* PR comments
2019-02-02 22:34:53 -08:00
Surekha 7baa33049c Introduce published segment cache in broker (#6901)
* Add published segment cache in broker

* Change the DataSegment interner so it's not based on DataSEgment's equals only and size is preserved if set

* Added a trueEquals to DataSegment class

* Use separate interner for realtime and historical segments

* Remove trueEquals as it's not used anymore, change log message

* PR comments

* PR comments

* Fix tests

* PR comments

* Few more modification to

* change the coordinator api
* removeall segments at once from MetadataSegmentView in order to serve a more consistent view of published segments
* Change the poll behaviour to avoid multiple poll execution at same time

* minor changes

* PR comments

* PR comments

* Make the segment cache in broker off by default

* Added a config to PlannerConfig
* Moved MetadataSegmentView to sql module

* Add doc for new planner config

* Update documentation

* PR comments

* some more changes

* PR comments

* fix test

* remove unintentional change, whether to synchronize on lifecycleLock is still in discussion in PR

* minor changes

* some changes to initialization

* use pollPeriodInMS

* Add boolean cachePopulated to check if first poll succeeds

* Remove poll from start()

* take the log message out of condition in stop()
2019-02-02 22:27:13 -08:00
Furkan KAMACI 61f165c23f Try-with-resources should be used since the new syntax is more readable. (#6944)
* Try-with-resources should be used since the new syntax is more readable.

* Fixed checkstyle error.
2019-02-03 10:42:28 +08:00
Justin Borromeo 7b58471394 Licensing stuff 2019-02-02 03:48:18 -08:00
Justin Borromeo 79e8319383 Move ScanResultValue timestamp comparator to a separate class for testing 2019-02-01 18:22:58 -08:00
Justin Borromeo 7a6080f636 Stuff for time-ordered scan query 2019-02-01 18:00:58 -08:00
Justin Borromeo 26930f8d20 It runs. 2019-02-01 16:38:49 -08:00
Justin Borromeo dd4ec1ac9c Need to form queries 2019-02-01 15:12:17 -08:00
Justin Borromeo 6430ef8e1b lol (#6985) 2019-02-01 14:21:13 -08:00
Justin Borromeo dba6e492a0 Merge branch 'master' into 6088-Create-Scan-Benchmark 2019-02-01 14:13:39 -08:00