9095 Commits

Author SHA1 Message Date
Justin Borromeo
7e872a8ebc Created an error message for when someone tries to time order a result
set > threshold limit
2019-02-06 15:36:24 -08:00
Justin Borromeo
e8a4b49044 Merge branch 'master' into 6088-Time-Ordering-On-Scans-V2 2019-02-06 15:05:11 -08:00
Justin Borromeo
305876a434 nit 2019-02-06 15:02:02 -08:00
Jihoon Son
8e3a58f723
Improve druid.storage.sse.kms.keyId and druid.s3.protocol (#7012)
* Improve druid.storage.sse.kms.keyId and druid.s3.protocol

* fix article
2019-02-06 15:00:51 -08:00
Justin Borromeo
6723243ed2 Create Scan Benchmark (#6986)
* Moved Scan Builder to Druids class and started on Scan Benchmark setup

* Need to form queries

* It runs.

* Remove todos

* Change number of benchmark iterations

* Changed benchmark params

* More param changes

* Made Jon's changes and removed TODOs

* Broke some long lines into two lines

* Decrease segment size for less memory usage

* Committing a param change to kick teamcity
2019-02-06 14:45:01 -08:00
Justin Borromeo
8212a21caf Improved conciseness 2019-02-06 14:40:35 -08:00
Furkan KAMACI
58f9507ccf Improper equals override is fixed to prevent NullPointerException (#6938)
* Improper equals override is fixed to prevent NullPointerException

* Fixed curly brace indentation.

* Test method is added for equals method of TaskLockPosse class.
2019-02-06 14:08:50 -08:00
Justin Borromeo
10b5e0ca93 . 2019-02-06 13:42:12 -08:00
Justin Borromeo
dfe4aa9681 Fixed codestyle and forbidden API errors 2019-02-06 13:41:18 -08:00
Justin Borromeo
148939e88b Merge branch '6088-Create-Scan-Benchmark' into 6088-Time-Ordering-On-Scans-V2 2019-02-06 13:26:17 -08:00
Justin Borromeo
5edbe2ae12 Merge github.com:apache/incubator-druid into 6088-Create-Scan-Benchmark 2019-02-06 13:18:55 -08:00
Justin Borromeo
60b7684db7 Committing a param change to kick teamcity 2019-02-06 13:02:13 -08:00
Justin Borromeo
4f51024b31 Wrote more tests for scan result value sort 2019-02-06 12:08:12 -08:00
Justin Borromeo
8b7d5f5081 Wrote tests for heapsort scan result values and fixed bug where iterator
wasn't returning elements in correct order
2019-02-06 11:55:09 -08:00
Justin Borromeo
b6d4df3864 Decrease segment size for less memory usage 2019-02-05 16:45:20 -08:00
anantmf
315ccb76b8 Fix for getSingleObjectSummary, replacing keyCount with objectSummaries().size (#7000)
* Instead of using keyCount, changing it to check the size of objectSummaries.

For issue:
https://github.com/apache/incubator-druid/issues/6980
https://github.com/apache/incubator-druid/issues/6980#issuecomment-460006580

* Changing another usage of keyCount with size of objectSummaries.

* Adding some comments to explain why using keyCount is not working as expected.
2019-02-05 15:45:44 -08:00
Surekha
ef451d3603 Add null checks in DruidSchema (#6830)
* Add null checks in DruidSchema

* Add unit tests

* Add VisibleForTesting annotation

* PR comments

* unused import
2019-02-05 13:42:20 -08:00
Justin Borromeo
d1a1793f36 nit 2019-02-05 12:40:26 -08:00
Jihoon Son
75c70c2ccc Add doc for S3 permissions settings (#7011)
* Add doc for S3 permissions settings

* add a comment about additional settings
2019-02-05 11:52:09 -08:00
Justin Borromeo
7deb06f6df Merge branch '6088-Create-Scan-Benchmark' into 6088-Time-Ordering-On-Scans-V2 2019-02-05 10:53:38 -08:00
Justin Borromeo
86c5eee13b Broke some long lines into two lines 2019-02-05 10:31:27 -08:00
Justin Borromeo
b7d3a4900a Merge branch 'master' into 6088-Time-Ordering-On-Scans-V2 2019-02-05 10:23:32 -08:00
Justin Borromeo
737a83321d Made Jon's changes and removed TODOs 2019-02-05 10:15:32 -08:00
Jonathan Wei
8bc5eaa908
Set version to 0.14.0-incubating-SNAPSHOT (#7003) 2019-02-04 19:36:20 -08:00
Egor Riashin
97b6407983 maintenance mode for Historical (#6349)
* maintenance mode for Historical

forbidden api fix, config deserialization fix

logging fix, unit tests

* addressed comments

* addressed comments

* a style fix

* addressed comments

* a unit-test fix due to recent code-refactoring

* docs & refactoring

* addressed comments

* addressed a LoadRule drop flaw

* post merge cleaning up
2019-02-04 18:11:00 -08:00
Justin Borromeo
796083f2bb Benchmark param change 2019-02-04 15:37:42 -08:00
Justin Borromeo
20c36644db More param changes 2019-02-04 15:36:35 -08:00
Justin Borromeo
9e6e71616b Changed benchmark params 2019-02-04 15:31:21 -08:00
Justin Borromeo
01b25ed112 Added time ordering to the scan benchmark 2019-02-04 14:36:18 -08:00
Justin Borromeo
432acaf085 Change number of benchmark iterations 2019-02-04 12:03:14 -08:00
Justin Borromeo
12e51a2721 Added TimestampComparator tests 2019-02-04 12:02:13 -08:00
David Glasser
7e48593b57 ParallelIndexSupervisorTask: don't warn about a default value (#6987)
Native batch indexing doesn't yet support the maxParseExceptions,
maxSavedParseExceptions, and logParseExceptions tuning config options, so
ParallelIndexSupervisorTask logs if these are set. But the default value for
maxParseExceptions is Integer.MAX_VALUE, which means that you'll get the
maxParseExceptions flavor of this warning even if you don't configure
maxParseExceptions.

This PR changes all three warnings to occur if you change the settings from the
default; this mostly affects the maxParseExceptions warning.
2019-02-04 12:00:26 -08:00
Justin Borromeo
e66339cd76 Remove todos 2019-02-04 10:56:41 -08:00
Justin Borromeo
ad731a362b Change benchmark 2019-02-04 10:55:56 -08:00
Justin Borromeo
989bd2d50e Merge branch '6088-Create-Scan-Benchmark' into 6088-Time-Ordering-On-Scans-V2 2019-02-04 10:46:38 -08:00
Roman Leventov
0e926e8652 Prohibit assigning concurrent maps into Map-typed variables and fields and fix a race condition in CoordinatorRuleManager (#6898)
* Prohibit assigning concurrent maps into Map-types variables and fields; Fix a race condition in CoordinatorRuleManager; improve logic in DirectDruidClient and ResourcePool

* Enforce that if compute(), computeIfAbsent(), computeIfPresent() or merge() is called on a ConcurrentHashMap, it's stored in a ConcurrentHashMap-typed variable, not ConcurrentMap; add comments explaining get()-before-computeIfAbsent() optimization; refactor Counters; fix a race condition in Intialization.java

* Remove unnecessary comment

* Checkstyle

* Fix getFromExtensions()

* Add a reference to the comment about guarded computeIfAbsent() optimization; IdentityHashMap optimization

* Fix UriCacheGeneratorTest

* Workaround issue with MaterializedViewQueryQueryToolChest

* Strengthen Appenderator's contract regarding concurrency
2019-02-04 09:18:12 -08:00
Vadim Ogievetsky
06d56c2abd Fixing little issues and CSS bugs in the new web console. (#6995) 2019-02-04 09:04:58 -08:00
Clint Wylie
6207b66e20 fix build (#6994) 2019-02-03 09:38:51 -08:00
Jonathan Wei
953b96d0a4 Add more sketch aggregator support in Druid SQL (#6951)
* Add more sketch aggregator support in Druid SQL

* Add docs

* Tweak module serde register

* Fix tests

* Checkstyle

* Test fix

* PR comment

* PR comment

* PR comments
2019-02-02 22:34:53 -08:00
Surekha
7baa33049c Introduce published segment cache in broker (#6901)
* Add published segment cache in broker

* Change the DataSegment interner so it's not based on DataSEgment's equals only and size is preserved if set

* Added a trueEquals to DataSegment class

* Use separate interner for realtime and historical segments

* Remove trueEquals as it's not used anymore, change log message

* PR comments

* PR comments

* Fix tests

* PR comments

* Few more modification to

* change the coordinator api
* removeall segments at once from MetadataSegmentView in order to serve a more consistent view of published segments
* Change the poll behaviour to avoid multiple poll execution at same time

* minor changes

* PR comments

* PR comments

* Make the segment cache in broker off by default

* Added a config to PlannerConfig
* Moved MetadataSegmentView to sql module

* Add doc for new planner config

* Update documentation

* PR comments

* some more changes

* PR comments

* fix test

* remove unintentional change, whether to synchronize on lifecycleLock is still in discussion in PR

* minor changes

* some changes to initialization

* use pollPeriodInMS

* Add boolean cachePopulated to check if first poll succeeds

* Remove poll from start()

* take the log message out of condition in stop()
2019-02-02 22:27:13 -08:00
Furkan KAMACI
61f165c23f Try-with-resources should be used since the new syntax is more readable. (#6944)
* Try-with-resources should be used since the new syntax is more readable.

* Fixed checkstyle error.
2019-02-03 10:42:28 +08:00
Justin Borromeo
7b58471394 Licensing stuff 2019-02-02 03:48:18 -08:00
Justin Borromeo
79e8319383 Move ScanResultValue timestamp comparator to a separate class for testing 2019-02-01 18:22:58 -08:00
Justin Borromeo
7a6080f636 Stuff for time-ordered scan query 2019-02-01 18:00:58 -08:00
Justin Borromeo
26930f8d20 It runs. 2019-02-01 16:38:49 -08:00
Justin Borromeo
dd4ec1ac9c Need to form queries 2019-02-01 15:12:17 -08:00
Justin Borromeo
6430ef8e1b lol (#6985) 2019-02-01 14:21:13 -08:00
Justin Borromeo
dba6e492a0 Merge branch 'master' into 6088-Create-Scan-Benchmark 2019-02-01 14:13:39 -08:00
Justin Borromeo
10e57d5f9e Moved Scan Builder to Druids class and started on Scan Benchmark setup 2019-02-01 14:04:13 -08:00
Jihoon Son
7d4cc28730 Fix node path for building the unified console (#6981) 2019-02-01 13:57:17 -08:00