Commit Graph

9139 Commits

Author SHA1 Message Date
Furkan KAMACI 58f9507ccf Improper equals override is fixed to prevent NullPointerException (#6938)
* Improper equals override is fixed to prevent NullPointerException

* Fixed curly brace indentation.

* Test method is added for equals method of TaskLockPosse class.
2019-02-06 14:08:50 -08:00
Justin Borromeo 10b5e0ca93 . 2019-02-06 13:42:12 -08:00
Justin Borromeo dfe4aa9681 Fixed codestyle and forbidden API errors 2019-02-06 13:41:18 -08:00
Justin Borromeo 148939e88b Merge branch '6088-Create-Scan-Benchmark' into 6088-Time-Ordering-On-Scans-V2 2019-02-06 13:26:17 -08:00
Justin Borromeo 5edbe2ae12 Merge github.com:apache/incubator-druid into 6088-Create-Scan-Benchmark 2019-02-06 13:18:55 -08:00
Justin Borromeo 60b7684db7 Committing a param change to kick teamcity 2019-02-06 13:02:13 -08:00
Justin Borromeo 4f51024b31 Wrote more tests for scan result value sort 2019-02-06 12:08:12 -08:00
Justin Borromeo 8b7d5f5081 Wrote tests for heapsort scan result values and fixed bug where iterator
wasn't returning elements in correct order
2019-02-06 11:55:09 -08:00
Justin Borromeo b6d4df3864 Decrease segment size for less memory usage 2019-02-05 16:45:20 -08:00
anantmf 315ccb76b8 Fix for getSingleObjectSummary, replacing keyCount with objectSummaries().size (#7000)
* Instead of using keyCount, changing it to check the size of objectSummaries.

For issue:
https://github.com/apache/incubator-druid/issues/6980
https://github.com/apache/incubator-druid/issues/6980#issuecomment-460006580

* Changing another usage of keyCount with size of objectSummaries.

* Adding some comments to explain why using keyCount is not working as expected.
2019-02-05 15:45:44 -08:00
Surekha ef451d3603 Add null checks in DruidSchema (#6830)
* Add null checks in DruidSchema

* Add unit tests

* Add VisibleForTesting annotation

* PR comments

* unused import
2019-02-05 13:42:20 -08:00
Justin Borromeo d1a1793f36 nit 2019-02-05 12:40:26 -08:00
Jihoon Son 75c70c2ccc Add doc for S3 permissions settings (#7011)
* Add doc for S3 permissions settings

* add a comment about additional settings
2019-02-05 11:52:09 -08:00
Justin Borromeo 7deb06f6df Merge branch '6088-Create-Scan-Benchmark' into 6088-Time-Ordering-On-Scans-V2 2019-02-05 10:53:38 -08:00
Justin Borromeo 86c5eee13b Broke some long lines into two lines 2019-02-05 10:31:27 -08:00
Justin Borromeo b7d3a4900a Merge branch 'master' into 6088-Time-Ordering-On-Scans-V2 2019-02-05 10:23:32 -08:00
Justin Borromeo 737a83321d Made Jon's changes and removed TODOs 2019-02-05 10:15:32 -08:00
Jonathan Wei 8bc5eaa908
Set version to 0.14.0-incubating-SNAPSHOT (#7003) 2019-02-04 19:36:20 -08:00
Egor Riashin 97b6407983 maintenance mode for Historical (#6349)
* maintenance mode for Historical

forbidden api fix, config deserialization fix

logging fix, unit tests

* addressed comments

* addressed comments

* a style fix

* addressed comments

* a unit-test fix due to recent code-refactoring

* docs & refactoring

* addressed comments

* addressed a LoadRule drop flaw

* post merge cleaning up
2019-02-04 18:11:00 -08:00
Justin Borromeo 796083f2bb Benchmark param change 2019-02-04 15:37:42 -08:00
Justin Borromeo 20c36644db More param changes 2019-02-04 15:36:35 -08:00
Justin Borromeo 9e6e71616b Changed benchmark params 2019-02-04 15:31:21 -08:00
Justin Borromeo 01b25ed112 Added time ordering to the scan benchmark 2019-02-04 14:36:18 -08:00
Justin Borromeo 432acaf085 Change number of benchmark iterations 2019-02-04 12:03:14 -08:00
Justin Borromeo 12e51a2721 Added TimestampComparator tests 2019-02-04 12:02:13 -08:00
David Glasser 7e48593b57 ParallelIndexSupervisorTask: don't warn about a default value (#6987)
Native batch indexing doesn't yet support the maxParseExceptions,
maxSavedParseExceptions, and logParseExceptions tuning config options, so
ParallelIndexSupervisorTask logs if these are set. But the default value for
maxParseExceptions is Integer.MAX_VALUE, which means that you'll get the
maxParseExceptions flavor of this warning even if you don't configure
maxParseExceptions.

This PR changes all three warnings to occur if you change the settings from the
default; this mostly affects the maxParseExceptions warning.
2019-02-04 12:00:26 -08:00
Justin Borromeo e66339cd76 Remove todos 2019-02-04 10:56:41 -08:00
Justin Borromeo ad731a362b Change benchmark 2019-02-04 10:55:56 -08:00
Justin Borromeo 989bd2d50e Merge branch '6088-Create-Scan-Benchmark' into 6088-Time-Ordering-On-Scans-V2 2019-02-04 10:46:38 -08:00
Roman Leventov 0e926e8652 Prohibit assigning concurrent maps into Map-typed variables and fields and fix a race condition in CoordinatorRuleManager (#6898)
* Prohibit assigning concurrent maps into Map-types variables and fields; Fix a race condition in CoordinatorRuleManager; improve logic in DirectDruidClient and ResourcePool

* Enforce that if compute(), computeIfAbsent(), computeIfPresent() or merge() is called on a ConcurrentHashMap, it's stored in a ConcurrentHashMap-typed variable, not ConcurrentMap; add comments explaining get()-before-computeIfAbsent() optimization; refactor Counters; fix a race condition in Intialization.java

* Remove unnecessary comment

* Checkstyle

* Fix getFromExtensions()

* Add a reference to the comment about guarded computeIfAbsent() optimization; IdentityHashMap optimization

* Fix UriCacheGeneratorTest

* Workaround issue with MaterializedViewQueryQueryToolChest

* Strengthen Appenderator's contract regarding concurrency
2019-02-04 09:18:12 -08:00
Vadim Ogievetsky 06d56c2abd Fixing little issues and CSS bugs in the new web console. (#6995) 2019-02-04 09:04:58 -08:00
Clint Wylie 6207b66e20 fix build (#6994) 2019-02-03 09:38:51 -08:00
Jonathan Wei 953b96d0a4 Add more sketch aggregator support in Druid SQL (#6951)
* Add more sketch aggregator support in Druid SQL

* Add docs

* Tweak module serde register

* Fix tests

* Checkstyle

* Test fix

* PR comment

* PR comment

* PR comments
2019-02-02 22:34:53 -08:00
Surekha 7baa33049c Introduce published segment cache in broker (#6901)
* Add published segment cache in broker

* Change the DataSegment interner so it's not based on DataSEgment's equals only and size is preserved if set

* Added a trueEquals to DataSegment class

* Use separate interner for realtime and historical segments

* Remove trueEquals as it's not used anymore, change log message

* PR comments

* PR comments

* Fix tests

* PR comments

* Few more modification to

* change the coordinator api
* removeall segments at once from MetadataSegmentView in order to serve a more consistent view of published segments
* Change the poll behaviour to avoid multiple poll execution at same time

* minor changes

* PR comments

* PR comments

* Make the segment cache in broker off by default

* Added a config to PlannerConfig
* Moved MetadataSegmentView to sql module

* Add doc for new planner config

* Update documentation

* PR comments

* some more changes

* PR comments

* fix test

* remove unintentional change, whether to synchronize on lifecycleLock is still in discussion in PR

* minor changes

* some changes to initialization

* use pollPeriodInMS

* Add boolean cachePopulated to check if first poll succeeds

* Remove poll from start()

* take the log message out of condition in stop()
2019-02-02 22:27:13 -08:00
Furkan KAMACI 61f165c23f Try-with-resources should be used since the new syntax is more readable. (#6944)
* Try-with-resources should be used since the new syntax is more readable.

* Fixed checkstyle error.
2019-02-03 10:42:28 +08:00
Justin Borromeo 7b58471394 Licensing stuff 2019-02-02 03:48:18 -08:00
Justin Borromeo 79e8319383 Move ScanResultValue timestamp comparator to a separate class for testing 2019-02-01 18:22:58 -08:00
Justin Borromeo 7a6080f636 Stuff for time-ordered scan query 2019-02-01 18:00:58 -08:00
Justin Borromeo 26930f8d20 It runs. 2019-02-01 16:38:49 -08:00
Justin Borromeo dd4ec1ac9c Need to form queries 2019-02-01 15:12:17 -08:00
Justin Borromeo 6430ef8e1b lol (#6985) 2019-02-01 14:21:13 -08:00
Justin Borromeo dba6e492a0 Merge branch 'master' into 6088-Create-Scan-Benchmark 2019-02-01 14:13:39 -08:00
Justin Borromeo 10e57d5f9e Moved Scan Builder to Druids class and started on Scan Benchmark setup 2019-02-01 14:04:13 -08:00
Jihoon Son 7d4cc28730 Fix node path for building the unified console (#6981) 2019-02-01 13:57:17 -08:00
Clint Wylie 7a5827e12e bloom filter sql aggregator (#6950)
* adds sql aggregator for bloom filter, adds complex value serde for sql results

* fix tests

* checkstyle

* fix copy-paste
2019-02-01 13:54:46 -08:00
lxqfy e45f9ea5e9 Update metrics.md (#6976) 2019-02-01 13:40:44 -08:00
jorbay-au 852fe86ea2 Remove repeated word in indexing-service.md (#6983) 2019-02-01 13:38:22 -08:00
Clint Wylie 5c0fbbda1b use System.err and System.out to print exit messages on CliPeon (#6975)
* use System.err and System.out to print exit messages on CliPeon

* more

* not necessarily a stopping error...
2019-02-01 18:54:14 +08:00
Roman Leventov f7df5fedcc Add several missing inspectRuntimeShape() calls (#6893)
* Add several missing inspectRuntimeShape() calls

* Add lgK to runtime shapes
2019-01-31 20:04:26 -08:00
Gian Merlino 4e426327bb Some adjustments to config examples. (#6973)
* Some adjustments to config examples.

- Add ExitOnOutOfMemoryError to jvm.config examples. It was added a
pretty long time ago (8u92) and is helpful since it prevents zombie
processes from hanging around. (OOMEs tend to bork things)
- Disable Broker caching and enable it on Historicals in example
configs. This config tends to scale better since it enables the
Historicals to merge results rather than sending everything by-segment
to the Broker. Also switch to "caffeine" cache from "local".
- Increase concurrency a bit for Broker example config.
- Enable SQL in the example config, a baby step towards making SQL
more of a thing. (It's still off by default in the code.)
- Reduce memory use a bit for the quickstart configs.
- Add example Router configs, in case someone wants to use that. One
reason might be to get the fancy new console (#6923).

* Add example Router configs.

* Fix up router example properties.

* Add router to quickstart supervise conf.
2019-01-31 17:59:39 -08:00