Apache
/
druid
mirror of https://github.com/apache/druid.git
1
0
Fork 0
Code Issues Packages Projects Releases Wiki Activity
5,543 Commits 111 Branches 503 Tags 721 MiB
Commit Graph

1124 Commits

Author SHA1 Message Date
Charles Allen 70e3108282 Multiple speed improvements revolving around topN with HLL 
Change serializer / deserializer for HyperLogLog
* Changed DirectDruidClient's InputStream handling. Is now ~10% faster for data heavy queries, and has lower variance in execution speed.
* Changed HLL Collector's toByteStream() method to be better optimized for small values. Is notably faster for small result quantities which fall into the sparse HLL bucket codepath.
  * No change for dense HLL which just uses a direct bytestream of the underlying byte data.

TopNNumericResultBuilder semi-aggressive loop unrolling for metricVals

Benchmark for HLL for sparse packing (small HLL bucket population):
HyperLogLogSerdeBenchmarkTest.benchmarkToByteBuffer[0]: [measured 100000 out of 100100 rounds, threads: 1 (sequential)]
 round: 0.00 [+- 0.00], round.block: 0.00 [+- 0.00], round.gc: 0.00 [+- 0.00], GC.calls: 216, GC.time: 0.42, time.total: 15.96, time.warmup: 0.22, time.bench: 15.74
HyperLogLogSerdeBenchmarkTest.benchmarkToByteBuffer[1]: [measured 100000 out of 100100 rounds, threads: 1 (sequential)]
 round: 0.00 [+- 0.00], round.block: 0.00 [+- 0.00], round.gc: 0.00 [+- 0.00], GC.calls: 217, GC.time: 0.45, time.total: 13.87, time.warmup: 0.02, time.bench: 13.85
HyperLogLogSerdeBenchmarkTest.benchmarkToByteBuffer[2]: [measured 100000 out of 100100 rounds, threads: 1 (sequential)]
 round: 0.00 [+- 0.00], round.block: 0.00 [+- 0.00], round.gc: 0.00 [+- 0.00], GC.calls: 55, GC.time: 0.16, time.total: 4.13, time.warmup: 0.00, time.bench: 4.12
HyperLogLogSerdeBenchmarkTest.benchmarkToByteBuffer[3]: [measured 100000 out of 100100 rounds, threads: 1 (sequential)]
 round: 0.00 [+- 0.00], round.block: 0.00 [+- 0.00], round.gc: 0.00 [+- 0.00], GC.calls: 55, GC.time: 0.16, time.total: 4.30, time.warmup: 0.00, time.bench: 4.30
HyperLogLogSerdeBenchmarkTest.benchmarkToByteBuffer[4]: [measured 100000 out of 100100 rounds, threads: 1 (sequential)]
 round: 0.00 [+- 0.00], round.block: 0.00 [+- 0.00], round.gc: 0.00 [+- 0.00], GC.calls: 8, GC.time: 0.03, time.total: 1.10, time.warmup: 0.00, time.bench: 1.09
HyperLogLogSerdeBenchmarkTest.benchmarkToByteBuffer[5]: [measured 100000 out of 100100 rounds, threads: 1 (sequential)]
 round: 0.00 [+- 0.00], round.block: 0.00 [+- 0.00], round.gc: 0.00 [+- 0.00], GC.calls: 8, GC.time: 0.03, time.total: 0.72, time.warmup: 0.00, time.bench: 0.72
HyperLogLogSerdeBenchmarkTest.benchmarkToByteBuffer[6]: [measured 100000 out of 100100 rounds, threads: 1 (sequential)]
 round: 0.00 [+- 0.00], round.block: 0.00 [+- 0.00], round.gc: 0.00 [+- 0.00], GC.calls: 1, GC.time: 0.00, time.total: 0.60, time.warmup: 0.00, time.bench: 0.60
HyperLogLogSerdeBenchmarkTest.benchmarkToByteBuffer[7]: [measured 100000 out of 100100 rounds, threads: 1 (sequential)]
 round: 0.00 [+- 0.00], round.block: 0.00 [+- 0.00], round.gc: 0.00 [+- 0.00], GC.calls: 2, GC.time: 0.01, time.total: 0.26, time.warmup: 0.00, time.bench: 0.25

Updates to HyperLogLogCollector toByteBuffer() based on code review

Removed changes from DirectDruidClient from this branch and put it in another branch.

Changed HyperLogLogCollector to have protected getters and setters

Remove unused ByteOrder from HyperLogLogCollector

Copyright header on HyperLogLogSerdeBenchmarkTest

Now with less ass!

Reformat in TopNNumericResultsBuilder. No code change

Removed unused import in HyperLogLogCollector

Replace AppendableByteArrayInputStream in DirectDruidClient
* Replace with SequenceInputStream fueled by an enumeration of ChannelBufferInputStream which directly wrap the response context ChannelBuffer

Modify TopNQueryQueryToolChest to use Arrays instead of Lists

Modify TopNQueryQueryToolChest to use Arrays instead of Lists

Revert accidental changes to DirectDruidClient

They should be in another merge request:
https://github.com/metamx/druid/pull/893

Fixes from code review
* Extracting names from AggregatorFactory classes now done with TopNQueryQueryToolChest.extractFactoryName
* Renamed variable in TopNNumericResultBuilder
 2014-11-24 16:02:00 -08:00
fjy 13cae41f6c Merge branch 'master' into refactor-examples 2014-11-24 11:00:26 -08:00
xvrl 9ced097abd Merge pull request #895 from metamx/fix-interval-retry 
A set of fixes to retry the query for missing segments in the timeline
 2014-11-24 10:23:02 -08:00
fjy c88aff6205 remove unneeded constant 2014-11-24 10:20:02 -08:00
fjy 9da66291e1 change naming to common config 2014-11-21 15:36:42 -08:00
Charles Allen 8f80d9e189 Update CompressedObjectStrategy to try-with-resources but print log error if error while closing 2014-11-21 11:35:11 -08:00
Charles Allen fc9a54ea48 Fix CompressedObjectStrategy LZFCompressor to ignore error on close of ResourceHolder 2014-11-21 10:49:43 -08:00
Charles Allen f8ce68565b Modified CompressedObjectStrategy to use 0xFF for Uncompressed 2014-11-21 10:33:53 -08:00
Charles Allen aa49e56ed6 Merge remote-tracking branch 'origin/master' into druid-0.7.x-compressionstrategy 2014-11-21 10:29:40 -08:00
fjy ef62bccdec ignore benchmark 2014-11-20 16:52:19 -08:00
nishantmonu51 e3260aa177 Filtered Aggregator fixes + enhancements 
- fix NPE on IncrementIndex
- refactor code to support AND, OR filter
- tests for AND & OR filter
- handling for missing column / null values
 2014-11-20 15:17:18 -08:00
fjy 47f5c1bd0a fix retry interval is stupid 2014-11-20 12:50:56 -08:00
fjy 3d9d989a9f A set of fixes to retry the query for missing intervals in the timeline 2014-11-20 12:04:37 -08:00
nishantmonu51 0ab34f86da Revert "fix filtered Aggregator" 
This reverts commit 6fd37ce023.
 2014-11-20 10:17:01 +05:30
nishantmonu51 6fd37ce023 fix filtered Aggregator 
fix filtered Aggregator
remove unused name parameter for filtered aggregator
add tests
 2014-11-20 09:29:26 +05:30
fjy a49e673122 put back another missing test 2014-11-19 16:55:20 -08:00
fjy 14668846aa add back some tests 2014-11-19 14:35:26 -08:00
fjy fdeab0c6af make Druid case sensitive 2014-11-19 14:27:31 -08:00
Fangjin Yang 590d31799e Merge pull request #876 from metamx/remove-backwards-compatible 
Remove backwards compatible
 2014-11-19 14:33:14 -07:00
Charles Allen 18f44beee9 CompressedObjectStrategy improvements 
* Added more unit tests
* Now properly uses safe / fast decompressor for LZ4
* Now chooses fastest lz4 instance instead of only looking at Java implmentations
* Encapsulate ResourceHolder in try-with-resources to make sure they close correctly
 2014-11-19 11:10:59 -08:00
Charles Allen ccc757dc64 Merge remote-tracking branch 'origin/master' into druid-0.7.x-compressionstrategy 2014-11-19 09:39:35 -08:00
Charles Allen 1bbc8fcbe5 Allow Smile to fall back to text 
* Modify SmileFactory to set the delegate to text option.
  * This option only occurs when a Reader type object is passed in to the deserialization stuff
  * This is needed by the X-Druid-Response-Context header return value, which is JSON
 2014-11-18 15:16:14 -08:00
Charles Allen 42517f5d37 Merge pull request #884 from metamx/optimize-topN-pruning 
optimise pruning of aggs
 2014-11-18 14:19:30 -08:00
xvrl a96eaeb036 Merge pull request #882 from metamx/now_with_OPEN_SOURCE 
Added src jar build to maven poms and re-formatted to conform to style guidelines.
 2014-11-18 13:00:04 -08:00
nishantmonu51 6023d602e6 optimise pruning of aggs 
optimise pruning of aggregators for topN
 2014-11-19 00:17:25 +05:30
Charles Allen dc66e1708e Added src jar build to maven poms and re-formatted to conform to style guidelines. 2014-11-18 09:05:30 -08:00
Xavier Léauté d914afe1cd make defaultVersion configurable for non-jar testing 2014-11-17 13:54:32 -08:00
nishantmonu51 0c2d06475d merge from master 2014-11-17 19:19:18 +05:30
nishantmonu51 cbffe3c648 merge from master and resolve conflicts 2014-11-17 18:07:08 +05:30
fjy d5c4282766 fix broken ut 2014-11-14 13:45:42 -08:00
xvrl e1e171ad20 Merge pull request #865 from metamx/fix-retry-qr 
Fix a couple of bugs with retry query runner after testing it locally
 2014-11-14 13:33:51 -08:00
fjy df1ad95075 remove useless config 2014-11-14 13:32:19 -08:00
fjy d641d41f9e address another cr 2014-11-14 13:29:59 -08:00
fjy 7736c3fc27 address cr 2014-11-14 13:28:32 -08:00
Fangjin Yang 6ee8029462 Merge pull request #866 from metamx/mutableBitmapBenchmark 
Add benchmarking for bitmaps
 2014-11-14 14:16:21 -07:00
xvrl a4fc64ca3f Merge pull request #856 from metamx/druid-845 
Fix query by segment
 2014-11-14 13:10:54 -08:00
Charles Allen 4b7ab23289 Remove getIntervalString from BySegmentResultValue 2014-11-14 13:03:48 -08:00
fjy bbc079b880 fix retry to actually return correct sequences 2014-11-14 12:10:04 -08:00
Charles Allen 648759e9f6 Add deserialization benchmark to BitmapCreationBenchmark 2014-11-13 13:43:14 -08:00
Charles Allen 483b2c7be0 Add copyright notice to BitmapCreationBenchmark 2014-11-13 12:55:02 -08:00
Charles Allen 228fb0cf40 Add benchmarking for bitmaps 
Here are the results on my laptop:

BitmapCreationBenchmark.testRandomAddition[0]: [measured 10 out of 20 rounds, threads: 1 (sequential)]
 round: 0.49 [+- 0.07], round.block: 0.00 [+- 0.00], round.gc: 0.00 [+- 0.00], GC.calls: 16, GC.time: 0.01, time.total: 9.91, time.warmup: 5.06, time.bench: 4.86
BitmapCreationBenchmark.testLinearAdditionDescending[0]: [measured 1000 out of 1010 rounds, threads: 1 (sequential)]
 round: 0.01 [+- 0.00], round.block: 0.00 [+- 0.00], round.gc: 0.00 [+- 0.00], GC.calls: 74, GC.time: 0.03, time.total: 5.82, time.warmup: 0.06, time.bench: 5.76
BitmapCreationBenchmark.testToImmutableByteArray[0]: [measured 1000 out of 1010 rounds, threads: 1 (sequential)]
 round: 0.00 [+- 0.00], round.block: 0.00 [+- 0.00], round.gc: 0.00 [+- 0.00], GC.calls: 0, GC.time: 0.00, time.total: 1.80, time.warmup: 0.02, time.bench: 1.78
BitmapCreationBenchmark.testRandomAddition[1]: [measured 10 out of 20 rounds, threads: 1 (sequential)]
 round: 0.00 [+- 0.00], round.block: 0.00 [+- 0.00], round.gc: 0.00 [+- 0.00], GC.calls: 0, GC.time: 0.00, time.total: 0.12, time.warmup: 0.08, time.bench: 0.04
BitmapCreationBenchmark.testLinearAdditionDescending[1]: [measured 1000 out of 1010 rounds, threads: 1 (sequential)]
 round: 0.00 [+- 0.00], round.block: 0.00 [+- 0.00], round.gc: 0.00 [+- 0.00], GC.calls: 10, GC.time: 0.01, time.total: 4.26, time.warmup: 0.04, time.bench: 4.22
BitmapCreationBenchmark.testToImmutableByteArray[1]: [measured 1000 out of 1010 rounds, threads: 1 (sequential)]
 round: 0.01 [+- 0.00], round.block: 0.00 [+- 0.00], round.gc: 0.00 [+- 0.00], GC.calls: 27, GC.time: 0.01, time.total: 5.11, time.warmup: 0.05, time.bench: 5.06
BitmapCreationBenchmark.testLinearAddition[0]: [measured 1000 out of 1010 rounds, threads: 1 (sequential)]
 round: 0.00 [+- 0.00], round.block: 0.00 [+- 0.00], round.gc: 0.00 [+- 0.00], GC.calls: 0, GC.time: 0.00, time.total: 3.48, time.warmup: 0.04, time.bench: 3.45
BitmapCreationBenchmark.testLinearAddition[1]: [measured 1000 out of 1010 rounds, threads: 1 (sequential)]
 round: 0.00 [+- 0.00], round.block: 0.00 [+- 0.00], round.gc: 0.00 [+- 0.00], GC.calls: 9, GC.time: 0.00, time.total: 2.95, time.warmup: 0.03, time.bench: 2.92
2014-11-13 12:47:23,995 INFO [main] io.druid.segment.data.BitmapCreationBenchmark - Entry [0] is io.druid.segment.data.ConciseBitmapSerdeFactory
2014-11-13 12:47:23,995 INFO [main] io.druid.segment.data.BitmapCreationBenchmark - Entry [1] is io.druid.segment.data.RoaringBitmapSerdeFactory
 2014-11-13 12:48:04 -08:00
fjy 6cdd6a6af7 change default settings for retry 2014-11-13 12:43:30 -08:00
fjy 2b0ed30344 add test 2014-11-13 12:38:35 -08:00
fjy 0d6816a037 Fix a couple of bugs with retry query runner after testing it locally 2014-11-13 11:53:29 -08:00
Fangjin Yang 4a3c0fc5c4 Merge pull request #848 from metamx/druid-0.7.x-fastTopN-rebase 
TopN performance improvements
 2014-11-13 11:56:20 -07:00
Charles Allen 34c3464bc9 Added more explanatory comments in PooledTopNAlgorithm 2014-11-13 10:47:56 -08:00
Charles Allen 9a9238a801 Merge remote-tracking branch 'origin/master' into druid-845 2014-11-13 10:04:56 -08:00
Xavier Léauté 19a37f773f test for groupBy order 2014-11-12 22:48:16 -08:00
Charles Allen 31fed7d329 Fix query by segment 
* Changed topN queries to use joda Interval instead of string values
* topN by segment now implements BySegmentResultValue<Result<TopNResultValue>> instead of BySegmentResultValue<TopNResultValue>
* Added a unit test which failed uner the prior implementation.
 2014-11-12 21:20:59 -08:00
Xavier Léauté 4ac1aaf90e Merge remote-tracking branch 'origin/master' into druid-0.7.x 
Conflicts:
	processing/src/main/java/io/druid/segment/QueryableIndexStorageAdapter.java
 2014-11-12 14:08:51 -08:00
Fangjin Yang 4b4f1c7d38 Merge pull request #846 from vikramakrishnan/cacheKeyFix 
Include origin when creating the cacheKey for period Grans
 2014-11-12 11:15:21 -07:00
Fangjin Yang 7ef19009db Merge pull request #834 from metamx/optimize-timestampchecking 
skip timestamp checking if not required, remove duplicate code
 2014-11-12 09:45:34 -07:00
Vikram Ramakrishnan 8151d14786 Include origin when creating the cacheKey for period Grans 2014-11-12 21:36:46 +05:30
Charles Allen 581e6830d5 Merge pull request #840 from metamx/powers-of-2-buffers 
make buffer size a power of 2 and optimize buffer lookup
 2014-11-11 19:54:00 -08:00
Xavier Léauté 60e98c35d7 workaround for annotations requiring class literals 2014-11-11 18:48:49 -08:00
Xavier Léauté b580269f6d Distinguish between default and legacy bitmaps 2014-11-11 18:35:45 -08:00
fjy bc5c56e441 fix default impl 2014-11-11 18:00:46 -08:00
fjy 63ca2375a3 remove dead code and cleanup some defaults 2014-11-11 17:57:24 -08:00
fjy 5629307548 address a few more comments on roaring 2014-11-11 17:50:03 -08:00
Charles Allen a89b539b4f Merge pull request #823 from metamx/roaring 
Configurable bitmap indexes: roaring and concise
 2014-11-11 17:26:38 -08:00
Xavier Léauté 007e57f876 make buffer size a power of 2 and optimize 2014-11-11 16:24:37 -08:00
fjy 1cc162727b address code review 2014-11-11 14:05:37 -08:00
Xavier Léauté 3f5449d40a loop unrolling provides no benefit for timeseries 2014-11-11 10:58:37 -08:00
Xavier Léauté e817db8b6c unroll timeseries aggregations + naming 2014-11-11 10:09:37 -08:00
fjy e6b7b03b5b fix conversion 2014-11-10 17:13:28 -08:00
fjy eb457c280e revert change 2014-11-10 17:00:53 -08:00
fjy 336c73bdc2 cleanup dead code 2014-11-10 16:53:13 -08:00
fjy df886fac1b fix 2014-11-10 16:49:27 -08:00
fjy d68bc3bdea cleanup unused imports 2014-11-10 16:15:28 -08:00
Charles Allen 92e71be864 Change TopNQueryRunnerBenchmark to use a ByteBuffer as per OffheapBufferPool 2014-11-10 15:40:54 -08:00
Charles Allen a093f3728a Reformat on TopNNumericResultBuilder 2014-11-10 15:26:09 -08:00
Charles Allen 2b0f4534bf Modify formatting in TopNQueryRunnerBenchmark 2014-11-10 15:17:26 -08:00
Charles Allen fc78f215c4 Aggressive dimValue unrolling in PooledTopNAlgorithm 2014-11-10 15:14:45 -08:00
fjy 6188315293 Merge branch 'master' into druid-0.7.x 
Conflicts:
	processing/src/test/java/io/druid/query/search/SearchQueryRunnerTest.java
 2014-11-10 14:52:10 -08:00
Xavier Léauté 49e878cf1a unroll multi-value dimensions 2014-11-10 14:21:56 -08:00
fjy df9be030db remove more legacy code 2014-11-10 14:09:00 -08:00
Charles Allen dbb6401fbe Aggressive unrolling in PooledTopNAlgorithm 2014-11-10 14:04:37 -08:00
fjy 72c355e8ae remove crazy handler code 2014-11-10 13:58:23 -08:00
fjy fc34858e95 fix things for real-time ingestion 2014-11-10 13:48:21 -08:00
fjy c5cc826998 make things actually work with roaring 2014-11-10 13:42:26 -08:00
fjy 358b2add17 make things actually work with roaring 2014-11-10 13:42:06 -08:00
Charles Allen 9ac8589143 Merge remote-tracking branch 'origin/druid-0.7.x-fastTopN-rebase' into druid-0.7.x-fastTopN-rebase 2014-11-10 12:55:08 -08:00
Xavier Léauté 70468400bf stupid mistake 2014-11-10 12:54:48 -08:00
Charles Allen bfc9d9f283 Merge remote-tracking branch 'origin/druid-0.7.x-fastTopN-rebase' into druid-0.7.x-fastTopN-rebase 2014-11-10 12:52:36 -08:00
Xavier Léauté 17d32a2561 try to benefit from out of order execution 2014-11-10 12:50:10 -08:00
Charles Allen 5aad149b24 Make some variables in PooledTopNAlgorithm final 2014-11-10 12:19:04 -08:00
nishantmonu51 0ca60e0334 skip timestamp checking, remove duplicate code 
1) No need compare all timestamps when all the values are within range
2) remove redundant code.
 2014-11-10 22:36:07 +05:30
fjy 3c21f62afd a whole bunch of fixes and some ugly conversion code 2014-11-07 16:03:22 -08:00
Charles Allen 7131cde710 Code reformat in CompressedObjectStrategy 2014-11-07 11:19:32 -08:00
Charles Allen 8bd6cf0d07 Updates to CompressedObjectStrategy to support more compression types 
* Compression types are not yet dynamically configurable.
* Added a benchmarking system for topN to test the compression
* Updated pom.xml to include junit benchmarking
* added an Uncompressed option
 2014-11-07 11:14:46 -08:00
Charles Allen f5f48c7756 Moved pom version for reals this time 2014-11-07 11:09:57 -08:00
Charles Allen 6af18931e5 Moveed processing/pom version into master pom 2014-11-07 11:08:23 -08:00
Charles Allen 75c79e7562 Added comments in TopNNumericResultBuilder 2014-11-07 11:01:46 -08:00
Charles Allen b4893b4490 Pre allocate LinkedHashMap in TopNNumericResultBuilder 2014-11-07 10:56:49 -08:00
Charles Allen bc92de233c Update dim name comparator in TopNNumericResultBuilder to better handle nulls 2014-11-07 10:54:10 -08:00
Charles Allen 0a562ebd77 Removed CompressedObjectStrategy from this pull request. will submit a new one later 2014-11-07 10:42:28 -08:00
Charles Allen 228614ddb5 Changed TopNNumericResultBuilder imports to not use package.*, but instead pulls all in explicitly 2014-11-07 10:34:28 -08:00
Charles Allen 956bdf80dd Removed negating comparator in TopNNumericResultBuilder 2014-11-07 10:29:06 -08:00
nishantmonu51 fd8eb7742b handle union query on realtime node 2014-11-07 23:27:50 +05:30
nishantmonu51 8bebb24fd5 Union Queries move merge to historical 2014-11-07 18:05:16 +05:30
Charles Allen ee019872f7 TopN performance improvements 
Re-factor scanAndAggregate in PooledTopN

* Loops are now a little bit tighter when looping over aggregates. This will hopefully assist in loop execution optimization.
* Pre-calculated the aggregate offsets instead of shifting them during runtime.
* Cursor loop could use some TLC, but would require a massive refactoring on how TopN queries are executed.
  * Any potential modifications to query workflow need to account for Stream vs Batch data, and that not all data will be array backed that comes in.

Change data storage type in TopNNumericResultBuilder.

  * Use PriorityQueue to store
  * Checks to see if should even bother adding to Queue before adding.
  * Re-orders Queue on build() call.
  * Ideally the order would be directly preserved on build(), but this is a close second.

Updates to CompressedObjectStrategy to support more compression types

 * Compression types are not yet dynamically configurable.
 * Added a benchmarking system for topN to test the compression
 * Updated pom.xml to include junit benchmarking
 * added an Uncompressed option
 2014-11-05 11:10:37 -08:00
fjy 253009d208 initial commit for roaring bitmaps 2014-11-04 17:04:40 -08:00
fjy 3e158f00ef fix random lowercasing in test 2014-10-30 10:56:39 -07:00
fjy 75da039396 fix search with same value in multiple columns 2014-10-30 10:43:42 -07:00
fjy 41ebdd2bae Merge branch 'druid-0.7.x' into fix-test 
Conflicts:
	processing/pom.xml
	processing/src/main/java/io/druid/segment/IndexMerger.java
	processing/src/main/java/io/druid/segment/MetricHolder.java
	processing/src/main/java/io/druid/segment/data/CompressedObjectStrategy.java
	processing/src/test/java/io/druid/query/groupby/GroupByQueryRunnerTest.java
	processing/src/test/java/io/druid/segment/SchemalessTestFull.java
	server/src/test/java/io/druid/client/CachingClusteredClientTest.java
	services/src/main/java/io/druid/cli/CliHistorical.java
 2014-10-24 11:07:52 -07:00
fjy bef74104d9 merge with 0.7.x and resolve any conflicts 2014-10-23 17:24:06 -07:00
fjy f790a05bd7 Merge branch 'master' into fix-test 
Conflicts:
	pom.xml
 2014-10-23 14:39:51 -07:00
fjy 1b3921faad fix for long columns 2014-10-23 14:33:25 -07:00
fjy 3b29e77866 [maven-release-plugin] prepare for next development iteration 2014-10-22 16:25:32 -07:00
fjy dcab2997f2 [maven-release-plugin] prepare release druid-0.6.160 2014-10-22 16:25:27 -07:00
fjy 94da2b6ad3 more logging for potential NPE because of segment unmapping 2014-10-22 16:16:10 -07:00
Xavier Léauté 4ec1e6ef13 simplify code a bit 2014-10-22 14:25:59 -07:00
nishantmonu51 0785baf54c review comments 2014-10-22 21:28:31 +05:30
nishantmonu51 6cb6ec39cf add support for not filter and add test 2014-10-22 10:25:24 +05:30
Xavier Léauté e24a6d222f we even have tests now 2014-10-21 17:46:24 -07:00
Xavier Léauté 23a6ab19b0 cleanup and add missing methods 2014-10-21 17:00:39 -07:00
nishantmonu51 56912adccb avoid typecasts 2014-10-21 16:47:50 -07:00
nishantmonu51 10db94101f skip value lookup 
compare ids instead of values
 2014-10-21 16:47:49 -07:00
Xavier Léauté 71831e49d9 rudimentary filtered aggregator 2014-10-21 16:47:49 -07:00
fjy bb90432862 Updated druid-api version to always sort dimension names, these tests needed fixing 2014-10-15 12:27:21 -04:00
Xavier Léauté 5b194ee5fe fix groupBy ordering casing bug 2014-10-15 12:16:17 -04:00
nishantmonu51 bce388fb27 merge changes from 0.7.x branch 2014-10-14 18:46:02 +05:30
fjy a4c8f04409 [maven-release-plugin] prepare for next development iteration 2014-10-13 12:50:45 -07:00
fjy 7fd1747ffa [maven-release-plugin] prepare release druid-0.6.159 2014-10-13 12:50:41 -07:00
nishantmonu51 454acd3f5a remove backwards compatible code 
1) remove backwards compatible and deprecated code
2) make hashed partitions spec default
 2014-10-13 19:30:44 +05:30
Xavier Léauté 592611f2ee fix cardinality aggregator serde 2014-10-09 17:06:29 -07:00
fjy d0fbd5f729 [maven-release-plugin] prepare for next development iteration 2014-10-09 14:24:53 -07:00
fjy dada60a301 [maven-release-plugin] prepare release druid-0.6.158 2014-10-09 14:24:48 -07:00
Xavier Léauté 290107ce7b remove unused jsonspec content type 2014-10-09 10:26:47 -07:00
fjy 8a4b32c482 [maven-release-plugin] prepare for next development iteration 2014-10-07 14:30:26 -07:00
fjy 96e29c2fdd [maven-release-plugin] prepare release druid-0.6.157 2014-10-07 14:30:20 -07:00
Xavier Léauté 30183d3b4d update druid-api for json parsing improvements 2014-10-07 13:25:02 -07:00
fjy fde510bbfc Merge pull request #779 from metamx/fix-histo 
Fix groupBy with histograms
 2014-10-07 11:53:24 -06:00
fjy d4217f1588 Merge pull request #777 from metamx/initialized-endpoint 
Historical endpoints accessible while loading + new loadstatus endpoint
 2014-10-06 17:35:58 -06:00
fjy c7b4d5b7b4 Merge branch 'master' into druid-0.7.x 
Conflicts:
	processing/src/test/java/io/druid/segment/filter/SpatialFilterTest.java
 2014-10-02 18:12:10 -07:00
fjy 816986f914 cleanup according to cr 2014-10-02 17:51:33 -07:00
fjy 1a67d6cb2e some tests for broken approx histo 2014-10-01 15:10:23 -07:00
nishantmonu51 09c41afb5b license header 2014-10-01 17:52:57 +05:30
nishantmonu51 ad75a21040 separate offheapIncrementalIndex implementation 2014-10-01 13:58:51 +05:30
fjy 267e861162 fix search 2014-09-30 13:36:55 -07:00
fjy 0f96a09939 use multi-val dims for metrics 2014-09-30 13:36:01 -07:00
nishantmonu51 358ff915bb fix merge conflicts 2014-09-30 22:19:18 +05:30
nishantmonu51 63fcd11864 fix compilation 2014-09-30 22:12:06 +05:30
nishantmonu51 2789536bed merge changes from druid-0.7.x 2014-09-30 22:05:49 +05:30
nishantmonu51 7ba914d379 review comments 2014-09-30 18:26:43 +05:30
nishantmonu51 61c7fd2e6e make ingestOffheap tuneable 2014-09-30 15:30:02 +05:30
Gian Merlino a3a2c8b86f Merge pull request #765 from metamx/fix-spatial 
fix spatial indexing with multiple spatial dims
 2014-09-29 17:23:29 -07:00
fjy e1c1e8997a address cr 2014-09-29 17:22:58 -07:00
fjy b57ac513bd Fixes #761 2014-09-29 14:56:43 -07:00
fjy 2b5ce89554 fix spatial indexing with multi spatial dims 2014-09-29 12:07:27 -07:00
nishantmonu51 3f66d3c167 review comments + mapdb deadlock fix 2014-09-29 21:49:12 +05:30
nishantmonu51 adb4a65e0a Merge branch 'offheap-incremental-index' into mapdb-branch 2014-09-29 12:38:31 +05:30
fjy eaf0a48b92 Merge branch 'master' into druid-0.7.x 
Conflicts:
	cassandra-storage/pom.xml
	common/pom.xml
	examples/pom.xml
	hdfs-storage/pom.xml
	histogram/pom.xml
	indexing-hadoop/pom.xml
	indexing-service/pom.xml
	kafka-eight/pom.xml
	kafka-seven/pom.xml
	pom.xml
	processing/pom.xml
	processing/src/main/java/io/druid/guice/PropertiesModule.java
	rabbitmq/pom.xml
	s3-extensions/pom.xml
	server/pom.xml
	services/pom.xml
 2014-09-26 11:39:24 -07:00
Xavier Léauté 77c9afe92f try to make javadoc happy 2014-09-23 18:53:29 -07:00
Xavier Léauté 7cfe86b7bc [maven-release-plugin] prepare for next development iteration 2014-09-23 16:31:25 -07:00
Xavier Léauté 80166314d9 [maven-release-plugin] prepare release druid-0.6.156 2014-09-23 16:31:21 -07:00
fjy 4235e14fc6 [maven-release-plugin] prepare for next development iteration 2014-09-23 16:12:43 -07:00
fjy 576e9b8a84 [maven-release-plugin] prepare release druid-0.6.155 2014-09-23 16:12:39 -07:00
fjy 0187c7f1c3 Merge pull request #748 from metamx/fix-columnschema-access 
fix column selector on varying incr index schema
 2014-09-23 11:25:46 -06:00
fjy 01bb7de3aa Merge pull request #756 from metamx/dimdim-comments 
add some comments about string interning
 2014-09-23 11:24:11 -06:00
nishantmonu51 f51ab84386 merge changes from druid-0.7.x 2014-09-22 23:48:45 +05:30
nishantmonu51 443e5788fb make OffheapIncrementalIndex tuneable 2014-09-22 19:26:10 +05:30
nishantmonu51 59aa8dc368 Merge branch 'offheap-incremental-index' into mapdb-branch 2014-09-19 23:09:35 +05:30
nishantmonu51 8c16377e9e add custom serializer 
add custom serializers
 2014-09-19 23:09:31 +05:30
nishantmonu51 8eb6466487 revert buffer size and add back rowFlushBoundary 2014-09-19 23:06:04 +05:30
Xavier Léauté e5a13544a0 fix column selector on varying incr index schema 2014-09-17 10:36:05 -07:00
fjy a17794a516 [maven-release-plugin] prepare for next development iteration 2014-09-16 15:22:55 -07:00
fjy f9c91ace3f [maven-release-plugin] prepare release druid-0.6.154 2014-09-16 15:22:49 -07:00
fjy cada14586a fix tests 2014-09-16 14:16:49 -07:00
fjy 8b5c2fcd82 fix compile error 2014-09-16 14:10:02 -07:00
xvrl ff3163516f Merge pull request #744 from metamx/fix-select 
fix select for non existant dimensions and metrics
 2014-09-16 14:03:51 -07:00
fjy 9e0a2fd487 fix select for non existant dimensions and metrics 2014-09-16 14:01:21 -07:00
Xavier Léauté 0c35fe2cf4 properly close reader 2014-09-15 13:00:06 -07:00
Xavier Léauté c8b8e3f6e9 negating compare is bad 2014-09-15 13:00:06 -07:00
Xavier Léauté 137ad50bf1 classes that should be static 2014-09-15 13:00:06 -07:00
Xavier Léauté e57e2d97ba make constants final 2014-09-15 12:53:40 -07:00
Xavier Léauté d4795ce927 fix missing charsets 2014-09-15 12:53:40 -07:00
Xavier Léauté cfa92e8217 fix incorrect nullable annotations 2014-09-15 12:13:52 -07:00
fjy 257fb3f18e fix retry logic 2014-09-12 15:47:41 -07:00
nishantmonu51 19bc77134a use hash map and in memory sort during index generation 
use hash map and in memory sort during index generation
 2014-09-12 23:39:02 +05:30
nishantmonu51 b94c5f0bd6 use putIfAbsent instead of get and put 
use putIfAbsent instead of get and put. reduce no. of calls to mapdb
decreases latency.
 2014-09-12 22:32:40 +05:30
fjy 469ccbbe5e Merge branch 'master' into druid-0.7.x 
Conflicts:
	cassandra-storage/pom.xml
	common/pom.xml
	examples/pom.xml
	hdfs-storage/pom.xml
	histogram/pom.xml
	indexing-hadoop/pom.xml
	indexing-service/pom.xml
	kafka-eight/pom.xml
	kafka-seven/pom.xml
	pom.xml
	processing/pom.xml
	processing/src/main/java/io/druid/query/FinalizeResultsQueryRunner.java
	processing/src/main/java/io/druid/query/UnionQueryRunner.java
	processing/src/main/java/io/druid/query/groupby/GroupByQueryRunnerFactory.java
	processing/src/main/java/io/druid/query/topn/TopNQueryEngine.java
	processing/src/main/java/io/druid/query/topn/TopNQueryRunnerFactory.java
	rabbitmq/pom.xml
	s3-extensions/pom.xml
	server/pom.xml
	server/src/test/java/io/druid/server/initialization/JettyTest.java
	services/pom.xml
 2014-09-11 16:20:50 -07:00
fjy e0c167753e Merge branch 'offheap-incremental-index' into mapdb-branch 2014-09-11 11:03:01 -07:00
nishantmonu51 c39eaf870b TimeAndDims optimise to store indexes 2014-09-11 16:13:37 +05:30
Xavier Léauté 32e256a842 [maven-release-plugin] prepare for next development iteration 2014-09-10 17:24:15 -07:00
Xavier Léauté f7dc13d240 [maven-release-plugin] prepare release druid-0.6.153 2014-09-10 17:24:11 -07:00
fjy fec7b43fcb make making v9 segments something completely configurable 2014-09-10 15:28:30 -07:00
nishantmonu51 2e3be39048 Merge branch 'offheap-incremental-index' into mapdb-branch 2014-09-10 11:06:54 +05:30
fjy 351afb8be7 allow legacy index generator 2014-09-09 17:04:35 -07:00
nishantmonu51 2b3ad4d742 append a random UUID to map name 
avoid collision with dimension name
 2014-09-09 23:20:19 +05:30
nishantmonu51 e7a5a01f7b cache size locally 2014-09-09 16:02:50 +05:30
nishantmonu51 178f002f05 indexing working with mapdb 2014-09-09 15:59:33 +05:30
Xavier Léauté 22553b0b28 [maven-release-plugin] prepare for next development iteration 2014-09-05 11:07:22 -07:00
Xavier Léauté 570a7a07bf [maven-release-plugin] prepare release druid-0.6.152 2014-09-05 11:07:14 -07:00
Xavier Léauté c255bd76a9 add a test for the previous fix 2014-09-05 11:03:35 -07:00
Xavier Léauté 5dd23b19c3 fix interval start before end in TimewarpOperator 2014-09-05 10:55:45 -07:00
fjy e24be03a38 [maven-release-plugin] prepare for next development iteration 2014-09-04 14:10:13 -07:00
fjy a933ea1866 [maven-release-plugin] prepare release druid-0.6.151 2014-09-04 14:10:09 -07:00
xvrl c4c54f7bc9 Merge pull request #721 from metamx/fix-union-query 
fix Union Query result merging
 2014-09-04 12:20:19 -07:00
nishantmonu51 0dec5826ed remove unused imports 2014-09-05 00:07:42 +05:30
nishantmonu51 90632ecf3a review comment - have separate methods 2014-09-05 00:03:36 +05:30
fjy f473e62f1f Merge pull request #714 from metamx/cleanup-javadocs 
Cleanup javadocs
 2014-09-04 12:28:10 -06:00
nishantmonu51 2772a78b3b review comments - move merging to tool chest 2014-09-04 11:31:04 +05:30
nishantmonu51 268a42bd79 fix Union Query result merging 2014-09-03 16:38:02 +05:30
nishantmonu51 c056960082 fix nested group by with hyperuniques 2014-09-01 10:50:09 +05:30
Xavier Léauté 75134335de fix last merge from master not being forward-compatible 2014-08-30 22:45:36 -07:00
Xavier Léauté 508e982190 Merge remote-tracking branch 'origin/master' into druid-0.7.x 
Conflicts:
	cassandra-storage/pom.xml
	common/pom.xml
	examples/config/historical/runtime.properties
	examples/config/overlord/runtime.properties
	examples/config/realtime/runtime.properties
	examples/pom.xml
	hdfs-storage/pom.xml
	histogram/pom.xml
	indexing-hadoop/pom.xml
	indexing-service/pom.xml
	kafka-eight/pom.xml
	kafka-seven/pom.xml
	pom.xml
	processing/pom.xml
	rabbitmq/pom.xml
	s3-extensions/pom.xml
	server/pom.xml
	server/src/main/java/io/druid/server/ClientQuerySegmentWalker.java
	services/pom.xml
 2014-08-30 22:42:36 -07:00
Gian Merlino 96d7b18163 [maven-release-plugin] prepare for next development iteration 2014-08-29 15:29:58 -07:00
Gian Merlino 860cf36ab1 [maven-release-plugin] prepare release druid-0.6.150 2014-08-29 15:29:54 -07:00
xvrl 068a5fe2aa Merge pull request #715 from metamx/aggregator-factory-equals-hashcode 
AggregatorFactory equals() and hashCode() methods.
 2014-08-29 15:22:00 -07:00
Gian Merlino 879e615314 AggregatorFactory equals() and hashCode() methods. 2014-08-29 15:02:04 -07:00
Xavier Léauté 58ab759fc6 remove unused imports 2014-08-29 14:03:47 -07:00
Xavier Léauté ac05836833 make Java 8 javadoc happy 2014-08-29 13:58:50 -07:00
Xavier Léauté daca178631 [maven-release-plugin] prepare for next development iteration 2014-08-29 10:43:19 -07:00
Xavier Léauté 0793a5aee6 [maven-release-plugin] prepare release druid-0.6.149 2014-08-29 10:43:15 -07:00
Xavier Léauté 674e7b0447 add tests for timeboundary timewarp 2014-08-28 17:59:56 -07:00
Xavier Léauté 743a9bfdac fix offset and nulls 2014-08-28 17:35:26 -07:00
Xavier Léauté 9f742a87a6 fix timewarp with different bounds 2014-08-28 17:15:45 -07:00
Xavier Léauté b0b39d6ec0 fix maxTime caching for individual bounds 2014-08-28 17:14:25 -07:00
Xavier Léauté 1d6842c7f0 [maven-release-plugin] prepare for next development iteration 2014-08-27 12:22:53 -07:00
Xavier Léauté 575951f99a [maven-release-plugin] prepare release druid-0.6.148 2014-08-27 12:22:49 -07:00
Gian Merlino 661e467d71 Fix groupBy/timeseries test, follow up to #705. 2014-08-26 22:36:08 -07:00
Xavier Léauté d15cd8fde7 add some comments about string interning 2014-08-26 21:33:38 -07:00
Gian Merlino 0eea1dc08d Test for desired timeseries zero-filling behavior. 2014-08-26 18:52:08 -07:00
Gian Merlino da48a713ae Fix overzealous timeseries zero-filling. 
When the index's maxTime is not aligned with the query granularity, gran.next
can cause an time extra bucket to get zero-filled. Truncating first prevents
that.
 2014-08-26 18:52:08 -07:00
Gian Merlino 28e3191503 Merge pull request #699 from metamx/postprocessing-operators 
query post-processing operators
 2014-08-26 17:23:37 -07:00
Xavier Léauté e39b6975fa rename t to now 2014-08-26 17:16:09 -07:00
Xavier Léauté a1c8b23ba7 fix duration granularity with non-zero origin 2014-08-26 11:47:07 -07:00
Xavier Léauté 7537cf4fb1 fix compound period granularity for exact periods 2014-08-26 10:34:08 -07:00
Xavier Léauté eb7dd6dc74 add documentation and tests 2014-08-26 09:53:46 -07:00
Xavier Léauté b52869ff28 default to noop 2014-08-25 17:17:35 -07:00
Xavier Léauté 047f793cb7 query post-processing operators 
- Allows post-processing of query results
- Example timewarp post-processing operator
 2014-08-25 16:55:46 -07:00
Xavier Léauté 3261fd879a fix groupBy deserialization 2014-08-22 13:43:56 -07:00
Xavier Léauté 72eb64a40c fix HLL equals when comparing sparse and dense 2014-08-22 13:42:46 -07:00
fjy 00f7077f50 [maven-release-plugin] prepare for next development iteration 2014-08-22 08:28:56 -07:00
fjy df488563e4 [maven-release-plugin] prepare release druid-0.6.147 2014-08-22 08:28:51 -07:00
fjy 172d0b6291 fix cache bug with group by query and uniques 2014-08-22 08:26:06 -07:00
fjy 197c80a694 fix logging 2014-08-21 13:40:55 -07:00
fjy 64ff5f0880 add back broken synch 2014-08-21 13:20:23 -07:00
fjy e5d1c390fa add extra logging 2014-08-21 13:17:29 -07:00
fjy df48b48e35 [maven-release-plugin] prepare for next development iteration 2014-08-21 12:50:33 -07:00
fjy 4f0b994037 [maven-release-plugin] prepare release druid-0.6.146 2014-08-21 12:50:30 -07:00
nishantmonu51 c216eb7340 remove ThreadSafeAggregator 2014-08-21 23:56:07 +05:30
nishantmonu51 67f4bbae74 fixes from review comments 
fix sync of aggs,
fix NPE in sink.isFull,
RealtimeTuningConfig lower the bufferSize to 256m
 2014-08-21 09:24:14 +05:30
Xavier Léauté 8933685884 log allocation of conversionBuf 2014-08-20 12:41:47 -07:00
fjy fd09b1b5f1 cleanup things according to cr 2014-08-20 09:49:21 -07:00
nishantmonu51 33354cf7fe replace maxRowsInMemory with BufferSize 2014-08-20 20:59:44 +05:30
nishantmonu51 e525562767 review comments - cleanup ColumnSelectorFactory 2014-08-20 15:04:43 +05:30
nishantmonu51 60906c3244 Revert "make valueType configurable" 
This reverts commit 6f60a3f604.
 2014-08-20 11:55:26 +05:30
nishantmonu51 0c77f549ff Revert "return correct valueType" 
This reverts commit 0b42ca4c75.
 2014-08-20 11:54:20 +05:30