Apache
/
druid
mirror of https://github.com/apache/druid.git
1
0
Fork 0
Code Issues Packages Projects Releases Wiki Activity
5,093 Commits 114 Branches 506 Tags 732 MiB
Commit Graph

289 Commits

Author SHA1 Message Date
nishantmonu51 0ab34f86da Revert "fix filtered Aggregator" 
This reverts commit 6fd37ce023.
 2014-11-20 10:17:01 +05:30
nishantmonu51 6fd37ce023 fix filtered Aggregator 
fix filtered Aggregator
remove unused name parameter for filtered aggregator
add tests
 2014-11-20 09:29:26 +05:30
fjy a49e673122 put back another missing test 2014-11-19 16:55:20 -08:00
fjy 14668846aa add back some tests 2014-11-19 14:35:26 -08:00
fjy fdeab0c6af make Druid case sensitive 2014-11-19 14:27:31 -08:00
Charles Allen 18f44beee9 CompressedObjectStrategy improvements 
* Added more unit tests
* Now properly uses safe / fast decompressor for LZ4
* Now chooses fastest lz4 instance instead of only looking at Java implmentations
* Encapsulate ResourceHolder in try-with-resources to make sure they close correctly
 2014-11-19 11:10:59 -08:00
nishantmonu51 0c2d06475d merge from master 2014-11-17 19:19:18 +05:30
nishantmonu51 cbffe3c648 merge from master and resolve conflicts 2014-11-17 18:07:08 +05:30
fjy d5c4282766 fix broken ut 2014-11-14 13:45:42 -08:00
xvrl e1e171ad20 Merge pull request #865 from metamx/fix-retry-qr 
Fix a couple of bugs with retry query runner after testing it locally
 2014-11-14 13:33:51 -08:00
Fangjin Yang 6ee8029462 Merge pull request #866 from metamx/mutableBitmapBenchmark 
Add benchmarking for bitmaps
 2014-11-14 14:16:21 -07:00
xvrl a4fc64ca3f Merge pull request #856 from metamx/druid-845 
Fix query by segment
 2014-11-14 13:10:54 -08:00
fjy bbc079b880 fix retry to actually return correct sequences 2014-11-14 12:10:04 -08:00
Charles Allen 648759e9f6 Add deserialization benchmark to BitmapCreationBenchmark 2014-11-13 13:43:14 -08:00
Charles Allen 483b2c7be0 Add copyright notice to BitmapCreationBenchmark 2014-11-13 12:55:02 -08:00
Charles Allen 228fb0cf40 Add benchmarking for bitmaps 
Here are the results on my laptop:

BitmapCreationBenchmark.testRandomAddition[0]: [measured 10 out of 20 rounds, threads: 1 (sequential)]
 round: 0.49 [+- 0.07], round.block: 0.00 [+- 0.00], round.gc: 0.00 [+- 0.00], GC.calls: 16, GC.time: 0.01, time.total: 9.91, time.warmup: 5.06, time.bench: 4.86
BitmapCreationBenchmark.testLinearAdditionDescending[0]: [measured 1000 out of 1010 rounds, threads: 1 (sequential)]
 round: 0.01 [+- 0.00], round.block: 0.00 [+- 0.00], round.gc: 0.00 [+- 0.00], GC.calls: 74, GC.time: 0.03, time.total: 5.82, time.warmup: 0.06, time.bench: 5.76
BitmapCreationBenchmark.testToImmutableByteArray[0]: [measured 1000 out of 1010 rounds, threads: 1 (sequential)]
 round: 0.00 [+- 0.00], round.block: 0.00 [+- 0.00], round.gc: 0.00 [+- 0.00], GC.calls: 0, GC.time: 0.00, time.total: 1.80, time.warmup: 0.02, time.bench: 1.78
BitmapCreationBenchmark.testRandomAddition[1]: [measured 10 out of 20 rounds, threads: 1 (sequential)]
 round: 0.00 [+- 0.00], round.block: 0.00 [+- 0.00], round.gc: 0.00 [+- 0.00], GC.calls: 0, GC.time: 0.00, time.total: 0.12, time.warmup: 0.08, time.bench: 0.04
BitmapCreationBenchmark.testLinearAdditionDescending[1]: [measured 1000 out of 1010 rounds, threads: 1 (sequential)]
 round: 0.00 [+- 0.00], round.block: 0.00 [+- 0.00], round.gc: 0.00 [+- 0.00], GC.calls: 10, GC.time: 0.01, time.total: 4.26, time.warmup: 0.04, time.bench: 4.22
BitmapCreationBenchmark.testToImmutableByteArray[1]: [measured 1000 out of 1010 rounds, threads: 1 (sequential)]
 round: 0.01 [+- 0.00], round.block: 0.00 [+- 0.00], round.gc: 0.00 [+- 0.00], GC.calls: 27, GC.time: 0.01, time.total: 5.11, time.warmup: 0.05, time.bench: 5.06
BitmapCreationBenchmark.testLinearAddition[0]: [measured 1000 out of 1010 rounds, threads: 1 (sequential)]
 round: 0.00 [+- 0.00], round.block: 0.00 [+- 0.00], round.gc: 0.00 [+- 0.00], GC.calls: 0, GC.time: 0.00, time.total: 3.48, time.warmup: 0.04, time.bench: 3.45
BitmapCreationBenchmark.testLinearAddition[1]: [measured 1000 out of 1010 rounds, threads: 1 (sequential)]
 round: 0.00 [+- 0.00], round.block: 0.00 [+- 0.00], round.gc: 0.00 [+- 0.00], GC.calls: 9, GC.time: 0.00, time.total: 2.95, time.warmup: 0.03, time.bench: 2.92
2014-11-13 12:47:23,995 INFO [main] io.druid.segment.data.BitmapCreationBenchmark - Entry [0] is io.druid.segment.data.ConciseBitmapSerdeFactory
2014-11-13 12:47:23,995 INFO [main] io.druid.segment.data.BitmapCreationBenchmark - Entry [1] is io.druid.segment.data.RoaringBitmapSerdeFactory
 2014-11-13 12:48:04 -08:00
fjy 2b0ed30344 add test 2014-11-13 12:38:35 -08:00
Fangjin Yang 4a3c0fc5c4 Merge pull request #848 from metamx/druid-0.7.x-fastTopN-rebase 
TopN performance improvements
 2014-11-13 11:56:20 -07:00
Charles Allen 9a9238a801 Merge remote-tracking branch 'origin/master' into druid-845 2014-11-13 10:04:56 -08:00
Xavier Léauté 19a37f773f test for groupBy order 2014-11-12 22:48:16 -08:00
Charles Allen 31fed7d329 Fix query by segment 
* Changed topN queries to use joda Interval instead of string values
* topN by segment now implements BySegmentResultValue<Result<TopNResultValue>> instead of BySegmentResultValue<TopNResultValue>
* Added a unit test which failed uner the prior implementation.
 2014-11-12 21:20:59 -08:00
Charles Allen 581e6830d5 Merge pull request #840 from metamx/powers-of-2-buffers 
make buffer size a power of 2 and optimize buffer lookup
 2014-11-11 19:54:00 -08:00
Charles Allen a89b539b4f Merge pull request #823 from metamx/roaring 
Configurable bitmap indexes: roaring and concise
 2014-11-11 17:26:38 -08:00
Xavier Léauté 007e57f876 make buffer size a power of 2 and optimize 2014-11-11 16:24:37 -08:00
fjy 1cc162727b address code review 2014-11-11 14:05:37 -08:00
Charles Allen 92e71be864 Change TopNQueryRunnerBenchmark to use a ByteBuffer as per OffheapBufferPool 2014-11-10 15:40:54 -08:00
Charles Allen 2b0f4534bf Modify formatting in TopNQueryRunnerBenchmark 2014-11-10 15:17:26 -08:00
fjy 6188315293 Merge branch 'master' into druid-0.7.x 
Conflicts:
	processing/src/test/java/io/druid/query/search/SearchQueryRunnerTest.java
 2014-11-10 14:52:10 -08:00
fjy fc34858e95 fix things for real-time ingestion 2014-11-10 13:48:21 -08:00
nishantmonu51 fd8eb7742b handle union query on realtime node 2014-11-07 23:27:50 +05:30
nishantmonu51 8bebb24fd5 Union Queries move merge to historical 2014-11-07 18:05:16 +05:30
Charles Allen ee019872f7 TopN performance improvements 
Re-factor scanAndAggregate in PooledTopN

* Loops are now a little bit tighter when looping over aggregates. This will hopefully assist in loop execution optimization.
* Pre-calculated the aggregate offsets instead of shifting them during runtime.
* Cursor loop could use some TLC, but would require a massive refactoring on how TopN queries are executed.
  * Any potential modifications to query workflow need to account for Stream vs Batch data, and that not all data will be array backed that comes in.

Change data storage type in TopNNumericResultBuilder.

  * Use PriorityQueue to store
  * Checks to see if should even bother adding to Queue before adding.
  * Re-orders Queue on build() call.
  * Ideally the order would be directly preserved on build(), but this is a close second.

Updates to CompressedObjectStrategy to support more compression types

 * Compression types are not yet dynamically configurable.
 * Added a benchmarking system for topN to test the compression
 * Updated pom.xml to include junit benchmarking
 * added an Uncompressed option
 2014-11-05 11:10:37 -08:00
fjy 253009d208 initial commit for roaring bitmaps 2014-11-04 17:04:40 -08:00
fjy 3e158f00ef fix random lowercasing in test 2014-10-30 10:56:39 -07:00
fjy 75da039396 fix search with same value in multiple columns 2014-10-30 10:43:42 -07:00
fjy bef74104d9 merge with 0.7.x and resolve any conflicts 2014-10-23 17:24:06 -07:00
fjy f790a05bd7 Merge branch 'master' into fix-test 
Conflicts:
	pom.xml
 2014-10-23 14:39:51 -07:00
nishantmonu51 6cb6ec39cf add support for not filter and add test 2014-10-22 10:25:24 +05:30
Xavier Léauté e24a6d222f we even have tests now 2014-10-21 17:46:24 -07:00
fjy bb90432862 Updated druid-api version to always sort dimension names, these tests needed fixing 2014-10-15 12:27:21 -04:00
Xavier Léauté 5b194ee5fe fix groupBy ordering casing bug 2014-10-15 12:16:17 -04:00
nishantmonu51 454acd3f5a remove backwards compatible code 
1) remove backwards compatible and deprecated code
2) make hashed partitions spec default
 2014-10-13 19:30:44 +05:30
Xavier Léauté 592611f2ee fix cardinality aggregator serde 2014-10-09 17:06:29 -07:00
Xavier Léauté 290107ce7b remove unused jsonspec content type 2014-10-09 10:26:47 -07:00
Xavier Léauté 30183d3b4d update druid-api for json parsing improvements 2014-10-07 13:25:02 -07:00
fjy fde510bbfc Merge pull request #779 from metamx/fix-histo 
Fix groupBy with histograms
 2014-10-07 11:53:24 -06:00
fjy c7b4d5b7b4 Merge branch 'master' into druid-0.7.x 
Conflicts:
	processing/src/test/java/io/druid/segment/filter/SpatialFilterTest.java
 2014-10-02 18:12:10 -07:00
fjy 816986f914 cleanup according to cr 2014-10-02 17:51:33 -07:00
fjy 1a67d6cb2e some tests for broken approx histo 2014-10-01 15:10:23 -07:00
nishantmonu51 ad75a21040 separate offheapIncrementalIndex implementation 2014-10-01 13:58:51 +05:30
nishantmonu51 358ff915bb fix merge conflicts 2014-09-30 22:19:18 +05:30
nishantmonu51 63fcd11864 fix compilation 2014-09-30 22:12:06 +05:30
nishantmonu51 2789536bed merge changes from druid-0.7.x 2014-09-30 22:05:49 +05:30
Gian Merlino a3a2c8b86f Merge pull request #765 from metamx/fix-spatial 
fix spatial indexing with multiple spatial dims
 2014-09-29 17:23:29 -07:00
fjy b57ac513bd Fixes #761 2014-09-29 14:56:43 -07:00
fjy 2b5ce89554 fix spatial indexing with multi spatial dims 2014-09-29 12:07:27 -07:00
nishantmonu51 adb4a65e0a Merge branch 'offheap-incremental-index' into mapdb-branch 2014-09-29 12:38:31 +05:30
fjy eaf0a48b92 Merge branch 'master' into druid-0.7.x 
Conflicts:
	cassandra-storage/pom.xml
	common/pom.xml
	examples/pom.xml
	hdfs-storage/pom.xml
	histogram/pom.xml
	indexing-hadoop/pom.xml
	indexing-service/pom.xml
	kafka-eight/pom.xml
	kafka-seven/pom.xml
	pom.xml
	processing/pom.xml
	processing/src/main/java/io/druid/guice/PropertiesModule.java
	rabbitmq/pom.xml
	s3-extensions/pom.xml
	server/pom.xml
	services/pom.xml
 2014-09-26 11:39:24 -07:00
nishantmonu51 f51ab84386 merge changes from druid-0.7.x 2014-09-22 23:48:45 +05:30
nishantmonu51 443e5788fb make OffheapIncrementalIndex tuneable 2014-09-22 19:26:10 +05:30
Xavier Léauté e5a13544a0 fix column selector on varying incr index schema 2014-09-17 10:36:05 -07:00
fjy cada14586a fix tests 2014-09-16 14:16:49 -07:00
fjy 8b5c2fcd82 fix compile error 2014-09-16 14:10:02 -07:00
fjy 9e0a2fd487 fix select for non existant dimensions and metrics 2014-09-16 14:01:21 -07:00
fjy 469ccbbe5e Merge branch 'master' into druid-0.7.x 
Conflicts:
	cassandra-storage/pom.xml
	common/pom.xml
	examples/pom.xml
	hdfs-storage/pom.xml
	histogram/pom.xml
	indexing-hadoop/pom.xml
	indexing-service/pom.xml
	kafka-eight/pom.xml
	kafka-seven/pom.xml
	pom.xml
	processing/pom.xml
	processing/src/main/java/io/druid/query/FinalizeResultsQueryRunner.java
	processing/src/main/java/io/druid/query/UnionQueryRunner.java
	processing/src/main/java/io/druid/query/groupby/GroupByQueryRunnerFactory.java
	processing/src/main/java/io/druid/query/topn/TopNQueryEngine.java
	processing/src/main/java/io/druid/query/topn/TopNQueryRunnerFactory.java
	rabbitmq/pom.xml
	s3-extensions/pom.xml
	server/pom.xml
	server/src/test/java/io/druid/server/initialization/JettyTest.java
	services/pom.xml
 2014-09-11 16:20:50 -07:00
Xavier Léauté c255bd76a9 add a test for the previous fix 2014-09-05 11:03:35 -07:00
xvrl c4c54f7bc9 Merge pull request #721 from metamx/fix-union-query 
fix Union Query result merging
 2014-09-04 12:20:19 -07:00
fjy f473e62f1f Merge pull request #714 from metamx/cleanup-javadocs 
Cleanup javadocs
 2014-09-04 12:28:10 -06:00
nishantmonu51 268a42bd79 fix Union Query result merging 2014-09-03 16:38:02 +05:30
nishantmonu51 c056960082 fix nested group by with hyperuniques 2014-09-01 10:50:09 +05:30
Xavier Léauté 75134335de fix last merge from master not being forward-compatible 2014-08-30 22:45:36 -07:00
Xavier Léauté 508e982190 Merge remote-tracking branch 'origin/master' into druid-0.7.x 
Conflicts:
	cassandra-storage/pom.xml
	common/pom.xml
	examples/config/historical/runtime.properties
	examples/config/overlord/runtime.properties
	examples/config/realtime/runtime.properties
	examples/pom.xml
	hdfs-storage/pom.xml
	histogram/pom.xml
	indexing-hadoop/pom.xml
	indexing-service/pom.xml
	kafka-eight/pom.xml
	kafka-seven/pom.xml
	pom.xml
	processing/pom.xml
	rabbitmq/pom.xml
	s3-extensions/pom.xml
	server/pom.xml
	server/src/main/java/io/druid/server/ClientQuerySegmentWalker.java
	services/pom.xml
 2014-08-30 22:42:36 -07:00
Xavier Léauté 58ab759fc6 remove unused imports 2014-08-29 14:03:47 -07:00
Xavier Léauté 674e7b0447 add tests for timeboundary timewarp 2014-08-28 17:59:56 -07:00
Gian Merlino 661e467d71 Fix groupBy/timeseries test, follow up to #705. 2014-08-26 22:36:08 -07:00
Gian Merlino 0eea1dc08d Test for desired timeseries zero-filling behavior. 2014-08-26 18:52:08 -07:00
Gian Merlino da48a713ae Fix overzealous timeseries zero-filling. 
When the index's maxTime is not aligned with the query granularity, gran.next
can cause an time extra bucket to get zero-filled. Truncating first prevents
that.
 2014-08-26 18:52:08 -07:00
Gian Merlino 28e3191503 Merge pull request #699 from metamx/postprocessing-operators 
query post-processing operators
 2014-08-26 17:23:37 -07:00
Xavier Léauté a1c8b23ba7 fix duration granularity with non-zero origin 2014-08-26 11:47:07 -07:00
Xavier Léauté 7537cf4fb1 fix compound period granularity for exact periods 2014-08-26 10:34:08 -07:00
Xavier Léauté eb7dd6dc74 add documentation and tests 2014-08-26 09:53:46 -07:00
Xavier Léauté 603152eb18 fix compilation with Java 8 2014-08-14 11:37:47 -07:00
fjy 18d3acd3a8 fix all manners of brokenness from nulls and empty strings 2014-08-14 11:11:57 -07:00
Xavier Léauté b448deeca0 fix compilation with Java 8 2014-08-12 14:02:08 -07:00
nishantmonu51 c6712739dc merge changes from druid-0.7.x 2014-08-12 15:47:42 +05:30
fjy b740987381 Merge pull request #616 from metamx/jisoo 
add retry feature on broker side; modify QueryRunner inferface and tests
 2014-08-11 15:36:16 -06:00
Xavier Léauté d198959c60 fix test harness 2014-08-08 14:05:10 -07:00
fjy 4af6a05378 Merge pull request #632 from metamx/lz4-compression 
Change default column compression from LZF to LZ4
 2014-08-08 14:24:00 -06:00
Xavier Léauté 991e1828b0 make LZ4 the default compression strategy 
- LZ4 is now hardwired to be the default strategy
- Rework tests to test all available compression strategies
 2014-08-06 15:50:07 -07:00
jisookim0513 6f58cd1156 fix merge conflcts; change test files 2014-08-04 17:45:56 -07:00
nishantmonu51 32b9290723 fix compilation 2014-07-31 16:20:58 +05:30
nishantmonu51 637bd35785 merge changes from druid-0.7.x 2014-07-31 16:07:22 +05:30
fjy 45e443bedb a few slight tweaks and bumping up curator version 2014-07-25 16:07:40 -07:00
fjy b70a6b1061 address code review comments 2014-07-16 13:52:06 -07:00
fjy 2f90bfcf9a rejiggering the cache for groupBy and select 2014-07-15 12:12:33 -07:00
nishantmonu51 f5f05e3a9b Sync changes from branch new-ingestion PR #599 
Sync and Resolve Conflicts
 2014-07-11 16:15:10 +05:30
jisookim0513 c4397bd65a fixed merge conflicts 2014-07-07 11:26:11 -07:00
Xavier Léauté c40a315c81 initial support for LZ4 compression 2014-07-07 11:01:08 -07:00
fjy 92bc340099 Merge branch 'master' into more-tests 2014-06-30 18:40:00 -07:00
fjy afa73a2d3a fix one of the TODOs and add test case 2014-06-30 17:29:19 -07:00