085ad8d345
Fix DataSourceMetadataResultValue serde.
2015-02-03 17:39:42 -08:00
1f12c5b2f1
[maven-release-plugin] prepare for next development iteration
2015-02-03 12:06:49 -08:00
e82d431be7
[maven-release-plugin] prepare release druid-0.7.0-rc2
2015-02-03 12:06:41 -08:00
4eff269536
Merge pull request #1079 from druid-io/cleanup-deps
...
Remove non friendly dependencies from Druid
2015-02-03 11:56:41 -08:00
3e5d338c8e
Remove non friendly dependencies from Druid
2015-02-03 11:36:08 -08:00
71b4c5fa86
Merge pull request #1076 from metamx/remove-threadlocals
...
remove thread-locals in GenericIndexed in favor of wrapped objects
2015-02-02 20:02:33 -08:00
cb2e300eba
remove thread-locals in GenericIndexed in favor of wrapped objects to reduce GC pressure
2015-02-02 15:59:30 -08:00
42eba986ce
Towards consistent null handling
...
This commit also includes
1) the addition of a context parameter on timeseries queries that allows it to ignore empty buckets instead of generating results for them
2) A cleanup of an unused method on an interface
2015-02-02 12:53:07 -08:00
92e616de11
Merge pull request #1077 from metamx/remove-unused-imports
...
remove unused imports
2015-02-02 10:45:27 -08:00
ba932bb1f2
remove unused imports
2015-02-02 21:53:39 +05:30
d05032b98a
towards a community led druid
2015-01-31 20:57:36 -08:00
f24a89a22a
fix NPE for topN over missing hyperUniques column
2015-01-27 16:12:41 -08:00
226dd91a31
Add a hash map for storing groupBy partition index
...
* Improves groupBy performance by approx 15%
2015-01-26 08:42:02 -08:00
1f94de22c6
[maven-release-plugin] prepare for next development iteration
2015-01-20 14:23:55 -08:00
17476edc31
[maven-release-plugin] prepare release druid-0.7.0-rc1
2015-01-20 14:23:51 -08:00
3d27747f7e
Upgrade to log4j2
...
Default behavior is as before.
Added documentation for how to enable synchronous logging for select chatty classes:
* io.druid.client.ServerInventoryView
* io.druid.client.BatchServerInventoryView
* io.druid.curator.inventory.CuratorInventoryManager
* com.metamx.http.client.pool.ChannelResourceFactory
2015-01-20 12:35:18 -08:00
91a79dbf95
Merge pull request #1031 from metamx/ingestmetadata-query
...
DataSourceMetadata query
2015-01-19 21:55:35 -08:00
7bb038756c
Account for very slow writer threads in IncrementalIndexTest
2015-01-17 13:02:59 -08:00
b4041c13e5
Merge pull request #1029 from metamx/fixChainedExecutionQueryRunnerTest
...
Address spurious test failures
2015-01-16 13:08:32 -08:00
3b3aad78cb
Merge pull request #1027 from metamx/concurrentOnHeapIncrementalIndexFix
...
Fix concurrency issues in OnheapIncrementalIndex
2015-01-16 12:54:42 -08:00
197af967ef
Fix concurrency issues in OnheapIncrementalIndex
...
* Was encountering weird errors when fast writes were coming in while queries were happening.
* Added unit tests which tend to cause concurrency query problems
2015-01-16 12:01:46 -08:00
ebafa2a786
Fix spurious test failures in ChainedExecutionQueryRunnerTest
2015-01-15 16:49:16 -08:00
5bfcc43377
Merge pull request #1008 from metamx/stringConversionJavaUtilUpdate
...
Update all String conversions to and from byte[] to use the java-util StringUtils functions
2015-01-15 13:50:27 -08:00
c7452b75f6
Merge branch 'master' into ingestmetadata-query
2015-01-15 18:00:31 +05:30
d5f4182de4
global test timeouts + fix test race condition
2015-01-07 23:36:57 -08:00
852e863425
Merge pull request #981 from druid-io/strictModuleTyping
...
Use Module instead of generic Object in Guice related items
2015-01-05 12:43:20 -08:00
b1b5c9099e
Update all String conversions to and from byte[] to use the java-util StringUtils functions
...
* Speedup of GroupBy with javaScript filters by ~10%
* Requires https://github.com/metamx/java-util/pull/15
2015-01-05 11:22:32 -08:00
3fc6cf918d
add test for large chunks
2015-01-02 14:31:22 -08:00
f2f9cbeca8
throw error rather than returning garbage results
2015-01-02 14:29:21 -08:00
071943a367
fix LZF compression with buffers exceeding LZF chunk size
2015-01-02 11:39:50 -08:00
f2439899e7
fix bitmap factory serde
2014-12-23 15:07:32 -08:00
27a3169312
increase test timeouts
2014-12-19 17:09:43 -08:00
971afab36f
Lengthen CompressionStrategyTest::testKnownSizeConcurrency() to have 2m timeout on its test to account for shared Jenkins build lag
2014-12-19 12:53:20 -08:00
7c8d4a7433
Use Module instead of generic Object in Guice related items
2014-12-19 10:54:06 -08:00
be507b8cb4
Merge pull request #943 from mrijke/partialdimextractfn-nullpointer
...
Fix NullPointerException in PartialDimExtractionFn
2014-12-16 12:29:27 -07:00
80e4b68ee7
review comments
2014-12-16 21:16:48 +05:30
b3fe91bb50
Merge pull request #830 from metamx/union-merge-on-historical
...
Union merge on historical
2014-12-15 13:36:47 -07:00
3cb7999eb9
i hate hadoop dependencies
2014-12-15 09:52:46 -08:00
a0d3579a92
add docs + fix tests
2014-12-11 17:58:01 +05:30
7ad03087c0
Merge branch 'master' into ingestmetadata-query
2014-12-11 16:54:38 +05:30
32b4f55b8a
review comments refactoring
2014-12-11 16:33:14 +05:30
3763357f6e
Ingest metadata query implementation
2014-12-10 19:44:00 +05:30
d6d3ec6846
Merge pull request #948 from metamx/ingestion-docs
...
Redocumenting ingestion
2014-12-09 15:30:03 -07:00
9596c11f42
address cr
2014-12-09 14:19:18 -08:00
1a1b0e6f23
merge from master and review comments
2014-12-09 13:16:45 +05:30
1392e2731f
Merge pull request #936 from metamx/cachingRunnerImprovements
...
General Caching Query Runners cleanup (40% query time reduction for HLL)
2014-12-08 14:07:52 -08:00
7b65f0635d
General Caching Query Runners cleanup
...
* Add type strictness to CachingClusteredClient.
* Add background caching to CachingClusteredClient. Gives between 0% and 5% query speed increase.
* Add @BackgroundCaching annotation for injected ExecutorService items
* Add `numBackgroundThreads' configuration options to CacheConfig (default 0 aka same thread legacy behavior)
* Add unit tests for CacheConfig
* Add an abstract caching query runner class, currently it doesn't do anything exceppt simply make the two caching queries distinct.
* Add caching to CachingQueryRunner. Gives up to a WHOPPING 40% reduction in query time on HLL queries
* Updated docs with more info on cache settings.
2014-12-08 13:29:32 -08:00
90670a9c7e
Fix NullPointerException in PartialDimExtractionFn by explicity checking for dimValue == null, attempt 2
2014-12-08 22:26:35 +01:00
bd9bbf396c
Fix NullPointerException in PartialDimExtractionFn by explicity checking for dimValue == null
2014-12-08 20:11:58 +01:00
ad23e49777
use fixed-size mapdb cache to avoid heap growing uncontrollably
2014-12-05 15:34:50 -08:00
7cd45a6e1f
IncrementalIndex throws exception if limit exceeded
...
- For now uses a hardcoded ratio of aggregator to timeanddim buffer sizes
- canAppendRow is a workaround for realtime index since the
Firehose currently does not have a way of rolling back the last event in
case of error
- canAppendRow needs a fudge factor; there is a race between checking
if we can add a row and actually adding a row, because of the way MapDB
reports its size.
2014-12-04 14:38:16 -08:00
c7dbe6116c
write byte data as is in smile
2014-12-04 10:57:56 -08:00
c21a82a697
upgrade LZ4 to operate directly on ByteBuffers
2014-12-04 10:57:56 -08:00
0c521e0a77
update joda-time and fix min/max instant
2014-12-04 10:57:56 -08:00
269a51964e
fix size calculation
2014-12-04 17:22:24 +05:30
4dc0fdba8a
consider mapped size in limit calculation & review comments
2014-12-03 23:47:30 +05:30
529e7e0272
Merge pull request #927 from metamx/speedup-smile-bytes
...
Improve Smile serde performance by writing binary data as is
2014-12-03 10:02:08 -08:00
0f5d5840da
Merge pull request #924 from metamx/update-joda-time
...
Update Joda-Time and fix min/max instant overflow
2014-12-03 09:15:39 -08:00
da8bd7836b
Introduce buffer size
2014-12-03 16:28:22 +05:30
5fece517fa
write byte data as is in smile
2014-12-03 00:01:01 -08:00
18f50097a9
upgrade LZ4 to operate directly on ByteBuffers
2014-12-02 23:53:56 -08:00
bc173d14fc
a whole bunch of cleanup and fixes
2014-12-02 17:32:05 -08:00
a79389a9e5
update joda-time and fix min/max instant
2014-12-02 17:27:22 -08:00
b65933ffb8
make tests parameterised
2014-12-02 23:55:29 +05:30
6dc69c2f30
code cleanups & formatting
2014-12-02 22:44:33 +05:30
eac776f1a7
tests passing with on heap incremental index
2014-12-02 22:29:28 +05:30
4eee7e69b9
fix cardinality aggregator caching
2014-11-26 15:00:37 -08:00
5bc1be5ba0
Merge pull request #850 from metamx/druid-0.7.x-compressionstrategy
...
Compression strategy changes
2014-11-25 12:58:39 -08:00
c6043afa32
Removed empty function from CompressionStrategyTest
2014-11-25 12:57:06 -08:00
6943db5251
Changed branching logic for LZFCompressor to return null only on error, and avoid checking in most circumstances
2014-11-25 12:53:11 -08:00
9f945c2216
Removed lz4Fast from CompressedObjectStrategy for compression since it is not currently used
2014-11-24 16:11:03 -08:00
70e3108282
Multiple speed improvements revolving around topN with HLL
...
Change serializer / deserializer for HyperLogLog
* Changed DirectDruidClient's InputStream handling. Is now ~10% faster for data heavy queries, and has lower variance in execution speed.
* Changed HLL Collector's toByteStream() method to be better optimized for small values. Is notably faster for small result quantities which fall into the sparse HLL bucket codepath.
* No change for dense HLL which just uses a direct bytestream of the underlying byte data.
TopNNumericResultBuilder semi-aggressive loop unrolling for metricVals
Benchmark for HLL for sparse packing (small HLL bucket population):
HyperLogLogSerdeBenchmarkTest.benchmarkToByteBuffer[0]: [measured 100000 out of 100100 rounds, threads: 1 (sequential)]
round: 0.00 [+- 0.00], round.block: 0.00 [+- 0.00], round.gc: 0.00 [+- 0.00], GC.calls: 216, GC.time: 0.42, time.total: 15.96, time.warmup: 0.22, time.bench: 15.74
HyperLogLogSerdeBenchmarkTest.benchmarkToByteBuffer[1]: [measured 100000 out of 100100 rounds, threads: 1 (sequential)]
round: 0.00 [+- 0.00], round.block: 0.00 [+- 0.00], round.gc: 0.00 [+- 0.00], GC.calls: 217, GC.time: 0.45, time.total: 13.87, time.warmup: 0.02, time.bench: 13.85
HyperLogLogSerdeBenchmarkTest.benchmarkToByteBuffer[2]: [measured 100000 out of 100100 rounds, threads: 1 (sequential)]
round: 0.00 [+- 0.00], round.block: 0.00 [+- 0.00], round.gc: 0.00 [+- 0.00], GC.calls: 55, GC.time: 0.16, time.total: 4.13, time.warmup: 0.00, time.bench: 4.12
HyperLogLogSerdeBenchmarkTest.benchmarkToByteBuffer[3]: [measured 100000 out of 100100 rounds, threads: 1 (sequential)]
round: 0.00 [+- 0.00], round.block: 0.00 [+- 0.00], round.gc: 0.00 [+- 0.00], GC.calls: 55, GC.time: 0.16, time.total: 4.30, time.warmup: 0.00, time.bench: 4.30
HyperLogLogSerdeBenchmarkTest.benchmarkToByteBuffer[4]: [measured 100000 out of 100100 rounds, threads: 1 (sequential)]
round: 0.00 [+- 0.00], round.block: 0.00 [+- 0.00], round.gc: 0.00 [+- 0.00], GC.calls: 8, GC.time: 0.03, time.total: 1.10, time.warmup: 0.00, time.bench: 1.09
HyperLogLogSerdeBenchmarkTest.benchmarkToByteBuffer[5]: [measured 100000 out of 100100 rounds, threads: 1 (sequential)]
round: 0.00 [+- 0.00], round.block: 0.00 [+- 0.00], round.gc: 0.00 [+- 0.00], GC.calls: 8, GC.time: 0.03, time.total: 0.72, time.warmup: 0.00, time.bench: 0.72
HyperLogLogSerdeBenchmarkTest.benchmarkToByteBuffer[6]: [measured 100000 out of 100100 rounds, threads: 1 (sequential)]
round: 0.00 [+- 0.00], round.block: 0.00 [+- 0.00], round.gc: 0.00 [+- 0.00], GC.calls: 1, GC.time: 0.00, time.total: 0.60, time.warmup: 0.00, time.bench: 0.60
HyperLogLogSerdeBenchmarkTest.benchmarkToByteBuffer[7]: [measured 100000 out of 100100 rounds, threads: 1 (sequential)]
round: 0.00 [+- 0.00], round.block: 0.00 [+- 0.00], round.gc: 0.00 [+- 0.00], GC.calls: 2, GC.time: 0.01, time.total: 0.26, time.warmup: 0.00, time.bench: 0.25
Updates to HyperLogLogCollector toByteBuffer() based on code review
Removed changes from DirectDruidClient from this branch and put it in another branch.
Changed HyperLogLogCollector to have protected getters and setters
Remove unused ByteOrder from HyperLogLogCollector
Copyright header on HyperLogLogSerdeBenchmarkTest
Now with less ass!
Reformat in TopNNumericResultsBuilder. No code change
Removed unused import in HyperLogLogCollector
Replace AppendableByteArrayInputStream in DirectDruidClient
* Replace with SequenceInputStream fueled by an enumeration of ChannelBufferInputStream which directly wrap the response context ChannelBuffer
Modify TopNQueryQueryToolChest to use Arrays instead of Lists
Modify TopNQueryQueryToolChest to use Arrays instead of Lists
Revert accidental changes to DirectDruidClient
They should be in another merge request:
https://github.com/metamx/druid/pull/893
Fixes from code review
* Extracting names from AggregatorFactory classes now done with TopNQueryQueryToolChest.extractFactoryName
* Renamed variable in TopNNumericResultBuilder
2014-11-24 16:02:00 -08:00
13cae41f6c
Merge branch 'master' into refactor-examples
2014-11-24 11:00:26 -08:00
9ced097abd
Merge pull request #895 from metamx/fix-interval-retry
...
A set of fixes to retry the query for missing segments in the timeline
2014-11-24 10:23:02 -08:00
c88aff6205
remove unneeded constant
2014-11-24 10:20:02 -08:00
9da66291e1
change naming to common config
2014-11-21 15:36:42 -08:00
8f80d9e189
Update CompressedObjectStrategy to try-with-resources but print log error if error while closing
2014-11-21 11:35:11 -08:00
fc9a54ea48
Fix CompressedObjectStrategy LZFCompressor to ignore error on close of ResourceHolder
2014-11-21 10:49:43 -08:00
f8ce68565b
Modified CompressedObjectStrategy to use 0xFF for Uncompressed
2014-11-21 10:33:53 -08:00
aa49e56ed6
Merge remote-tracking branch 'origin/master' into druid-0.7.x-compressionstrategy
2014-11-21 10:29:40 -08:00
ef62bccdec
ignore benchmark
2014-11-20 16:52:19 -08:00
e3260aa177
Filtered Aggregator fixes + enhancements
...
- fix NPE on IncrementIndex
- refactor code to support AND, OR filter
- tests for AND & OR filter
- handling for missing column / null values
2014-11-20 15:17:18 -08:00
47f5c1bd0a
fix retry interval is stupid
2014-11-20 12:50:56 -08:00
3d9d989a9f
A set of fixes to retry the query for missing intervals in the timeline
2014-11-20 12:04:37 -08:00
0ab34f86da
Revert "fix filtered Aggregator"
...
This reverts commit 6fd37ce023
2014-11-20 10:17:01 +05:30
6fd37ce023
fix filtered Aggregator
...
fix filtered Aggregator
remove unused name parameter for filtered aggregator
add tests
2014-11-20 09:29:26 +05:30
a49e673122
put back another missing test
2014-11-19 16:55:20 -08:00
14668846aa
add back some tests
2014-11-19 14:35:26 -08:00
fdeab0c6af
make Druid case sensitive
2014-11-19 14:27:31 -08:00
590d31799e
Merge pull request #876 from metamx/remove-backwards-compatible
...
Remove backwards compatible
2014-11-19 14:33:14 -07:00
18f44beee9
CompressedObjectStrategy improvements
...
* Added more unit tests
* Now properly uses safe / fast decompressor for LZ4
* Now chooses fastest lz4 instance instead of only looking at Java implmentations
* Encapsulate ResourceHolder in try-with-resources to make sure they close correctly
2014-11-19 11:10:59 -08:00
ccc757dc64
Merge remote-tracking branch 'origin/master' into druid-0.7.x-compressionstrategy
2014-11-19 09:39:35 -08:00
1bbc8fcbe5
Allow Smile to fall back to text
...
* Modify SmileFactory to set the delegate to text option.
* This option only occurs when a Reader type object is passed in to the deserialization stuff
* This is needed by the X-Druid-Response-Context header return value, which is JSON
2014-11-18 15:16:14 -08:00
42517f5d37
Merge pull request #884 from metamx/optimize-topN-pruning
...
optimise pruning of aggs
2014-11-18 14:19:30 -08:00
a96eaeb036
Merge pull request #882 from metamx/now_with_OPEN_SOURCE
...
Added src jar build to maven poms and re-formatted to conform to style guidelines.
2014-11-18 13:00:04 -08:00
6023d602e6
optimise pruning of aggs
...
optimise pruning of aggregators for topN
2014-11-19 00:17:25 +05:30
dc66e1708e
Added src jar build to maven poms and re-formatted to conform to style guidelines.
2014-11-18 09:05:30 -08:00
d914afe1cd
make defaultVersion configurable for non-jar testing
2014-11-17 13:54:32 -08:00
0c2d06475d
merge from master
2014-11-17 19:19:18 +05:30
cbffe3c648
merge from master and resolve conflicts
2014-11-17 18:07:08 +05:30
d5c4282766
fix broken ut
2014-11-14 13:45:42 -08:00
e1e171ad20
Merge pull request #865 from metamx/fix-retry-qr
...
Fix a couple of bugs with retry query runner after testing it locally
2014-11-14 13:33:51 -08:00
df1ad95075
remove useless config
2014-11-14 13:32:19 -08:00
d641d41f9e
address another cr
2014-11-14 13:29:59 -08:00
7736c3fc27
address cr
2014-11-14 13:28:32 -08:00
6ee8029462
Merge pull request #866 from metamx/mutableBitmapBenchmark
...
Add benchmarking for bitmaps
2014-11-14 14:16:21 -07:00
a4fc64ca3f
Merge pull request #856 from metamx/druid-845
...
Fix query by segment
2014-11-14 13:10:54 -08:00
4b7ab23289
Remove getIntervalString from BySegmentResultValue
2014-11-14 13:03:48 -08:00
bbc079b880
fix retry to actually return correct sequences
2014-11-14 12:10:04 -08:00
648759e9f6
Add deserialization benchmark to BitmapCreationBenchmark
2014-11-13 13:43:14 -08:00
483b2c7be0
Add copyright notice to BitmapCreationBenchmark
2014-11-13 12:55:02 -08:00
228fb0cf40
Add benchmarking for bitmaps
...
Here are the results on my laptop:
BitmapCreationBenchmark.testRandomAddition[0]: [measured 10 out of 20 rounds, threads: 1 (sequential)]
round: 0.49 [+- 0.07], round.block: 0.00 [+- 0.00], round.gc: 0.00 [+- 0.00], GC.calls: 16, GC.time: 0.01, time.total: 9.91, time.warmup: 5.06, time.bench: 4.86
BitmapCreationBenchmark.testLinearAdditionDescending[0]: [measured 1000 out of 1010 rounds, threads: 1 (sequential)]
round: 0.01 [+- 0.00], round.block: 0.00 [+- 0.00], round.gc: 0.00 [+- 0.00], GC.calls: 74, GC.time: 0.03, time.total: 5.82, time.warmup: 0.06, time.bench: 5.76
BitmapCreationBenchmark.testToImmutableByteArray[0]: [measured 1000 out of 1010 rounds, threads: 1 (sequential)]
round: 0.00 [+- 0.00], round.block: 0.00 [+- 0.00], round.gc: 0.00 [+- 0.00], GC.calls: 0, GC.time: 0.00, time.total: 1.80, time.warmup: 0.02, time.bench: 1.78
BitmapCreationBenchmark.testRandomAddition[1]: [measured 10 out of 20 rounds, threads: 1 (sequential)]
round: 0.00 [+- 0.00], round.block: 0.00 [+- 0.00], round.gc: 0.00 [+- 0.00], GC.calls: 0, GC.time: 0.00, time.total: 0.12, time.warmup: 0.08, time.bench: 0.04
BitmapCreationBenchmark.testLinearAdditionDescending[1]: [measured 1000 out of 1010 rounds, threads: 1 (sequential)]
round: 0.00 [+- 0.00], round.block: 0.00 [+- 0.00], round.gc: 0.00 [+- 0.00], GC.calls: 10, GC.time: 0.01, time.total: 4.26, time.warmup: 0.04, time.bench: 4.22
BitmapCreationBenchmark.testToImmutableByteArray[1]: [measured 1000 out of 1010 rounds, threads: 1 (sequential)]
round: 0.01 [+- 0.00], round.block: 0.00 [+- 0.00], round.gc: 0.00 [+- 0.00], GC.calls: 27, GC.time: 0.01, time.total: 5.11, time.warmup: 0.05, time.bench: 5.06
BitmapCreationBenchmark.testLinearAddition[0]: [measured 1000 out of 1010 rounds, threads: 1 (sequential)]
round: 0.00 [+- 0.00], round.block: 0.00 [+- 0.00], round.gc: 0.00 [+- 0.00], GC.calls: 0, GC.time: 0.00, time.total: 3.48, time.warmup: 0.04, time.bench: 3.45
BitmapCreationBenchmark.testLinearAddition[1]: [measured 1000 out of 1010 rounds, threads: 1 (sequential)]
round: 0.00 [+- 0.00], round.block: 0.00 [+- 0.00], round.gc: 0.00 [+- 0.00], GC.calls: 9, GC.time: 0.00, time.total: 2.95, time.warmup: 0.03, time.bench: 2.92
2014-11-13 12:47:23,995 INFO [main] io.druid.segment.data.BitmapCreationBenchmark - Entry [0] is io.druid.segment.data.ConciseBitmapSerdeFactory
2014-11-13 12:47:23,995 INFO [main] io.druid.segment.data.BitmapCreationBenchmark - Entry [1] is io.druid.segment.data.RoaringBitmapSerdeFactory
2014-11-13 12:48:04 -08:00
6cdd6a6af7
change default settings for retry
2014-11-13 12:43:30 -08:00
2b0ed30344
add test
2014-11-13 12:38:35 -08:00
0d6816a037
Fix a couple of bugs with retry query runner after testing it locally
2014-11-13 11:53:29 -08:00
4a3c0fc5c4
Merge pull request #848 from metamx/druid-0.7.x-fastTopN-rebase
...
TopN performance improvements
2014-11-13 11:56:20 -07:00
34c3464bc9
Added more explanatory comments in PooledTopNAlgorithm
2014-11-13 10:47:56 -08:00
9a9238a801
Merge remote-tracking branch 'origin/master' into druid-845
2014-11-13 10:04:56 -08:00
19a37f773f
test for groupBy order
2014-11-12 22:48:16 -08:00
31fed7d329
Fix query by segment
...
* Changed topN queries to use joda Interval instead of string values
* topN by segment now implements BySegmentResultValue<Result<TopNResultValue>> instead of BySegmentResultValue<TopNResultValue>
* Added a unit test which failed uner the prior implementation.
2014-11-12 21:20:59 -08:00
4ac1aaf90e
Merge remote-tracking branch 'origin/master' into druid-0.7.x
...
Conflicts:
processing/src/main/java/io/druid/segment/QueryableIndexStorageAdapter.java
2014-11-12 14:08:51 -08:00
4b4f1c7d38
Merge pull request #846 from vikramakrishnan/cacheKeyFix
...
Include origin when creating the cacheKey for period Grans
2014-11-12 11:15:21 -07:00
7ef19009db
Merge pull request #834 from metamx/optimize-timestampchecking
...
skip timestamp checking if not required, remove duplicate code
2014-11-12 09:45:34 -07:00
8151d14786
Include origin when creating the cacheKey for period Grans
2014-11-12 21:36:46 +05:30
581e6830d5
Merge pull request #840 from metamx/powers-of-2-buffers
...
make buffer size a power of 2 and optimize buffer lookup
2014-11-11 19:54:00 -08:00
60e98c35d7
workaround for annotations requiring class literals
2014-11-11 18:48:49 -08:00
b580269f6d
Distinguish between default and legacy bitmaps
2014-11-11 18:35:45 -08:00
bc5c56e441
fix default impl
2014-11-11 18:00:46 -08:00
63ca2375a3
remove dead code and cleanup some defaults
2014-11-11 17:57:24 -08:00
5629307548
address a few more comments on roaring
2014-11-11 17:50:03 -08:00
a89b539b4f
Merge pull request #823 from metamx/roaring
...
Configurable bitmap indexes: roaring and concise
2014-11-11 17:26:38 -08:00
007e57f876
make buffer size a power of 2 and optimize
2014-11-11 16:24:37 -08:00
1cc162727b
address code review
2014-11-11 14:05:37 -08:00
3f5449d40a
loop unrolling provides no benefit for timeseries
2014-11-11 10:58:37 -08:00
e817db8b6c
unroll timeseries aggregations + naming
2014-11-11 10:09:37 -08:00
e6b7b03b5b
fix conversion
2014-11-10 17:13:28 -08:00
eb457c280e
revert change
2014-11-10 17:00:53 -08:00
336c73bdc2
cleanup dead code
2014-11-10 16:53:13 -08:00
df886fac1b
fix
2014-11-10 16:49:27 -08:00
d68bc3bdea
cleanup unused imports
2014-11-10 16:15:28 -08:00
92e71be864
Change TopNQueryRunnerBenchmark to use a ByteBuffer as per OffheapBufferPool
2014-11-10 15:40:54 -08:00
a093f3728a
Reformat on TopNNumericResultBuilder
2014-11-10 15:26:09 -08:00
2b0f4534bf
Modify formatting in TopNQueryRunnerBenchmark
2014-11-10 15:17:26 -08:00
fc78f215c4
Aggressive dimValue unrolling in PooledTopNAlgorithm
2014-11-10 15:14:45 -08:00
6188315293
Merge branch 'master' into druid-0.7.x
...
Conflicts:
processing/src/test/java/io/druid/query/search/SearchQueryRunnerTest.java
2014-11-10 14:52:10 -08:00
49e878cf1a
unroll multi-value dimensions
2014-11-10 14:21:56 -08:00
df9be030db
remove more legacy code
2014-11-10 14:09:00 -08:00
dbb6401fbe
Aggressive unrolling in PooledTopNAlgorithm
2014-11-10 14:04:37 -08:00
72c355e8ae
remove crazy handler code
2014-11-10 13:58:23 -08:00
fc34858e95
fix things for real-time ingestion
2014-11-10 13:48:21 -08:00
c5cc826998
make things actually work with roaring
2014-11-10 13:42:26 -08:00
358b2add17
make things actually work with roaring
2014-11-10 13:42:06 -08:00
9ac8589143
Merge remote-tracking branch 'origin/druid-0.7.x-fastTopN-rebase' into druid-0.7.x-fastTopN-rebase
2014-11-10 12:55:08 -08:00
70468400bf
stupid mistake
2014-11-10 12:54:48 -08:00
bfc9d9f283
Merge remote-tracking branch 'origin/druid-0.7.x-fastTopN-rebase' into druid-0.7.x-fastTopN-rebase
2014-11-10 12:52:36 -08:00
17d32a2561
try to benefit from out of order execution
2014-11-10 12:50:10 -08:00
5aad149b24
Make some variables in PooledTopNAlgorithm final
2014-11-10 12:19:04 -08:00
0ca60e0334
skip timestamp checking, remove duplicate code
...
1) No need compare all timestamps when all the values are within range
2) remove redundant code.
2014-11-10 22:36:07 +05:30
3c21f62afd
a whole bunch of fixes and some ugly conversion code
2014-11-07 16:03:22 -08:00
7131cde710
Code reformat in CompressedObjectStrategy
2014-11-07 11:19:32 -08:00
8bd6cf0d07
Updates to CompressedObjectStrategy to support more compression types
...
* Compression types are not yet dynamically configurable.
* Added a benchmarking system for topN to test the compression
* Updated pom.xml to include junit benchmarking
* added an Uncompressed option
2014-11-07 11:14:46 -08:00
f5f48c7756
Moved pom version for reals this time
2014-11-07 11:09:57 -08:00
6af18931e5
Moveed processing/pom version into master pom
2014-11-07 11:08:23 -08:00
75c79e7562
Added comments in TopNNumericResultBuilder
2014-11-07 11:01:46 -08:00
b4893b4490
Pre allocate LinkedHashMap in TopNNumericResultBuilder
2014-11-07 10:56:49 -08:00
bc92de233c
Update dim name comparator in TopNNumericResultBuilder to better handle nulls
2014-11-07 10:54:10 -08:00
0a562ebd77
Removed CompressedObjectStrategy from this pull request. will submit a new one later
2014-11-07 10:42:28 -08:00
228614ddb5
Changed TopNNumericResultBuilder imports to not use package.*, but instead pulls all in explicitly
2014-11-07 10:34:28 -08:00
956bdf80dd
Removed negating comparator in TopNNumericResultBuilder
2014-11-07 10:29:06 -08:00
fd8eb7742b
handle union query on realtime node
2014-11-07 23:27:50 +05:30
8bebb24fd5
Union Queries move merge to historical
2014-11-07 18:05:16 +05:30
ee019872f7
TopN performance improvements
...
Re-factor scanAndAggregate in PooledTopN
* Loops are now a little bit tighter when looping over aggregates. This will hopefully assist in loop execution optimization.
* Pre-calculated the aggregate offsets instead of shifting them during runtime.
* Cursor loop could use some TLC, but would require a massive refactoring on how TopN queries are executed.
* Any potential modifications to query workflow need to account for Stream vs Batch data, and that not all data will be array backed that comes in.
Change data storage type in TopNNumericResultBuilder.
* Use PriorityQueue to store
* Checks to see if should even bother adding to Queue before adding.
* Re-orders Queue on build() call.
* Ideally the order would be directly preserved on build(), but this is a close second.
Updates to CompressedObjectStrategy to support more compression types
* Compression types are not yet dynamically configurable.
* Added a benchmarking system for topN to test the compression
* Updated pom.xml to include junit benchmarking
* added an Uncompressed option
2014-11-05 11:10:37 -08:00
253009d208
initial commit for roaring bitmaps
2014-11-04 17:04:40 -08:00
3e158f00ef
fix random lowercasing in test
2014-10-30 10:56:39 -07:00
75da039396
fix search with same value in multiple columns
2014-10-30 10:43:42 -07:00
41ebdd2bae
Merge branch 'druid-0.7.x' into fix-test
...
Conflicts:
processing/pom.xml
processing/src/main/java/io/druid/segment/IndexMerger.java
processing/src/main/java/io/druid/segment/MetricHolder.java
processing/src/main/java/io/druid/segment/data/CompressedObjectStrategy.java
processing/src/test/java/io/druid/query/groupby/GroupByQueryRunnerTest.java
processing/src/test/java/io/druid/segment/SchemalessTestFull.java
server/src/test/java/io/druid/client/CachingClusteredClientTest.java
services/src/main/java/io/druid/cli/CliHistorical.java
2014-10-24 11:07:52 -07:00
bef74104d9
merge with 0.7.x and resolve any conflicts
2014-10-23 17:24:06 -07:00
f790a05bd7
Merge branch 'master' into fix-test
...
Conflicts:
pom.xml
2014-10-23 14:39:51 -07:00
1b3921faad
fix for long columns
2014-10-23 14:33:25 -07:00
3b29e77866
[maven-release-plugin] prepare for next development iteration
2014-10-22 16:25:32 -07:00
dcab2997f2
[maven-release-plugin] prepare release druid-0.6.160
2014-10-22 16:25:27 -07:00
94da2b6ad3
more logging for potential NPE because of segment unmapping
2014-10-22 16:16:10 -07:00
4ec1e6ef13
simplify code a bit
2014-10-22 14:25:59 -07:00
0785baf54c
review comments
2014-10-22 21:28:31 +05:30
6cb6ec39cf
add support for not filter and add test
2014-10-22 10:25:24 +05:30
e24a6d222f
we even have tests now
2014-10-21 17:46:24 -07:00
23a6ab19b0
cleanup and add missing methods
2014-10-21 17:00:39 -07:00
56912adccb
avoid typecasts
2014-10-21 16:47:50 -07:00
10db94101f
skip value lookup
...
compare ids instead of values
2014-10-21 16:47:49 -07:00
71831e49d9
rudimentary filtered aggregator
2014-10-21 16:47:49 -07:00
bb90432862
Updated druid-api version to always sort dimension names, these tests needed fixing
2014-10-15 12:27:21 -04:00
5b194ee5fe
fix groupBy ordering casing bug
2014-10-15 12:16:17 -04:00
bce388fb27
merge changes from 0.7.x branch
2014-10-14 18:46:02 +05:30
a4c8f04409
[maven-release-plugin] prepare for next development iteration
2014-10-13 12:50:45 -07:00
7fd1747ffa
[maven-release-plugin] prepare release druid-0.6.159
2014-10-13 12:50:41 -07:00
454acd3f5a
remove backwards compatible code
...
1) remove backwards compatible and deprecated code
2) make hashed partitions spec default
2014-10-13 19:30:44 +05:30
592611f2ee
fix cardinality aggregator serde
2014-10-09 17:06:29 -07:00
d0fbd5f729
[maven-release-plugin] prepare for next development iteration
2014-10-09 14:24:53 -07:00
dada60a301
[maven-release-plugin] prepare release druid-0.6.158
2014-10-09 14:24:48 -07:00
290107ce7b
remove unused jsonspec content type
2014-10-09 10:26:47 -07:00
Gian Merlino
fjy
Xavier Léauté
Xavier Léauté
Eric Tschetter
nishantmonu51
Charles Allen
Maarten Rijke
Vikram Ramakrishnan
Charles Allen