Apache
/
druid
mirror of https://github.com/apache/druid.git
1
0
Fork 0
Code Issues Packages Projects Releases Wiki Activity
7,107 Commits 111 Branches 503 Tags 721 MiB
Commit Graph

1475 Commits

Author SHA1 Message Date
jon-wei cdceaf2d26 Fix IncrementalIndexAdapter getRows() Iterable 2015-11-12 13:10:42 -08:00
Charles Allen af34e9c8cb Add the druid artifact version to metrics when emitted 2015-11-12 12:11:27 -08:00
binlijin 286b8f8c6f optimize index merge 2015-11-12 11:08:54 +08:00
Xavier Léauté fa6142e217 cleanup and remove unused imports 2015-11-11 12:25:21 -08:00
dclim fd0935ecb9 fix spatial dimension transformer to work with hadoop 2015-11-10 19:16:51 -07:00
Slim Bouguerra c511273efd adding in filter 2015-11-06 16:23:24 -06:00
Charles Allen 929b981710 Change DefaultObjectMapper to NOT overwrite final fields unless explicitly asked to 2015-11-05 18:10:13 -08:00
fjy 8f231fd3e3 cleanup druid codebase 2015-11-04 13:59:53 -08:00
Gian Merlino 8defe29270 Merge pull request #1901 from guobingkun/fix_typo_and_rename 
Fix metadata typo and rename default extension directory
 2015-11-03 14:02:11 -08:00
Bingkun Guo 962f65cc76 fix metadata typo and rename default extension directory 2015-11-03 14:50:42 -06:00
Fangjin Yang cec09a9967 Merge pull request #1804 from himanshug/objectify_index_creators 
static to non-static conversion for methods in Index[Merger/Maker/IO]
 2015-11-03 11:25:32 -08:00
Himanshu Gupta 8b67417ac8 make methods in Index[Merger,Maker,IO] non-static so that they can have 
appropriate ObjectMapper injected instead of creating one statically
 2015-11-02 23:24:26 -06:00
navis.ryu e03fc2032f changed equals/hashCode implementation 2015-11-02 17:21:35 +09:00
navis.ryu 69c86716d6 addressed comments 2015-11-02 14:23:13 +09:00
navis.ryu 032c3e986d Make 'search' filter have a case sensitive option(#1878) 2015-10-30 16:38:54 +09:00
Fangjin Yang 25a0eb7ed5 Merge pull request #1799 from dclim/nested-groupby-aggregator-fix 
Support multiple outer aggregators of same type and provide more help…
 2015-10-29 18:01:31 -07:00
Xavier Léauté 59872bd0cd Merge pull request #1809 from metamx/fifoPriorityExecutorService 
Make PrioritizedExecutorService optionally FIFO
 2015-10-27 15:19:32 -07:00
Charles Allen 060402a216 Merge pull request #1855 from himanshug/fix_having_specs 
fix [GreaterThan,LessThan,Equals] HavingSpecs
 2015-10-27 14:46:04 -07:00
Charles Allen ecdafa87c5 Make PrioritizedExecutorService optionally FIFO 2015-10-27 14:16:22 -07:00
Himanshu Gupta a71c7270b9 making [GreaterThan,LessThan,Equals] HavingSpecs more robust by carefully using long vs float for comparison 2015-10-27 13:15:13 -05:00
Fangjin Yang 5a082b2f5e Merge pull request #1824 from metamx/UniformGranularitySpecHashEquals 
Add hashCode and equals to UniformGranularitySpec
 2015-10-26 09:34:01 -07:00
Fangjin Yang 5f23703216 Merge pull request #1638 from guobingkun/remove_maven_client_code 
Remove Maven client at runtime + Provide a way to load Druid extensions through local file system
 2015-10-26 09:30:05 -07:00
Nishant 7cecc55045 Add segment merge time as a metric 
Add merge and persist cpu time

Fix typo

review comment

move cpu time measuring to VMUtils

review comments.
 2015-10-22 12:28:03 +05:30
Bingkun Guo 4914925d65 New extension loading mechanism 
1) Remove maven client from downloading extensions at runtime.
2) Provide a way to load Druid extensions and hadoop dependencies through file system.
3) Refactor pull-deps so that it can download extensions into extension directories.
4) Add documents on how to use this new extension loading mechanism.
5) Change the way how Druid tarball is generated. Now all the extensions + hadoop-client 2.3.0
are packaged within the Druid tarball.
 2015-10-21 14:22:36 -05:00
Xavier Léauté e4ac78e43d bump next snapshot to 0.9.0 2015-10-20 13:46:13 -07:00
dclim 46ecdfa757 add comment explaining logic 2015-10-15 16:04:06 -06:00
Xavier Léauté 4c2c7a2c37 update version to 0.8.3 2015-10-14 21:40:55 -07:00
Charles Allen f432b8e3f9 Add hashCode and equals to UniformGranularitySpec 
* Also add hashCode != 0 to AllGranularity and NoneGranularity
 2015-10-13 16:42:21 -07:00
Gian Merlino c9d6994040 Merge pull request #1821 from himanshug/storage_adapter_update 
cache max data timestamp in QueryableIndexStorageAdapter
 2015-10-13 10:52:43 -07:00
Himanshu Gupta 490de1f98a support multiple non-consecutive intervals in outer query of nested group-by 2015-10-13 10:16:06 -05:00
Himanshu Gupta fbba30eb60 cache max data timestamp in QueryableIndexStorageAdapter so that TimestampCheckingOffset 
does not have to get it per cursor.
 2015-10-12 15:34:22 -05:00
Charles Allen 8ed5d2c06a Add hashCode and equals to stock lookups 2015-10-12 10:29:39 -07:00
Himanshu Gupta 2737fd83f5 in the IndexSizeExceededException put maxRowCount to confirm if it is correctly picked up from configuration 2015-10-06 15:23:14 -05:00
Himanshu Gupta 8654732ef6 make IndexSizeExceededException constructor take formatString and arguments than just fixed String 
like ISE, IAE etc
 2015-10-06 13:44:22 -05:00
dclim f4e0a76820 Support multiple outer aggregators of same type and provide more helpful exception when the same inner aggregator is referenced by multiple types of outer aggregators 2015-10-01 15:15:12 -06:00
Gian Merlino 774765dc40 GroupByQueryRunnerTest for hyperUnique finalizing post aggregators 2015-10-01 00:09:29 -04:00
Gian Merlino e3bb93e8c7 Revert "Merge pull request #1781 from dclim/nested-groupby-multiple-same-aggregator-fix-v2" 
This reverts commit dae488b7c0, reversing
changes made to 397be4b897.
 2015-10-01 00:05:59 -04:00
dclim 8e20a1e1f3 Use DoubleSumAggregatorFactory instead of CountAggregatorFactory, add test for non-integers 2015-09-30 17:11:39 -06:00
David Lim 70ae5ca922 Fix failure in nested groupBy with multiple aggregators with same fieldName 
Version 2 - Throws an exception if an outer query references an
aggregator that doesn't exist in the inner query, and then uses the
inner query aggregator names to form the columns for the intermediate
incremental index.

Also deleted all the getRequiredColumns() methods which are no longer
being used.

We do something wacky by adding an aggregator factory for the post
aggregators when building the intermediate incremental index, otherwise
queries on post aggregate results fail because the data isn't in the
incremental index.

Closes #1419
 2015-09-30 15:43:11 -06:00
Charles Allen 8199ecf1a4 Merge pull request #1782 from jon-wei/smq_cachekey 
Add analysisTypes to SegmentMetadataQuery cache key
 2015-09-29 15:51:35 -07:00
jon-wei 41ff271339 Add analysisTypes to SegmentMetadataQuery cache key 2015-09-29 14:33:35 -07:00
Charles Allen 2d847ad654 Merge pull request #1730 from metamx/union-queries-fix 
fix #1727 - Union bySegment queries fix
 2015-09-29 12:23:25 -07:00
Nishant 573aa96bd6 fix #1727 - Union bySegment queries fix 
Fixes #1727.
revert to doing merging for results for union queries on broker.

revert unrelated changes

Add test for union query runner

Add test

remove unused imports

fix imports

fix renamed file

fix test

update docs.
 2015-09-29 23:32:36 +05:30
Gian Merlino 62d4ced4dd Separate ListColumnIncluderator cache key parts with nul bytes 2015-09-29 13:59:58 -04:00
jon-wei e6a6284ebd Allow SegmentMetadataQuery to skip cardinality and size calculations 2015-09-22 13:51:55 -07:00
Gian Merlino aaa8a88464 Merge pull request #1739 from jon-wei/segment_realtime 
Allow SegmentAnalyzer to read columns from StorageAdapter, allow SegmentMetadataQuery to query IncrementalIndexSegments on realtime node
 2015-09-17 18:36:53 -07:00
Charles Allen df4c2bab10 Soften concurrency requirements on IncrementalIndexTest 2015-09-17 15:51:07 -07:00
jon-wei 367c50d4ba Allow SegmentAnalyzer to read columns from StorageAdapter, allow SegmentMetadataQuery to query IncrementalIndexSegments on realtime node 2015-09-16 18:39:31 -07:00
Charles Allen 6e1eb3b7fe Add better concurrency testing to IncrementalIndexTest 2015-09-16 14:04:20 -07:00
Gian Merlino 9705c5139b Merge pull request #1732 from jon-wei/segmentmeta 
Add support for a configurable default segment history period for segmentMetadata queries and GET /datasources/<datasourceName> lookups
 2015-09-16 12:36:25 -07:00
Fangjin Yang 8b071a7230 Merge pull request #1710 from metamx/incrementalIndexConcurrentTestLatching 
Add some basic latching to concurrency testing in IncrementalIndexTest
 2015-09-15 13:55:52 -07:00
jon-wei 193fb4fdfc Add support for a configurable default segment history period for segmentMetadata queries and GET /datasources/<datasourceName> lookups 2015-09-14 19:41:42 -07:00
Charles Allen bd605a097e Merge pull request #1731 from metamx/regex-extraction-npe 
fix NPE with regex extraction function
 2015-09-14 15:55:05 -07:00
Xavier Léauté 08a527d01a fix NPE with regex extraction function 2015-09-14 14:45:30 -07:00
Charles Allen e569f4b6a7 Add dimension extraction functionality to SearchQuery 
* Add IdentityExtractionFn
 2015-09-14 11:36:15 -07:00
Himanshu 5ff92664f8 Merge pull request #1696 from metamx/cpuTimeReporting 
Add CPU time to metrics for segment scanning.
 2015-09-14 10:53:55 -05:00
Fangjin Yang 34ef81572d Merge pull request #1700 from himanshug/update_agg_test_helper 
update indexing in the helper to use multiple persists and merge
 2015-09-14 06:56:29 -07:00
Charles Allen 8d3cdd8572 Don't check for sortedness if we already know GenericIndexedWriter isn't sorted 2015-09-11 16:32:09 -07:00
Charles Allen d6849805ea Add some basic latching to concurrency testing in IncrementalIndexTest 2015-09-10 10:06:51 -07:00
Himanshu Gupta 5da58e48e0 use Rule based TemporaryFolder for cleanup of temp directory/files 2015-09-09 11:10:33 -05:00
Himanshu Gupta 44911039c5 update indexing in the helper to use multiple persists and final merge to 
catch further issues in aggregator implementations
 2015-09-09 11:10:33 -05:00
Charles Allen fcf5cae81d Add CPU time to metrics for segment scanning. 2015-09-08 13:34:19 -07:00
cheddar 4f61b42f40 Merge pull request #1578 from b-slim/fix_extraction_filter_2 
Fix UT and documentation to the extraction filter
 2015-09-01 10:46:20 -07:00
Himanshu 04ff6cd355 Merge pull request #1685 from gianm/close-loudly 
Close output streams and channels loudly when creating segments.
 2015-08-28 23:32:22 -05:00
Gian Merlino 940e1aa3eb Replace funky imports with standard ones. 
1) Lots of Guava imports were not coming from the actual Guava
2) junit.framework.Assert should be org.junit.Assert
 2015-08-28 18:02:05 -07:00
Gian Merlino 7d6fa2ba50 Close output streams and channels loudly when creating segments. 2015-08-28 17:14:03 -07:00
Himanshu Gupta 2e0dd1d792 adding UTs and addressing review comments to 
firehoseV2 addition to Realtime[Manager|Plumber],
essential segment metadata persist support,
kafka-simple-consumer-firehose extension patch
 2015-08-27 20:50:46 -05:00
lvjq 2237a8cf0f kafka 8 simple consumer firehose 2015-08-27 20:50:46 -05:00
Charles Allen c1388a1685 Merge pull request #1632 from Hailei/fix-subquery-innerquery-demension 
Inner Query  should build on sub query
 2015-08-27 10:25:38 -07:00
Gian Merlino 2a866f49df Downgrade Jackson to 2.4.6. 2015-08-26 18:25:55 -07:00
Charles Allen 24aa762c79 Add test for #1632 2015-08-25 20:50:30 -07:00
Xavier Léauté 51f6a9a2c9 update jackson to 2.6.1 2015-08-25 16:07:01 -07:00
Himanshu Gupta c57c07f28a add ability for client code to provide InputStream of input data in addition to File 
It would be needed when input data file does not reside in the same jar
but you could still use getResourceAsStream() to read the data inside a file
 2015-08-20 00:54:58 -05:00
Xavier Léauté 3b2e41e42a update for next release 2015-08-18 17:16:46 -07:00
Slim Bouguerra 7549f02578 support the case filter value is null 2015-08-17 15:09:37 -05:00
zhanghailei 234a958817 Inner Query should build on sub query 2015-08-17 18:18:26 +08:00
Charles Allen db19d2d547 Revert "Update to guice 4.0" 2015-08-14 09:26:07 -07:00
Charles Allen be89105621 Merge pull request #1602 from metamx/more-code-cleanup 
Some perf Improvements in Broker
 2015-08-11 13:51:49 -07:00
Xavier Léauté fbdb841928 Merge pull request #1603 from metamx/optimize-lexicographic-topN 
Optimizations for LexicographicTopNs
 2015-08-11 13:35:34 -07:00
Nishant b8d8a8da9e Optimisations for LexicographicTopNs 
initial review for perf optimizations for lexicographic TopNs

fix compilation

create map with proper size

review comment

review comment

review comments
 2015-08-12 00:37:48 +05:30
Charles Allen 7e61216287 Update to guice 4.0 
- Mark a lot of `@Provides` methods as final since guice 4.0 disallows overriding them
 2015-08-10 13:57:18 -07:00
Slim Bouguerra f0bc362981 clean code if is not needed anymore 2015-08-07 12:38:41 -05:00
Slim Bouguerra 64d638a386 optimize makeMatcher 2015-08-06 17:04:36 -05:00
Nishant 1a46c4c71c avoid creating mergeSeqence when not required 2015-08-06 14:25:13 +05:30
Slim Bouguerra 83de5a4716 addressing reviewers comments 2015-08-03 09:03:28 -05:00
Slim Bouguerra dda0790a60 Fix extractionFilter by implementing make matcher 
Fix getBitmapIndex to consider the case were dim is null
Unit Test for exractionFn with empty result and null_column
UT for TopN queries with Extraction filter
refactor in Extractiuon fileter makematcher for realtime segment and clean code in b/processing/src/test/java/io/druid/query/groupby/GroupByQueryRunnerTest.java
fix to make sure that empty string are converted to null
 2015-08-03 09:02:17 -05:00
Himanshu Gupta d11d9b6c45 dont waste memory in storing all lines from input 
CharSource.readLines() reads all lines from input into a in-memory list
Since we need an iterator here, so this wastage can be easily prevented
 2015-07-20 21:59:38 -05:00
Fangjin Yang 0481c8ca26 Merge pull request #1406 from zhaown/fix-breaking-while-exceeding-max-intermediate-rows 
Fix breaking while exceeding max intermediate rows.
 2015-07-20 13:41:22 -07:00
Himanshu Gupta f7a92db332 generic byte[] serde for InputRow 2015-07-20 12:01:53 -05:00
Himanshu Gupta 0439e8ec23 adding serde methods for intermediate aggregation object to ComplexMetricSerde 
This provides the alternative to using ComplexMetricSerde.getObjectStrategy()
and using the serde methods from ObjectStrategy as that usage pattern is deprecated.
 2015-07-20 12:01:53 -05:00
zhaown 524b05f073 Fix breaking while exceeding max intermediate rows. 2015-07-19 10:41:53 +08:00
Fangjin Yang e21195f987 Merge pull request #1469 from guobingkun/table_config 
Inconsistent property names for "druid.metadata.storage.tables.xxx"
 2015-07-17 07:43:19 -07:00
Himanshu 19af3bc9bc Merge pull request #1535 from metamx/alphanum-docs-tests 
Update alphanumeric sort docs + more tests / examples
 2015-07-16 22:09:41 -05:00
Xavier Léauté 2c464ad936 correct reference in docs + more tests / examples 2015-07-16 19:50:05 -07:00
Xavier Léauté 9616c10b1d remove import static 2015-07-16 17:46:21 -07:00
Xavier Léauté c1308203b8 Merge pull request #1532 from metamx/fixTopNDimExtractionDoubleApply 
Fix TopN dimension extractions being applied twice
 2015-07-16 13:39:02 -07:00
Xavier Léauté 3a0793aaf9 Merge pull request #1533 from metamx/extraCheckGroupByDimExtraction 
Add more unit tests for group by
 2015-07-15 21:09:00 -07:00
Charles Allen 7d0b77c261 Add more unit tests for group by 2015-07-15 20:15:21 -07:00
Xavier Léauté a15a2c4047 fix histogram aggregator cache key 2015-07-15 17:33:36 -07:00
Charles Allen 9092c665b7 Fix TopN dimension extractions being applied twice 2015-07-15 16:58:15 -07:00
Charles Allen 456ad9ffba Merge pull request #1529 from metamx/update-versions 
inrement version
 2015-07-15 13:25:31 -07:00
Xavier Léauté 4cfb00bc8a inrement version 2015-07-15 13:09:05 -07:00
Charles Allen 5eadd395e2 Move lots of executor service creation to Execs 2015-07-14 15:38:49 -07:00
Nishant 184b12bee8 fix groupBy caching to work with renamed aggregators 
Issue - while storing results in cache we store the event map which
contains aggregator names mapped to values. Now when someone fire same
query after renaming aggs, the cache key will be same but the event
will contain metric values mapped to older names which leads to wrong
results.
Fix - modify cache to not store raw event but the actual list of values
only.

review comments + fix dimension renaming

review comment
 2015-07-09 11:48:26 +05:30
Xavier Léauté 9789417612 ModuleList is already part of Initialization 2015-07-01 11:37:40 -07:00
Xavier Léauté 2c463ae435 Merge pull request #1489 from metamx/moveTestPackages 
Move some test packages
 2015-07-01 11:18:09 -07:00
Charles Allen 5e19a615f1 Add coments to DimExtractionTopNAlgorithm 2015-07-01 10:32:45 -07:00
Charles Allen 7a2a8a3d6e Move extraction tests to more reasonable package 2015-07-01 10:30:50 -07:00
Bingkun Guo 4a0ae7d8d5 Fix inconsistent druid property names for "druid.metadata.storage.tables.xxx" between document and code 2015-06-29 10:12:30 -05:00
Xavier Léauté 28fa1642b9 add node time metrics to DirectDruidClient 2015-06-26 17:57:44 -07:00
Xavier Léauté 36b4453789 Merge pull request #1455 from druid-io/fix-protobuf 
Fix protobuf impl and docs
 2015-06-22 23:15:40 -07:00
nishant f9cdb0ad61 test for #1120 
Make the changes described in #1120 to add test for the issue described
there.
 2015-06-21 23:34:21 +05:30
fjy 9c74993559 fix protobuf impl and docs 2015-06-20 21:59:38 -07:00
Xavier Léauté 0a5bb909a2 [maven-release-plugin] prepare for next development iteration 2015-06-18 17:35:19 -07:00
Xavier Léauté 59c6b2b279 [maven-release-plugin] prepare release druid-0.8.0-rc1 2015-06-18 17:35:14 -07:00
Charles Allen 6230ac90ae Use IndexMerger for conversion 2015-06-10 11:34:58 -07:00
Xavier Léauté 395ba79f8b Merge pull request #1403 from metamx/mergerMakerTests 
Improvements around resource handling in IndexMerger / IndexIO / QueryableIndex
 2015-06-04 15:59:10 -07:00
Charles Allen ed8eb5c991 Improvements around resource handling in IndexMerger / IndexIO / QueryableIndex 
* Fix resource leak in `io.druid.segment.IndexIO.DefaultIndexIOHandler#validateTwoSegments(java.io.File, java.io.File)`
* Un-deprecate `close()` in `QueryableIndex` and make it inherit `Closeable`
* Fix resource leaks in various unit tests
* Add `CloserRule` for closing out resources
 2015-06-04 14:18:27 -07:00
Himanshu 50ad0e6474 Merge pull request #1412 from pjain1/alphaNumericTopN_NPE_fix 
NPE fix for TopN query with alphaNumericTopN metric spec
 2015-06-04 09:49:31 -05:00
Parag Jain a7b09e857c NPE fix for alphaNumericTopN when pervious stop is not specified 2015-06-04 09:30:31 -05:00
Xavier Léauté 35e2fde18e Merge pull request #1386 from himanshug/aggregation_testing1 
General class for testing any Aggregation Implementation
 2015-06-03 23:43:36 -07:00
Xavier Léauté 92d7316ed8 Merge pull request #1414 from metamx/timeout2TIMEOUT 
Replace "timeout" with QueryContextKeys.TIMEOUT
 2015-06-02 17:11:09 -07:00
Charles Allen 1c4d42bc15 Replace "timeout" with QueryContextKeys.TIMEOUT 2015-06-02 14:49:21 -07:00
Charles Allen f48db09e35 Add optimizations for ExtractionFn by enabling MANY_TO_ONE vs ONE_TO_ONE codepaths 
* Also adds LookupExtractionFn and MapLookupExtractor which takes in an explicit mapping of renames
* Add injective to javascript extraction fn
 2015-06-02 12:22:56 -07:00
Himanshu Gupta 215c1ab01e UTs for hyperUnique aggregation 2015-06-01 12:52:40 -05:00
Himanshu Gupta 160d5fe6b7 a general class for testing any [complex] aggregation implementation 2015-06-01 12:52:40 -05:00
Charles Allen 55292bba13 Add more IndexMergerTests 2015-05-28 18:18:20 -07:00
Charles Allen 1ebe622c7d Add checkin GroupByQuery for null DimensionSpec in dimension list 2015-05-28 14:55:34 -07:00
Xavier Léauté f9c624c7db Merge pull request #1361 from mrijke/groupby-limithavingorder-unittest 
GroupBy Query with Having/Limit/Orderingspec inconsistencies (UnitTest)
 2015-05-27 14:49:18 -07:00
Xavier Léauté 1a3f04f0ed Merge pull request #1354 from metamx/multi-valued-dimension-compression 
Enabling compression for multiValued dimension
 2015-05-26 23:43:53 -07:00
Charles Allen fd64c24e43 Fix roaring extraction filter on empty values 2015-05-26 13:54:18 -07:00
nishant 81415282aa Enabling compression for multiValued dimension 
Add test and refactoring

Add benchmark tests
 2015-05-27 00:09:14 +05:30
Charles Allen e97d22a10a Fix Extraction Filter cast problems for empty results 2015-05-22 15:20:11 -07:00
Charles Allen e1399b7ce4 Add unit test to show breaking Dimension Extraction Filter 2015-05-22 15:02:11 -07:00
Xavier Léauté 75c092ccb1 Merge pull request #1375 from metamx/MetricManipulatorFnInstances 
Modify MetricManipulatorFns to use instanced classes
 2015-05-22 15:56:47 -04:00
Charles Allen 042653ebcb Modify MetricManipulatorFns to use instanced classes 2015-05-22 12:38:38 -07:00
Himanshu Gupta 723df735e9 force eagerness of processing of SegmentMetadata queries on the processing executor by converting the Sequence into List 2015-05-22 13:46:26 -05:00
Himanshu Gupta 5852b64852 adding UT for SegmentMetadata bySegment query which catches following regression caused by commit 55ebf0cfdf 
it fails when we issue the SegmentMetadataQuery by setting {"bySegment" : true} in context with exception -
java.lang.ClassCastException: io.druid.query.Result cannot be cast to io.druid.query.metadata.metadata.SegmentAnalysis
at io.druid.query.metadata.SegmentMetadataQueryQueryToolChest$4.compare(SegmentMetadataQueryQueryToolChest.java:222) ~[druid-processing-0.7.3-SNAPSHOT.jar:0.7.3-SNAPSHOT]
at com.google.common.collect.NullsFirstOrdering.compare(NullsFirstOrdering.java:44) ~[guava-16.0.1.jar:?]
at com.metamx.common.guava.MergeIterator$1.compare(MergeIterator.java:46) ~[java-util-0.27.0.jar:?]
at com.metamx.common.guava.MergeIterator$1.compare(MergeIterator.java:42) ~[java-util-0.27.0.jar:?]
at java.util.PriorityQueue.siftUpUsingComparator(PriorityQueue.java:649) ~[?:1.7.0_80]
 2015-05-22 13:45:54 -05:00
Himanshu Gupta da0cc32bc8 Revert commit 55ebf0cfdf 
which caused following regression
 it fails when we issue the SegmentMetadataQuery by setting {"bySegment" : true} in context with exception -
java.lang.ClassCastException: io.druid.query.Result cannot be cast to io.druid.query.metadata.metadata.SegmentAnalysis
at io.druid.query.metadata.SegmentMetadataQueryQueryToolChest$4.compare(SegmentMetadataQueryQueryToolChest.java:222) ~[druid-processing-0.7.3-SNAPSHOT.jar:0.7.3-SNAPSHOT]
at com.google.common.collect.NullsFirstOrdering.compare(NullsFirstOrdering.java:44) ~[guava-16.0.1.jar:?]
at com.metamx.common.guava.MergeIterator$1.compare(MergeIterator.java:46) ~[java-util-0.27.0.jar:?]
at com.metamx.common.guava.MergeIterator$1.compare(MergeIterator.java:42) ~[java-util-0.27.0.jar:?]
at java.util.PriorityQueue.siftUpUsingComparator(PriorityQueue.java:649) ~[?:1.7.0_80]
 2015-05-22 13:39:34 -05:00
Maarten Rijke 82da479464 Fix for GroupBy with Having+Limit+Orderspec 
* Inverted function arguments to compose postProcFn for GroupBy queries
    with havingspec + limitspec.
  * Replaced query.getLimitSpec() with null in GroupByQueryToolChest's
    mergeGroupByResults
  * Added unittest to verify functionality
 2015-05-19 18:35:48 +02:00
Himanshu Gupta 2fd3e9e8e5 return size = 0 in ColumnAnalysis if its unknown 
that is if complex agg did not implement inputSizeFn() so
that segment metadata query shows atleast some information.
also instead of COMPLEX, return type of data stored.
 2015-05-15 20:11:56 -05:00
Xavier Léauté 3c3db7229c Merge pull request #1355 from himanshug/long_max_min_aggregators 
Long max/min aggregators
 2015-05-13 12:08:11 -07:00
Himanshu Gupta cebb550796 additional UTs for [DoubleMax/DoubleMin] aggregation 2015-05-13 09:25:41 -05:00
Himanshu Gupta d0ec945129 adding aliases doubleMax and doubleMin for max and min respectively 
renamed all [Max/Min]*.java to [DoubleMax/DoubleMin]*.java and created [Max/Min]AggregatorFactory.java which can be removed when we dont need the min/max aggregator type backward compatibility
 2015-05-13 09:25:41 -05:00
Himanshu Gupta 2de38f7d29 UTs for long[Max/Min] aggregation 2015-05-13 09:25:22 -05:00
Himanshu Gupta 00436f93e2 long max/min aggregators implementation 2015-05-13 09:25:22 -05:00
fjy 7a6acf5c1b update pom to 0.8 2015-05-11 19:41:58 -06:00
Xavier Léauté 33265d63e1 Merge pull request #1262 from metamx/fix-null-dimension 
fix handling of dimension having only null values
 2015-05-06 13:51:26 -07:00
nishant 34be1e96fa fix NPE 
review comments

Add test

fix test for java8
 2015-05-05 23:11:13 +05:30
Neo 8f8400e24e fix handling of dimension having only null values 
fixes #1211

fix value matcher

more improvements

more fixes for partial null column

fix handling of dimension having only null values

fixes #1211

fix value matcher

more improvements

more fixes for partial null column

review comment

IndexMaker speedups
* About 15% speedup

Conflicts:
	processing/src/main/java/io/druid/segment/IndexMaker.java

fix handling of dimension having only null values

fixes #1211

fix value matcher

more improvements

more fixes for partial null column

fix handling of dimension having only null values

fixes #1211

fix value matcher

more improvements

more fixes for partial null column

review comment

review comments

review comment

fix failing tests

review comment

fix compilation
 2015-05-04 22:07:45 +05:30
nishant 50158357ff fixes #1330 
fixes #1330,
Avoid creating Period instance as creating a Period from Long.MAX_VALUE
throws arithmetic exception.
After this query metric will emit duration in seconds instead of
minutes.
 2015-05-04 20:34:28 +05:30
Xavier Léauté 721505c017 Merge pull request #1208 from druid-io/rework-metrics 
Schemaless metrics + additional metrics for things we care about
 2015-04-27 15:04:54 -07:00
fjy 963e5765bf Schemaless metrics + additional metrics for things we care about 2015-04-27 13:39:40 -07:00
Charles Allen 27016c0289 Fix IndexIO segment validator to account for timestamp mismatches. 2015-04-27 12:42:16 -07:00
Charles Allen 633fdb029e Add option to ConvertSegmentTask to skip validation 
* Validation is enabled by default
 2015-04-27 08:37:55 -07:00
Charles Allen 303727e6a9 IndexMaker speedups 
* About 15% speedup

Conflicts:
	processing/src/main/java/io/druid/segment/IndexMaker.java
 2015-04-23 13:19:21 -07:00
Charles Allen f2300430d1 Cleanup some code in index creation. 
* Add some unit tests
* Add io.druid.segment.IndexMerger.reprocess for quick re-indexing of data
* Add dim-value validation to validation checker (instead of ONLY index #)
* General code refactoring to make things a little easier to read
 2015-04-23 12:41:42 -07:00
Xavier Léauté 7939f43681 Merge pull request #1296 from druid-io/limit-test 
Add test for order by metric and limit across multiple days
 2015-04-22 11:28:06 -07:00
fjy 97d87a06d0 Add another test for limit across multiple days 2015-04-22 11:27:37 -07:00
Fangjin Yang 28f69d6bd3 Merge pull request #1299 from metamx/improve-filter-datasource-metadata 
Improve filtering of segments for dataSourceMetadataQuery
 2015-04-22 11:07:35 -07:00
Xavier Léauté a0a28de551 fix serde issue when pulling timestamps from cache 2015-04-22 11:03:26 -07:00
Xavier Léauté 2b4406671e Merge pull request #1301 from druid-io/fix-type 
fix count agg factory type
 2015-04-21 09:24:20 -07:00
fjy 7805357ab1 fix count agg factory type 2015-04-21 09:23:04 -07:00
nishant bb8c0cb50b Improve filtering of segments for dataSourceMetadataQuery 
dataSourceMetadataQuery only needs to be executed on latest segments at
present, modify filterSegments and add test.
 2015-04-21 09:31:13 +05:30
Xavier Léauté f73f14ab91 Merge pull request #1297 from metamx/versionConverterTaskUpdates 
Update VersionConverterTask for IndexSpec and allowing Forced updates
 2015-04-20 16:44:35 -07:00
Charles Allen 7479ac9012 Update VersionConverterTask for IndexSepc and allowing Forced updates 2015-04-20 16:17:06 -07:00
fjy d260515a43 update druid-api version 2015-04-17 14:58:35 -07:00
Bingkun Guo cf155e4eba Fix an issue that after broker forwards GroupByQuery to historical, havingSpec is still applied 
on postAggregations which are removed in the forwarded query.

Add a unit test to replicate the issue.
Add a query that can replicate this issue into integration test.
 2015-04-17 13:00:41 -05:00
fjy f0a19349bf fix up some comments for contributed test 2015-04-16 15:07:09 -07:00
Fangjin Yang 90b17a5259 Merge pull request #1285 from venkateshk/limitspec-tests 
Unit test to surface bug with limit-spec order by over specific query intervals
 2015-04-16 13:52:58 -07:00
Xavier Léauté 1d153674b6 remove overzealous check for backwards compatibility 2015-04-15 22:11:55 -07:00
Xavier Léauté ea5572d001 Merge pull request #1271 from metamx/strictErrorChecking 
Add stricter checking for potential coding errors
 2015-04-15 15:21:41 -07:00
Charles Allen abdeaa0746 Add stricter checking for potential coding errors 
Can use via `mvn clean compile test-compile -P strict'
 2015-04-15 14:52:25 -07:00
vkavuluri a2ba5b6183 Unit test to surface bug with limit-spec order by over specific query intervals 2015-04-15 06:31:22 -07:00
Xavier Léauté 3a3046ccf3 add support for dimension compression 
- compression for single-value dimensions using CompressedVSizeIntsIndexedSupplier
- makes dimension compression configurable via IndexSpec
- IndexSpec also enables configuring bitmap and metric compression
 2015-04-14 10:44:18 -07:00
Xavier Léauté bafc5114b4 add toString, equals, and hashCode to BitmapSerdeFactory 2015-04-14 10:44:18 -07:00
Xavier Léauté d20128b89b add compressed variable-size ints column type 2015-04-14 10:44:18 -07:00
Xavier Léauté ce928d9636 add compressed ints column type 2015-04-14 10:44:17 -07:00
Xavier Léauté 5c23679238 add WritableSupplier and IndexedMultivalue 2015-04-14 10:44:17 -07:00
Xavier Léauté 1abb9cce7c make IndexedInts closeable + add fill method 2015-04-14 10:44:17 -07:00
Xavier Léauté ed0d49933e fix memory leak in CompressedXXXIndexedSupplierTest 2015-04-14 10:44:16 -07:00
Xavier Léauté 6790e6cf0f add fromList to CompressedLongsIndexedSupplier 2015-04-14 10:44:16 -07:00
Eric Tschetter 7517f0d0f0 Add some javadoc to the two Query processing interfaces to help aid in implementations of new Queries. 
Also, remove some comments that did not have enough context to actually make sense to anyone but the original author (at least, I hope they make sense to the author, I definitely don't know what was being said).
 2015-04-09 18:11:42 -07:00
Fangjin Yang 208e307915 Merge pull request #1251 from metamx/uriSegmentLoaders 
Revert "Revert "Overhaul of SegmentPullers to add consistency and retries""
 2015-03-30 17:43:51 -07:00
fjy aea7f9d192 [maven-release-plugin] prepare for next development iteration 2015-03-30 16:35:24 -07:00
fjy 060d7aef03 [maven-release-plugin] prepare release druid-0.7.1 2015-03-30 16:35:20 -07:00
Charles Allen 1c6cbea89c Revert "Revert "Overhaul of SegmentPullers to add consistency and retries"" 
This reverts commit f904bc7858.
 2015-03-30 13:40:04 -07:00
Fangjin Yang f904bc7858 Revert "Overhaul of SegmentPullers to add consistency and retries" 2015-03-30 13:15:50 -07:00
Charles Allen 6d407e8677 Add URI handling to SegmentPullers 
* Requires https://github.com/druid-io/druid-api/pull/37
* Requires https://github.com/metamx/java-util/pull/22
* Moves the puller logic to use a more standard workflow going through java-util helpers instead of re-writing the handlers for each impl
  * General workflow goes like this: 1) LoadSpec makes sure the correct Puller is called with the correct parameters. 2) The Puller sets up general information like how to make an InputStream, how to find a file name (for .gz files for example), and when to retry. 3) CompressionUtils does most of the heavy lifting when it can
 2015-03-30 12:33:23 -07:00
Fangjin Yang e5653f0752 Merge pull request #1190 from vigiglobe/master 
Fix NPE when partionNumber 0 does not exist.
 2015-03-26 13:25:39 -07:00
Xavier Léauté 389ea4c32f Merge pull request #1245 from b-slim/fix_injector_plus_ut 
Bug fix @DruidSecondaryModule plus unit test
 2015-03-26 10:04:44 -07:00
Fangjin Yang a9c47de571 Merge pull request #1243 from metamx/fix-union-timeline-lookup 
fixes TimeboundaryQuery and DataSourceMetadata queries returning wrong values for union queries
 2015-03-26 10:02:56 -07:00
Slim Bouguerra 1e6be7796e bug fix @DruidSecondaryModule plus unit test 2015-03-26 10:44:52 -05:00
nishantmonu51 638bf9d4e9 return sorted List of TimeLineObjectHolder 2015-03-26 11:51:09 +05:30
msprunck 942c17a2aa Remove timeline chunk count assumptions. 
* Replace with generic iterables
 2015-03-24 22:40:49 +01:00
Prajwal Tuladhar 9983216871 use https maven repo URL to download dependencies 2015-03-20 14:09:07 -04:00
fjy b389cfe404 [maven-release-plugin] prepare for next development iteration 2015-03-19 12:38:17 -07:00
fjy 60e7d543cc [maven-release-plugin] prepare release druid-0.7.1-rc1 2015-03-19 12:38:13 -07:00
nishantmonu51 39e60b3405 fix race in groupByParallelQueryRunner 
add UT and use a queue for better concurrency
 2015-03-17 20:57:05 +05:30
Xavier Léauté 127b6fd857 Merge pull request #1172 from himanshug/segment_metadata_eager 
force eager the processing of segment metadata query on the processing executor
 2015-03-12 10:19:48 -07:00
Xavier Léauté 0a5a3fe2dc fix file missing from rebase 2015-03-11 17:30:11 -07:00
Xavier Léauté e01ed16030 serde tests + equals/hashCode fixes for extraction functions 2015-03-11 16:48:28 -07:00
Xavier Léauté d3f5bddc5c Add ability to apply extraction functions to the time dimension 
- Moves DimExtractionFn under a more generic ExtractionFn interface to
  support extracting dimension values other than strings
- pushes down extractionFn to the storage adapter from query engine
- 'dimExtractionFn' parameter has been deprecated in favor of 'extractionFn'
- adds a TimeFormatExtractionFn, allowing to project the '__time' dimension
- JavascriptDimExtractionFn renamed to JavascriptExtractionFn, adding
  support for any dimension value types that map directly to Javascript
- update documentation for time column extraction and related changes
 2015-03-11 16:45:42 -07:00
Himanshu Gupta 55ebf0cfdf force eager the processing of segment metadata query on the processing threadpool by using ChainedExecutionQueryRunner in SegmentMetadataQueryRunnerFactory.mergeRunners(..) 2015-03-11 12:58:58 -05:00
Xavier Léauté 217e674063 Handling aggregators and post aggregators with duplicate names 
* add test for same-name groupBy hyperUniques post-agg
* add test for same-name post-agg in groupby with approx histogram
* Fixes https://github.com/druid-io/druid/issues/1045
* Throws an error if post aggs and aggs do not have unique names
* Add more groupBy tests for Having filters
 2015-03-10 17:10:43 -07:00
Fangjin Yang 0b467624ec Merge pull request #694 from druid-io/arithmetic-op-strategies 
normal division & configurable ordering for ArithmeticPostAggregator
 2015-03-10 13:48:27 -07:00
Fangjin Yang 2abdce1dc0 Merge pull request #1180 from metamx/logging-groupBy-NPE 
add null check early to catch root cause for groupBy NPE while running bySegment query
 2015-03-09 09:16:33 -07:00
nishantmonu51 6e935cca0a add null check early to catch root cause 2015-03-09 21:10:28 +05:30
Xavier Léauté 0d47c0c36d normal division and configurable ordering for ArithmeticPostAggregator 
Fixes #510
 2015-03-04 12:44:24 -08:00
Fangjin Yang d685e2ab04 Merge pull request #1165 from friedhardware/fix-NPerror-select 
Added null check for the pagingSpec on a Select Query.
 2015-03-02 14:17:06 -08:00
Fangjin Yang e8605c63a9 Merge pull request #1150 from himanshug/broker-parallel-chunk-process 
interval chunk query runner now processes individual chunk in a threadpool
 2015-03-02 13:50:23 -08:00
Himanshu Gupta 29039fd541 interval chunk query runner now processes individual chunk in a thread pool and prints metrics query/time per chunk 2015-03-02 15:45:09 -06:00
Joshua Schumacher e6130e0fdc Added null check for the pagingSpec on a Select Query. 2015-03-02 12:41:59 -08:00
Fangjin Yang 005f4da2c0 Merge pull request #1143 from metamx/update-rhino-1.7rc5 
Update Rhino to 1.7RC5
 2015-02-25 12:50:23 -08:00
Xavier Léauté b167dcf82c [maven-release-plugin] prepare for next development iteration 2015-02-23 14:28:06 -08:00
Xavier Léauté e81ac2ba43 [maven-release-plugin] prepare release druid-0.7.0 2015-02-23 14:27:58 -08:00
James Estes 562de6c621 Update docs and examples for log4j2 usage. 
- Put configs early in classpath in examples so log4j2.xml will get picked up properly
- Add an example log4j2.xml file.
- Update Logging doc.
 2015-02-19 11:40:56 -07:00
Xavier Léauté c4d721fffd update Rhino to 1.7RC5 2015-02-19 09:48:18 -08:00
Xavier Léauté 78df7f6165 Move Druid release artifacts to Sonatype 
- Switch to using Druid parent POM
- Add required fields for Sonatype
- Common plugin versions and settings have been moved to the parent pom
- Cleanup artifacts and POMs for consistent formatting
- Remove org.hyperic.sigar dependency and update docs to reflect necessary jars to add at runtime when sigar is needed
 2015-02-13 14:26:31 -08:00
fjy d29740ed9f [maven-release-plugin] prepare for next development iteration 2015-02-12 16:16:00 -08:00
fjy 211fd15b7e [maven-release-plugin] prepare release druid-0.7.0-rc3 2015-02-12 16:15:56 -08:00
Fangjin Yang 90bc62eb5c Merge pull request #1108 from metamx/improve-groupby-perf 
Improve groupby by removing conversion to case insensitive row
 2015-02-12 11:45:20 -08:00
nishantmonu51 15cf432b74 remove conversion to case insensitive row 
this is not required after death to casing in 0.7
 2015-02-11 19:40:36 +05:30
Xavier Léauté c5e99bf6ec Merge pull request #1105 from metamx/fixEmptyExtractionFilter 
Fix empty results on ExtractionFilter.
 2015-02-10 14:25:58 -08:00
Charles Allen b9cb311a52 Fix empty results on ExtractionFilter. 
* Now returns empty results rather than erroring out
* Added unit tests for multiples case
 2015-02-10 14:04:38 -08:00
fjy 708759e1e0 Update http-client to 1.0.0 2015-02-10 13:36:47 -08:00
Xavier Léauté a7dcaffb53 fix `__time` column selector for incremental index 
- also adds tests for selecting the time column
 2015-02-06 12:06:05 -08:00
Fangjin Yang 42e902b6e3 Merge pull request #1090 from metamx/alphanum-attribution 
update code attribution
 2015-02-04 15:51:34 -08:00
Xavier Léauté 0fbc6071c9 update code attribution 2015-02-04 15:28:44 -08:00
Fangjin Yang 25cf15824b Merge pull request #1085 from gianm/dsmrv-fix 
DataSourceMetadataResultValue fixes and JodaUtils adjustments.
 2015-02-03 17:51:33 -08:00
Gian Merlino 085ad8d345 Fix DataSourceMetadataResultValue serde. 2015-02-03 17:39:42 -08:00
fjy 1f12c5b2f1 [maven-release-plugin] prepare for next development iteration 2015-02-03 12:06:49 -08:00
fjy e82d431be7 [maven-release-plugin] prepare release druid-0.7.0-rc2 2015-02-03 12:06:41 -08:00
Xavier Léauté 4eff269536 Merge pull request #1079 from druid-io/cleanup-deps 
Remove non friendly dependencies from Druid
 2015-02-03 11:56:41 -08:00
fjy 3e5d338c8e Remove non friendly dependencies from Druid 2015-02-03 11:36:08 -08:00
Fangjin Yang 71b4c5fa86 Merge pull request #1076 from metamx/remove-threadlocals 
remove thread-locals in GenericIndexed in favor of wrapped objects
 2015-02-02 20:02:33 -08:00
Xavier Léauté cb2e300eba remove thread-locals in GenericIndexed in favor of wrapped objects to reduce GC pressure 2015-02-02 15:59:30 -08:00
Eric Tschetter 42eba986ce Towards consistent null handling 
This commit also includes
1) the addition of a context parameter on timeseries queries that allows it to ignore empty buckets instead of generating results for them
2) A cleanup of an unused method on an interface
 2015-02-02 12:53:07 -08:00
Fangjin Yang 92e616de11 Merge pull request #1077 from metamx/remove-unused-imports 
remove unused imports
 2015-02-02 10:45:27 -08:00
nishantmonu51 ba932bb1f2 remove unused imports 2015-02-02 21:53:39 +05:30
fjy d05032b98a towards a community led druid 2015-01-31 20:57:36 -08:00
Xavier Léauté f24a89a22a fix NPE for topN over missing hyperUniques column 2015-01-27 16:12:41 -08:00
Charles Allen 226dd91a31 Add a hash map for storing groupBy partition index 
* Improves groupBy performance by approx 15%
 2015-01-26 08:42:02 -08:00
fjy 1f94de22c6 [maven-release-plugin] prepare for next development iteration 2015-01-20 14:23:55 -08:00
fjy 17476edc31 [maven-release-plugin] prepare release druid-0.7.0-rc1 2015-01-20 14:23:51 -08:00
Charles Allen 3d27747f7e Upgrade to log4j2 
Default behavior is as before.
Added documentation for how to enable synchronous logging for select chatty classes:
* io.druid.client.ServerInventoryView
* io.druid.client.BatchServerInventoryView
* io.druid.curator.inventory.CuratorInventoryManager
* com.metamx.http.client.pool.ChannelResourceFactory
 2015-01-20 12:35:18 -08:00
Fangjin Yang 91a79dbf95 Merge pull request #1031 from metamx/ingestmetadata-query 
DataSourceMetadata query
 2015-01-19 21:55:35 -08:00
Charles Allen 7bb038756c Account for very slow writer threads in IncrementalIndexTest 2015-01-17 13:02:59 -08:00
Fangjin Yang b4041c13e5 Merge pull request #1029 from metamx/fixChainedExecutionQueryRunnerTest 
Address spurious test failures
 2015-01-16 13:08:32 -08:00
Xavier Léauté 3b3aad78cb Merge pull request #1027 from metamx/concurrentOnHeapIncrementalIndexFix 
Fix concurrency issues in OnheapIncrementalIndex
 2015-01-16 12:54:42 -08:00
Charles Allen 197af967ef Fix concurrency issues in OnheapIncrementalIndex 
* Was encountering weird errors when fast writes were coming in while queries were happening.
* Added unit tests which tend to cause concurrency query problems
 2015-01-16 12:01:46 -08:00
Charles Allen ebafa2a786 Fix spurious test failures in ChainedExecutionQueryRunnerTest 2015-01-15 16:49:16 -08:00
Fangjin Yang 5bfcc43377 Merge pull request #1008 from metamx/stringConversionJavaUtilUpdate 
Update all String conversions to and from byte[] to use the java-util StringUtils functions
 2015-01-15 13:50:27 -08:00
nishantmonu51 c7452b75f6 Merge branch 'master' into ingestmetadata-query 2015-01-15 18:00:31 +05:30
Xavier Léauté d5f4182de4 global test timeouts + fix test race condition 2015-01-07 23:36:57 -08:00
Fangjin Yang 852e863425 Merge pull request #981 from druid-io/strictModuleTyping 
Use Module instead of generic Object in Guice related items
 2015-01-05 12:43:20 -08:00
Charles Allen b1b5c9099e Update all String conversions to and from byte[] to use the java-util StringUtils functions 
* Speedup of GroupBy with javaScript filters by ~10%
* Requires https://github.com/metamx/java-util/pull/15
 2015-01-05 11:22:32 -08:00
Xavier Léauté 3fc6cf918d add test for large chunks 2015-01-02 14:31:22 -08:00
Xavier Léauté f2f9cbeca8 throw error rather than returning garbage results 2015-01-02 14:29:21 -08:00
Xavier Léauté 071943a367 fix LZF compression with buffers exceeding LZF chunk size 2015-01-02 11:39:50 -08:00
Xavier Léauté f2439899e7 fix bitmap factory serde 2014-12-23 15:07:32 -08:00
Xavier Léauté 27a3169312 increase test timeouts 2014-12-19 17:09:43 -08:00
Charles Allen 971afab36f Lengthen CompressionStrategyTest::testKnownSizeConcurrency() to have 2m timeout on its test to account for shared Jenkins build lag 2014-12-19 12:53:20 -08:00
Charles Allen 7c8d4a7433 Use Module instead of generic Object in Guice related items 2014-12-19 10:54:06 -08:00
Fangjin Yang be507b8cb4 Merge pull request #943 from mrijke/partialdimextractfn-nullpointer 
Fix NullPointerException in PartialDimExtractionFn
 2014-12-16 12:29:27 -07:00
nishantmonu51 80e4b68ee7 review comments 2014-12-16 21:16:48 +05:30
Fangjin Yang b3fe91bb50 Merge pull request #830 from metamx/union-merge-on-historical 
Union merge on historical
 2014-12-15 13:36:47 -07:00
fjy 3cb7999eb9 i hate hadoop dependencies 2014-12-15 09:52:46 -08:00
nishantmonu51 a0d3579a92 add docs + fix tests 2014-12-11 17:58:01 +05:30
nishantmonu51 7ad03087c0 Merge branch 'master' into ingestmetadata-query 2014-12-11 16:54:38 +05:30
nishantmonu51 32b4f55b8a review comments refactoring 2014-12-11 16:33:14 +05:30
nishantmonu51 3763357f6e Ingest metadata query implementation 2014-12-10 19:44:00 +05:30
Fangjin Yang d6d3ec6846 Merge pull request #948 from metamx/ingestion-docs 
Redocumenting ingestion
 2014-12-09 15:30:03 -07:00
fjy 9596c11f42 address cr 2014-12-09 14:19:18 -08:00
nishantmonu51 1a1b0e6f23 merge from master and review comments 2014-12-09 13:16:45 +05:30
xvrl 1392e2731f Merge pull request #936 from metamx/cachingRunnerImprovements 
General Caching Query Runners cleanup (40% query time reduction for HLL)
 2014-12-08 14:07:52 -08:00
Charles Allen 7b65f0635d General Caching Query Runners cleanup 
* Add type strictness to CachingClusteredClient.
* Add background caching to CachingClusteredClient. Gives between 0% and 5% query speed increase.
* Add @BackgroundCaching annotation for injected ExecutorService items
* Add `numBackgroundThreads' configuration options to CacheConfig (default 0 aka same thread legacy behavior)
* Add unit tests for CacheConfig
* Add an abstract caching query runner class, currently it doesn't do anything exceppt simply make the two caching queries distinct.
* Add caching to CachingQueryRunner. Gives up to a WHOPPING 40% reduction in query time on HLL queries
* Updated docs with more info on cache settings.
 2014-12-08 13:29:32 -08:00
Maarten Rijke 90670a9c7e Fix NullPointerException in PartialDimExtractionFn by explicity checking for dimValue == null, attempt 2 2014-12-08 22:26:35 +01:00
Maarten Rijke bd9bbf396c Fix NullPointerException in PartialDimExtractionFn by explicity checking for dimValue == null 2014-12-08 20:11:58 +01:00
Xavier Léauté ad23e49777 use fixed-size mapdb cache to avoid heap growing uncontrollably 2014-12-05 15:34:50 -08:00
Xavier Léauté 7cd45a6e1f IncrementalIndex throws exception if limit exceeded 
- For now uses a hardcoded ratio of aggregator to timeanddim buffer sizes
- canAppendRow is a workaround for realtime index since the
Firehose currently does not have a way of rolling back the last event in
case of error
- canAppendRow needs a fudge factor; there is a race between checking
if we can add a row and actually adding a row, because of the way MapDB
reports its size.
 2014-12-04 14:38:16 -08:00
Xavier Léauté c7dbe6116c write byte data as is in smile 2014-12-04 10:57:56 -08:00
Xavier Léauté c21a82a697 upgrade LZ4 to operate directly on ByteBuffers 2014-12-04 10:57:56 -08:00
Xavier Léauté 0c521e0a77 update joda-time and fix min/max instant 2014-12-04 10:57:56 -08:00
nishantmonu51 269a51964e fix size calculation 2014-12-04 17:22:24 +05:30
nishantmonu51 4dc0fdba8a consider mapped size in limit calculation & review comments 2014-12-03 23:47:30 +05:30
Charles Allen 529e7e0272 Merge pull request #927 from metamx/speedup-smile-bytes 
Improve Smile serde performance by writing binary data as is
 2014-12-03 10:02:08 -08:00
Charles Allen 0f5d5840da Merge pull request #924 from metamx/update-joda-time 
Update Joda-Time and fix min/max instant overflow
 2014-12-03 09:15:39 -08:00
nishantmonu51 da8bd7836b Introduce buffer size 2014-12-03 16:28:22 +05:30
Xavier Léauté 5fece517fa write byte data as is in smile 2014-12-03 00:01:01 -08:00
Xavier Léauté 18f50097a9 upgrade LZ4 to operate directly on ByteBuffers 2014-12-02 23:53:56 -08:00
fjy bc173d14fc a whole bunch of cleanup and fixes 2014-12-02 17:32:05 -08:00
Xavier Léauté a79389a9e5 update joda-time and fix min/max instant 2014-12-02 17:27:22 -08:00
nishantmonu51 b65933ffb8 make tests parameterised 2014-12-02 23:55:29 +05:30
nishantmonu51 6dc69c2f30 code cleanups & formatting 2014-12-02 22:44:33 +05:30
nishantmonu51 eac776f1a7 tests passing with on heap incremental index 2014-12-02 22:29:28 +05:30
Xavier Léauté 4eee7e69b9 fix cardinality aggregator caching 2014-11-26 15:00:37 -08:00
xvrl 5bc1be5ba0 Merge pull request #850 from metamx/druid-0.7.x-compressionstrategy 
Compression strategy changes
 2014-11-25 12:58:39 -08:00
Charles Allen c6043afa32 Removed empty function from CompressionStrategyTest 2014-11-25 12:57:06 -08:00
Charles Allen 6943db5251 Changed branching logic for LZFCompressor to return null only on error, and avoid checking in most circumstances 2014-11-25 12:53:11 -08:00