1338 Commits

Author SHA1 Message Date
Charles Allen
508734c8b0 Long constant reformatting in tests l --> L 2016-01-27 08:59:19 -08:00
Gian Merlino
b1e6c01762 Make LookupExtractor abstract methods public, they have to work across classloaders. 2016-01-26 23:08:03 -08:00
Himanshu
3844658fb5 Merge pull request #2323 from druid-io/update-druidapi
Update druid-api to 0.3.16
2016-01-26 13:02:10 -06:00
Himanshu Gupta
09d3678667 adding single threaded indexing and querying test for IncrementalIndex 2016-01-23 00:17:14 -06:00
Charles Allen
0000b9fc62 Remove sorting in ProtoBufInputRowParserTest
Due to processing/src/test/java/io/druid/data/input/ProtoBufInputRowParserTest.java
2016-01-22 16:02:25 -08:00
Himanshu Gupta
2f7f5119cf older segments might not have field bitmapSerdeFactory for dimension columns and we must use appropriate default 2016-01-22 13:28:25 -06:00
binlijin
1d1f4d996d Merge pull request #2111 from binlijin/optimize-create-inverted-indexes
optimize create inverted indexes
2016-01-22 11:36:27 +08:00
binlijin
55f7dd4629 optimize create inverted indexes 2016-01-22 10:40:09 +08:00
Gian Merlino
d416279c14 SegmentMetadataQuery support for returning aggregators. 2016-01-21 17:27:25 -08:00
Fangjin Yang
5a9cd89059 Merge pull request #2305 from gianm/segment-metadata-query-multivalues
Add StorageAdapter#getColumnTypeName, and various SegmentMetadataQuery adjustments
2016-01-21 17:22:34 -08:00
Gian Merlino
e5913be90e Merge pull request #2257 from tubemogul/index-merge-bug
Adds support for empty merge metrics. fixes #2256
2016-01-21 16:38:00 -08:00
Gian Merlino
87c8046c6c Add StorageAdapter#getColumnTypeName, and various SegmentMetadataQuery adjustments.
SegmentMetadataQuery stuff:

- Simplify implementation of SegmentAnalyzer.
- Fix type names for realtime complex columns; this used to try to merge a nice type
  name (like "hyperUnique") from mmapped segments with the word "COMPLEX" from incremental
  index segments, leading to a merge failure. Now it always uses the nice name.
- Add hasMultipleValues to ColumnAnalysis.
- Add tests for both mmapped and incremental index segments.
- Update docs to include errorMessage.
2016-01-21 15:50:33 -08:00
Fangjin Yang
3f998117a6 Merge pull request #2306 from jon-wei/inherit2
More specific null/empty str handling in IndexMerger
2016-01-21 14:36:09 -08:00
Michael Schiff
1e44445f06 Adds support for empty merge metrics. fixes #2256 2016-01-21 13:21:37 -08:00
jon-wei
459a236067 More specific null/empty str handling in IndexMerger 2016-01-21 12:24:38 -08:00
Slim
201539260c Merge pull request #2076 from b-slim/issue_2010_upper_lower_extractionFN
adding lower and upper extraction fn
2016-01-21 09:58:07 -06:00
Slim Bouguerra
78feb3a13e adding lower and upper extraction fn 2016-01-21 08:59:05 -06:00
Gian Merlino
5a932d28c1 Merge pull request #2288 from tubemogul/index-merge-bug2
Null check in IncrementalIndexAdapter.getDimValueLookup()
2016-01-20 17:07:15 -08:00
Nishant
59ea186af7 fix reference counting for segments 2016-01-20 17:24:21 +05:30
Michael Schiff
50ceec78a2 null check in IncrementalIndexAdapter.getDimValueLookup() 2016-01-19 23:19:28 -08:00
jon-wei
bc1e9b27c8 Consolidate IndexMergerTest and IndexMergerV9Test 2016-01-19 16:28:35 -08:00
jon-wei
747343e621 Preserve dimension order across indexes during ingestion 2016-01-19 13:34:11 -08:00
Fangjin Yang
0c31f007fc Merge pull request #1728 from himanshug/aggregators_in_segment_metadata
Store AggregatorFactory[] in segment metadata
2016-01-19 12:55:49 -08:00
Himanshu Gupta
a99aef29a1 adding aggregators to segment metadata 2016-01-19 14:23:39 -06:00
Himanshu Gupta
52eb0f04a7 adding a new method getMergingFactory(..) to AggregatorFactory 2016-01-18 22:03:46 -06:00
Himanshu Gupta
77fc86c015 making AggregatorFactory abstract class 2016-01-18 22:03:46 -06:00
Himanshu Gupta
164b0aad7a removing Map<String,Object> segmentMetadata from methods in Index[Maker/Merger] and using Metadata class
instead of a Map to store segment metadata
2016-01-18 22:03:46 -06:00
zhxiaog
3459a202ce fixed #1873, add ability to express CONCAT as an extractionFn 2016-01-18 15:03:17 -08:00
Keuntae Park
238dd3be3c support cascade execution of extraction filters in extraction dimension spec 2016-01-18 11:10:19 +09:00
Fangjin Yang
f6a1a4ae20 Merge pull request #2138 from KurtYoung/feature-build-v9
build v9 directly
2016-01-16 13:35:46 -06:00
Kurt Young
82ff98c2bf add config for build v9 directly and update docs 2016-01-16 11:26:34 +08:00
Kurt Young
1f2168fae5 add IndexMergerV9
add unit tests for IndexMergerV9 and fix some bugs

add more unit tests and fix bugs

handle null values and add more tests

minor changes & use LoggingProgressIndicator in IndexGeneratorReducer

make some static class public from IndexMerger

minor changes and add some comments

changes for comments
2016-01-16 11:25:28 +08:00
Kurt Young
bb50d2a2b2 add some streaming writers 2016-01-16 11:25:26 +08:00
Fangjin Yang
e0932ba1c2 Merge pull request #2267 from himanshug/fix_topn_multi_val_filter
Remap id's returned in XXXFilteredDimensionSpec.getRow() as per reduced cardinality
2016-01-14 17:06:54 -08:00
Fangjin Yang
7704699b40 Merge pull request #2265 from navis/strlen-dimension-ignored
Strlen sort spec ignores dimension
2016-01-14 17:06:33 -08:00
Himanshu Gupta
ae6a111444 fix XXXFilteredDimensionSpec to remap the dictionary encodings as per new cardinality 2016-01-13 22:25:02 -06:00
binlijin
a3140b2548 fix topN filtering on multi-valued dimension bug 2016-01-13 22:25:02 -06:00
navis.ryu
ea9fabdf2f Strlen sort spec ignores dimension 2016-01-14 11:05:44 +09:00
Fangjin Yang
4c014c1574 Merge pull request #2228 from metamx/incremental-index-mem2
Improve heap usage for IncrementalIndex
2016-01-13 14:48:03 -08:00
navis.ryu
18479bb757 time-descending result of timeseries queries 2016-01-13 12:23:01 +09:00
Fangjin Yang
d7ad93debc Merge pull request #2221 from binlijin/topN_minTopNThreshold
Allow change minTopNThreshold per topN query
2016-01-12 16:22:20 -08:00
Nishant
4863e2ca4f cache metric selectors instead of creating new ones for every metric in each row
clear selectors on close.

Add comments about thread safety.
2016-01-13 00:45:23 +05:30
Nishant
dfe6abb721 Merge pull request #2250 from himanshug/agg_test_helper_fix
remove redundant registering of json modules in AggregationTestHelper
2016-01-12 11:42:00 +05:30
navis.ryu
976ebc45c0 Simplify information in IncrementalIndex 2016-01-12 10:18:11 +09:00
Himanshu Gupta
b973604bf8 remove redundant registering of json modules in AggregationTestHelper 2016-01-11 19:03:22 -06:00
Xavier Léauté
46a7f2660d fix casing to be consistent with other classes 2016-01-08 10:19:06 -08:00
Fangjin Yang
d0b10c29d7 Merge pull request #2197 from metamx/clearIncIndexClose
Make OnHeapIncrementalIndex clean maps on close()
2016-01-07 15:43:47 -08:00
Gian Merlino
4ecd901a1a Merge pull request #2219 from himanshug/identity_extraction_fn_singleton
make IdentityExtractionFn singleton
2016-01-07 10:08:28 -08:00
Fangjin Yang
aaea95ed1b Merge pull request #2207 from himanshug/theta_sketch_select_query
fix bug for thetaSketch metric not working with select queries
2016-01-07 09:46:09 -08:00
binlijin
010c6e959c add test 2016-01-07 18:01:46 +08:00