Apache
/
druid
mirror of https://github.com/apache/druid.git
1
0
Fork 0
Code Issues Packages Projects Releases Wiki Activity
6,974 Commits 111 Branches 503 Tags 721 MiB
Commit Graph

1425 Commits

Author SHA1 Message Date
navis.ryu dd2375477a Support min/max values for metadata query (#2208) 2016-02-12 09:35:58 +09:00
Gian Merlino 2d037ef05e Merge pull request #2453 from DreamLab/fix/topn_sorting_anomaly 
Fix for unstable behavior of HyperLogLog comparator
 2016-02-11 16:05:34 -08:00
navis.ryu 4d63196535 Support dimension spec for select query 2016-02-12 08:54:28 +09:00
Himanshu 47d48e1e67 Merge pull request #2452 from gianm/print-properties 
PropertiesModule: Print properties, processors, totalMemory on startup.
 2016-02-11 16:49:34 -06:00
turu f277a54a5c removed unsafe heuristics from hll compareTo and provided unit test for regression 2016-02-11 23:46:24 +01:00
Slim 368988d187 Merge pull request #2291 from druid-io/lookupManager 
Promoting LookupExtractor state and LookupExtractorFactory to be a first class druid state object.
 2016-02-11 16:07:27 -06:00
Gian Merlino 29f7758e74 PropertiesModule: Print properties, processors, totalMemory on startup. 2016-02-11 13:51:08 -08:00
Slim Bouguerra 4e119b7a24 Adding lookup ref manager and lookup dimension spec impl 2016-02-11 12:11:51 -06:00
Jaehong Choi 2f2e2ff5b9 support alphanumeric sorting for dimensional columns in groupby 2016-02-11 17:31:28 +09:00
Keuntae Park 05a144e39a fix crash with filtered aggregator at ingestion time 
- only for selector filter because extraction filter is not supported as
  cardinality is not fixed at ingestion time
 2016-02-11 11:25:33 +09:00
Fangjin Yang b1673ee90e Merge pull request #2409 from gianm/smq-merged-thing 
SegmentMetadataQuery: Retain segment id when merging, if possible.
 2016-02-08 15:43:39 -08:00
Fangjin Yang c9c20bb7f3 Merge pull request #2395 from metamx/fixExtractionDimFilterNullTest 
Actually check cache key null checking in ExtractionDimFilterTest
 2016-02-08 14:10:52 -08:00
Gian Merlino bd9c04244f SegmentMetadataQuery: Retain segment id when merging, if possible. 
This is helpful on realtime nodes, where two analyses from two different hydrants
are merged together but they are actually from the same segment.
 2016-02-08 13:07:02 -08:00
Himanshu Gupta 9fe1b28ee5 provide configuration to enable usage of Off heap merging for groupBy query 2016-02-05 14:18:06 -06:00
Himanshu Gupta b40c342cd1 make Global stupid pool cache size configurable 2016-02-05 14:18:06 -06:00
Himanshu Gupta 72a1e730a2 OffheapIncrementalIndex updates to do the aggregation merging off-heap 2016-02-05 14:17:05 -06:00
Himanshu Gupta 907dd77483 OffheapIncrementalIndex a copy/paste of OnheapIncrementalIndex 2016-02-05 14:02:31 -06:00
Charles Allen aac5f9b2c9 Actually check cache key null checking in ExtractionDimFilterTest 2016-02-04 09:44:13 -08:00
fjy 1aa363cea7 new quickstart 2016-02-04 09:37:38 -08:00
Fangjin Yang da77591129 Merge pull request #2392 from metamx/fix2391 
Allow ExtractionDimFilter value to be null
 2016-02-03 17:47:14 -08:00
Charles Allen d4f00096ff Allow ExtractionDimFilter value to be null 
* Fixes #2391
 2016-02-03 15:51:47 -08:00
Himanshu Gupta 6e7d90cf56 UTs for DefaultLimitSpec 2016-02-03 15:59:12 -06:00
Himanshu Gupta 29e0d7f971 lazily create comparators for row columns when needed 2016-02-03 13:38:20 -06:00
navis.ryu 1d602be0f9 Replace string[] with int[] for dimensions 2016-02-03 15:03:22 +09:00
binlijin a5ef30ff84 optimize topn on particular situation 2016-02-02 14:20:09 +08:00
Himanshu 93c50d8538 Merge pull request #2094 from navis/simplify-index-merge 
Simplifying dimension merging
 2016-01-29 11:23:14 -06:00
navis.ryu 55a888ea2f time-descending result of select queries 2016-01-29 10:06:05 +09:00
navis.ryu dd774ef4dd one-pass merging of dictionary & index 2016-01-29 10:03:53 +09:00
Himanshu edd7ce58aa Merge pull request #2348 from AlexanderSaydakov/fix-aggregator-test-helper 
fixed createIndex
 2016-01-28 16:01:36 -06:00
saydakov e0860661b1 fixed createIndex 2016-01-28 13:20:50 -08:00
Nishant 99017f4518 Merge pull request #2326 from navis/use-reverse-iterator 
use reverse-iterator if possible
 2016-01-28 19:48:38 +05:30
Nishant 3880f54b87 Merge pull request #2332 from himanshug/configurable_partial 
make populateUncoveredIntervals a configuration in query context
 2016-01-28 10:34:35 +05:30
navis.ryu 7324ece8f9 use reverse-iterator if possible 2016-01-28 09:04:55 +09:00
Xavier Léauté 5a3642bb93 Merge pull request #2247 from metamx/pedanticBuild 
Enable strict building in travis
 2016-01-27 10:27:03 -08:00
Xavier Léauté 2e5004095a Merge pull request #2341 from gianm/smq-test 
SegmentMetadataQuery: Fix merging of ColumnAnalysis errors.
 2016-01-27 09:37:06 -08:00
Charles Allen 508734c8b0 Long constant reformatting in tests `l` --> `L` 2016-01-27 08:59:19 -08:00
Gian Merlino b1e6c01762 Make LookupExtractor abstract methods public, they have to work across classloaders. 2016-01-26 23:08:03 -08:00
Gian Merlino 795343f7ef SegmentMetadataQuery: Fix merging of ColumnAnalysis errors. 
Also add tests for:
- ColumnAnalysis folding
- Mixed mmap/incremental merging
 2016-01-26 17:16:26 -08:00
Himanshu Gupta 3719b6e3c8 make populateUncoveredIntervals a configuration in query context 2016-01-26 15:13:45 -06:00
Himanshu 3844658fb5 Merge pull request #2323 from druid-io/update-druidapi 
Update druid-api to 0.3.16
 2016-01-26 13:02:10 -06:00
Himanshu Gupta 09d3678667 adding single threaded indexing and querying test for IncrementalIndex 2016-01-23 00:17:14 -06:00
Charles Allen 0000b9fc62 Remove sorting in ProtoBufInputRowParserTest 
Due to processing/src/test/java/io/druid/data/input/ProtoBufInputRowParserTest.java
 2016-01-22 16:02:25 -08:00
Himanshu Gupta 2f7f5119cf older segments might not have field bitmapSerdeFactory for dimension columns and we must use appropriate default 2016-01-22 13:28:25 -06:00
binlijin 1d1f4d996d Merge pull request #2111 from binlijin/optimize-create-inverted-indexes 
optimize create inverted indexes
 2016-01-22 11:36:27 +08:00
binlijin 55f7dd4629 optimize create inverted indexes 2016-01-22 10:40:09 +08:00
Gian Merlino d416279c14 SegmentMetadataQuery support for returning aggregators. 2016-01-21 17:27:25 -08:00
Fangjin Yang 5a9cd89059 Merge pull request #2305 from gianm/segment-metadata-query-multivalues 
Add StorageAdapter#getColumnTypeName, and various SegmentMetadataQuery adjustments
 2016-01-21 17:22:34 -08:00
Gian Merlino e5913be90e Merge pull request #2257 from tubemogul/index-merge-bug 
Adds support for empty merge metrics. fixes #2256
 2016-01-21 16:38:00 -08:00
Gian Merlino 87c8046c6c Add StorageAdapter#getColumnTypeName, and various SegmentMetadataQuery adjustments. 
SegmentMetadataQuery stuff:

- Simplify implementation of SegmentAnalyzer.
- Fix type names for realtime complex columns; this used to try to merge a nice type
  name (like "hyperUnique") from mmapped segments with the word "COMPLEX" from incremental
  index segments, leading to a merge failure. Now it always uses the nice name.
- Add hasMultipleValues to ColumnAnalysis.
- Add tests for both mmapped and incremental index segments.
- Update docs to include errorMessage.
 2016-01-21 15:50:33 -08:00
Fangjin Yang 3f998117a6 Merge pull request #2306 from jon-wei/inherit2 
More specific null/empty str handling in IndexMerger
 2016-01-21 14:36:09 -08:00
Michael Schiff 1e44445f06 Adds support for empty merge metrics. fixes #2256 2016-01-21 13:21:37 -08:00
jon-wei 459a236067 More specific null/empty str handling in IndexMerger 2016-01-21 12:24:38 -08:00
Slim 201539260c Merge pull request #2076 from b-slim/issue_2010_upper_lower_extractionFN 
adding lower and upper extraction fn
 2016-01-21 09:58:07 -06:00
Slim Bouguerra 78feb3a13e adding lower and upper extraction fn 2016-01-21 08:59:05 -06:00
Gian Merlino 5a932d28c1 Merge pull request #2288 from tubemogul/index-merge-bug2 
Null check in IncrementalIndexAdapter.getDimValueLookup()
 2016-01-20 17:07:15 -08:00
Nishant 59ea186af7 fix reference counting for segments 2016-01-20 17:24:21 +05:30
Michael Schiff 50ceec78a2 null check in IncrementalIndexAdapter.getDimValueLookup() 2016-01-19 23:19:28 -08:00
jon-wei bc1e9b27c8 Consolidate IndexMergerTest and IndexMergerV9Test 2016-01-19 16:28:35 -08:00
jon-wei 747343e621 Preserve dimension order across indexes during ingestion 2016-01-19 13:34:11 -08:00
Fangjin Yang 0c31f007fc Merge pull request #1728 from himanshug/aggregators_in_segment_metadata 
Store AggregatorFactory[] in segment metadata
 2016-01-19 12:55:49 -08:00
Himanshu Gupta a99aef29a1 adding aggregators to segment metadata 2016-01-19 14:23:39 -06:00
Himanshu Gupta 52eb0f04a7 adding a new method getMergingFactory(..) to AggregatorFactory 2016-01-18 22:03:46 -06:00
Himanshu Gupta 77fc86c015 making AggregatorFactory abstract class 2016-01-18 22:03:46 -06:00
Himanshu Gupta 164b0aad7a removing Map<String,Object> segmentMetadata from methods in Index[Maker/Merger] and using Metadata class 
instead of a Map to store segment metadata
 2016-01-18 22:03:46 -06:00
zhxiaog 3459a202ce fixed #1873, add ability to express CONCAT as an extractionFn 2016-01-18 15:03:17 -08:00
Keuntae Park 238dd3be3c support cascade execution of extraction filters in extraction dimension spec 2016-01-18 11:10:19 +09:00
Fangjin Yang f6a1a4ae20 Merge pull request #2138 from KurtYoung/feature-build-v9 
build v9 directly
 2016-01-16 13:35:46 -06:00
Kurt Young 82ff98c2bf add config for build v9 directly and update docs 2016-01-16 11:26:34 +08:00
Kurt Young 1f2168fae5 add IndexMergerV9 
add unit tests for IndexMergerV9 and fix some bugs

add more unit tests and fix bugs

handle null values and add more tests

minor changes & use LoggingProgressIndicator in IndexGeneratorReducer

make some static class public from IndexMerger

minor changes and add some comments

changes for comments
 2016-01-16 11:25:28 +08:00
Kurt Young bb50d2a2b2 add some streaming writers 2016-01-16 11:25:26 +08:00
Fangjin Yang e0932ba1c2 Merge pull request #2267 from himanshug/fix_topn_multi_val_filter 
Remap id's returned in XXXFilteredDimensionSpec.getRow() as per reduced cardinality
 2016-01-14 17:06:54 -08:00
Fangjin Yang 7704699b40 Merge pull request #2265 from navis/strlen-dimension-ignored 
Strlen sort spec ignores dimension
 2016-01-14 17:06:33 -08:00
Himanshu Gupta ae6a111444 fix XXXFilteredDimensionSpec to remap the dictionary encodings as per new cardinality 2016-01-13 22:25:02 -06:00
binlijin a3140b2548 fix topN filtering on multi-valued dimension bug 2016-01-13 22:25:02 -06:00
navis.ryu ea9fabdf2f Strlen sort spec ignores dimension 2016-01-14 11:05:44 +09:00
Fangjin Yang 4c014c1574 Merge pull request #2228 from metamx/incremental-index-mem2 
Improve heap usage for IncrementalIndex
 2016-01-13 14:48:03 -08:00
navis.ryu 18479bb757 time-descending result of timeseries queries 2016-01-13 12:23:01 +09:00
Fangjin Yang d7ad93debc Merge pull request #2221 from binlijin/topN_minTopNThreshold 
Allow change minTopNThreshold per topN query
 2016-01-12 16:22:20 -08:00
Nishant 4863e2ca4f cache metric selectors instead of creating new ones for every metric in each row 
clear selectors on close.

Add comments about thread safety.
 2016-01-13 00:45:23 +05:30
Nishant dfe6abb721 Merge pull request #2250 from himanshug/agg_test_helper_fix 
remove redundant registering of json modules in AggregationTestHelper
 2016-01-12 11:42:00 +05:30
navis.ryu 976ebc45c0 Simplify information in IncrementalIndex 2016-01-12 10:18:11 +09:00
Himanshu Gupta b973604bf8 remove redundant registering of json modules in AggregationTestHelper 2016-01-11 19:03:22 -06:00
Xavier Léauté 46a7f2660d fix casing to be consistent with other classes 2016-01-08 10:19:06 -08:00
Fangjin Yang d0b10c29d7 Merge pull request #2197 from metamx/clearIncIndexClose 
Make OnHeapIncrementalIndex clean maps on close()
 2016-01-07 15:43:47 -08:00
Gian Merlino 4ecd901a1a Merge pull request #2219 from himanshug/identity_extraction_fn_singleton 
make IdentityExtractionFn singleton
 2016-01-07 10:08:28 -08:00
Fangjin Yang aaea95ed1b Merge pull request #2207 from himanshug/theta_sketch_select_query 
fix bug for thetaSketch metric not working with select queries
 2016-01-07 09:46:09 -08:00
binlijin 010c6e959c add test 2016-01-07 18:01:46 +08:00
binlijin a6bfcc5bfd Allow change minTopNThreshold per topN query 2016-01-07 14:51:00 +08:00
Fangjin Yang 4cc81d3eff Merge pull request #2096 from b-slim/add_use_case_unapply 
Add use case unapply
 2016-01-06 21:58:12 -08:00
Himanshu Gupta 217079d0c7 make IdentityExtractionFn singleton 2016-01-06 22:29:07 -06:00
Himanshu 902f51433d Merge pull request #2125 from mangeshpardeshiyahoo/master 
Add extraction function support for Dimension Selector
 2016-01-06 14:22:26 -06:00
Mangesh Pardeshi 75ee952197 Add extraction function support for dimension Selector 2016-01-06 13:47:07 -06:00
Slim Bouguerra 032d3bf6e6 Optimization of extraction filter by reversing the lookup 2016-01-06 11:16:11 -06:00
Himanshu Gupta 3f048f0b15 adding support to execute Select queries in AggregationTestHelper so that Select query based UTs can be written for complex aggregator implementations 2016-01-05 21:54:55 -06:00
Charles Allen 91fc32749b Make OnHeapIncrementalIndex clean maps on close() 2016-01-04 11:18:16 -08:00
Himanshu Gupta b47d807738 Add support for filtering at DimensionSpec level so that multivalued dimensions can be filtered correctly 
also adding UTs for multi-valued dimensions
 2015-12-30 17:59:47 -06:00
Himanshu Gupta fa5c3bb014 adding decorate(DimensionSelector) to DimensionSpec to enable support for arbitrary filtering/transformations to returned dimension values 2015-12-30 15:06:24 -06:00
Nishant b68265399c Merge pull request #2168 from druid-io/remove-indexmaker 
Remove IndexMaker
 2015-12-30 12:24:29 +05:30
Fangjin Yang e14ad74088 Merge pull request #1936 from b-slim/between_range_with_predicat 
adding Upper/Lower Bound Filter
 2015-12-29 10:11:22 -08:00
fjy faf421726b remove IndexMaker 2015-12-28 14:19:02 -08:00