Apache
/
druid
mirror of https://github.com/apache/druid.git
1
0
Fork 0
Code Issues Packages Projects Releases Wiki Activity
6,906 Commits 113 Branches 506 Tags 718 MiB
Commit Graph

1411 Commits

Author SHA1 Message Date
Himanshu Gupta b40c342cd1 make Global stupid pool cache size configurable 2016-02-05 14:18:06 -06:00
Himanshu Gupta 72a1e730a2 OffheapIncrementalIndex updates to do the aggregation merging off-heap 2016-02-05 14:17:05 -06:00
Himanshu Gupta 907dd77483 OffheapIncrementalIndex a copy/paste of OnheapIncrementalIndex 2016-02-05 14:02:31 -06:00
Charles Allen aac5f9b2c9 Actually check cache key null checking in ExtractionDimFilterTest 2016-02-04 09:44:13 -08:00
fjy 1aa363cea7 new quickstart 2016-02-04 09:37:38 -08:00
Fangjin Yang da77591129 Merge pull request #2392 from metamx/fix2391 
Allow ExtractionDimFilter value to be null
 2016-02-03 17:47:14 -08:00
Charles Allen d4f00096ff Allow ExtractionDimFilter value to be null 
* Fixes #2391
 2016-02-03 15:51:47 -08:00
Himanshu Gupta 6e7d90cf56 UTs for DefaultLimitSpec 2016-02-03 15:59:12 -06:00
Himanshu Gupta 29e0d7f971 lazily create comparators for row columns when needed 2016-02-03 13:38:20 -06:00
navis.ryu 1d602be0f9 Replace string[] with int[] for dimensions 2016-02-03 15:03:22 +09:00
binlijin a5ef30ff84 optimize topn on particular situation 2016-02-02 14:20:09 +08:00
Himanshu 93c50d8538 Merge pull request #2094 from navis/simplify-index-merge 
Simplifying dimension merging
 2016-01-29 11:23:14 -06:00
navis.ryu 55a888ea2f time-descending result of select queries 2016-01-29 10:06:05 +09:00
navis.ryu dd774ef4dd one-pass merging of dictionary & index 2016-01-29 10:03:53 +09:00
Himanshu edd7ce58aa Merge pull request #2348 from AlexanderSaydakov/fix-aggregator-test-helper 
fixed createIndex
 2016-01-28 16:01:36 -06:00
saydakov e0860661b1 fixed createIndex 2016-01-28 13:20:50 -08:00
Nishant 99017f4518 Merge pull request #2326 from navis/use-reverse-iterator 
use reverse-iterator if possible
 2016-01-28 19:48:38 +05:30
Nishant 3880f54b87 Merge pull request #2332 from himanshug/configurable_partial 
make populateUncoveredIntervals a configuration in query context
 2016-01-28 10:34:35 +05:30
navis.ryu 7324ece8f9 use reverse-iterator if possible 2016-01-28 09:04:55 +09:00
Xavier Léauté 5a3642bb93 Merge pull request #2247 from metamx/pedanticBuild 
Enable strict building in travis
 2016-01-27 10:27:03 -08:00
Xavier Léauté 2e5004095a Merge pull request #2341 from gianm/smq-test 
SegmentMetadataQuery: Fix merging of ColumnAnalysis errors.
 2016-01-27 09:37:06 -08:00
Charles Allen 508734c8b0 Long constant reformatting in tests `l` --> `L` 2016-01-27 08:59:19 -08:00
Gian Merlino b1e6c01762 Make LookupExtractor abstract methods public, they have to work across classloaders. 2016-01-26 23:08:03 -08:00
Gian Merlino 795343f7ef SegmentMetadataQuery: Fix merging of ColumnAnalysis errors. 
Also add tests for:
- ColumnAnalysis folding
- Mixed mmap/incremental merging
 2016-01-26 17:16:26 -08:00
Himanshu Gupta 3719b6e3c8 make populateUncoveredIntervals a configuration in query context 2016-01-26 15:13:45 -06:00
Himanshu 3844658fb5 Merge pull request #2323 from druid-io/update-druidapi 
Update druid-api to 0.3.16
 2016-01-26 13:02:10 -06:00
Himanshu Gupta 09d3678667 adding single threaded indexing and querying test for IncrementalIndex 2016-01-23 00:17:14 -06:00
Charles Allen 0000b9fc62 Remove sorting in ProtoBufInputRowParserTest 
Due to processing/src/test/java/io/druid/data/input/ProtoBufInputRowParserTest.java
 2016-01-22 16:02:25 -08:00
Himanshu Gupta 2f7f5119cf older segments might not have field bitmapSerdeFactory for dimension columns and we must use appropriate default 2016-01-22 13:28:25 -06:00
binlijin 1d1f4d996d Merge pull request #2111 from binlijin/optimize-create-inverted-indexes 
optimize create inverted indexes
 2016-01-22 11:36:27 +08:00
binlijin 55f7dd4629 optimize create inverted indexes 2016-01-22 10:40:09 +08:00
Gian Merlino d416279c14 SegmentMetadataQuery support for returning aggregators. 2016-01-21 17:27:25 -08:00
Fangjin Yang 5a9cd89059 Merge pull request #2305 from gianm/segment-metadata-query-multivalues 
Add StorageAdapter#getColumnTypeName, and various SegmentMetadataQuery adjustments
 2016-01-21 17:22:34 -08:00
Gian Merlino e5913be90e Merge pull request #2257 from tubemogul/index-merge-bug 
Adds support for empty merge metrics. fixes #2256
 2016-01-21 16:38:00 -08:00
Gian Merlino 87c8046c6c Add StorageAdapter#getColumnTypeName, and various SegmentMetadataQuery adjustments. 
SegmentMetadataQuery stuff:

- Simplify implementation of SegmentAnalyzer.
- Fix type names for realtime complex columns; this used to try to merge a nice type
  name (like "hyperUnique") from mmapped segments with the word "COMPLEX" from incremental
  index segments, leading to a merge failure. Now it always uses the nice name.
- Add hasMultipleValues to ColumnAnalysis.
- Add tests for both mmapped and incremental index segments.
- Update docs to include errorMessage.
 2016-01-21 15:50:33 -08:00
Fangjin Yang 3f998117a6 Merge pull request #2306 from jon-wei/inherit2 
More specific null/empty str handling in IndexMerger
 2016-01-21 14:36:09 -08:00
Michael Schiff 1e44445f06 Adds support for empty merge metrics. fixes #2256 2016-01-21 13:21:37 -08:00
jon-wei 459a236067 More specific null/empty str handling in IndexMerger 2016-01-21 12:24:38 -08:00
Slim 201539260c Merge pull request #2076 from b-slim/issue_2010_upper_lower_extractionFN 
adding lower and upper extraction fn
 2016-01-21 09:58:07 -06:00
Slim Bouguerra 78feb3a13e adding lower and upper extraction fn 2016-01-21 08:59:05 -06:00
Gian Merlino 5a932d28c1 Merge pull request #2288 from tubemogul/index-merge-bug2 
Null check in IncrementalIndexAdapter.getDimValueLookup()
 2016-01-20 17:07:15 -08:00
Nishant 59ea186af7 fix reference counting for segments 2016-01-20 17:24:21 +05:30
Michael Schiff 50ceec78a2 null check in IncrementalIndexAdapter.getDimValueLookup() 2016-01-19 23:19:28 -08:00
jon-wei bc1e9b27c8 Consolidate IndexMergerTest and IndexMergerV9Test 2016-01-19 16:28:35 -08:00
jon-wei 747343e621 Preserve dimension order across indexes during ingestion 2016-01-19 13:34:11 -08:00
Fangjin Yang 0c31f007fc Merge pull request #1728 from himanshug/aggregators_in_segment_metadata 
Store AggregatorFactory[] in segment metadata
 2016-01-19 12:55:49 -08:00
Himanshu Gupta a99aef29a1 adding aggregators to segment metadata 2016-01-19 14:23:39 -06:00
Himanshu Gupta 52eb0f04a7 adding a new method getMergingFactory(..) to AggregatorFactory 2016-01-18 22:03:46 -06:00
Himanshu Gupta 77fc86c015 making AggregatorFactory abstract class 2016-01-18 22:03:46 -06:00
Himanshu Gupta 164b0aad7a removing Map<String,Object> segmentMetadata from methods in Index[Maker/Merger] and using Metadata class 
instead of a Map to store segment metadata
 2016-01-18 22:03:46 -06:00
zhxiaog 3459a202ce fixed #1873, add ability to express CONCAT as an extractionFn 2016-01-18 15:03:17 -08:00
Keuntae Park 238dd3be3c support cascade execution of extraction filters in extraction dimension spec 2016-01-18 11:10:19 +09:00
Fangjin Yang f6a1a4ae20 Merge pull request #2138 from KurtYoung/feature-build-v9 
build v9 directly
 2016-01-16 13:35:46 -06:00
Kurt Young 82ff98c2bf add config for build v9 directly and update docs 2016-01-16 11:26:34 +08:00
Kurt Young 1f2168fae5 add IndexMergerV9 
add unit tests for IndexMergerV9 and fix some bugs

add more unit tests and fix bugs

handle null values and add more tests

minor changes & use LoggingProgressIndicator in IndexGeneratorReducer

make some static class public from IndexMerger

minor changes and add some comments

changes for comments
 2016-01-16 11:25:28 +08:00
Kurt Young bb50d2a2b2 add some streaming writers 2016-01-16 11:25:26 +08:00
Fangjin Yang e0932ba1c2 Merge pull request #2267 from himanshug/fix_topn_multi_val_filter 
Remap id's returned in XXXFilteredDimensionSpec.getRow() as per reduced cardinality
 2016-01-14 17:06:54 -08:00
Fangjin Yang 7704699b40 Merge pull request #2265 from navis/strlen-dimension-ignored 
Strlen sort spec ignores dimension
 2016-01-14 17:06:33 -08:00
Himanshu Gupta ae6a111444 fix XXXFilteredDimensionSpec to remap the dictionary encodings as per new cardinality 2016-01-13 22:25:02 -06:00
binlijin a3140b2548 fix topN filtering on multi-valued dimension bug 2016-01-13 22:25:02 -06:00
navis.ryu ea9fabdf2f Strlen sort spec ignores dimension 2016-01-14 11:05:44 +09:00
Fangjin Yang 4c014c1574 Merge pull request #2228 from metamx/incremental-index-mem2 
Improve heap usage for IncrementalIndex
 2016-01-13 14:48:03 -08:00
navis.ryu 18479bb757 time-descending result of timeseries queries 2016-01-13 12:23:01 +09:00
Fangjin Yang d7ad93debc Merge pull request #2221 from binlijin/topN_minTopNThreshold 
Allow change minTopNThreshold per topN query
 2016-01-12 16:22:20 -08:00
Nishant 4863e2ca4f cache metric selectors instead of creating new ones for every metric in each row 
clear selectors on close.

Add comments about thread safety.
 2016-01-13 00:45:23 +05:30
Nishant dfe6abb721 Merge pull request #2250 from himanshug/agg_test_helper_fix 
remove redundant registering of json modules in AggregationTestHelper
 2016-01-12 11:42:00 +05:30
navis.ryu 976ebc45c0 Simplify information in IncrementalIndex 2016-01-12 10:18:11 +09:00
Himanshu Gupta b973604bf8 remove redundant registering of json modules in AggregationTestHelper 2016-01-11 19:03:22 -06:00
Xavier Léauté 46a7f2660d fix casing to be consistent with other classes 2016-01-08 10:19:06 -08:00
Fangjin Yang d0b10c29d7 Merge pull request #2197 from metamx/clearIncIndexClose 
Make OnHeapIncrementalIndex clean maps on close()
 2016-01-07 15:43:47 -08:00
Gian Merlino 4ecd901a1a Merge pull request #2219 from himanshug/identity_extraction_fn_singleton 
make IdentityExtractionFn singleton
 2016-01-07 10:08:28 -08:00
Fangjin Yang aaea95ed1b Merge pull request #2207 from himanshug/theta_sketch_select_query 
fix bug for thetaSketch metric not working with select queries
 2016-01-07 09:46:09 -08:00
binlijin 010c6e959c add test 2016-01-07 18:01:46 +08:00
binlijin a6bfcc5bfd Allow change minTopNThreshold per topN query 2016-01-07 14:51:00 +08:00
Fangjin Yang 4cc81d3eff Merge pull request #2096 from b-slim/add_use_case_unapply 
Add use case unapply
 2016-01-06 21:58:12 -08:00
Himanshu Gupta 217079d0c7 make IdentityExtractionFn singleton 2016-01-06 22:29:07 -06:00
Himanshu 902f51433d Merge pull request #2125 from mangeshpardeshiyahoo/master 
Add extraction function support for Dimension Selector
 2016-01-06 14:22:26 -06:00
Mangesh Pardeshi 75ee952197 Add extraction function support for dimension Selector 2016-01-06 13:47:07 -06:00
Slim Bouguerra 032d3bf6e6 Optimization of extraction filter by reversing the lookup 2016-01-06 11:16:11 -06:00
Himanshu Gupta 3f048f0b15 adding support to execute Select queries in AggregationTestHelper so that Select query based UTs can be written for complex aggregator implementations 2016-01-05 21:54:55 -06:00
Charles Allen 91fc32749b Make OnHeapIncrementalIndex clean maps on close() 2016-01-04 11:18:16 -08:00
Himanshu Gupta b47d807738 Add support for filtering at DimensionSpec level so that multivalued dimensions can be filtered correctly 
also adding UTs for multi-valued dimensions
 2015-12-30 17:59:47 -06:00
Himanshu Gupta fa5c3bb014 adding decorate(DimensionSelector) to DimensionSpec to enable support for arbitrary filtering/transformations to returned dimension values 2015-12-30 15:06:24 -06:00
Nishant b68265399c Merge pull request #2168 from druid-io/remove-indexmaker 
Remove IndexMaker
 2015-12-30 12:24:29 +05:30
Fangjin Yang e14ad74088 Merge pull request #1936 from b-slim/between_range_with_predicat 
adding Upper/Lower Bound Filter
 2015-12-29 10:11:22 -08:00
fjy faf421726b remove IndexMaker 2015-12-28 14:19:02 -08:00
Gian Merlino 83f4130b5f SegmentMetadataQuery merging fixes. 
- Fix merging when the INTERVALS analysisType is disabled, and add a test.
- Remove transformFn from CombiningSequence, use MappingSequence instead. transformFn did
  not work for "accumulate" anyway, which made the tests wrong (the intervals should have
  been condensed, but were not).
- Add analysisTypes to the Druids segmentMetadataQuery builder to make testing simpler.
 2015-12-22 07:57:10 -08:00
Robin dded4441d3 for completeness, add unit test for groupby/having with unrecognized type 2015-12-21 12:06:56 -06:00
Himanshu Gupta e1631967e3 adding comments to explain merge failure in segmentMetadata query 2015-12-19 11:39:24 -06:00
Himanshu Gupta 7ecad1be24 Fix and UT for testing segment analysis merge 2015-12-19 00:24:02 -06:00
Fangjin Yang 7019d3c421 Merge pull request #2107 from jon-wei/fix_smq 
More efficient SegmentMetadataQuery
 2015-12-18 16:40:47 -08:00
Fangjin Yang 14229ba0f2 Merge pull request #1922 from metamx/jsonIgnoresFinalFields 
Change DefaultObjectMapper to NOT overwrite final fields unless explicitly asked to
 2015-12-18 15:38:32 -08:00
Fangjin Yang 71f554bf80 Merge pull request #2101 from himanshug/fix_extraction_dim_filter_cache_key 
add extractionFn bytes to cache key in ExtractionDimFilter
 2015-12-18 12:05:43 -08:00
Fangjin Yang 9e6874cc7e Merge pull request #2084 from binlijin/master 
minor optimize IndexMerger's MMappedIndexRowIterable
 2015-12-18 11:42:55 -08:00
Bingkun cc21a5fac7 Merge pull request #1999 from himanshug/remove_min_max_aggs 
remove min/max aggregator factory
 2015-12-18 13:38:52 -06:00
jon-wei 356b07c6c3 More efficient SegmentMetadataQuery 2015-12-17 12:46:23 -08:00
Jonathan Wei f8cf84f466 Merge pull request #1995 from himanshug/num_rows_seg_metadata_query 
add numRows to segment metadata query response
 2015-12-17 12:23:46 -08:00
Himanshu Gupta 82ea348003 add extractionFn bytes to cache key in ExtractionDimFilter 2015-12-16 14:00:38 -06:00
Himanshu 628643d80e Merge pull request #2091 from rasahner/noDefaultForGroupbyHaving 
take away default for groupBy/having
 2015-12-16 01:07:40 -06:00
sahner 3441cf3110 take away default for groupBy/having 2015-12-15 10:32:45 -06:00