Gian Merlino
6171e078c8
Improve NPE message in LookupDimensionSpec when lookup does not exist. ( #3065 )
...
The message used to be empty, which made things hard to debug.
2016-06-02 19:59:12 -07:00
John Wang
e662efa79f
segment interface refactor for proposal 2965 ( #2990 )
2016-05-26 20:36:41 -07:00
Kurt Young
b5bd406597
fix #2991 : race condition in OnheapIncrementalIndex#addToFacts ( #3002 )
...
* fix #2991 : race condition in OnheapIncrementalIndex#addToFacts
* add missing header
* handle parseExceptions when first doing first agg
2016-05-25 19:05:46 -07:00
Jonathan Wei
b72c54c4f8
Add benchmark data generator, basic ingestion/persist/merge/query benchmarks ( #2875 )
2016-05-25 16:39:37 -07:00
Dave Li
dcabd4b1ee
Add lookup optimization for InDimFilter ( #2938 )
...
* Add lookup optimization for InDimFilter
* tests for in filter with lookup extraction fn
* refactor
* refactor2 and modified filter test
* make optimizeLookup private
2016-05-19 16:29:16 -07:00
Charles Allen
15ccf451f9
Move QueryGranularity static fields to QueryGranularities ( #2980 )
...
* Move QueryGranularity static fields to QueryGranularityUtil
* Fixes #2979
* Add test showing #2979
* change name to QueryGranularities
2016-05-17 16:23:48 -07:00
Charles Allen
fb01db4db7
[QTL] Allows RegisteredLookupExtractionFn to find its lookups lazily ( #2971 )
...
* Allows RegisteredLookupExtractionFn to find its lookups lazily
* Use raw variables instead of AtomicReference
* Make sure to use volatile
* Remove extra local variable.
* Move from BAOS to ByteBuffer
2016-05-17 11:29:39 -07:00
Himanshu
d3e9c47a5f
use correct ObjectMapper in Index[IO/Merger] in AggregationTestHelper and minor fix in theta sketch SketchMergeAggregatorFactory.getMergingFactory(..) ( #2943 )
2016-05-13 10:06:31 +05:30
Himanshu
d821144738
at historicals GpBy query mergeResults does not need merging as results are already merged by GroupByQueryRunnerFactory.mergeRunners(..) ( #2962 )
2016-05-12 17:41:24 -07:00
Gian Merlino
01bebf432a
GroupByQuery: Multi-value dimension tests. ( #2959 )
2016-05-12 11:31:50 -07:00
Charles Allen
a31348450f
Add toString for LookupConfig ( #2935 )
...
* Helps with operations and getting where the snapshot dir is
2016-05-09 18:20:00 -07:00
Dave Li
79a54283d4
Optimize filter for timeseries, search, and select queries ( #2931 )
...
* Optimize filter for timeseries, search, and select queries
* exception at failed toolchest type check
* took out query type check
* java7 error fix and test improvement
2016-05-09 11:04:06 -07:00
Slim
8b570ab130
make it clear what LookupExtractorFactory start/stop methods return ( #2925 )
2016-05-05 10:38:40 -07:00
David Lim
b489f63698
Supervisor for KafkaIndexTask ( #2656 )
...
* supervisor for kafka indexing tasks
* cr changes
2016-05-04 23:13:13 -07:00
Himanshu
8e2742b7e8
adding QueryGranularity to segment metadata and optionally expose same from segmentMetadata query ( #2873 )
2016-05-03 11:31:10 -07:00
Gian Merlino
40e595c7a0
Remove types from TimeAndDims, they aren't needed. ( #2865 )
2016-05-03 13:10:25 -05:00
binlijin
841be5c61f
periodically emit metric segment/scan/pending ( #2854 )
2016-05-02 22:38:13 -07:00
Navis Ryu
2729fea84d
Fix parsing fail of segment id with datasource containing underscore ( #2797 )
...
* Fix parsing fail of segment id with underscored datasource (Fix for #2786 )
* addressed comment
* renamed and moved code into api. added log4 dependency for tests
* addressed comments
* fixed test fails
2016-05-02 22:37:28 -07:00
Gian Merlino
90ce03c66f
Fix integer overflow in SegmentMetadataQuery numRows. ( #2890 )
2016-04-27 14:37:04 -07:00
Gian Merlino
6dc7688a29
TimeAndDims equals/hashCode implementation. ( #2870 )
...
Adapted from #2692 , thanks @navis for original implementation.
2016-04-22 08:45:20 +08:00
Himanshu
3cfd9c64c9
make singleThreaded groupBy query config overridable at query time ( #2828 )
...
* make isSingleThreaded groupBy query processing overridable at query time
* refactor code in GroupByMergedQueryRunner to make processing of single threaded and parallel merging of runners consistent
2016-04-21 17:12:58 -07:00
Slim
984a518c9f
Merge pull request #2734 from b-slim/LookupIntrospection2
...
[QTL][Lookup] adding introspection endpoint
2016-04-21 12:15:57 -05:00
Gian Merlino
c74391e54c
JavaScript: Ability to disable. ( #2853 )
...
Fixes #2852 .
2016-04-21 09:43:15 -05:00
Gian Merlino
7d3e55717d
Reduce cost of various toFilter calls. ( #2860 )
...
These happen once per segment and so it's better if they don't do
as much work.
2016-04-21 04:28:46 +08:00
Gian Merlino
59460b17cc
Add Filters.matchPredicate helper, use it where appropriate. ( #2851 )
...
This approach simplifies code and is generally faster, due to skipping
unnecessary dictionary lookups (see #2850 ).
2016-04-19 15:54:32 -07:00
Xavier Léauté
b2745befb7
remove obsolete comment ( #2858 )
2016-04-19 13:06:58 -07:00
Jisoo Kim
7b65ca7889
refactor ClientQuerySegmentWalker ( #2837 )
...
* refactor ClientQuerySegmentWalker
* add header to FluentQueryRunnerBuilder
* refactor QueryRunnerTestHelper
2016-04-18 14:00:47 -07:00
Gian Merlino
7c0b1dde3a
DimensionPredicateFilter: Skip unnecessary dictionary lookup. ( #2850 )
2016-04-18 12:38:25 -07:00
Jonathan Wei
b534f7203c
Fix performance regression from #2753 in IndexMerger ( #2841 )
2016-04-14 21:39:41 -07:00
Jonathan Wei
a26134575b
Fix NPE in TopNLexicographicResultBuilder.addEntry() ( #2835 )
2016-04-13 17:27:16 -07:00
Fangjin Yang
abd951df1a
Document how to use roaring bitmaps ( #2824 )
...
* Document how to use roaring bitmaps
This fixes #2408 .
While not all indexSpec properties are explained, it does explain how roaring bitmaps can be turned on.
* fix
* fix
* fix
* fix
2016-04-12 19:28:02 -07:00
michaelschiff
db35dd7508
fix issue #2744 . Check for null before combining metrics ( #2774 )
2016-04-12 14:46:31 -07:00
Nishant
1bf1dd03a0
Merge pull request #2812 from mrijke/fix-missing-equals-hashcode-filters
...
Add missing equals/hashcode to JS, Regex and SearchQuery DimFilters
2016-04-12 12:00:23 +05:30
Charles Allen
21e406613c
Merge pull request #2809 from metamx/fix2694
...
Fix test for snapshot taker to better check for lookup perist failure
2016-04-11 14:52:47 -07:00
Maarten Rijke
de68d6b7c4
Add missing equals/hashcode to JS, Regex and SearchQuery DimFilters
...
This commits adds missing equals() and hashcode() methods to
the JavascriptDimFilter, RegexDimFilter and the SearchQueryDimFilter.
2016-04-11 12:16:24 +02:00
Nishant
bbb326decf
Merge pull request #2799 from b-slim/fix_snapshot
...
MapLookupFactory need to be Ser/Desr ready.
2016-04-07 13:22:34 +05:30
Slim Bouguerra
bf1eafc4e1
remove all the mock lookupFactory
2016-04-06 15:37:52 -05:00
Slim Bouguerra
59eb2490a0
MapLookupFactory need to be Ser/Desr.
2016-04-06 15:02:18 -05:00
Charles Allen
f915a59138
Merge pull request #2691 from metamx/lookupExtrFn
...
Add ExtractionFn to LookupExtractor bridge
2016-04-06 09:13:08 -07:00
jon-wei
051fd6c0eb
Remove extra println from InFilter
2016-04-05 14:55:49 -07:00
Fangjin Yang
289bb6f885
Merge pull request #2690 from jon-wei/filter_support
...
Allow filters to use extraction functions
2016-04-05 15:40:15 -06:00
jon-wei
0e481d6f93
Allow filters to use extraction functions
2016-04-05 13:24:56 -07:00
Gian Merlino
e060a9f283
Additional ExtractionFn null-handling adjustments.
...
Followup to comments on #2771 .
2016-04-01 18:35:26 -07:00
Fangjin Yang
18b9ea62cf
Merge pull request #2771 from gianm/extractionfn-stuff
...
Various ExtractionFn null handling fixes.
2016-04-01 16:35:46 -07:00
Gian Merlino
23d66e5ff9
Merge pull request #2765 from navis/invalid-encode-nullstring
...
Null string is encoded as "null" in incremental index
2016-04-01 14:43:40 -07:00
Gian Merlino
b6e4d8b2c1
Various ExtractionFn null handling fixes.
...
- JavaScriptExtractionFn shouldn't pass empty strings to its JS functions
- Upper/LowerExtractionFn properly handles null Objects (DimExtractionFn's implementation works here)
- MatchingDimExtractionFn properly returns nulls rather than empties
- RegexDimExtractionFn properly attempts matching on nulls and empties
- SearchQuerySpecDimExtractionFn properly returns nulls when passed empties
2016-04-01 14:34:47 -07:00
Fangjin Yang
eea7a47870
Merge pull request #2576 from navis/paging-from-next
...
Add option for select query to get next page without modifying returned paging identifiers
2016-04-01 13:50:36 -07:00
Fangjin Yang
4eb5a2c4f1
Merge pull request #2715 from navis/stringformat-null-handling
...
stringFormat extractionFn should be able to return null on null values (Fix for #2706 )
2016-04-01 13:45:28 -07:00
Gian Merlino
23364a47fd
BaseFilterTest: Test optimized filters too.
2016-04-01 12:44:59 -07:00
navis.ryu
077522a46f
stringFormat extractionFn should be able to return null on null values (Fix for #2706 )
2016-04-01 13:40:56 +09:00
navis.ryu
f0e55f5d31
Null string is encoded as "null" in incremental index
2016-04-01 09:47:15 +09:00
navis.ryu
29bb00535b
Add option for select query to get next page without modifying returned paging identifiers
2016-04-01 09:03:03 +09:00
Gian Merlino
5f9240fcbc
Merge pull request #2577 from navis/native-in-filter
...
Implement native in filter
2016-03-30 20:02:54 -07:00
Fangjin Yang
3d68da94fe
Merge pull request #2661 from navis/utf8-estimated-length
...
Utility method for length estimation of utf8
2016-03-30 19:56:14 -07:00
navis.ryu
108535fd07
Implement native in filter (Fix for #2577 )
2016-03-31 10:10:57 +09:00
navis.ryu
e0cfd9ee19
Utility method for length estimation of utf8
2016-03-31 10:07:00 +09:00
jon-wei
5503bf1b38
Remove unnecessary type check in TimeAndDimsComp
2016-03-30 17:54:15 -07:00
Fangjin Yang
95733a362f
Merge pull request #2753 from gianm/null-filtering-multi-value-columns
...
More consistent empty-set filtering behavior on multi-value columns.
2016-03-29 18:52:25 -07:00
Charles Allen
95d42cfd9e
Merge pull request #2758 from pjain1/fix_npe_in_filter
...
handle null values in In Filter
2016-03-29 17:53:02 -07:00
Gian Merlino
1853f36e9f
More consistent empty-set filtering behavior on multi-value columns.
...
The behavior is now that filters on "null" will match rows with no
values. The behavior in the past was inconsistent; sometimes these
filters would match and sometimes they wouldn't.
Adds tests for this behavior to SelectorFilterTest and
BoundFilterTest, for query-level filters and filtered aggregates.
Fixes #2750 .
2016-03-29 15:32:13 -07:00
Parag Jain
d892918a3d
handle null values in In Filter
2016-03-29 17:03:26 -05:00
Fangjin Yang
e023df2b92
Merge pull request #2754 from gianm/i-dont-get-it
...
Remove error suppression code from IncrementalIndexAdapter.
2016-03-28 19:29:53 -07:00
Gian Merlino
c7ff0d698e
Remove error suppression code from IncrementalIndexAdapter.
2016-03-28 18:40:27 -07:00
fjy
c418a55638
cleanup distinct count agg
2016-03-28 17:29:41 -07:00
Fangjin Yang
9cb197adec
Merge pull request #2722 from himanshug/fix_hadoop_jar_upload
...
config to explicitly specify classpath for hadoop container during hadoop ingestion
2016-03-28 14:49:03 -07:00
Charles Allen
4a98c4fbac
Fix LookupExtractionFn equals and hashCode
2016-03-28 13:14:43 -07:00
Charles Allen
0ee861d0da
Add ExtractionFn to LookupExtractor bridge
2016-03-28 13:14:43 -07:00
Fangjin Yang
7fe277e6da
Merge pull request #2727 from gianm/optimize-bound-filter
...
BoundFilter optimizations, and related interface changes.
2016-03-26 18:59:05 -07:00
Fangjin Yang
0dae28b6af
Merge pull request #2729 from jon-wei/fix_hyperunique_comparator
...
Fix HyperUniquesAggregatorFactory comparator
2016-03-26 15:39:35 -07:00
Gian Merlino
2970b49adc
BoundFilter optimizations, and related interface changes.
...
BoundFilter:
- For lexicographic bounds, use bitmapIndex.getIndex to find the start and end points,
then union all bitmaps between those points.
- For alphanumeric bounds, iterate through dimValues, and union all bitmaps for values
matching the predicate.
- Change behavior for nulls: it used to be that the BoundFilter would never match nulls,
now it matches nulls if "" is allowed by the lower limit and not excluded by the
upper limit.
Interface changes:
- BitmapIndex: add `int getIndex(value)` to make it possible to get the index for a
value without retrieving the bitmap.
- BitmapIndex: remove `ImmutableBitmap getBitmap(value)`, change callers to `getBitmap(getIndex(value))`.
- BitmapIndexSelector: allow retrieving the underlying BitmapIndex through getBitmapIndex.
- Clarified contract of indexOf in Indexed, GenericIndexed.
Also added tests for SelectorFilter, NotFilter, and BoundFilter.
2016-03-25 14:11:48 -07:00
jon-wei
9afaa2b94a
Fix HyperUniquesAggregatorFactory comparator
2016-03-25 12:36:42 -07:00
Gian Merlino
4ac9e03161
Fix predicate-based ValueMatcher behavior for IncrementalIndex on missing columns.
...
Missing columns should be treated the same as columns containing 100% nulls.
2016-03-25 10:23:59 -07:00
Himanshu Gupta
e78a469fb7
UTs for ExtensionsConfig
2016-03-25 10:51:28 -05:00
Himanshu Gupta
004b00bb96
config to explicitly specify classpath for hadoop container during hadoop ingestion
2016-03-25 10:51:28 -05:00
Nishant
0b03c9405f
Merge pull request #2614 from sirpkt/calendric_gran
...
Support week, month, quarter, and year in query granularity
2016-03-24 16:21:01 -07:00
Himanshu
56343c6cdc
Merge pull request #2704 from navis/simple-optimize
...
optimize single elemented and/or filter
2016-03-24 16:13:48 -05:00
Gian Merlino
713062053c
Filters: Add filter.toFilter method, use that instead of the instanceof chain in Filters.
...
I believe that the instanceof chain in Filters exists because in the past, Filter
and DimFilter were in different packages (DimFilter was in druid-client and Filter
was in druid-processing). And since druid-client didn't depend on druid-processing,
DimFilter couldn't have a toFilter method. But now it can.
2016-03-23 17:03:49 -07:00
Gian Merlino
dd86198902
All Filters should work with FilteredAggregators.
...
This removes Filter.makeMatcher(ColumnSelectorFactory) and adds a
ValueMatcherFactory implementation to FilteredAggregatorFactory so it can
take advantage of existing makeMatcher(ValueMatcherFactory) implementations.
This patch also removes the Bound-based method from ValueMatcherFactory. Its
only user was the SpatialFilter, which could use the Predicate-based method.
Fixes #2604 .
2016-03-23 12:24:01 -07:00
binlijin
57d78d3293
clean tmp file when index merge fail
2016-03-23 10:55:12 +08:00
navis.ryu
91f6be4884
optimize single elemented and/or filter
2016-03-23 09:29:15 +09:00
Gian Merlino
ff25325f3b
Improved docs for multi-value dimensions.
...
- Add central doc for multi-value dimensions, with some content from other docs.
- Link to multi-value dimension doc from topN and groupBy docs.
- Fixes a broken link from dimensionspecs.md, which was presciently already
linking to this nonexistent doc.
- Resolve inconsistent naming in docs & code (sometimes "multi-valued", sometimes
"multi-value") in favor of "multi-value".
2016-03-22 14:40:55 -07:00
jon-wei
a59c9ee1b1
Support use of DimensionSchema class in DimensionsSpec
2016-03-21 13:12:04 -07:00
Keuntae Park
7f29f2ac3b
support week, month, quarter, year in query granularity
2016-03-21 17:41:53 +09:00
Charles Allen
5da9a280b6
Query Time Lookup - Dynamic Configuration
2016-03-18 09:45:05 -07:00
Gian Merlino
738dcd8cd9
Update version to 0.9.1-SNAPSHOT.
...
Fixes #2462
2016-03-17 10:34:20 -07:00
Slim
cf342d8d3c
Merge pull request #2517 from b-slim/adding_lookup_snapshot_utility
...
[QTL][Lookup] lookup module with the snapshot utility
2016-03-17 11:39:47 -05:00
Slim Bouguerra
0c86b29ef0
lookup module with the snapshot utility
2016-03-17 09:20:41 -05:00
Charles Allen
2ac8a22173
Merge pull request #2579 from metamx/closerIsCloser
...
Make CloserRule use guava's Closer
2016-03-14 17:18:19 -07:00
Charles Allen
a64979463f
Make CloserRule use guava's Closer
2016-03-14 15:01:24 -07:00
Fangjin Yang
06813b510a
Merge pull request #2571 from himanshug/gp_by_avoid_sort
...
avoid sort while doing groupBy merging when possible
2016-03-14 14:46:51 -07:00
Fangjin Yang
dbdbacaa18
Merge pull request #2260 from navis/cardinality-for-searchquery
...
Support cardinality for search query
2016-03-14 13:24:40 -07:00
Slim
8cc3582e70
Merge pull request #2644 from metamx/optimize-timeboundary
...
optimize timeboundary for min or max bound
2016-03-13 13:16:24 -05:00
navis.ryu
be341bf4e3
Support cardinality for search query (Fix for #2260 )
2016-03-12 09:51:01 +09:00
Xavier Léauté
6f0d6ef0e9
optimize timeboundary for min or max bound
2016-03-11 14:11:47 -08:00
Gian Merlino
8a11161b20
Plumbers: Move plumber.add out of try/catch for ParseException.
...
The incremental indexes handle that now so it's not necessary.
Also, add debug logging and more detailed exceptions to the incremental
indexes for the case where there are parse exceptions during aggregation.
2016-03-10 16:39:26 -08:00
Himanshu Gupta
dc0214bddb
while GroupBy merging use unsorted facts in IncrementalIndex wherever possible
2016-03-10 16:11:48 -06:00
Himanshu Gupta
02dfd5cd80
update IncrementalIndex to support unsorted facts map that can be used in groupBy merging to improve performance
2016-03-10 16:11:48 -06:00
Xavier Léauté
90d7409e1a
Merge pull request #2611 from himanshug/gp_by_max_limit
...
only allow lowering maxResults and maxIntermediateRows from groupBy query context
2016-03-10 13:44:13 -08:00
Gian Merlino
a2b1652787
Clarify parser docs.
...
- Clarify what parseSpecs are used for.
- Avro, Protobuf should use timeAndDims parseSpecs.
- Hadoop jobs should use hadoopyString string parsers.
2016-03-10 08:45:04 -08:00
Fangjin Yang
68cffe1d91
Merge pull request #2615 from gianm/timeseries-skipEmptyBuckets-cache
...
Fix caching of skipEmptyBuckets for TimeseriesQuery.
2016-03-09 18:45:59 -08:00
Gian Merlino
708bc674fa
Make specifying query context booleans more consistent.
...
Before, some needed to be strings and some needed to be real booleans. Now
they can all be either one.
2016-03-08 19:38:26 -08:00
Gian Merlino
40dad6dff4
Fix caching of skipEmptyBuckets for TimeseriesQuery.
2016-03-08 19:22:12 -08:00
Himanshu Gupta
ca5de3f583
only allow lowering maxResults and maxIntermediateRows from groupBy query context
2016-03-08 15:03:59 -06:00
Himanshu Gupta
099acb4966
allow groupBy max[Intermediate]Rows limit be overridable by context
2016-03-07 15:22:41 -06:00
Himanshu Gupta
c544ebf25e
reintroducing the safety check removed in commit-1d602be so that dim value ids are less than cardinality
2016-03-03 23:34:23 -06:00
Bingkun Guo
4a58462fc7
update querySegmentSpec when passing query to getQueryRunner
...
After finding the FireChief for a specific partition, Druid will need to find the specific queryRunner for each segment being queried by passing the query to FireChief. Currently Druid is passing the original query that contains all the segments need to be queried, it's possible that fireChief.getQueryRunner(query) returns more than 1 queryRunner because query.getIntervals() is not specific to a single segment.
In this patch, for each segment being queried, Druid will update the query with its corresponding SpecificSegmentSpec.
2016-03-02 16:44:56 -06:00
Nishant
31b502773a
Merge pull request #2480 from navis/pagingfail-over-segments
...
Select query cannot span to next segment with paging
2016-03-01 11:42:41 +05:30
Fangjin Yang
e5c25725c0
Merge pull request #2562 from himanshug/fix_2556
...
with nested GpBy query outer query results need to be further merged
2016-02-29 12:17:33 -08:00
Himanshu Gupta
0722ced413
with GpBy query outer query results need to be further merged
2016-02-29 10:16:25 -06:00
navis.ryu
b1ff920831
Lazily initialize predicate for bound filter
2016-02-29 15:35:52 +09:00
navis.ryu
5f1e60324a
Added more complex test case with versioned segments
2016-02-29 14:48:24 +09:00
navis.ryu
2686bfa394
Select query cannot span to next segment with paging
2016-02-29 00:01:46 +09:00
Fangjin Yang
29d29ba98d
Merge pull request #2263 from jon-wei/flex_dims3
...
Allow IncrementalIndex to store Long/Float dimensions
2016-02-25 17:23:02 -08:00
jon-wei
c17ce02467
Allow IncrementalIndex to store Long/Float dimensions
2016-02-24 13:51:57 -08:00
jon-wei
fd3782522c
Rename 'replaceMissingValues...' parameters in RegexExtractionFn
2016-02-24 13:12:56 -08:00
Nishant
fb7eae34ed
Merge pull request #2249 from metamx/workerExpanded
...
Use Worker instead of ZkWorker whenever possible
2016-02-24 13:23:22 +05:30
Charles Allen
ac13a5942a
Use Worker instead of ZkWorker whenver possible
...
* Moves last run task state information to Worker
* Makes WorkerTaskRunner a TaskRunner which has interfaces to help with getting information about a Worker
2016-02-23 15:02:03 -08:00
Gian Merlino
3534483433
Better handling of ParseExceptions.
...
Two changes:
- Allow IncrementalIndex to suppress ParseExceptions on "aggregate".
- Add "reportParseExceptions" option to realtime tuning configs. By default this is "false".
Behavior of the counters should now be:
- processed: Number of rows indexed, including rows where some fields could be parsed and some could not.
- thrownAway: Number of rows thrown away due to rejection policy.
- unparseable: Number of rows thrown away due to being completely unparseable (no fields salvageable at all).
If "reportParseExceptions" is true then "unparseable" will always be zero (because a parse error would
cause an exception to be thrown). In addition, "processed" will only include fully parseable rows
(because even partial parse failures will cause exceptions to be thrown).
Fixes #2510 .
2016-02-23 10:11:43 -08:00
Fangjin Yang
3bdd757024
Merge pull request #1773 from b-slim/log_details
...
Adding downstream source when throwing QueryInterruptedException
2016-02-22 10:16:07 -08:00
Slim Bouguerra
77925cc061
adding downstream source of QueryInterruptedException
2016-02-20 13:05:14 -06:00
Fangjin Yang
8ee81947cd
Merge pull request #2494 from himanshug/fix_timeseries
...
do not drop post-aggs in TimeseriesQueryToolChest.makePreComputeManipulatorFn
2016-02-20 10:37:32 -08:00
Gian Merlino
d25c46cb9f
Add comparator to HyperUniquesFinalizingPostAggregator.
...
This makes it possible to do groupBys with clauses like "HAVING uniques > 10".
Beforehand you couldn't do it with either an aggregator (because it returns
an HLLV1 which the havingSpec can't understand) or a finalized postaggregator
(because it didn't have a comparator).
Now you can at least do it with a finalizing postaggregator. Trying it with
the aggregator alone still doesn't work.
Added some topN and groupBy tests verifying the comparator, and added an
@Ignore test that should pass if havingSpecs are made work on the aggregator
directly.
2016-02-19 08:36:08 -08:00
Himanshu Gupta
11b0117422
do not drop post-aggs in timeseries query tool chest makePreComputeManipulatorFn like other query types
2016-02-17 20:51:35 -06:00
Jaehong Choi
32b9d57b23
handle a failing UT in GroupByQueryRunnerTest after merging into the master
2016-02-16 16:56:57 +09:00
Jaehong Choi
b25bca85bc
Merge branch 'master' of https://github.com/druid-io/druid into support-alphanumeric-dimensional-sort-in-gropu-by
2016-02-16 16:42:05 +09:00
Jaehong Choi
e89afc901b
delete System.out.println() in test code
2016-02-16 15:26:37 +09:00
Navis Ryu
cd315627c9
Merge pull request #2393 from CHOIJAEHONG1/support-alphanumeric-dimensional-sort-in-gropu-by
...
support alphanumeric sorting for dimensional columns in groupby (#2393 )
2016-02-16 14:11:30 +09:00
Slim
16092eb5e2
Merge pull request #2464 from gianm/print-properties
...
Make startup properties logging optional.
2016-02-14 15:11:35 -06:00
Gian Merlino
e0c049c0b0
Make startup properties logging optional.
...
Off by default, but enabled in the example config files. See also #2452 .
2016-02-12 14:12:16 -08:00
Himanshu Gupta
da5fcd0124
before facts get it , indexAndOffsets should already know about it
2016-02-12 13:32:06 -06:00
Jonathan Wei
d63eec65a1
Merge pull request #2208 from navis/metadataquery-minmax
...
Support min/max values for metadata query
2016-02-11 17:28:07 -08:00
Jonathan Wei
e1b022eac9
Merge pull request #2349 from navis/dimensionspec-for-selectquery
...
Support dimension spec for select query
2016-02-11 16:38:16 -08:00
navis.ryu
dd2375477a
Support min/max values for metadata query ( #2208 )
2016-02-12 09:35:58 +09:00
Gian Merlino
2d037ef05e
Merge pull request #2453 from DreamLab/fix/topn_sorting_anomaly
...
Fix for unstable behavior of HyperLogLog comparator
2016-02-11 16:05:34 -08:00
navis.ryu
4d63196535
Support dimension spec for select query
2016-02-12 08:54:28 +09:00
Himanshu
47d48e1e67
Merge pull request #2452 from gianm/print-properties
...
PropertiesModule: Print properties, processors, totalMemory on startup.
2016-02-11 16:49:34 -06:00
turu
f277a54a5c
removed unsafe heuristics from hll compareTo and provided unit test for regression
2016-02-11 23:46:24 +01:00
Slim
368988d187
Merge pull request #2291 from druid-io/lookupManager
...
Promoting LookupExtractor state and LookupExtractorFactory to be a first class druid state object.
2016-02-11 16:07:27 -06:00
Gian Merlino
29f7758e74
PropertiesModule: Print properties, processors, totalMemory on startup.
2016-02-11 13:51:08 -08:00
Slim Bouguerra
4e119b7a24
Adding lookup ref manager and lookup dimension spec impl
2016-02-11 12:11:51 -06:00
Jaehong Choi
2f2e2ff5b9
support alphanumeric sorting for dimensional columns in groupby
2016-02-11 17:31:28 +09:00
Keuntae Park
05a144e39a
fix crash with filtered aggregator at ingestion time
...
- only for selector filter because extraction filter is not supported as
cardinality is not fixed at ingestion time
2016-02-11 11:25:33 +09:00
Fangjin Yang
b1673ee90e
Merge pull request #2409 from gianm/smq-merged-thing
...
SegmentMetadataQuery: Retain segment id when merging, if possible.
2016-02-08 15:43:39 -08:00
Fangjin Yang
c9c20bb7f3
Merge pull request #2395 from metamx/fixExtractionDimFilterNullTest
...
Actually check cache key null checking in ExtractionDimFilterTest
2016-02-08 14:10:52 -08:00
Gian Merlino
bd9c04244f
SegmentMetadataQuery: Retain segment id when merging, if possible.
...
This is helpful on realtime nodes, where two analyses from two different hydrants
are merged together but they are actually from the same segment.
2016-02-08 13:07:02 -08:00
Himanshu Gupta
9fe1b28ee5
provide configuration to enable usage of Off heap merging for groupBy query
2016-02-05 14:18:06 -06:00
Himanshu Gupta
b40c342cd1
make Global stupid pool cache size configurable
2016-02-05 14:18:06 -06:00
Himanshu Gupta
72a1e730a2
OffheapIncrementalIndex updates to do the aggregation merging off-heap
2016-02-05 14:17:05 -06:00
Himanshu Gupta
907dd77483
OffheapIncrementalIndex a copy/paste of OnheapIncrementalIndex
2016-02-05 14:02:31 -06:00
Charles Allen
aac5f9b2c9
Actually check cache key null checking in ExtractionDimFilterTest
2016-02-04 09:44:13 -08:00