navis.ryu
b1ff920831
Lazily initialize predicate for bound filter
2016-02-29 15:35:52 +09:00
navis.ryu
5f1e60324a
Added more complex test case with versioned segments
2016-02-29 14:48:24 +09:00
navis.ryu
2686bfa394
Select query cannot span to next segment with paging
2016-02-29 00:01:46 +09:00
Fangjin Yang
29d29ba98d
Merge pull request #2263 from jon-wei/flex_dims3
...
Allow IncrementalIndex to store Long/Float dimensions
2016-02-25 17:23:02 -08:00
jon-wei
c17ce02467
Allow IncrementalIndex to store Long/Float dimensions
2016-02-24 13:51:57 -08:00
jon-wei
fd3782522c
Rename 'replaceMissingValues...' parameters in RegexExtractionFn
2016-02-24 13:12:56 -08:00
Nishant
fb7eae34ed
Merge pull request #2249 from metamx/workerExpanded
...
Use Worker instead of ZkWorker whenever possible
2016-02-24 13:23:22 +05:30
Charles Allen
ac13a5942a
Use Worker instead of ZkWorker whenver possible
...
* Moves last run task state information to Worker
* Makes WorkerTaskRunner a TaskRunner which has interfaces to help with getting information about a Worker
2016-02-23 15:02:03 -08:00
Gian Merlino
3534483433
Better handling of ParseExceptions.
...
Two changes:
- Allow IncrementalIndex to suppress ParseExceptions on "aggregate".
- Add "reportParseExceptions" option to realtime tuning configs. By default this is "false".
Behavior of the counters should now be:
- processed: Number of rows indexed, including rows where some fields could be parsed and some could not.
- thrownAway: Number of rows thrown away due to rejection policy.
- unparseable: Number of rows thrown away due to being completely unparseable (no fields salvageable at all).
If "reportParseExceptions" is true then "unparseable" will always be zero (because a parse error would
cause an exception to be thrown). In addition, "processed" will only include fully parseable rows
(because even partial parse failures will cause exceptions to be thrown).
Fixes #2510 .
2016-02-23 10:11:43 -08:00
Fangjin Yang
3bdd757024
Merge pull request #1773 from b-slim/log_details
...
Adding downstream source when throwing QueryInterruptedException
2016-02-22 10:16:07 -08:00
Slim Bouguerra
77925cc061
adding downstream source of QueryInterruptedException
2016-02-20 13:05:14 -06:00
Fangjin Yang
8ee81947cd
Merge pull request #2494 from himanshug/fix_timeseries
...
do not drop post-aggs in TimeseriesQueryToolChest.makePreComputeManipulatorFn
2016-02-20 10:37:32 -08:00
Gian Merlino
d25c46cb9f
Add comparator to HyperUniquesFinalizingPostAggregator.
...
This makes it possible to do groupBys with clauses like "HAVING uniques > 10".
Beforehand you couldn't do it with either an aggregator (because it returns
an HLLV1 which the havingSpec can't understand) or a finalized postaggregator
(because it didn't have a comparator).
Now you can at least do it with a finalizing postaggregator. Trying it with
the aggregator alone still doesn't work.
Added some topN and groupBy tests verifying the comparator, and added an
@Ignore test that should pass if havingSpecs are made work on the aggregator
directly.
2016-02-19 08:36:08 -08:00
Himanshu Gupta
11b0117422
do not drop post-aggs in timeseries query tool chest makePreComputeManipulatorFn like other query types
2016-02-17 20:51:35 -06:00
Jaehong Choi
32b9d57b23
handle a failing UT in GroupByQueryRunnerTest after merging into the master
2016-02-16 16:56:57 +09:00
Jaehong Choi
b25bca85bc
Merge branch 'master' of https://github.com/druid-io/druid into support-alphanumeric-dimensional-sort-in-gropu-by
2016-02-16 16:42:05 +09:00
Jaehong Choi
e89afc901b
delete System.out.println() in test code
2016-02-16 15:26:37 +09:00
Navis Ryu
cd315627c9
Merge pull request #2393 from CHOIJAEHONG1/support-alphanumeric-dimensional-sort-in-gropu-by
...
support alphanumeric sorting for dimensional columns in groupby (#2393 )
2016-02-16 14:11:30 +09:00
Slim
16092eb5e2
Merge pull request #2464 from gianm/print-properties
...
Make startup properties logging optional.
2016-02-14 15:11:35 -06:00
Gian Merlino
e0c049c0b0
Make startup properties logging optional.
...
Off by default, but enabled in the example config files. See also #2452 .
2016-02-12 14:12:16 -08:00
Himanshu Gupta
da5fcd0124
before facts get it , indexAndOffsets should already know about it
2016-02-12 13:32:06 -06:00
Jonathan Wei
d63eec65a1
Merge pull request #2208 from navis/metadataquery-minmax
...
Support min/max values for metadata query
2016-02-11 17:28:07 -08:00
Jonathan Wei
e1b022eac9
Merge pull request #2349 from navis/dimensionspec-for-selectquery
...
Support dimension spec for select query
2016-02-11 16:38:16 -08:00
navis.ryu
dd2375477a
Support min/max values for metadata query ( #2208 )
2016-02-12 09:35:58 +09:00
Gian Merlino
2d037ef05e
Merge pull request #2453 from DreamLab/fix/topn_sorting_anomaly
...
Fix for unstable behavior of HyperLogLog comparator
2016-02-11 16:05:34 -08:00
navis.ryu
4d63196535
Support dimension spec for select query
2016-02-12 08:54:28 +09:00
Himanshu
47d48e1e67
Merge pull request #2452 from gianm/print-properties
...
PropertiesModule: Print properties, processors, totalMemory on startup.
2016-02-11 16:49:34 -06:00
turu
f277a54a5c
removed unsafe heuristics from hll compareTo and provided unit test for regression
2016-02-11 23:46:24 +01:00
Slim
368988d187
Merge pull request #2291 from druid-io/lookupManager
...
Promoting LookupExtractor state and LookupExtractorFactory to be a first class druid state object.
2016-02-11 16:07:27 -06:00
Gian Merlino
29f7758e74
PropertiesModule: Print properties, processors, totalMemory on startup.
2016-02-11 13:51:08 -08:00
Slim Bouguerra
4e119b7a24
Adding lookup ref manager and lookup dimension spec impl
2016-02-11 12:11:51 -06:00
Jaehong Choi
2f2e2ff5b9
support alphanumeric sorting for dimensional columns in groupby
2016-02-11 17:31:28 +09:00
Keuntae Park
05a144e39a
fix crash with filtered aggregator at ingestion time
...
- only for selector filter because extraction filter is not supported as
cardinality is not fixed at ingestion time
2016-02-11 11:25:33 +09:00
Fangjin Yang
b1673ee90e
Merge pull request #2409 from gianm/smq-merged-thing
...
SegmentMetadataQuery: Retain segment id when merging, if possible.
2016-02-08 15:43:39 -08:00
Fangjin Yang
c9c20bb7f3
Merge pull request #2395 from metamx/fixExtractionDimFilterNullTest
...
Actually check cache key null checking in ExtractionDimFilterTest
2016-02-08 14:10:52 -08:00
Gian Merlino
bd9c04244f
SegmentMetadataQuery: Retain segment id when merging, if possible.
...
This is helpful on realtime nodes, where two analyses from two different hydrants
are merged together but they are actually from the same segment.
2016-02-08 13:07:02 -08:00
Himanshu Gupta
9fe1b28ee5
provide configuration to enable usage of Off heap merging for groupBy query
2016-02-05 14:18:06 -06:00
Himanshu Gupta
b40c342cd1
make Global stupid pool cache size configurable
2016-02-05 14:18:06 -06:00
Himanshu Gupta
72a1e730a2
OffheapIncrementalIndex updates to do the aggregation merging off-heap
2016-02-05 14:17:05 -06:00
Himanshu Gupta
907dd77483
OffheapIncrementalIndex a copy/paste of OnheapIncrementalIndex
2016-02-05 14:02:31 -06:00
Charles Allen
aac5f9b2c9
Actually check cache key null checking in ExtractionDimFilterTest
2016-02-04 09:44:13 -08:00
fjy
1aa363cea7
new quickstart
2016-02-04 09:37:38 -08:00
Fangjin Yang
da77591129
Merge pull request #2392 from metamx/fix2391
...
Allow ExtractionDimFilter value to be null
2016-02-03 17:47:14 -08:00
Charles Allen
d4f00096ff
Allow ExtractionDimFilter value to be null
...
* Fixes #2391
2016-02-03 15:51:47 -08:00
Himanshu Gupta
6e7d90cf56
UTs for DefaultLimitSpec
2016-02-03 15:59:12 -06:00
Himanshu Gupta
29e0d7f971
lazily create comparators for row columns when needed
2016-02-03 13:38:20 -06:00
navis.ryu
1d602be0f9
Replace string[] with int[] for dimensions
2016-02-03 15:03:22 +09:00
binlijin
a5ef30ff84
optimize topn on particular situation
2016-02-02 14:20:09 +08:00
Himanshu
93c50d8538
Merge pull request #2094 from navis/simplify-index-merge
...
Simplifying dimension merging
2016-01-29 11:23:14 -06:00
navis.ryu
55a888ea2f
time-descending result of select queries
2016-01-29 10:06:05 +09:00
navis.ryu
dd774ef4dd
one-pass merging of dictionary & index
2016-01-29 10:03:53 +09:00
Himanshu
edd7ce58aa
Merge pull request #2348 from AlexanderSaydakov/fix-aggregator-test-helper
...
fixed createIndex
2016-01-28 16:01:36 -06:00
saydakov
e0860661b1
fixed createIndex
2016-01-28 13:20:50 -08:00
Nishant
99017f4518
Merge pull request #2326 from navis/use-reverse-iterator
...
use reverse-iterator if possible
2016-01-28 19:48:38 +05:30
Nishant
3880f54b87
Merge pull request #2332 from himanshug/configurable_partial
...
make populateUncoveredIntervals a configuration in query context
2016-01-28 10:34:35 +05:30
navis.ryu
7324ece8f9
use reverse-iterator if possible
2016-01-28 09:04:55 +09:00
Xavier Léauté
5a3642bb93
Merge pull request #2247 from metamx/pedanticBuild
...
Enable strict building in travis
2016-01-27 10:27:03 -08:00
Xavier Léauté
2e5004095a
Merge pull request #2341 from gianm/smq-test
...
SegmentMetadataQuery: Fix merging of ColumnAnalysis errors.
2016-01-27 09:37:06 -08:00
Charles Allen
508734c8b0
Long constant reformatting in tests `l` --> `L`
2016-01-27 08:59:19 -08:00
Gian Merlino
b1e6c01762
Make LookupExtractor abstract methods public, they have to work across classloaders.
2016-01-26 23:08:03 -08:00
Gian Merlino
795343f7ef
SegmentMetadataQuery: Fix merging of ColumnAnalysis errors.
...
Also add tests for:
- ColumnAnalysis folding
- Mixed mmap/incremental merging
2016-01-26 17:16:26 -08:00
Himanshu Gupta
3719b6e3c8
make populateUncoveredIntervals a configuration in query context
2016-01-26 15:13:45 -06:00
Himanshu
3844658fb5
Merge pull request #2323 from druid-io/update-druidapi
...
Update druid-api to 0.3.16
2016-01-26 13:02:10 -06:00
Himanshu Gupta
09d3678667
adding single threaded indexing and querying test for IncrementalIndex
2016-01-23 00:17:14 -06:00
Charles Allen
0000b9fc62
Remove sorting in ProtoBufInputRowParserTest
...
Due to processing/src/test/java/io/druid/data/input/ProtoBufInputRowParserTest.java
2016-01-22 16:02:25 -08:00
Himanshu Gupta
2f7f5119cf
older segments might not have field bitmapSerdeFactory for dimension columns and we must use appropriate default
2016-01-22 13:28:25 -06:00
binlijin
1d1f4d996d
Merge pull request #2111 from binlijin/optimize-create-inverted-indexes
...
optimize create inverted indexes
2016-01-22 11:36:27 +08:00
binlijin
55f7dd4629
optimize create inverted indexes
2016-01-22 10:40:09 +08:00
Gian Merlino
d416279c14
SegmentMetadataQuery support for returning aggregators.
2016-01-21 17:27:25 -08:00
Fangjin Yang
5a9cd89059
Merge pull request #2305 from gianm/segment-metadata-query-multivalues
...
Add StorageAdapter#getColumnTypeName, and various SegmentMetadataQuery adjustments
2016-01-21 17:22:34 -08:00
Gian Merlino
e5913be90e
Merge pull request #2257 from tubemogul/index-merge-bug
...
Adds support for empty merge metrics. fixes #2256
2016-01-21 16:38:00 -08:00
Gian Merlino
87c8046c6c
Add StorageAdapter#getColumnTypeName, and various SegmentMetadataQuery adjustments.
...
SegmentMetadataQuery stuff:
- Simplify implementation of SegmentAnalyzer.
- Fix type names for realtime complex columns; this used to try to merge a nice type
name (like "hyperUnique") from mmapped segments with the word "COMPLEX" from incremental
index segments, leading to a merge failure. Now it always uses the nice name.
- Add hasMultipleValues to ColumnAnalysis.
- Add tests for both mmapped and incremental index segments.
- Update docs to include errorMessage.
2016-01-21 15:50:33 -08:00
Fangjin Yang
3f998117a6
Merge pull request #2306 from jon-wei/inherit2
...
More specific null/empty str handling in IndexMerger
2016-01-21 14:36:09 -08:00
Michael Schiff
1e44445f06
Adds support for empty merge metrics. fixes #2256
2016-01-21 13:21:37 -08:00
jon-wei
459a236067
More specific null/empty str handling in IndexMerger
2016-01-21 12:24:38 -08:00
Slim
201539260c
Merge pull request #2076 from b-slim/issue_2010_upper_lower_extractionFN
...
adding lower and upper extraction fn
2016-01-21 09:58:07 -06:00
Slim Bouguerra
78feb3a13e
adding lower and upper extraction fn
2016-01-21 08:59:05 -06:00
Gian Merlino
5a932d28c1
Merge pull request #2288 from tubemogul/index-merge-bug2
...
Null check in IncrementalIndexAdapter.getDimValueLookup()
2016-01-20 17:07:15 -08:00
Nishant
59ea186af7
fix reference counting for segments
2016-01-20 17:24:21 +05:30
Michael Schiff
50ceec78a2
null check in IncrementalIndexAdapter.getDimValueLookup()
2016-01-19 23:19:28 -08:00
jon-wei
bc1e9b27c8
Consolidate IndexMergerTest and IndexMergerV9Test
2016-01-19 16:28:35 -08:00
jon-wei
747343e621
Preserve dimension order across indexes during ingestion
2016-01-19 13:34:11 -08:00
Fangjin Yang
0c31f007fc
Merge pull request #1728 from himanshug/aggregators_in_segment_metadata
...
Store AggregatorFactory[] in segment metadata
2016-01-19 12:55:49 -08:00
Himanshu Gupta
a99aef29a1
adding aggregators to segment metadata
2016-01-19 14:23:39 -06:00
Himanshu Gupta
52eb0f04a7
adding a new method getMergingFactory(..) to AggregatorFactory
2016-01-18 22:03:46 -06:00
Himanshu Gupta
77fc86c015
making AggregatorFactory abstract class
2016-01-18 22:03:46 -06:00
Himanshu Gupta
164b0aad7a
removing Map<String,Object> segmentMetadata from methods in Index[Maker/Merger] and using Metadata class
...
instead of a Map to store segment metadata
2016-01-18 22:03:46 -06:00
zhxiaog
3459a202ce
fixed #1873 , add ability to express CONCAT as an extractionFn
2016-01-18 15:03:17 -08:00
Keuntae Park
238dd3be3c
support cascade execution of extraction filters in extraction dimension spec
2016-01-18 11:10:19 +09:00
Fangjin Yang
f6a1a4ae20
Merge pull request #2138 from KurtYoung/feature-build-v9
...
build v9 directly
2016-01-16 13:35:46 -06:00
Kurt Young
82ff98c2bf
add config for build v9 directly and update docs
2016-01-16 11:26:34 +08:00
Kurt Young
1f2168fae5
add IndexMergerV9
...
add unit tests for IndexMergerV9 and fix some bugs
add more unit tests and fix bugs
handle null values and add more tests
minor changes & use LoggingProgressIndicator in IndexGeneratorReducer
make some static class public from IndexMerger
minor changes and add some comments
changes for comments
2016-01-16 11:25:28 +08:00
Kurt Young
bb50d2a2b2
add some streaming writers
2016-01-16 11:25:26 +08:00
Fangjin Yang
e0932ba1c2
Merge pull request #2267 from himanshug/fix_topn_multi_val_filter
...
Remap id's returned in XXXFilteredDimensionSpec.getRow() as per reduced cardinality
2016-01-14 17:06:54 -08:00
Fangjin Yang
7704699b40
Merge pull request #2265 from navis/strlen-dimension-ignored
...
Strlen sort spec ignores dimension
2016-01-14 17:06:33 -08:00
Himanshu Gupta
ae6a111444
fix XXXFilteredDimensionSpec to remap the dictionary encodings as per new cardinality
2016-01-13 22:25:02 -06:00
binlijin
a3140b2548
fix topN filtering on multi-valued dimension bug
2016-01-13 22:25:02 -06:00
navis.ryu
ea9fabdf2f
Strlen sort spec ignores dimension
2016-01-14 11:05:44 +09:00
Fangjin Yang
4c014c1574
Merge pull request #2228 from metamx/incremental-index-mem2
...
Improve heap usage for IncrementalIndex
2016-01-13 14:48:03 -08:00
navis.ryu
18479bb757
time-descending result of timeseries queries
2016-01-13 12:23:01 +09:00
Fangjin Yang
d7ad93debc
Merge pull request #2221 from binlijin/topN_minTopNThreshold
...
Allow change minTopNThreshold per topN query
2016-01-12 16:22:20 -08:00
Nishant
4863e2ca4f
cache metric selectors instead of creating new ones for every metric in each row
...
clear selectors on close.
Add comments about thread safety.
2016-01-13 00:45:23 +05:30
Nishant
dfe6abb721
Merge pull request #2250 from himanshug/agg_test_helper_fix
...
remove redundant registering of json modules in AggregationTestHelper
2016-01-12 11:42:00 +05:30
navis.ryu
976ebc45c0
Simplify information in IncrementalIndex
2016-01-12 10:18:11 +09:00
Himanshu Gupta
b973604bf8
remove redundant registering of json modules in AggregationTestHelper
2016-01-11 19:03:22 -06:00
Xavier Léauté
46a7f2660d
fix casing to be consistent with other classes
2016-01-08 10:19:06 -08:00
Fangjin Yang
d0b10c29d7
Merge pull request #2197 from metamx/clearIncIndexClose
...
Make OnHeapIncrementalIndex clean maps on close()
2016-01-07 15:43:47 -08:00
Gian Merlino
4ecd901a1a
Merge pull request #2219 from himanshug/identity_extraction_fn_singleton
...
make IdentityExtractionFn singleton
2016-01-07 10:08:28 -08:00
Fangjin Yang
aaea95ed1b
Merge pull request #2207 from himanshug/theta_sketch_select_query
...
fix bug for thetaSketch metric not working with select queries
2016-01-07 09:46:09 -08:00
binlijin
010c6e959c
add test
2016-01-07 18:01:46 +08:00
binlijin
a6bfcc5bfd
Allow change minTopNThreshold per topN query
2016-01-07 14:51:00 +08:00
Fangjin Yang
4cc81d3eff
Merge pull request #2096 from b-slim/add_use_case_unapply
...
Add use case unapply
2016-01-06 21:58:12 -08:00
Himanshu Gupta
217079d0c7
make IdentityExtractionFn singleton
2016-01-06 22:29:07 -06:00
Himanshu
902f51433d
Merge pull request #2125 from mangeshpardeshiyahoo/master
...
Add extraction function support for Dimension Selector
2016-01-06 14:22:26 -06:00
Mangesh Pardeshi
75ee952197
Add extraction function support for dimension Selector
2016-01-06 13:47:07 -06:00
Slim Bouguerra
032d3bf6e6
Optimization of extraction filter by reversing the lookup
2016-01-06 11:16:11 -06:00
Himanshu Gupta
3f048f0b15
adding support to execute Select queries in AggregationTestHelper so that Select query based UTs can be written for complex aggregator implementations
2016-01-05 21:54:55 -06:00
Charles Allen
91fc32749b
Make OnHeapIncrementalIndex clean maps on close()
2016-01-04 11:18:16 -08:00
Himanshu Gupta
b47d807738
Add support for filtering at DimensionSpec level so that multivalued dimensions can be filtered correctly
...
also adding UTs for multi-valued dimensions
2015-12-30 17:59:47 -06:00
Himanshu Gupta
fa5c3bb014
adding decorate(DimensionSelector) to DimensionSpec to enable support for arbitrary filtering/transformations to returned dimension values
2015-12-30 15:06:24 -06:00
Nishant
b68265399c
Merge pull request #2168 from druid-io/remove-indexmaker
...
Remove IndexMaker
2015-12-30 12:24:29 +05:30
Fangjin Yang
e14ad74088
Merge pull request #1936 from b-slim/between_range_with_predicat
...
adding Upper/Lower Bound Filter
2015-12-29 10:11:22 -08:00
fjy
faf421726b
remove IndexMaker
2015-12-28 14:19:02 -08:00
Gian Merlino
83f4130b5f
SegmentMetadataQuery merging fixes.
...
- Fix merging when the INTERVALS analysisType is disabled, and add a test.
- Remove transformFn from CombiningSequence, use MappingSequence instead. transformFn did
not work for "accumulate" anyway, which made the tests wrong (the intervals should have
been condensed, but were not).
- Add analysisTypes to the Druids segmentMetadataQuery builder to make testing simpler.
2015-12-22 07:57:10 -08:00
Robin
dded4441d3
for completeness, add unit test for groupby/having with unrecognized type
2015-12-21 12:06:56 -06:00
Himanshu Gupta
e1631967e3
adding comments to explain merge failure in segmentMetadata query
2015-12-19 11:39:24 -06:00
Himanshu Gupta
7ecad1be24
Fix and UT for testing segment analysis merge
2015-12-19 00:24:02 -06:00
Fangjin Yang
7019d3c421
Merge pull request #2107 from jon-wei/fix_smq
...
More efficient SegmentMetadataQuery
2015-12-18 16:40:47 -08:00
Fangjin Yang
14229ba0f2
Merge pull request #1922 from metamx/jsonIgnoresFinalFields
...
Change DefaultObjectMapper to NOT overwrite final fields unless explicitly asked to
2015-12-18 15:38:32 -08:00
Fangjin Yang
71f554bf80
Merge pull request #2101 from himanshug/fix_extraction_dim_filter_cache_key
...
add extractionFn bytes to cache key in ExtractionDimFilter
2015-12-18 12:05:43 -08:00
Fangjin Yang
9e6874cc7e
Merge pull request #2084 from binlijin/master
...
minor optimize IndexMerger's MMappedIndexRowIterable
2015-12-18 11:42:55 -08:00
Bingkun
cc21a5fac7
Merge pull request #1999 from himanshug/remove_min_max_aggs
...
remove min/max aggregator factory
2015-12-18 13:38:52 -06:00
jon-wei
356b07c6c3
More efficient SegmentMetadataQuery
2015-12-17 12:46:23 -08:00
Jonathan Wei
f8cf84f466
Merge pull request #1995 from himanshug/num_rows_seg_metadata_query
...
add numRows to segment metadata query response
2015-12-17 12:23:46 -08:00
Himanshu Gupta
82ea348003
add extractionFn bytes to cache key in ExtractionDimFilter
2015-12-16 14:00:38 -06:00
Himanshu
628643d80e
Merge pull request #2091 from rasahner/noDefaultForGroupbyHaving
...
take away default for groupBy/having
2015-12-16 01:07:40 -06:00
sahner
3441cf3110
take away default for groupBy/having
2015-12-15 10:32:45 -06:00
Fangjin Yang
e7f06cf61c
Merge pull request #2075 from jon-wei/regex_extract
...
Configurable value replacement on match failure for RegexExtractionFn
2015-12-14 19:10:50 -08:00
jon-wei
c88f75df7c
Configurable value replacement on match failure for RegexExtractionFn
2015-12-14 17:57:41 -08:00
binlijin
362bea1090
minor optimize IndexMerger's MMappedIndexRowIterable
2015-12-11 15:04:46 +08:00
Xavier Léauté
d531e69d1a
Merge pull request #2079 from binlijin/master
...
reduce bytearray copy to minimal optimize VSizeIndexedWriter
2015-12-10 21:30:09 -08:00
Slim Bouguerra
77afdf25e3
adding Bound Filter
2015-12-10 08:47:21 -06:00
Slim Bouguerra
ee1a39801a
adding bulk lookup and reverse lookup
2015-12-10 08:29:41 -06:00
binlijin
0eafbd55b2
reduce bytearray copy to minimal optimize VSizeIndexedWriter
2015-12-10 16:34:39 +08:00
Fangjin Yang
f4ba13a1ac
Merge pull request #2029 from b-slim/add_reverse_fn
...
Adding reverse lookup function to LookupExtractor.
2015-12-09 12:50:13 -08:00
Xavier Léauté
9015a68c03
Merge pull request #2002 from navis/DRUID-2001
...
fixed #2001 GenericIndexed.fromIterable compares all values even when it's not sorted
2015-12-09 08:56:49 -08:00
Slim Bouguerra
85f339b687
introduction and implem of reverse lookup function unApply.
2015-12-09 10:02:57 -06:00
Nishant
6c23d8edb4
Merge pull request #2043 from mangeshpardeshiyahoo/master
...
Add dimension selector support for groupby/having filters
2015-12-08 12:08:53 +05:30
Mangesh Pardeshi
d7ce120929
Add dimension selector support for groupby/having quries
2015-12-08 01:51:11 +00:00
Himanshu Gupta
431469e9c1
remove min/max aggregator factory which are replaced by double[min/max] aggregator factories
2015-12-05 22:36:49 -06:00
Himanshu Gupta
62ba9ade37
unifying license header in all java files
2015-12-05 22:16:23 -06:00
Gian Merlino
d21a640695
Merge pull request #2034 from b-slim/fix_cache_key
...
Fix getCacheKey for DimFilters
2015-12-04 09:13:06 -08:00
Slim Bouguerra
fb4ff3cf54
fix getCacheKey
2015-12-04 08:07:08 -06:00
Charles Allen
9d02f47201
Update IncrementalIndexTest copyright notice
2015-12-03 18:03:08 -08:00
Charles Allen
be8c6fafb0
Merge pull request #2017 from tubemogul/issue/63
...
fixes issue #63
2015-12-03 18:01:11 -08:00
Gian Merlino
045df54404
Merge pull request #1961 from metamx/druidMetricsVersion
...
Add the druid artifact version to metrics when emitted
2015-12-03 17:34:57 -08:00
Michael Schiff
b6cc2428e1
fixes issue #63
2015-12-03 17:30:47 -08:00
Himanshu
0eab8417cb
Merge pull request #2008 from codingwhatever/regex-search-query
...
Regex search query
2015-12-03 09:57:34 -06:00
Sam Groth
596b7ebd9a
Adding RegexSearchQuerySpec
2015-12-03 09:16:02 -06:00
Himanshu
d02be6194d
Merge pull request #1967 from metamx/realtime-metrics-improvements
...
Add datasource and taskId to metrics emitted by peons
2015-12-02 23:48:13 -06:00
Himanshu
00c6027777
Merge pull request #1986 from metamx/substring
...
fixes #1874 adding a substring extraction function, tests, and documentation
2015-12-02 23:45:47 -06:00
Clint Wylie
68ef5f437a
fixes #1874 adding a substring extraction function, tests, and documentation
2015-12-01 23:50:32 -08:00
navis.ryu
87357a0534
fixed #2001 GenericIndexed.fromIterable compares all values even when it's not sorted
2015-12-02 15:11:14 +09:00
Nishant
1eb8211346
Add datasource and taskId to metrics emitted by peons
...
This PR adds the datasource and taskId to the jvm and sys metrics
emitted by the peons.
fix spelling
review comment
review comment
2015-12-01 23:20:59 +05:30
Gian Merlino
cd2cff24ff
Fix serde for FragmentSearchQuerySpec and add some tests.
2015-11-30 17:34:35 -08:00
navis.ryu
c73418c181
fixed #2003 ColumnSelectorBitmapIndexSelector throws NPE for dimension not supporting bitmap
2015-11-24 10:45:36 +09:00
Himanshu Gupta
7a89b2e1a6
add numRows to segment metadata query response
2015-11-20 01:25:02 -06:00
Himanshu
d93640bfcb
Merge pull request #1974 from jon-wei/dim_order_merge
...
Allow IndexMerger to use non-lexicographic dim order when merging indexes
2015-11-18 19:51:34 -06:00
Xavier Léauté
e3e6159336
Merge pull request #1985 from metamx/FixLookupCacheKey
...
Change LookupExtractionFn cache key to be unique
2015-11-18 10:13:55 -08:00
Charles Allen
7abe999418
Change LookupExtractionFn cache key to be unique
2015-11-17 18:02:40 -08:00
jon-wei
4afc62be29
Allow IndexMerger to use non-lexicographic dim order when merging indexes
2015-11-17 13:02:31 -08:00
Xavier Léauté
d7eb2f717e
enable query caching on intermediate realtime persists
2015-11-17 10:58:00 -08:00
Gian Merlino
57f213d536
Better toString for groupBy, segmentMetadata queries.
2015-11-16 12:54:59 -08:00
jon-wei
cdceaf2d26
Fix IncrementalIndexAdapter getRows() Iterable
2015-11-12 13:10:42 -08:00
Charles Allen
af34e9c8cb
Add the druid artifact version to metrics when emitted
2015-11-12 12:11:27 -08:00
binlijin
286b8f8c6f
optimize index merge
2015-11-12 11:08:54 +08:00
Xavier Léauté
fa6142e217
cleanup and remove unused imports
2015-11-11 12:25:21 -08:00
dclim
fd0935ecb9
fix spatial dimension transformer to work with hadoop
2015-11-10 19:16:51 -07:00
Slim Bouguerra
c511273efd
adding in filter
2015-11-06 16:23:24 -06:00
Charles Allen
929b981710
Change DefaultObjectMapper to NOT overwrite final fields unless explicitly asked to
2015-11-05 18:10:13 -08:00
fjy
8f231fd3e3
cleanup druid codebase
2015-11-04 13:59:53 -08:00
Gian Merlino
8defe29270
Merge pull request #1901 from guobingkun/fix_typo_and_rename
...
Fix metadata typo and rename default extension directory
2015-11-03 14:02:11 -08:00
Bingkun Guo
962f65cc76
fix metadata typo and rename default extension directory
2015-11-03 14:50:42 -06:00
Fangjin Yang
cec09a9967
Merge pull request #1804 from himanshug/objectify_index_creators
...
static to non-static conversion for methods in Index[Merger/Maker/IO]
2015-11-03 11:25:32 -08:00
Himanshu Gupta
8b67417ac8
make methods in Index[Merger,Maker,IO] non-static so that they can have
...
appropriate ObjectMapper injected instead of creating one statically
2015-11-02 23:24:26 -06:00
navis.ryu
e03fc2032f
changed equals/hashCode implementation
2015-11-02 17:21:35 +09:00
navis.ryu
69c86716d6
addressed comments
2015-11-02 14:23:13 +09:00
navis.ryu
032c3e986d
Make 'search' filter have a case sensitive option( #1878 )
2015-10-30 16:38:54 +09:00
Fangjin Yang
25a0eb7ed5
Merge pull request #1799 from dclim/nested-groupby-aggregator-fix
...
Support multiple outer aggregators of same type and provide more help…
2015-10-29 18:01:31 -07:00
Xavier Léauté
59872bd0cd
Merge pull request #1809 from metamx/fifoPriorityExecutorService
...
Make PrioritizedExecutorService optionally FIFO
2015-10-27 15:19:32 -07:00
Charles Allen
060402a216
Merge pull request #1855 from himanshug/fix_having_specs
...
fix [GreaterThan,LessThan,Equals] HavingSpecs
2015-10-27 14:46:04 -07:00
Charles Allen
ecdafa87c5
Make PrioritizedExecutorService optionally FIFO
2015-10-27 14:16:22 -07:00
Himanshu Gupta
a71c7270b9
making [GreaterThan,LessThan,Equals] HavingSpecs more robust by carefully using long vs float for comparison
2015-10-27 13:15:13 -05:00
Fangjin Yang
5a082b2f5e
Merge pull request #1824 from metamx/UniformGranularitySpecHashEquals
...
Add hashCode and equals to UniformGranularitySpec
2015-10-26 09:34:01 -07:00
Fangjin Yang
5f23703216
Merge pull request #1638 from guobingkun/remove_maven_client_code
...
Remove Maven client at runtime + Provide a way to load Druid extensions through local file system
2015-10-26 09:30:05 -07:00
Nishant
7cecc55045
Add segment merge time as a metric
...
Add merge and persist cpu time
Fix typo
review comment
move cpu time measuring to VMUtils
review comments.
2015-10-22 12:28:03 +05:30
Bingkun Guo
4914925d65
New extension loading mechanism
...
1) Remove maven client from downloading extensions at runtime.
2) Provide a way to load Druid extensions and hadoop dependencies through file system.
3) Refactor pull-deps so that it can download extensions into extension directories.
4) Add documents on how to use this new extension loading mechanism.
5) Change the way how Druid tarball is generated. Now all the extensions + hadoop-client 2.3.0
are packaged within the Druid tarball.
2015-10-21 14:22:36 -05:00
Xavier Léauté
e4ac78e43d
bump next snapshot to 0.9.0
2015-10-20 13:46:13 -07:00
dclim
46ecdfa757
add comment explaining logic
2015-10-15 16:04:06 -06:00
Xavier Léauté
4c2c7a2c37
update version to 0.8.3
2015-10-14 21:40:55 -07:00
Charles Allen
f432b8e3f9
Add hashCode and equals to UniformGranularitySpec
...
* Also add hashCode != 0 to AllGranularity and NoneGranularity
2015-10-13 16:42:21 -07:00
Gian Merlino
c9d6994040
Merge pull request #1821 from himanshug/storage_adapter_update
...
cache max data timestamp in QueryableIndexStorageAdapter
2015-10-13 10:52:43 -07:00
Himanshu Gupta
490de1f98a
support multiple non-consecutive intervals in outer query of nested group-by
2015-10-13 10:16:06 -05:00
Himanshu Gupta
fbba30eb60
cache max data timestamp in QueryableIndexStorageAdapter so that TimestampCheckingOffset
...
does not have to get it per cursor.
2015-10-12 15:34:22 -05:00
Charles Allen
8ed5d2c06a
Add hashCode and equals to stock lookups
2015-10-12 10:29:39 -07:00
Himanshu Gupta
2737fd83f5
in the IndexSizeExceededException put maxRowCount to confirm if it is correctly picked up from configuration
2015-10-06 15:23:14 -05:00
Himanshu Gupta
8654732ef6
make IndexSizeExceededException constructor take formatString and arguments than just fixed String
...
like ISE, IAE etc
2015-10-06 13:44:22 -05:00
dclim
f4e0a76820
Support multiple outer aggregators of same type and provide more helpful exception when the same inner aggregator is referenced by multiple types of outer aggregators
2015-10-01 15:15:12 -06:00
Gian Merlino
774765dc40
GroupByQueryRunnerTest for hyperUnique finalizing post aggregators
2015-10-01 00:09:29 -04:00
Gian Merlino
e3bb93e8c7
Revert "Merge pull request #1781 from dclim/nested-groupby-multiple-same-aggregator-fix-v2"
...
This reverts commit dae488b7c0
, reversing
changes made to 397be4b897
.
2015-10-01 00:05:59 -04:00
dclim
8e20a1e1f3
Use DoubleSumAggregatorFactory instead of CountAggregatorFactory, add test for non-integers
2015-09-30 17:11:39 -06:00
David Lim
70ae5ca922
Fix failure in nested groupBy with multiple aggregators with same fieldName
...
Version 2 - Throws an exception if an outer query references an
aggregator that doesn't exist in the inner query, and then uses the
inner query aggregator names to form the columns for the intermediate
incremental index.
Also deleted all the getRequiredColumns() methods which are no longer
being used.
We do something wacky by adding an aggregator factory for the post
aggregators when building the intermediate incremental index, otherwise
queries on post aggregate results fail because the data isn't in the
incremental index.
Closes #1419
2015-09-30 15:43:11 -06:00
Charles Allen
8199ecf1a4
Merge pull request #1782 from jon-wei/smq_cachekey
...
Add analysisTypes to SegmentMetadataQuery cache key
2015-09-29 15:51:35 -07:00
jon-wei
41ff271339
Add analysisTypes to SegmentMetadataQuery cache key
2015-09-29 14:33:35 -07:00
Charles Allen
2d847ad654
Merge pull request #1730 from metamx/union-queries-fix
...
fix #1727 - Union bySegment queries fix
2015-09-29 12:23:25 -07:00
Nishant
573aa96bd6
fix #1727 - Union bySegment queries fix
...
Fixes #1727 .
revert to doing merging for results for union queries on broker.
revert unrelated changes
Add test for union query runner
Add test
remove unused imports
fix imports
fix renamed file
fix test
update docs.
2015-09-29 23:32:36 +05:30
Gian Merlino
62d4ced4dd
Separate ListColumnIncluderator cache key parts with nul bytes
2015-09-29 13:59:58 -04:00
jon-wei
e6a6284ebd
Allow SegmentMetadataQuery to skip cardinality and size calculations
2015-09-22 13:51:55 -07:00
Gian Merlino
aaa8a88464
Merge pull request #1739 from jon-wei/segment_realtime
...
Allow SegmentAnalyzer to read columns from StorageAdapter, allow SegmentMetadataQuery to query IncrementalIndexSegments on realtime node
2015-09-17 18:36:53 -07:00
Charles Allen
df4c2bab10
Soften concurrency requirements on IncrementalIndexTest
2015-09-17 15:51:07 -07:00
jon-wei
367c50d4ba
Allow SegmentAnalyzer to read columns from StorageAdapter, allow SegmentMetadataQuery to query IncrementalIndexSegments on realtime node
2015-09-16 18:39:31 -07:00
Charles Allen
6e1eb3b7fe
Add better concurrency testing to IncrementalIndexTest
2015-09-16 14:04:20 -07:00
Gian Merlino
9705c5139b
Merge pull request #1732 from jon-wei/segmentmeta
...
Add support for a configurable default segment history period for segmentMetadata queries and GET /datasources/<datasourceName> lookups
2015-09-16 12:36:25 -07:00
Fangjin Yang
8b071a7230
Merge pull request #1710 from metamx/incrementalIndexConcurrentTestLatching
...
Add some basic latching to concurrency testing in IncrementalIndexTest
2015-09-15 13:55:52 -07:00
jon-wei
193fb4fdfc
Add support for a configurable default segment history period for segmentMetadata queries and GET /datasources/<datasourceName> lookups
2015-09-14 19:41:42 -07:00
Charles Allen
bd605a097e
Merge pull request #1731 from metamx/regex-extraction-npe
...
fix NPE with regex extraction function
2015-09-14 15:55:05 -07:00
Xavier Léauté
08a527d01a
fix NPE with regex extraction function
2015-09-14 14:45:30 -07:00
Charles Allen
e569f4b6a7
Add dimension extraction functionality to SearchQuery
...
* Add IdentityExtractionFn
2015-09-14 11:36:15 -07:00
Himanshu
5ff92664f8
Merge pull request #1696 from metamx/cpuTimeReporting
...
Add CPU time to metrics for segment scanning.
2015-09-14 10:53:55 -05:00
Fangjin Yang
34ef81572d
Merge pull request #1700 from himanshug/update_agg_test_helper
...
update indexing in the helper to use multiple persists and merge
2015-09-14 06:56:29 -07:00
Charles Allen
8d3cdd8572
Don't check for sortedness if we already know GenericIndexedWriter isn't sorted
2015-09-11 16:32:09 -07:00
Charles Allen
d6849805ea
Add some basic latching to concurrency testing in IncrementalIndexTest
2015-09-10 10:06:51 -07:00
Himanshu Gupta
5da58e48e0
use Rule based TemporaryFolder for cleanup of temp directory/files
2015-09-09 11:10:33 -05:00
Himanshu Gupta
44911039c5
update indexing in the helper to use multiple persists and final merge to
...
catch further issues in aggregator implementations
2015-09-09 11:10:33 -05:00
Charles Allen
fcf5cae81d
Add CPU time to metrics for segment scanning.
2015-09-08 13:34:19 -07:00
cheddar
4f61b42f40
Merge pull request #1578 from b-slim/fix_extraction_filter_2
...
Fix UT and documentation to the extraction filter
2015-09-01 10:46:20 -07:00
Himanshu
04ff6cd355
Merge pull request #1685 from gianm/close-loudly
...
Close output streams and channels loudly when creating segments.
2015-08-28 23:32:22 -05:00
Gian Merlino
940e1aa3eb
Replace funky imports with standard ones.
...
1) Lots of Guava imports were not coming from the actual Guava
2) junit.framework.Assert should be org.junit.Assert
2015-08-28 18:02:05 -07:00
Gian Merlino
7d6fa2ba50
Close output streams and channels loudly when creating segments.
2015-08-28 17:14:03 -07:00
Himanshu Gupta
2e0dd1d792
adding UTs and addressing review comments to
...
firehoseV2 addition to Realtime[Manager|Plumber],
essential segment metadata persist support,
kafka-simple-consumer-firehose extension patch
2015-08-27 20:50:46 -05:00
lvjq
2237a8cf0f
kafka 8 simple consumer firehose
2015-08-27 20:50:46 -05:00
Charles Allen
c1388a1685
Merge pull request #1632 from Hailei/fix-subquery-innerquery-demension
...
Inner Query should build on sub query
2015-08-27 10:25:38 -07:00
Gian Merlino
2a866f49df
Downgrade Jackson to 2.4.6.
2015-08-26 18:25:55 -07:00
Charles Allen
24aa762c79
Add test for #1632
2015-08-25 20:50:30 -07:00
Xavier Léauté
51f6a9a2c9
update jackson to 2.6.1
2015-08-25 16:07:01 -07:00
Himanshu Gupta
c57c07f28a
add ability for client code to provide InputStream of input data in addition to File
...
It would be needed when input data file does not reside in the same jar
but you could still use getResourceAsStream() to read the data inside a file
2015-08-20 00:54:58 -05:00
Xavier Léauté
3b2e41e42a
update for next release
2015-08-18 17:16:46 -07:00
Slim Bouguerra
7549f02578
support the case filter value is null
2015-08-17 15:09:37 -05:00
zhanghailei
234a958817
Inner Query should build on sub query
2015-08-17 18:18:26 +08:00
Charles Allen
db19d2d547
Revert "Update to guice 4.0"
2015-08-14 09:26:07 -07:00