Apache
/
druid
mirror of https://github.com/apache/druid.git
1
0
Fork 0
Code Issues Packages Projects Releases Wiki Activity
6,577 Commits 111 Branches 503 Tags 728 MiB
Commit Graph

1333 Commits

Author SHA1 Message Date
Slim Bouguerra 032d3bf6e6 Optimization of extraction filter by reversing the lookup 2016-01-06 11:16:11 -06:00
Himanshu Gupta 3f048f0b15 adding support to execute Select queries in AggregationTestHelper so that Select query based UTs can be written for complex aggregator implementations 2016-01-05 21:54:55 -06:00
Charles Allen 91fc32749b Make OnHeapIncrementalIndex clean maps on close() 2016-01-04 11:18:16 -08:00
Himanshu Gupta b47d807738 Add support for filtering at DimensionSpec level so that multivalued dimensions can be filtered correctly 
also adding UTs for multi-valued dimensions
 2015-12-30 17:59:47 -06:00
Himanshu Gupta fa5c3bb014 adding decorate(DimensionSelector) to DimensionSpec to enable support for arbitrary filtering/transformations to returned dimension values 2015-12-30 15:06:24 -06:00
Nishant b68265399c Merge pull request #2168 from druid-io/remove-indexmaker 
Remove IndexMaker
 2015-12-30 12:24:29 +05:30
Fangjin Yang e14ad74088 Merge pull request #1936 from b-slim/between_range_with_predicat 
adding Upper/Lower Bound Filter
 2015-12-29 10:11:22 -08:00
fjy faf421726b remove IndexMaker 2015-12-28 14:19:02 -08:00
Gian Merlino 83f4130b5f SegmentMetadataQuery merging fixes. 
- Fix merging when the INTERVALS analysisType is disabled, and add a test.
- Remove transformFn from CombiningSequence, use MappingSequence instead. transformFn did
  not work for "accumulate" anyway, which made the tests wrong (the intervals should have
  been condensed, but were not).
- Add analysisTypes to the Druids segmentMetadataQuery builder to make testing simpler.
 2015-12-22 07:57:10 -08:00
Robin dded4441d3 for completeness, add unit test for groupby/having with unrecognized type 2015-12-21 12:06:56 -06:00
Himanshu Gupta e1631967e3 adding comments to explain merge failure in segmentMetadata query 2015-12-19 11:39:24 -06:00
Himanshu Gupta 7ecad1be24 Fix and UT for testing segment analysis merge 2015-12-19 00:24:02 -06:00
Fangjin Yang 7019d3c421 Merge pull request #2107 from jon-wei/fix_smq 
More efficient SegmentMetadataQuery
 2015-12-18 16:40:47 -08:00
Fangjin Yang 14229ba0f2 Merge pull request #1922 from metamx/jsonIgnoresFinalFields 
Change DefaultObjectMapper to NOT overwrite final fields unless explicitly asked to
 2015-12-18 15:38:32 -08:00
Fangjin Yang 71f554bf80 Merge pull request #2101 from himanshug/fix_extraction_dim_filter_cache_key 
add extractionFn bytes to cache key in ExtractionDimFilter
 2015-12-18 12:05:43 -08:00
Fangjin Yang 9e6874cc7e Merge pull request #2084 from binlijin/master 
minor optimize IndexMerger's MMappedIndexRowIterable
 2015-12-18 11:42:55 -08:00
Bingkun cc21a5fac7 Merge pull request #1999 from himanshug/remove_min_max_aggs 
remove min/max aggregator factory
 2015-12-18 13:38:52 -06:00
jon-wei 356b07c6c3 More efficient SegmentMetadataQuery 2015-12-17 12:46:23 -08:00
Jonathan Wei f8cf84f466 Merge pull request #1995 from himanshug/num_rows_seg_metadata_query 
add numRows to segment metadata query response
 2015-12-17 12:23:46 -08:00
Himanshu Gupta 82ea348003 add extractionFn bytes to cache key in ExtractionDimFilter 2015-12-16 14:00:38 -06:00
Himanshu 628643d80e Merge pull request #2091 from rasahner/noDefaultForGroupbyHaving 
take away default for groupBy/having
 2015-12-16 01:07:40 -06:00
sahner 3441cf3110 take away default for groupBy/having 2015-12-15 10:32:45 -06:00
Fangjin Yang e7f06cf61c Merge pull request #2075 from jon-wei/regex_extract 
Configurable value replacement on match failure for RegexExtractionFn
 2015-12-14 19:10:50 -08:00
jon-wei c88f75df7c Configurable value replacement on match failure for RegexExtractionFn 2015-12-14 17:57:41 -08:00
binlijin 362bea1090 minor optimize IndexMerger's MMappedIndexRowIterable 2015-12-11 15:04:46 +08:00
Xavier Léauté d531e69d1a Merge pull request #2079 from binlijin/master 
reduce bytearray copy to minimal optimize VSizeIndexedWriter
 2015-12-10 21:30:09 -08:00
Slim Bouguerra 77afdf25e3 adding Bound Filter 2015-12-10 08:47:21 -06:00
Slim Bouguerra ee1a39801a adding bulk lookup and reverse lookup 2015-12-10 08:29:41 -06:00
binlijin 0eafbd55b2 reduce bytearray copy to minimal optimize VSizeIndexedWriter 2015-12-10 16:34:39 +08:00
Fangjin Yang f4ba13a1ac Merge pull request #2029 from b-slim/add_reverse_fn 
Adding reverse lookup function to LookupExtractor.
 2015-12-09 12:50:13 -08:00
Xavier Léauté 9015a68c03 Merge pull request #2002 from navis/DRUID-2001 
fixed #2001 GenericIndexed.fromIterable compares all values even when it's not sorted
 2015-12-09 08:56:49 -08:00
Slim Bouguerra 85f339b687 introduction and implem of reverse lookup function unApply. 2015-12-09 10:02:57 -06:00
Nishant 6c23d8edb4 Merge pull request #2043 from mangeshpardeshiyahoo/master 
Add dimension selector support for groupby/having filters
 2015-12-08 12:08:53 +05:30
Mangesh Pardeshi d7ce120929 Add dimension selector support for groupby/having quries 2015-12-08 01:51:11 +00:00
Himanshu Gupta 431469e9c1 remove min/max aggregator factory which are replaced by double[min/max] aggregator factories 2015-12-05 22:36:49 -06:00
Himanshu Gupta 62ba9ade37 unifying license header in all java files 2015-12-05 22:16:23 -06:00
Gian Merlino d21a640695 Merge pull request #2034 from b-slim/fix_cache_key 
Fix getCacheKey for DimFilters
 2015-12-04 09:13:06 -08:00
Slim Bouguerra fb4ff3cf54 fix getCacheKey 2015-12-04 08:07:08 -06:00
Charles Allen 9d02f47201 Update IncrementalIndexTest copyright notice 2015-12-03 18:03:08 -08:00
Charles Allen be8c6fafb0 Merge pull request #2017 from tubemogul/issue/63 
fixes issue #63
 2015-12-03 18:01:11 -08:00
Gian Merlino 045df54404 Merge pull request #1961 from metamx/druidMetricsVersion 
Add the druid artifact version to metrics when emitted
 2015-12-03 17:34:57 -08:00
Michael Schiff b6cc2428e1 fixes issue #63 2015-12-03 17:30:47 -08:00
Himanshu 0eab8417cb Merge pull request #2008 from codingwhatever/regex-search-query 
Regex search query
 2015-12-03 09:57:34 -06:00
Sam Groth 596b7ebd9a Adding RegexSearchQuerySpec 2015-12-03 09:16:02 -06:00
Himanshu d02be6194d Merge pull request #1967 from metamx/realtime-metrics-improvements 
Add datasource and taskId to metrics emitted by peons
 2015-12-02 23:48:13 -06:00
Himanshu 00c6027777 Merge pull request #1986 from metamx/substring 
fixes #1874 adding a substring extraction function, tests, and documentation
 2015-12-02 23:45:47 -06:00
Clint Wylie 68ef5f437a fixes #1874 adding a substring extraction function, tests, and documentation 2015-12-01 23:50:32 -08:00
navis.ryu 87357a0534 fixed #2001 GenericIndexed.fromIterable compares all values even when it's not sorted 2015-12-02 15:11:14 +09:00
Nishant 1eb8211346 Add datasource and taskId to metrics emitted by peons 
This PR adds the datasource and taskId to the jvm and sys metrics
emitted by the peons.

fix spelling

review comment

review comment
 2015-12-01 23:20:59 +05:30
Gian Merlino cd2cff24ff Fix serde for FragmentSearchQuerySpec and add some tests. 2015-11-30 17:34:35 -08:00
navis.ryu c73418c181 fixed #2003 ColumnSelectorBitmapIndexSelector throws NPE for dimension not supporting bitmap 2015-11-24 10:45:36 +09:00
Himanshu Gupta 7a89b2e1a6 add numRows to segment metadata query response 2015-11-20 01:25:02 -06:00
Himanshu d93640bfcb Merge pull request #1974 from jon-wei/dim_order_merge 
Allow IndexMerger to use non-lexicographic dim order when merging indexes
 2015-11-18 19:51:34 -06:00
Xavier Léauté e3e6159336 Merge pull request #1985 from metamx/FixLookupCacheKey 
Change LookupExtractionFn cache key to be unique
 2015-11-18 10:13:55 -08:00
Charles Allen 7abe999418 Change LookupExtractionFn cache key to be unique 2015-11-17 18:02:40 -08:00
jon-wei 4afc62be29 Allow IndexMerger to use non-lexicographic dim order when merging indexes 2015-11-17 13:02:31 -08:00
Xavier Léauté d7eb2f717e enable query caching on intermediate realtime persists 2015-11-17 10:58:00 -08:00
Gian Merlino 57f213d536 Better toString for groupBy, segmentMetadata queries. 2015-11-16 12:54:59 -08:00
jon-wei cdceaf2d26 Fix IncrementalIndexAdapter getRows() Iterable 2015-11-12 13:10:42 -08:00
Charles Allen af34e9c8cb Add the druid artifact version to metrics when emitted 2015-11-12 12:11:27 -08:00
binlijin 286b8f8c6f optimize index merge 2015-11-12 11:08:54 +08:00
Xavier Léauté fa6142e217 cleanup and remove unused imports 2015-11-11 12:25:21 -08:00
dclim fd0935ecb9 fix spatial dimension transformer to work with hadoop 2015-11-10 19:16:51 -07:00
Slim Bouguerra c511273efd adding in filter 2015-11-06 16:23:24 -06:00
Charles Allen 929b981710 Change DefaultObjectMapper to NOT overwrite final fields unless explicitly asked to 2015-11-05 18:10:13 -08:00
fjy 8f231fd3e3 cleanup druid codebase 2015-11-04 13:59:53 -08:00
Gian Merlino 8defe29270 Merge pull request #1901 from guobingkun/fix_typo_and_rename 
Fix metadata typo and rename default extension directory
 2015-11-03 14:02:11 -08:00
Bingkun Guo 962f65cc76 fix metadata typo and rename default extension directory 2015-11-03 14:50:42 -06:00
Fangjin Yang cec09a9967 Merge pull request #1804 from himanshug/objectify_index_creators 
static to non-static conversion for methods in Index[Merger/Maker/IO]
 2015-11-03 11:25:32 -08:00
Himanshu Gupta 8b67417ac8 make methods in Index[Merger,Maker,IO] non-static so that they can have 
appropriate ObjectMapper injected instead of creating one statically
 2015-11-02 23:24:26 -06:00
navis.ryu e03fc2032f changed equals/hashCode implementation 2015-11-02 17:21:35 +09:00
navis.ryu 69c86716d6 addressed comments 2015-11-02 14:23:13 +09:00
navis.ryu 032c3e986d Make 'search' filter have a case sensitive option(#1878) 2015-10-30 16:38:54 +09:00
Fangjin Yang 25a0eb7ed5 Merge pull request #1799 from dclim/nested-groupby-aggregator-fix 
Support multiple outer aggregators of same type and provide more help…
 2015-10-29 18:01:31 -07:00
Xavier Léauté 59872bd0cd Merge pull request #1809 from metamx/fifoPriorityExecutorService 
Make PrioritizedExecutorService optionally FIFO
 2015-10-27 15:19:32 -07:00
Charles Allen 060402a216 Merge pull request #1855 from himanshug/fix_having_specs 
fix [GreaterThan,LessThan,Equals] HavingSpecs
 2015-10-27 14:46:04 -07:00
Charles Allen ecdafa87c5 Make PrioritizedExecutorService optionally FIFO 2015-10-27 14:16:22 -07:00
Himanshu Gupta a71c7270b9 making [GreaterThan,LessThan,Equals] HavingSpecs more robust by carefully using long vs float for comparison 2015-10-27 13:15:13 -05:00
Fangjin Yang 5a082b2f5e Merge pull request #1824 from metamx/UniformGranularitySpecHashEquals 
Add hashCode and equals to UniformGranularitySpec
 2015-10-26 09:34:01 -07:00
Fangjin Yang 5f23703216 Merge pull request #1638 from guobingkun/remove_maven_client_code 
Remove Maven client at runtime + Provide a way to load Druid extensions through local file system
 2015-10-26 09:30:05 -07:00
Nishant 7cecc55045 Add segment merge time as a metric 
Add merge and persist cpu time

Fix typo

review comment

move cpu time measuring to VMUtils

review comments.
 2015-10-22 12:28:03 +05:30
Bingkun Guo 4914925d65 New extension loading mechanism 
1) Remove maven client from downloading extensions at runtime.
2) Provide a way to load Druid extensions and hadoop dependencies through file system.
3) Refactor pull-deps so that it can download extensions into extension directories.
4) Add documents on how to use this new extension loading mechanism.
5) Change the way how Druid tarball is generated. Now all the extensions + hadoop-client 2.3.0
are packaged within the Druid tarball.
 2015-10-21 14:22:36 -05:00
Xavier Léauté e4ac78e43d bump next snapshot to 0.9.0 2015-10-20 13:46:13 -07:00
dclim 46ecdfa757 add comment explaining logic 2015-10-15 16:04:06 -06:00
Xavier Léauté 4c2c7a2c37 update version to 0.8.3 2015-10-14 21:40:55 -07:00
Charles Allen f432b8e3f9 Add hashCode and equals to UniformGranularitySpec 
* Also add hashCode != 0 to AllGranularity and NoneGranularity
 2015-10-13 16:42:21 -07:00
Gian Merlino c9d6994040 Merge pull request #1821 from himanshug/storage_adapter_update 
cache max data timestamp in QueryableIndexStorageAdapter
 2015-10-13 10:52:43 -07:00
Himanshu Gupta 490de1f98a support multiple non-consecutive intervals in outer query of nested group-by 2015-10-13 10:16:06 -05:00
Himanshu Gupta fbba30eb60 cache max data timestamp in QueryableIndexStorageAdapter so that TimestampCheckingOffset 
does not have to get it per cursor.
 2015-10-12 15:34:22 -05:00
Charles Allen 8ed5d2c06a Add hashCode and equals to stock lookups 2015-10-12 10:29:39 -07:00
Himanshu Gupta 2737fd83f5 in the IndexSizeExceededException put maxRowCount to confirm if it is correctly picked up from configuration 2015-10-06 15:23:14 -05:00
Himanshu Gupta 8654732ef6 make IndexSizeExceededException constructor take formatString and arguments than just fixed String 
like ISE, IAE etc
 2015-10-06 13:44:22 -05:00
dclim f4e0a76820 Support multiple outer aggregators of same type and provide more helpful exception when the same inner aggregator is referenced by multiple types of outer aggregators 2015-10-01 15:15:12 -06:00
Gian Merlino 774765dc40 GroupByQueryRunnerTest for hyperUnique finalizing post aggregators 2015-10-01 00:09:29 -04:00
Gian Merlino e3bb93e8c7 Revert "Merge pull request #1781 from dclim/nested-groupby-multiple-same-aggregator-fix-v2" 
This reverts commit dae488b7c0, reversing
changes made to 397be4b897.
 2015-10-01 00:05:59 -04:00
dclim 8e20a1e1f3 Use DoubleSumAggregatorFactory instead of CountAggregatorFactory, add test for non-integers 2015-09-30 17:11:39 -06:00
David Lim 70ae5ca922 Fix failure in nested groupBy with multiple aggregators with same fieldName 
Version 2 - Throws an exception if an outer query references an
aggregator that doesn't exist in the inner query, and then uses the
inner query aggregator names to form the columns for the intermediate
incremental index.

Also deleted all the getRequiredColumns() methods which are no longer
being used.

We do something wacky by adding an aggregator factory for the post
aggregators when building the intermediate incremental index, otherwise
queries on post aggregate results fail because the data isn't in the
incremental index.

Closes #1419
 2015-09-30 15:43:11 -06:00
Charles Allen 8199ecf1a4 Merge pull request #1782 from jon-wei/smq_cachekey 
Add analysisTypes to SegmentMetadataQuery cache key
 2015-09-29 15:51:35 -07:00
jon-wei 41ff271339 Add analysisTypes to SegmentMetadataQuery cache key 2015-09-29 14:33:35 -07:00
Charles Allen 2d847ad654 Merge pull request #1730 from metamx/union-queries-fix 
fix #1727 - Union bySegment queries fix
 2015-09-29 12:23:25 -07:00