Commit Graph

7024 Commits

Author SHA1 Message Date
Fangjin Yang 7fe277e6da Merge pull request #2727 from gianm/optimize-bound-filter
BoundFilter optimizations, and related interface changes.
2016-03-26 18:59:05 -07:00
Fangjin Yang 0dae28b6af Merge pull request #2729 from jon-wei/fix_hyperunique_comparator
Fix HyperUniquesAggregatorFactory comparator
2016-03-26 15:39:35 -07:00
Fangjin Yang a0216dcf7d Merge pull request #2735 from metamx/fixlookupDocs
Move lookup docs that are in druid-proper back into lookups.md
2016-03-26 15:38:48 -07:00
Fangjin Yang 3c4691aa5a Merge pull request #2741 from gianm/examples-wiki
Downgrade geoip2, exclude com.google.http-client.
2016-03-25 23:08:38 -07:00
Xavier Léauté 01f3221a62 Merge pull request #2665 from jisookim0513/remove-druid-server-serialization
remove serialization of DruidServer
2016-03-25 15:54:05 -07:00
Gian Merlino 977e867ad8 Downgrade geoip2, exclude com.google.http-client.
Reverts "Update com.maxmind.geoip2 to 2.6.0" and exclude the google http client
from com.maxmind.geoip2. This should satisfy the original need from #2646 (wanting
to run Druid along with an upgraded com.google.http-client) while preventing
Jackson conflicts pointed out in #2717.

Fixes #2717.

This reverts commit 21b7572533.
2016-03-25 14:43:22 -07:00
Gian Merlino 2970b49adc BoundFilter optimizations, and related interface changes.
BoundFilter:

- For lexicographic bounds, use bitmapIndex.getIndex to find the start and end points,
  then union all bitmaps between those points.
- For alphanumeric bounds, iterate through dimValues, and union all bitmaps for values
  matching the predicate.
- Change behavior for nulls: it used to be that the BoundFilter would never match nulls,
  now it matches nulls if "" is allowed by the lower limit and not excluded by the
  upper limit.

Interface changes:

- BitmapIndex: add `int getIndex(value)` to make it possible to get the index for a
  value without retrieving the bitmap.
- BitmapIndex: remove `ImmutableBitmap getBitmap(value)`, change callers to `getBitmap(getIndex(value))`.
- BitmapIndexSelector: allow retrieving the underlying BitmapIndex through getBitmapIndex.
- Clarified contract of indexOf in Indexed, GenericIndexed.

Also added tests for SelectorFilter, NotFilter, and BoundFilter.
2016-03-25 14:11:48 -07:00
Jonathan Wei 02d0c704b1 Merge pull request #2737 from gianm/fix-iisa-valueMatcher-predicate
Fix predicate-based ValueMatcher behavior for IncrementalIndex on missing columns.
2016-03-25 14:11:18 -07:00
jon-wei 9afaa2b94a Fix HyperUniquesAggregatorFactory comparator 2016-03-25 12:36:42 -07:00
Himanshu 2dff26512d Merge pull request #2733 from gianm/docs-datasketches-broken-link
Fix broken link in datasketches-aggregators.md.
2016-03-25 14:35:49 -05:00
jisookim 0d3c5a3b6c remove serialization of Druid Server and add tests for ServersResource 2016-03-25 12:27:27 -07:00
Charles Allen ab324e4ac0 Move lookup docs that are in druid-proper back into lookups.md 2016-03-25 10:46:50 -07:00
Gian Merlino 4ac9e03161 Fix predicate-based ValueMatcher behavior for IncrementalIndex on missing columns.
Missing columns should be treated the same as columns containing 100% nulls.
2016-03-25 10:23:59 -07:00
Gian Merlino 6d18382fb2 Fix broken link in datasketches-aggregators.md. 2016-03-25 09:32:40 -07:00
Fangjin Yang ef0bc5d64f Merge pull request #2726 from gianm/move-flatten-json-benchmark-test
Move FlattenJSONBenchmarkUtilTest into the correct location.
2016-03-25 08:00:05 -07:00
Fangjin Yang 980943074b Merge pull request #2724 from guobingkun/conditional_multi_bind
Conditional multi bind
2016-03-24 17:09:27 -07:00
Nishant 0b03c9405f Merge pull request #2614 from sirpkt/calendric_gran
Support week, month, quarter, and year in query granularity
2016-03-24 16:21:01 -07:00
Gian Merlino 1869c350d8 Move FlattenJSONBenchmarkUtilTest into the correct location. 2016-03-24 15:47:54 -07:00
Bingkun Guo 62edbab434 Conditional multi bind
Usage example:
       ConditionalMultibind.create(props, binder, Animal.class)
                           .conditionBinder("animal.type", Predicates.equalTo("cat"), Cat.class)
                           .conditionBinder("animal.type", Predicates.equalTo("dog"), Dog.class);

At binding time, this will check the value set for property "animal.type" in props. If the value is "cat", it will
add a binding to Cat.class. If the value is "dog", it will add a binding to Dog.class.

At ingestion time, you will get the items that satisfy their corresponding predicates by calling
injector.getInstance(Key.get(new TypeLiteral<Set<Animal>>(){}))
2016-03-24 16:23:22 -05:00
Himanshu 56343c6cdc Merge pull request #2704 from navis/simple-optimize
optimize single elemented and/or filter
2016-03-24 16:13:48 -05:00
Himanshu f26e73d133 Merge pull request #2720 from gianm/druid-api
Move druid-api into the druid repo.
2016-03-24 15:51:10 -05:00
Himanshu fb25c083ec Merge pull request #2721 from guobingkun/refine_doc
refine description for mergeBytesLimit
2016-03-24 15:48:57 -05:00
Fangjin Yang d4a96843f9 Merge pull request #2710 from himanshug/fix_sketch_estimate_comparator
fix SketchEstimate post aggregator's getComparator() and test changes to verify same
2016-03-24 13:34:32 -07:00
Bingkun Guo 0fa04305a6 refine description for mergeBytesLimit 2016-03-24 13:17:24 -05:00
Gian Merlino 7e7a886f65 Move druid-api into the druid repo.
This is from druid-api-0.3.17, as of commit 51884f1d05d5512cacaf62cedfbb28c6ab2535cf
in the druid-api repo.
2016-03-24 11:04:34 -07:00
Himanshu Gupta 4aead38130 fix SketchEstimate post aggregator's getComparator() and test changes to verify same 2016-03-24 10:11:06 -05:00
Gian Merlino bfc0ae7d2a Merge pull request #2714 from rasahner/DOC_minorFix
dynamic config endpoint is at coordinator
2016-03-23 18:43:56 -07:00
Fangjin Yang 8750f4b58c Merge pull request #2713 from gianm/filter-extensions
Filters: Add filter.toFilter method, use that instead of the instanceof chain in Filters.
2016-03-23 18:04:08 -07:00
Gian Merlino 713062053c Filters: Add filter.toFilter method, use that instead of the instanceof chain in Filters.
I believe that the instanceof chain in Filters exists because in the past, Filter
and DimFilter were in different packages (DimFilter was in druid-client and Filter
was in druid-processing). And since druid-client didn't depend on druid-processing,
DimFilter couldn't have a toFilter method. But now it can.
2016-03-23 17:03:49 -07:00
Robin 448e0127b9 dynamic config endpoint is at coordinator 2016-03-23 17:22:19 -05:00
Fangjin Yang a5d5529749 Merge pull request #2711 from gianm/filtered-aggregator-impls
All Filters should work with FilteredAggregators.
2016-03-23 13:37:21 -07:00
Gian Merlino dd86198902 All Filters should work with FilteredAggregators.
This removes Filter.makeMatcher(ColumnSelectorFactory) and adds a
ValueMatcherFactory implementation to FilteredAggregatorFactory so it can
take advantage of existing makeMatcher(ValueMatcherFactory) implementations.

This patch also removes the Bound-based method from ValueMatcherFactory. Its
only user was the SpatialFilter, which could use the Predicate-based method.

Fixes #2604.
2016-03-23 12:24:01 -07:00
Bingkun Guo a6af1974f7 Merge pull request #2699 from gianm/docs-fix-broken-links
Fix a bunch of broken links in the docs.
2016-03-23 13:14:29 -05:00
Himanshu 8a473111c5 Merge pull request #2650 from rasahner/IT_index_hadoop
add test for batch indexing from hadoop
2016-03-23 13:04:55 -05:00
Gian Merlino 2dfd3877c0 Fix a bunch of broken links in the docs. 2016-03-23 10:21:28 -07:00
Fangjin Yang bbf08fcc24 Merge pull request #2675 from binlijin/clean_temp_file
clean tmp file when index merge fail
2016-03-23 09:09:07 -07:00
Fangjin Yang 9956df5609 Merge pull request #2705 from kilida/patch-1
Duplicate statement in ReservoirSegmentSamplerTest.java
2016-03-23 08:42:27 -07:00
Fangjin Yang d1f8f2b2fd Merge pull request #2698 from druid-io/fix-ext-docs
refactor extensions into their own docs
2016-03-22 22:04:12 -07:00
binlijin 57d78d3293 clean tmp file when index merge fail 2016-03-23 10:55:12 +08:00
kilida f25b2ed6f8 Duplicate statement in ReservoirSegmentSamplerTest.java 2016-03-22 22:14:36 -04:00
fjy 943cbe6e76 refactor extensions into their own docs 2016-03-22 18:54:10 -07:00
Fangjin Yang 041350c31b Merge pull request #2701 from gianm/mvd-docs
Improved docs for multi-value dimensions.
2016-03-22 18:09:37 -07:00
navis.ryu 91f6be4884 optimize single elemented and/or filter 2016-03-23 09:29:15 +09:00
Gian Merlino 451c0bc6d8 Merge pull request #2702 from pjain1/improve_docs
how to query in the querying section, correct default for select strategy, formatting
2016-03-22 16:40:35 -07:00
Fangjin Yang 826b371259 Merge pull request #2697 from guobingkun/remove_duplicate_version_converter
remove duplicated DruidCoordinatorVersionConverter
2016-03-22 15:48:09 -07:00
Parag Jain 39ecb9929d how to query, correct default for select strategy, formatting 2016-03-22 17:06:15 -05:00
Gian Merlino ff25325f3b Improved docs for multi-value dimensions.
- Add central doc for multi-value dimensions, with some content from other docs.
- Link to multi-value dimension doc from topN and groupBy docs.
- Fixes a broken link from dimensionspecs.md, which was presciently already
  linking to this nonexistent doc.
- Resolve inconsistent naming in docs & code (sometimes "multi-valued", sometimes
  "multi-value") in favor of "multi-value".
2016-03-22 14:40:55 -07:00
Bingkun Guo a6e9ff48ec Merge pull request #2688 from pjain1/props_cli
do not inject properties directly in module
2016-03-22 15:27:19 -05:00
Bingkun Guo 3778adf1f4 remove duplicated DruidCoordinatorVersionConverter 2016-03-22 14:45:52 -05:00
Parag Jain 7b93195dc6 do not inject properties directly in module 2016-03-22 14:30:10 -05:00