Commit Graph

7167 Commits

Author SHA1 Message Date
Gian Merlino 4ac9e03161 Fix predicate-based ValueMatcher behavior for IncrementalIndex on missing columns.
Missing columns should be treated the same as columns containing 100% nulls.
2016-03-25 10:23:59 -07:00
Bingkun Guo 0872448ff0 make Coordinator IndexingService helpers pluggable
Fixes #2682
IndexingService helpers are added according to the settings in runtime.properties.
Rather than having all the config.isXXX checks there, it makes sense to have a pluggable
approach for allowing the dynamic configuration to bring in implementations for helpers
without having to have hard-coded sets of available helpers. Plus, it will also make it possible for extensions to plug helpers in.

With https://github.com/druid-io/druid-api/pull/76, we could conditionally bind a helper to Coordinator's runlist.
The condition is driven by the value set in the runtime.properties.
2016-03-25 11:48:54 -05:00
Gian Merlino 6d18382fb2 Fix broken link in datasketches-aggregators.md. 2016-03-25 09:32:40 -07:00
Himanshu Gupta e78a469fb7 UTs for ExtensionsConfig 2016-03-25 10:51:28 -05:00
Himanshu Gupta 004b00bb96 config to explicitly specify classpath for hadoop container during hadoop ingestion 2016-03-25 10:51:28 -05:00
Fangjin Yang ef0bc5d64f Merge pull request #2726 from gianm/move-flatten-json-benchmark-test
Move FlattenJSONBenchmarkUtilTest into the correct location.
2016-03-25 08:00:05 -07:00
Fangjin Yang 980943074b Merge pull request #2724 from guobingkun/conditional_multi_bind
Conditional multi bind
2016-03-24 17:09:27 -07:00
Nishant 0b03c9405f Merge pull request #2614 from sirpkt/calendric_gran
Support week, month, quarter, and year in query granularity
2016-03-24 16:21:01 -07:00
Gian Merlino 1869c350d8 Move FlattenJSONBenchmarkUtilTest into the correct location. 2016-03-24 15:47:54 -07:00
Bingkun Guo 62edbab434 Conditional multi bind
Usage example:
       ConditionalMultibind.create(props, binder, Animal.class)
                           .conditionBinder("animal.type", Predicates.equalTo("cat"), Cat.class)
                           .conditionBinder("animal.type", Predicates.equalTo("dog"), Dog.class);

At binding time, this will check the value set for property "animal.type" in props. If the value is "cat", it will
add a binding to Cat.class. If the value is "dog", it will add a binding to Dog.class.

At ingestion time, you will get the items that satisfy their corresponding predicates by calling
injector.getInstance(Key.get(new TypeLiteral<Set<Animal>>(){}))
2016-03-24 16:23:22 -05:00
Himanshu 56343c6cdc Merge pull request #2704 from navis/simple-optimize
optimize single elemented and/or filter
2016-03-24 16:13:48 -05:00
Himanshu f26e73d133 Merge pull request #2720 from gianm/druid-api
Move druid-api into the druid repo.
2016-03-24 15:51:10 -05:00
Himanshu fb25c083ec Merge pull request #2721 from guobingkun/refine_doc
refine description for mergeBytesLimit
2016-03-24 15:48:57 -05:00
Fangjin Yang d4a96843f9 Merge pull request #2710 from himanshug/fix_sketch_estimate_comparator
fix SketchEstimate post aggregator's getComparator() and test changes to verify same
2016-03-24 13:34:32 -07:00
Bingkun Guo 0fa04305a6 refine description for mergeBytesLimit 2016-03-24 13:17:24 -05:00
Gian Merlino 7e7a886f65 Move druid-api into the druid repo.
This is from druid-api-0.3.17, as of commit 51884f1d05d5512cacaf62cedfbb28c6ab2535cf
in the druid-api repo.
2016-03-24 11:04:34 -07:00
Himanshu Gupta 4aead38130 fix SketchEstimate post aggregator's getComparator() and test changes to verify same 2016-03-24 10:11:06 -05:00
binlijin 2729efca71 implement special distinctcount 2016-03-24 11:11:11 +08:00
Gian Merlino bfc0ae7d2a Merge pull request #2714 from rasahner/DOC_minorFix
dynamic config endpoint is at coordinator
2016-03-23 18:43:56 -07:00
Fangjin Yang 8750f4b58c Merge pull request #2713 from gianm/filter-extensions
Filters: Add filter.toFilter method, use that instead of the instanceof chain in Filters.
2016-03-23 18:04:08 -07:00
Gian Merlino 713062053c Filters: Add filter.toFilter method, use that instead of the instanceof chain in Filters.
I believe that the instanceof chain in Filters exists because in the past, Filter
and DimFilter were in different packages (DimFilter was in druid-client and Filter
was in druid-processing). And since druid-client didn't depend on druid-processing,
DimFilter couldn't have a toFilter method. But now it can.
2016-03-23 17:03:49 -07:00
Robin 448e0127b9 dynamic config endpoint is at coordinator 2016-03-23 17:22:19 -05:00
Fangjin Yang a5d5529749 Merge pull request #2711 from gianm/filtered-aggregator-impls
All Filters should work with FilteredAggregators.
2016-03-23 13:37:21 -07:00
Gian Merlino dd86198902 All Filters should work with FilteredAggregators.
This removes Filter.makeMatcher(ColumnSelectorFactory) and adds a
ValueMatcherFactory implementation to FilteredAggregatorFactory so it can
take advantage of existing makeMatcher(ValueMatcherFactory) implementations.

This patch also removes the Bound-based method from ValueMatcherFactory. Its
only user was the SpatialFilter, which could use the Predicate-based method.

Fixes #2604.
2016-03-23 12:24:01 -07:00
Bingkun Guo a6af1974f7 Merge pull request #2699 from gianm/docs-fix-broken-links
Fix a bunch of broken links in the docs.
2016-03-23 13:14:29 -05:00
Himanshu 8a473111c5 Merge pull request #2650 from rasahner/IT_index_hadoop
add test for batch indexing from hadoop
2016-03-23 13:04:55 -05:00
Gian Merlino 2dfd3877c0 Fix a bunch of broken links in the docs. 2016-03-23 10:21:28 -07:00
Fangjin Yang bbf08fcc24 Merge pull request #2675 from binlijin/clean_temp_file
clean tmp file when index merge fail
2016-03-23 09:09:07 -07:00
Fangjin Yang 9956df5609 Merge pull request #2705 from kilida/patch-1
Duplicate statement in ReservoirSegmentSamplerTest.java
2016-03-23 08:42:27 -07:00
Fangjin Yang d1f8f2b2fd Merge pull request #2698 from druid-io/fix-ext-docs
refactor extensions into their own docs
2016-03-22 22:04:12 -07:00
binlijin 57d78d3293 clean tmp file when index merge fail 2016-03-23 10:55:12 +08:00
kilida f25b2ed6f8 Duplicate statement in ReservoirSegmentSamplerTest.java 2016-03-22 22:14:36 -04:00
fjy 943cbe6e76 refactor extensions into their own docs 2016-03-22 18:54:10 -07:00
Fangjin Yang 041350c31b Merge pull request #2701 from gianm/mvd-docs
Improved docs for multi-value dimensions.
2016-03-22 18:09:37 -07:00
navis.ryu 91f6be4884 optimize single elemented and/or filter 2016-03-23 09:29:15 +09:00
Gian Merlino 451c0bc6d8 Merge pull request #2702 from pjain1/improve_docs
how to query in the querying section, correct default for select strategy, formatting
2016-03-22 16:40:35 -07:00
Fangjin Yang 826b371259 Merge pull request #2697 from guobingkun/remove_duplicate_version_converter
remove duplicated DruidCoordinatorVersionConverter
2016-03-22 15:48:09 -07:00
Parag Jain 39ecb9929d how to query, correct default for select strategy, formatting 2016-03-22 17:06:15 -05:00
Gian Merlino ff25325f3b Improved docs for multi-value dimensions.
- Add central doc for multi-value dimensions, with some content from other docs.
- Link to multi-value dimension doc from topN and groupBy docs.
- Fixes a broken link from dimensionspecs.md, which was presciently already
  linking to this nonexistent doc.
- Resolve inconsistent naming in docs & code (sometimes "multi-valued", sometimes
  "multi-value") in favor of "multi-value".
2016-03-22 14:40:55 -07:00
Bingkun Guo a6e9ff48ec Merge pull request #2688 from pjain1/props_cli
do not inject properties directly in module
2016-03-22 15:27:19 -05:00
Bingkun Guo 3778adf1f4 remove duplicated DruidCoordinatorVersionConverter 2016-03-22 14:45:52 -05:00
Parag Jain 7b93195dc6 do not inject properties directly in module 2016-03-22 14:30:10 -05:00
Himanshu 00d7021291 Merge pull request #2607 from jon-wei/dim_schema
Support use of DimensionSchema class in DimensionsSpec
2016-03-22 11:53:46 -05:00
Himanshu 3220b109ad Merge pull request #2570 from binlijin/single_dimension_partitioning
Single dimension hash-based partitioning
2016-03-22 11:51:06 -05:00
binlijin bce600f5d5 Single dimension hash-based partitioning 2016-03-22 13:15:33 +08:00
jon-wei a59c9ee1b1 Support use of DimensionSchema class in DimensionsSpec 2016-03-21 13:12:04 -07:00
Keuntae Park 7f29f2ac3b support week, month, quarter, year in query granularity 2016-03-21 17:41:53 +09:00
Charles Allen 527b728f3e Merge pull request #2689 from metamx/fix-broadcast-lockup
fix servlet startup sequence, fixes #2681
2016-03-19 09:38:31 -07:00
Nishant 11b8d1ed70 Merge pull request #2686 from gianm/fix-analysistypes-docs
Fix analysisTypes docs for SegmentMetadataQuery.
2016-03-18 16:15:38 -07:00
Xavier Léauté 25967d0ed8 fix servlet startup sequence, fixes #2681 2016-03-18 15:06:15 -07:00