7133 Commits

Author SHA1 Message Date
Parag Jain
89a8277ae2 Merge pull request #2712 from guobingkun/make_runnables_pluggable
make Coordinator IndexingService helpers pluggable
2016-03-28 12:18:18 -05:00
Charles Allen
05151bc325 Fix LookupCoordinatorManagerTest for alerts
* Also fixes bad alerting on missing nodes
2016-03-28 09:41:47 -07:00
Gian Merlino
dbdfcd2443 Fix extension reference in Kafka namespaced lookup docs.
The reference to io.druid.extensions:kafka-extraction-namespace is wrong (should
be druid-kafka-extraction-namespace) and unnecessary (the extension id is written
at the top of the doc file).
2016-03-28 09:23:24 -07:00
Fangjin Yang
7fe277e6da Merge pull request #2727 from gianm/optimize-bound-filter
BoundFilter optimizations, and related interface changes.
2016-03-26 18:59:05 -07:00
Fangjin Yang
0dae28b6af Merge pull request #2729 from jon-wei/fix_hyperunique_comparator
Fix HyperUniquesAggregatorFactory comparator
2016-03-26 15:39:35 -07:00
Fangjin Yang
a0216dcf7d Merge pull request #2735 from metamx/fixlookupDocs
Move lookup docs that are in druid-proper back into lookups.md
2016-03-26 15:38:48 -07:00
Fangjin Yang
3c4691aa5a Merge pull request #2741 from gianm/examples-wiki
Downgrade geoip2, exclude com.google.http-client.
2016-03-25 23:08:38 -07:00
Xavier Léauté
01f3221a62 Merge pull request #2665 from jisookim0513/remove-druid-server-serialization
remove serialization of DruidServer
2016-03-25 15:54:05 -07:00
Gian Merlino
977e867ad8 Downgrade geoip2, exclude com.google.http-client.
Reverts "Update com.maxmind.geoip2 to 2.6.0" and exclude the google http client
from com.maxmind.geoip2. This should satisfy the original need from #2646 (wanting
to run Druid along with an upgraded com.google.http-client) while preventing
Jackson conflicts pointed out in #2717.

Fixes #2717.

This reverts commit 21b7572533592f1700f86379483d87e9e340f2a7.
2016-03-25 14:43:22 -07:00
Gian Merlino
2970b49adc BoundFilter optimizations, and related interface changes.
BoundFilter:

- For lexicographic bounds, use bitmapIndex.getIndex to find the start and end points,
  then union all bitmaps between those points.
- For alphanumeric bounds, iterate through dimValues, and union all bitmaps for values
  matching the predicate.
- Change behavior for nulls: it used to be that the BoundFilter would never match nulls,
  now it matches nulls if "" is allowed by the lower limit and not excluded by the
  upper limit.

Interface changes:

- BitmapIndex: add `int getIndex(value)` to make it possible to get the index for a
  value without retrieving the bitmap.
- BitmapIndex: remove `ImmutableBitmap getBitmap(value)`, change callers to `getBitmap(getIndex(value))`.
- BitmapIndexSelector: allow retrieving the underlying BitmapIndex through getBitmapIndex.
- Clarified contract of indexOf in Indexed, GenericIndexed.

Also added tests for SelectorFilter, NotFilter, and BoundFilter.
2016-03-25 14:11:48 -07:00
Jonathan Wei
02d0c704b1 Merge pull request #2737 from gianm/fix-iisa-valueMatcher-predicate
Fix predicate-based ValueMatcher behavior for IncrementalIndex on missing columns.
2016-03-25 14:11:18 -07:00
jon-wei
9afaa2b94a Fix HyperUniquesAggregatorFactory comparator 2016-03-25 12:36:42 -07:00
Himanshu
2dff26512d Merge pull request #2733 from gianm/docs-datasketches-broken-link
Fix broken link in datasketches-aggregators.md.
2016-03-25 14:35:49 -05:00
jisookim
0d3c5a3b6c remove serialization of Druid Server and add tests for ServersResource 2016-03-25 12:27:27 -07:00
Gian Merlino
ee4bb96855 Fix handling of InterruptedException in WorkerTaskMonitor's mainLoop.
I believe this will fix #2664.
2016-03-25 12:17:33 -07:00
Charles Allen
ab324e4ac0 Move lookup docs that are in druid-proper back into lookups.md 2016-03-25 10:46:50 -07:00
Gian Merlino
4ac9e03161 Fix predicate-based ValueMatcher behavior for IncrementalIndex on missing columns.
Missing columns should be treated the same as columns containing 100% nulls.
2016-03-25 10:23:59 -07:00
Bingkun Guo
0872448ff0 make Coordinator IndexingService helpers pluggable
Fixes #2682
IndexingService helpers are added according to the settings in runtime.properties.
Rather than having all the config.isXXX checks there, it makes sense to have a pluggable
approach for allowing the dynamic configuration to bring in implementations for helpers
without having to have hard-coded sets of available helpers. Plus, it will also make it possible for extensions to plug helpers in.

With https://github.com/druid-io/druid-api/pull/76, we could conditionally bind a helper to Coordinator's runlist.
The condition is driven by the value set in the runtime.properties.
2016-03-25 11:48:54 -05:00
Gian Merlino
6d18382fb2 Fix broken link in datasketches-aggregators.md. 2016-03-25 09:32:40 -07:00
Himanshu Gupta
e78a469fb7 UTs for ExtensionsConfig 2016-03-25 10:51:28 -05:00
Himanshu Gupta
004b00bb96 config to explicitly specify classpath for hadoop container during hadoop ingestion 2016-03-25 10:51:28 -05:00
Fangjin Yang
ef0bc5d64f Merge pull request #2726 from gianm/move-flatten-json-benchmark-test
Move FlattenJSONBenchmarkUtilTest into the correct location.
2016-03-25 08:00:05 -07:00
Fangjin Yang
980943074b Merge pull request #2724 from guobingkun/conditional_multi_bind
Conditional multi bind
2016-03-24 17:09:27 -07:00
Nishant
0b03c9405f Merge pull request #2614 from sirpkt/calendric_gran
Support week, month, quarter, and year in query granularity
2016-03-24 16:21:01 -07:00
Gian Merlino
1869c350d8 Move FlattenJSONBenchmarkUtilTest into the correct location. 2016-03-24 15:47:54 -07:00
Bingkun Guo
62edbab434 Conditional multi bind
Usage example:
       ConditionalMultibind.create(props, binder, Animal.class)
                           .conditionBinder("animal.type", Predicates.equalTo("cat"), Cat.class)
                           .conditionBinder("animal.type", Predicates.equalTo("dog"), Dog.class);

At binding time, this will check the value set for property "animal.type" in props. If the value is "cat", it will
add a binding to Cat.class. If the value is "dog", it will add a binding to Dog.class.

At ingestion time, you will get the items that satisfy their corresponding predicates by calling
injector.getInstance(Key.get(new TypeLiteral<Set<Animal>>(){}))
2016-03-24 16:23:22 -05:00
Himanshu
56343c6cdc Merge pull request #2704 from navis/simple-optimize
optimize single elemented and/or filter
2016-03-24 16:13:48 -05:00
Himanshu
f26e73d133 Merge pull request #2720 from gianm/druid-api
Move druid-api into the druid repo.
2016-03-24 15:51:10 -05:00
Himanshu
fb25c083ec Merge pull request #2721 from guobingkun/refine_doc
refine description for mergeBytesLimit
2016-03-24 15:48:57 -05:00
Fangjin Yang
d4a96843f9 Merge pull request #2710 from himanshug/fix_sketch_estimate_comparator
fix SketchEstimate post aggregator's getComparator() and test changes to verify same
2016-03-24 13:34:32 -07:00
Bingkun Guo
0fa04305a6 refine description for mergeBytesLimit 2016-03-24 13:17:24 -05:00
Gian Merlino
7e7a886f65 Move druid-api into the druid repo.
This is from druid-api-0.3.17, as of commit 51884f1d05d5512cacaf62cedfbb28c6ab2535cf
in the druid-api repo.
2016-03-24 11:04:34 -07:00
Himanshu Gupta
4aead38130 fix SketchEstimate post aggregator's getComparator() and test changes to verify same 2016-03-24 10:11:06 -05:00
binlijin
2729efca71 implement special distinctcount 2016-03-24 11:11:11 +08:00
Gian Merlino
bfc0ae7d2a Merge pull request #2714 from rasahner/DOC_minorFix
dynamic config endpoint is at coordinator
2016-03-23 18:43:56 -07:00
Fangjin Yang
8750f4b58c Merge pull request #2713 from gianm/filter-extensions
Filters: Add filter.toFilter method, use that instead of the instanceof chain in Filters.
2016-03-23 18:04:08 -07:00
Gian Merlino
713062053c Filters: Add filter.toFilter method, use that instead of the instanceof chain in Filters.
I believe that the instanceof chain in Filters exists because in the past, Filter
and DimFilter were in different packages (DimFilter was in druid-client and Filter
was in druid-processing). And since druid-client didn't depend on druid-processing,
DimFilter couldn't have a toFilter method. But now it can.
2016-03-23 17:03:49 -07:00
Robin
448e0127b9 dynamic config endpoint is at coordinator 2016-03-23 17:22:19 -05:00
Fangjin Yang
a5d5529749 Merge pull request #2711 from gianm/filtered-aggregator-impls
All Filters should work with FilteredAggregators.
2016-03-23 13:37:21 -07:00
Gian Merlino
dd86198902 All Filters should work with FilteredAggregators.
This removes Filter.makeMatcher(ColumnSelectorFactory) and adds a
ValueMatcherFactory implementation to FilteredAggregatorFactory so it can
take advantage of existing makeMatcher(ValueMatcherFactory) implementations.

This patch also removes the Bound-based method from ValueMatcherFactory. Its
only user was the SpatialFilter, which could use the Predicate-based method.

Fixes #2604.
2016-03-23 12:24:01 -07:00
Bingkun Guo
a6af1974f7 Merge pull request #2699 from gianm/docs-fix-broken-links
Fix a bunch of broken links in the docs.
2016-03-23 13:14:29 -05:00
Himanshu
8a473111c5 Merge pull request #2650 from rasahner/IT_index_hadoop
add test for batch indexing from hadoop
2016-03-23 13:04:55 -05:00
Gian Merlino
2dfd3877c0 Fix a bunch of broken links in the docs. 2016-03-23 10:21:28 -07:00
Fangjin Yang
bbf08fcc24 Merge pull request #2675 from binlijin/clean_temp_file
clean tmp file when index merge fail
2016-03-23 09:09:07 -07:00
Fangjin Yang
9956df5609 Merge pull request #2705 from kilida/patch-1
Duplicate statement in ReservoirSegmentSamplerTest.java
2016-03-23 08:42:27 -07:00
Fangjin Yang
d1f8f2b2fd Merge pull request #2698 from druid-io/fix-ext-docs
refactor extensions into their own docs
2016-03-22 22:04:12 -07:00
binlijin
57d78d3293 clean tmp file when index merge fail 2016-03-23 10:55:12 +08:00
kilida
f25b2ed6f8 Duplicate statement in ReservoirSegmentSamplerTest.java 2016-03-22 22:14:36 -04:00
fjy
943cbe6e76 refactor extensions into their own docs 2016-03-22 18:54:10 -07:00
Fangjin Yang
041350c31b Merge pull request #2701 from gianm/mvd-docs
Improved docs for multi-value dimensions.
2016-03-22 18:09:37 -07:00