Commit Graph

7214 Commits

Author SHA1 Message Date
navis.ryu 108535fd07 Implement native in filter (Fix for #2577) 2016-03-31 10:10:57 +09:00
navis.ryu e0cfd9ee19 Utility method for length estimation of utf8 2016-03-31 10:07:00 +09:00
jon-wei 5503bf1b38 Remove unnecessary type check in TimeAndDimsComp 2016-03-30 17:54:15 -07:00
fjy 14dbc431ef clean up for extensions docs 2016-03-30 17:14:58 -07:00
Fangjin Yang 595d359c3b Merge pull request #2618 from navis/fix-task-validation
Add missing modules for validating tasks
2016-03-30 08:34:57 -07:00
Fangjin Yang a8b28879f1 Merge pull request #2369 from du00cs/master
[Feature] Extension: Offline Ingestion with limited Parquet Support
2016-03-29 23:19:35 -07:00
navis.ryu c868d374c0 Add missing modules for validating tasks (Fix for #2618) 2016-03-30 14:39:20 +09:00
Fangjin Yang 23a8830bc2 Merge pull request #2757 from druid-io/fix-conf
Update libraries.md
2016-03-29 21:32:01 -07:00
DuNinglin [杜宁林] 0f67ff7dfb reoganize code folder according to recent upstream folder changes, seperate it from avro code and take it into extensions-conrib. docs rewite too 2016-03-30 11:21:41 +08:00
Fangjin Yang 95733a362f Merge pull request #2753 from gianm/null-filtering-multi-value-columns
More consistent empty-set filtering behavior on multi-value columns.
2016-03-29 18:52:25 -07:00
Charles Allen 95d42cfd9e Merge pull request #2758 from pjain1/fix_npe_in_filter
handle null values in In Filter
2016-03-29 17:53:02 -07:00
Gian Merlino 1853f36e9f More consistent empty-set filtering behavior on multi-value columns.
The behavior is now that filters on "null" will match rows with no
values. The behavior in the past was inconsistent; sometimes these
filters would match and sometimes they wouldn't.

Adds tests for this behavior to SelectorFilterTest and
BoundFilterTest, for query-level filters and filtered aggregates.

Fixes #2750.
2016-03-29 15:32:13 -07:00
r4ruchir 4bff008d65 Update libraries.md
Adding embedded-druid information in helper libraries
2016-03-29 15:16:36 -07:00
Parag Jain d892918a3d handle null values in In Filter 2016-03-29 17:03:26 -05:00
Fangjin Yang 77fff5709e Merge pull request #2756 from gianm/rabbit-bbirp
RabbitMQ: Swap SIRP to BBIRP so it can use more kinds of parsers.
2016-03-29 11:54:35 -07:00
Gian Merlino 9877c570d3 RabbitMQ: Swap SIRP to BBIRP so it can use more kinds of parsers. 2016-03-29 08:42:06 -07:00
Fangjin Yang 1e02eeab13 Merge pull request #2683 from metamx/default_retry
Better defaults for Retry policy for task actions
2016-03-29 08:02:59 -07:00
Nishant 0081bb87f0 Merge pull request #2755 from gianm/scary-warning
Overlord: Avoid a scary Jersey warning.
2016-03-29 07:59:53 -07:00
Fangjin Yang e023df2b92 Merge pull request #2754 from gianm/i-dont-get-it
Remove error suppression code from IncrementalIndexAdapter.
2016-03-28 19:29:53 -07:00
Gian Merlino 195c9c5240 Overlord: Avoid a scary Jersey warning.
Avoids the following message from being printed on Overlord startup:

WARNING: Parameter 1 of type io.druid.indexing.common.actions.TaskActionHolder<T> from
public <T> javax.ws.rs.core.Response io.druid.indexing.overlord.http.OverlordResource.doAction
(io.druid.indexing.common.actions.TaskActionHolder<T>) is not resolvable to a concrete type
2016-03-28 19:08:56 -07:00
binlijin 5120b8ab5c Merge pull request #2752 from druid-io/cleanup
cleanup distinct count agg
2016-03-29 09:58:18 +08:00
Gian Merlino c7ff0d698e Remove error suppression code from IncrementalIndexAdapter. 2016-03-28 18:40:27 -07:00
fjy c418a55638 cleanup distinct count agg 2016-03-28 17:29:41 -07:00
Fangjin Yang 62c1dc7a09 Merge pull request #2602 from binlijin/distinctcount
implement special distinctcount
2016-03-28 17:20:17 -07:00
Fangjin Yang c2284929dc Merge pull request #2739 from gianm/fix-wtmtest-failure
Fix handling of InterruptedException in WorkerTaskMonitor's mainLoop.
2016-03-28 14:52:10 -07:00
Fangjin Yang 9cb197adec Merge pull request #2722 from himanshug/fix_hadoop_jar_upload
config to explicitly specify classpath for hadoop container during hadoop ingestion
2016-03-28 14:49:03 -07:00
Fangjin Yang 7a84c267f7 Merge pull request #2743 from metamx/fixlookuptest
Fix LookupCoordinatorManager and Test for alerts
2016-03-28 14:41:37 -07:00
Charles Allen 4764e86409 Add docs for RegisteredDimensionExtractionFn 2016-03-28 13:27:49 -07:00
Charles Allen 4a98c4fbac Fix LookupExtractionFn equals and hashCode 2016-03-28 13:14:43 -07:00
Charles Allen 0ee861d0da Add ExtractionFn to LookupExtractor bridge 2016-03-28 13:14:43 -07:00
Parag Jain e2cabc93ca Merge pull request #2747 from gianm/fix-docs-kafka-lookup
Fix extension reference in Kafka namespaced lookup docs.
2016-03-28 12:41:36 -05:00
Parag Jain 89a8277ae2 Merge pull request #2712 from guobingkun/make_runnables_pluggable
make Coordinator IndexingService helpers pluggable
2016-03-28 12:18:18 -05:00
Charles Allen 05151bc325 Fix LookupCoordinatorManagerTest for alerts
* Also fixes bad alerting on missing nodes
2016-03-28 09:41:47 -07:00
Gian Merlino dbdfcd2443 Fix extension reference in Kafka namespaced lookup docs.
The reference to io.druid.extensions:kafka-extraction-namespace is wrong (should
be druid-kafka-extraction-namespace) and unnecessary (the extension id is written
at the top of the doc file).
2016-03-28 09:23:24 -07:00
Fangjin Yang 7fe277e6da Merge pull request #2727 from gianm/optimize-bound-filter
BoundFilter optimizations, and related interface changes.
2016-03-26 18:59:05 -07:00
Fangjin Yang 0dae28b6af Merge pull request #2729 from jon-wei/fix_hyperunique_comparator
Fix HyperUniquesAggregatorFactory comparator
2016-03-26 15:39:35 -07:00
Fangjin Yang a0216dcf7d Merge pull request #2735 from metamx/fixlookupDocs
Move lookup docs that are in druid-proper back into lookups.md
2016-03-26 15:38:48 -07:00
Fangjin Yang 3c4691aa5a Merge pull request #2741 from gianm/examples-wiki
Downgrade geoip2, exclude com.google.http-client.
2016-03-25 23:08:38 -07:00
Xavier Léauté 01f3221a62 Merge pull request #2665 from jisookim0513/remove-druid-server-serialization
remove serialization of DruidServer
2016-03-25 15:54:05 -07:00
Gian Merlino 977e867ad8 Downgrade geoip2, exclude com.google.http-client.
Reverts "Update com.maxmind.geoip2 to 2.6.0" and exclude the google http client
from com.maxmind.geoip2. This should satisfy the original need from #2646 (wanting
to run Druid along with an upgraded com.google.http-client) while preventing
Jackson conflicts pointed out in #2717.

Fixes #2717.

This reverts commit 21b7572533.
2016-03-25 14:43:22 -07:00
Gian Merlino 2970b49adc BoundFilter optimizations, and related interface changes.
BoundFilter:

- For lexicographic bounds, use bitmapIndex.getIndex to find the start and end points,
  then union all bitmaps between those points.
- For alphanumeric bounds, iterate through dimValues, and union all bitmaps for values
  matching the predicate.
- Change behavior for nulls: it used to be that the BoundFilter would never match nulls,
  now it matches nulls if "" is allowed by the lower limit and not excluded by the
  upper limit.

Interface changes:

- BitmapIndex: add `int getIndex(value)` to make it possible to get the index for a
  value without retrieving the bitmap.
- BitmapIndex: remove `ImmutableBitmap getBitmap(value)`, change callers to `getBitmap(getIndex(value))`.
- BitmapIndexSelector: allow retrieving the underlying BitmapIndex through getBitmapIndex.
- Clarified contract of indexOf in Indexed, GenericIndexed.

Also added tests for SelectorFilter, NotFilter, and BoundFilter.
2016-03-25 14:11:48 -07:00
Jonathan Wei 02d0c704b1 Merge pull request #2737 from gianm/fix-iisa-valueMatcher-predicate
Fix predicate-based ValueMatcher behavior for IncrementalIndex on missing columns.
2016-03-25 14:11:18 -07:00
jon-wei 9afaa2b94a Fix HyperUniquesAggregatorFactory comparator 2016-03-25 12:36:42 -07:00
Himanshu 2dff26512d Merge pull request #2733 from gianm/docs-datasketches-broken-link
Fix broken link in datasketches-aggregators.md.
2016-03-25 14:35:49 -05:00
jisookim 0d3c5a3b6c remove serialization of Druid Server and add tests for ServersResource 2016-03-25 12:27:27 -07:00
Gian Merlino ee4bb96855 Fix handling of InterruptedException in WorkerTaskMonitor's mainLoop.
I believe this will fix #2664.
2016-03-25 12:17:33 -07:00
Charles Allen ab324e4ac0 Move lookup docs that are in druid-proper back into lookups.md 2016-03-25 10:46:50 -07:00
Gian Merlino 4ac9e03161 Fix predicate-based ValueMatcher behavior for IncrementalIndex on missing columns.
Missing columns should be treated the same as columns containing 100% nulls.
2016-03-25 10:23:59 -07:00
Bingkun Guo 0872448ff0 make Coordinator IndexingService helpers pluggable
Fixes #2682
IndexingService helpers are added according to the settings in runtime.properties.
Rather than having all the config.isXXX checks there, it makes sense to have a pluggable
approach for allowing the dynamic configuration to bring in implementations for helpers
without having to have hard-coded sets of available helpers. Plus, it will also make it possible for extensions to plug helpers in.

With https://github.com/druid-io/druid-api/pull/76, we could conditionally bind a helper to Coordinator's runlist.
The condition is driven by the value set in the runtime.properties.
2016-03-25 11:48:54 -05:00
Gian Merlino 6d18382fb2 Fix broken link in datasketches-aggregators.md. 2016-03-25 09:32:40 -07:00