mirror of https://github.com/apache/druid.git
Lookups: Inherit "injective" from registered lookups, improve docs. (#5316)
Code changes: - In the lookup-based extractionFns, inherit injective property from the lookup itself if not specified. Doc changes: - Add a "Query execution" section to the lookups doc explaining how injective lookups and their optimizations work. - Remove scary warnings against using registeredLookup extractionFns. They are necessary and important since they work with filters and function cascades -- two things that the dimension specs do not do. They deserve to be first class citizens. - Move the "registeredLookup" fn above the "lookup" fn. It's probably more commonly used, so the docs read better this way.
This commit is contained in:
Gian Merlino
GitHub
parent
1fffc681d2
commit
ed47a1e1a9
|
|
@ -71,7 +71,7 @@ The parameters are as follows
|
|||
|--------|-----------|--------|-------|
|
||||
|`extractionNamespace`|Specifies how to populate the local cache. See below|Yes|-|
|
||||
|`firstCacheTimeout`|How long to wait (in ms) for the first run of the cache to populate. 0 indicates to not wait|No|`0` (do not wait)|
|
||||
|`injective`|If the underlying map is injective (keys and values are unique) then optimizations can occur internally by setting this to `true`|No|`false`|
|
||||
|`injective`|If the underlying map is [injective](../../querying/lookups.html#query-execution) (keys and values are unique) then optimizations can occur internally by setting this to `true`|No|`false`|
|
||||
|
||||
If `firstCacheTimeout` is set to a non-zero value, it should be less than `druid.manager.lookups.hostUpdateTimeout`. If `firstCacheTimeout` is NOT set, then management is essentially asynchronous and does not know if a lookup succeeded or failed in starting. In such a case logs from the lookup nodes should be monitored for repeated failures.
|
||||
|
||||
|
|
|
|||
|
|
@ -95,8 +95,6 @@ The default values are `replaceMissingValueWith = null` and `retainMissingValue
|
|||
|
||||
It is illegal to set `retainMissingValue = true` and also specify a `replaceMissingValueWith`.
|
||||
|
||||
A property of `injective` specifies if optimizations can be used which assume there is no combining of multiple names into one. For example: If ABC123 is the only key that maps to SomeCompany, that can be optimized since it is a unique lookup. But if both ABC123 and DEF456 BOTH map to SomeCompany, then that is NOT a unique lookup. Setting this value to true and setting `retainMissingValue` to FALSE (the default) may cause undesired behavior.
|
||||
|
||||
A property `optimize` can be supplied to allow optimization of lookup based extraction filter (by default `optimize = true`).
|
||||
|
||||
The second kind where it is not possible to pass at query time due to their size, will be based on an external lookup table or resource that is already registered via configuration file or/and coordinator.
|
||||
|
|
@ -316,11 +314,94 @@ Example for the `__time` dimension:
|
|||
JavaScript-based functionality is disabled by default. Please refer to the Druid <a href="../development/javascript.html">JavaScript programming guide</a> for guidelines about using Druid's JavaScript functionality, including instructions on how to enable it.
|
||||
</div>
|
||||
|
||||
### Lookup extraction function
|
||||
### Registered lookup extraction function
|
||||
|
||||
Lookups are a concept in Druid where dimension values are (optionally) replaced with new values.
|
||||
For more documentation on using lookups, please see [here](../querying/lookups.html).
|
||||
Explicit lookups allow you to specify a set of keys and values to use when performing the extraction.
|
||||
For more documentation on using lookups, please see [Lookups](../querying/lookups.html).
|
||||
The "registeredLookup" extraction function lets you refer to a lookup that has been registered in the cluster-wide
|
||||
configuration.
|
||||
|
||||
An example:
|
||||
|
||||
```json
|
||||
{
|
||||
"type":"registeredLookup",
|
||||
"lookup":"some_lookup_name",
|
||||
"retainMissingValue":true
|
||||
}
|
||||
```
|
||||
|
||||
A property of `retainMissingValue` and `replaceMissingValueWith` can be specified at query time to hint how to handle
|
||||
missing values. Setting `replaceMissingValueWith` to `""` has the same effect as setting it to `null` or omitting the
|
||||
property. Setting `retainMissingValue` to true will use the dimension's original value if it is not found in the lookup.
|
||||
The default values are `replaceMissingValueWith = null` and `retainMissingValue = false` which causes missing values to
|
||||
be treated as missing.
|
||||
|
||||
It is illegal to set `retainMissingValue = true` and also specify a `replaceMissingValueWith`.
|
||||
|
||||
A property of `injective` can override the lookup's own sense of whether or not it is
|
||||
[injective](lookups.html#query-execution). If left unspecified, Druid will use the registered cluster-wide lookup
|
||||
configuration.
|
||||
|
||||
A property `optimize` can be supplied to allow optimization of lookup based extraction filter (by default `optimize = true`).
|
||||
The optimization layer will run on the broker and it will rewrite the extraction filter as clause of selector filters.
|
||||
For instance the following filter
|
||||
|
||||
```json
|
||||
{
|
||||
"filter": {
|
||||
"type": "selector",
|
||||
"dimension": "product",
|
||||
"value": "bar_1",
|
||||
"extractionFn": {
|
||||
"type": "registeredLookup",
|
||||
"optimize": true,
|
||||
"lookup": "some_lookup_name"
|
||||
}
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
will be rewritten as the following simpler query, assuming a lookup that maps "product_1" and "product_3" to the value
|
||||
"bar_1":
|
||||
|
||||
```json
|
||||
{
|
||||
"filter":{
|
||||
"type":"or",
|
||||
"fields":[
|
||||
{
|
||||
"filter":{
|
||||
"type":"selector",
|
||||
"dimension":"product",
|
||||
"value":"product_1"
|
||||
}
|
||||
},
|
||||
{
|
||||
"filter":{
|
||||
"type":"selector",
|
||||
"dimension":"product",
|
||||
"value":"product_3"
|
||||
}
|
||||
}
|
||||
]
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
A null dimension value can be mapped to a specific value by specifying the empty string as the key in your lookup file.
|
||||
This allows distinguishing between a null dimension and a lookup resulting in a null.
|
||||
For example, specifying `{"":"bar","bat":"baz"}` with dimension values `[null, "foo", "bat"]` and replacing missing values with `"oof"` will yield results of `["bar", "oof", "baz"]`.
|
||||
Omitting the empty string key will cause the missing value to take over. For example, specifying `{"bat":"baz"}` with dimension values `[null, "foo", "bat"]` and replacing missing values with `"oof"` will yield results of `["oof", "oof", "baz"]`.
|
||||
|
||||
### Inline lookup extraction function
|
||||
|
||||
Lookups are a concept in Druid where dimension values are (optionally) replaced with new values.
|
||||
For more documentation on using lookups, please see [Lookups](../querying/lookups.html).
|
||||
The "lookup" extraction function lets you specify an inline lookup map without registering one in the cluster-wide
|
||||
configuration.
|
||||
|
||||
Examples:
|
||||
|
||||
```json
|
||||
{
|
||||
|
|
@ -347,108 +428,10 @@ Explicit lookups allow you to specify a set of keys and values to use when perfo
|
|||
}
|
||||
```
|
||||
|
||||
```json
|
||||
{
|
||||
"type":"lookup",
|
||||
"lookup":{"type":"namespace","namespace":"some_lookup"},
|
||||
"replaceMissingValueWith":"Unknown",
|
||||
"injective":false
|
||||
}
|
||||
```
|
||||
The inline lookup should be of type `map`.
|
||||
|
||||
```json
|
||||
{
|
||||
"type":"lookup",
|
||||
"lookup":{"type":"namespace","namespace":"some_lookup"},
|
||||
"retainMissingValue":true,
|
||||
"injective":false
|
||||
}
|
||||
```
|
||||
|
||||
A lookup can be of type `namespace` or `map`. A `map` lookup is passed as part of the query.
|
||||
A `namespace` lookup is populated on all the nodes which handle queries as per [lookups](../querying/lookups.html)
|
||||
|
||||
A property of `retainMissingValue` and `replaceMissingValueWith` can be specified at query time to hint how to handle missing values. Setting `replaceMissingValueWith` to `""` has the same effect as setting it to `null` or omitting the property. Setting `retainMissingValue` to true will use the dimension's original value if it is not found in the lookup. The default values are `replaceMissingValueWith = null` and `retainMissingValue = false` which causes missing values to be treated as missing.
|
||||
|
||||
It is illegal to set `retainMissingValue = true` and also specify a `replaceMissingValueWith`.
|
||||
|
||||
A property of `injective` specifies if optimizations can be used which assume there is no combining of multiple names into one. For example: If ABC123 is the only key that maps to SomeCompany, that can be optimized since it is a unique lookup. But if both ABC123 and DEF456 BOTH map to SomeCompany, then that is NOT a unique lookup. Setting this value to true and setting `retainMissingValue` to FALSE (the default) may cause undesired behavior.
|
||||
|
||||
A property `optimize` can be supplied to allow optimization of lookup based extraction filter (by default `optimize = true`).
|
||||
The optimization layer will run on the broker and it will rewrite the extraction filter as clause of selector filters.
|
||||
For instance the following filter
|
||||
|
||||
```json
|
||||
{
|
||||
"filter": {
|
||||
"type": "selector",
|
||||
"dimension": "product",
|
||||
"value": "bar_1",
|
||||
"extractionFn": {
|
||||
"type": "lookup",
|
||||
"optimize": true,
|
||||
"lookup": {
|
||||
"type": "map",
|
||||
"map": {
|
||||
"product_1": "bar_1",
|
||||
"product_3": "bar_1"
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
will be rewritten as
|
||||
|
||||
```json
|
||||
{
|
||||
"filter":{
|
||||
"type":"or",
|
||||
"fields":[
|
||||
{
|
||||
"filter":{
|
||||
"type":"selector",
|
||||
"dimension":"product",
|
||||
"value":"product_1"
|
||||
}
|
||||
},
|
||||
{
|
||||
"filter":{
|
||||
"type":"selector",
|
||||
"dimension":"product",
|
||||
"value":"product_3"
|
||||
}
|
||||
}
|
||||
]
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
A null dimension value can be mapped to a specific value by specifying the empty string as the key.
|
||||
This allows distinguishing between a null dimension and a lookup resulting in a null.
|
||||
For example, specifying `{"":"bar","bat":"baz"}` with dimension values `[null, "foo", "bat"]` and replacing missing values with `"oof"` will yield results of `["bar", "oof", "baz"]`.
|
||||
Omitting the empty string key will cause the missing value to take over. For example, specifying `{"bat":"baz"}` with dimension values `[null, "foo", "bat"]` and replacing missing values with `"oof"` will yield results of `["oof", "oof", "baz"]`.
|
||||
|
||||
### Registered Lookup Extraction Function
|
||||
|
||||
While it is recommended that the [lookup dimension spec](#lookup-dimensionspecs) be used whenever possible, any lookup that is registered for use as a lookup dimension spec can be used as a dimension extraction.
|
||||
|
||||
The specification for dimension extraction using dimension specification named lookups is formatted as per the following example:
|
||||
|
||||
```json
|
||||
{
|
||||
"type":"registeredLookup",
|
||||
"lookup":"some_lookup_name",
|
||||
"retainMissingValue":true,
|
||||
"injective":false
|
||||
}
|
||||
```
|
||||
|
||||
All the flags for [lookup extraction function](#lookup-extraction-function) apply here as well.
|
||||
|
||||
In general, the dimension specification should be used. This dimension **extraction** implementation is made available for testing, validation, and transitioning from dimension extraction to the dimension specification style lookups.
|
||||
There is also a chance that a feature uses dimension extraction in such a way that it is not applied to dimension specification lookups. Such a scenario should be brought to the attention of the development mailing list.
|
||||
The properties `retainMissingValue`, `replaceMissingValueWith`, `injective`, and `optimize` behave similarly to the
|
||||
[registered lookup extraction function](#registered-lookup-extraction-function).
|
||||
|
||||
### Cascade Extraction Function
|
||||
|
||||
|
|
|
|||
|
|
@ -8,15 +8,17 @@ layout: doc_page
|
|||
Lookups are an <a href="../development/experimental.html">experimental</a> feature.
|
||||
</div>
|
||||
|
||||
Lookups are a concept in Druid where dimension values are (optionally) replaced with new values.
|
||||
See [dimension specs](../querying/dimensionspecs.html) for more information. For the purpose of these documents,
|
||||
a "key" refers to a dimension value to match, and a "value" refers to its replacement.
|
||||
So if you wanted to rename `appid-12345` to `Super Mega Awesome App` then the key would be `appid-12345` and the value
|
||||
would be `Super Mega Awesome App`.
|
||||
Lookups are a concept in Druid where dimension values are (optionally) replaced with new values, allowing join-like
|
||||
functionality. Applying lookups in Druid is similar to joining a dimension table in a data warehouse. See
|
||||
[dimension specs](../querying/dimensionspecs.html) for more information. For the purpose of these documents, a "key"
|
||||
refers to a dimension value to match, and a "value" refers to its replacement. So if you wanted to map
|
||||
`appid-12345` to `Super Mega Awesome App` then the key would be `appid-12345` and the value would be
|
||||
`Super Mega Awesome App`.
|
||||
|
||||
It is worth noting that lookups support use cases where keys map to unique values (injective) such as a country code and
|
||||
a country name, and also supports use cases where multiple IDs map to the same value, e.g. multiple app-ids belonging to
|
||||
a single account manager.
|
||||
It is worth noting that lookups support not just use cases where keys map one-to-one to unique values, such as country
|
||||
code and country name, but also support use cases where multiple IDs map to the same value, e.g. multiple app-ids
|
||||
mapping to a single account manager. When lookups are one-to-one, Druid is able to apply additional optimizations at
|
||||
query time; see [Query execution](#query-execution) below for more details.
|
||||
|
||||
Lookups do not have history. They always use the current data. This means that if the chief account manager for a
|
||||
particular app-id changes, and you issue a query with a lookup to store the app-id to account manager relationship,
|
||||
|
|
@ -33,6 +35,38 @@ Other lookup types are available as extensions, including:
|
|||
- Globally cached lookups from local files, remote URIs, or JDBC through [lookups-cached-global](../development/extensions-core/lookups-cached-global.html).
|
||||
- Globally cached lookups from a Kafka topic through [kafka-extraction-namespace](../development/extensions-core/kafka-extraction-namespace.html).
|
||||
|
||||
Query Execution
|
||||
---------------
|
||||
When executing an aggregation query involving lookups, Druid can decide to apply lookups either while scanning and
|
||||
aggregating rows, or to apply them after aggregation is complete. It is more efficient to apply lookups after
|
||||
aggregation is complete, so Druid will do this if it can. Druid decides this by checking if the lookup is marked
|
||||
as "injective" or not. In general, you should set this property for any lookup that is naturally one-to-one, to allow
|
||||
Druid to run your queries as fast as possible.
|
||||
|
||||
Injective lookups should include _all_ possible keys that may show up in your dataset, and should also map all keys to
|
||||
_unique values_. This matters because non-injective lookups may map different keys to the same value, which must be
|
||||
accounted for during aggregation, lest query results contain two result values that should have been aggregated into
|
||||
one.
|
||||
|
||||
This lookup is injective (assuming it contains all possible keys from your data):
|
||||
|
||||
```
|
||||
1 -> Foo
|
||||
2 -> Bar
|
||||
3 -> Billy
|
||||
```
|
||||
|
||||
But this one is not, since both "2" and "3" map to the same key:
|
||||
|
||||
```
|
||||
1 -> Foo
|
||||
2 -> Bar
|
||||
3 -> Bar
|
||||
```
|
||||
|
||||
To tell Druid that your lookup is injective, you must specify `"injective" : true` in the lookup configuration. Druid
|
||||
will not detect this automatically.
|
||||
|
||||
Dynamic Configuration
|
||||
---------------------
|
||||
<div class="note caution">
|
||||
|
|
|
|||
|
|
@ -37,18 +37,14 @@ public class LookupExtractionFn extends FunctionalExtraction
|
|||
{
|
||||
private final LookupExtractor lookup;
|
||||
private final boolean optimize;
|
||||
// Thes are retained for auto generated hashCode and Equals
|
||||
private final boolean retainMissingValue;
|
||||
private final String replaceMissingValueWith;
|
||||
private final boolean injective;
|
||||
|
||||
@JsonCreator
|
||||
public LookupExtractionFn(
|
||||
@JsonProperty("lookup") final LookupExtractor lookup,
|
||||
@JsonProperty("retainMissingValue") final boolean retainMissingValue,
|
||||
@Nullable @JsonProperty("replaceMissingValueWith") final String replaceMissingValueWith,
|
||||
@JsonProperty("injective") final boolean injective,
|
||||
@JsonProperty("optimize") Boolean optimize
|
||||
@JsonProperty("injective") final Boolean injective,
|
||||
@JsonProperty("optimize") final Boolean optimize
|
||||
)
|
||||
{
|
||||
super(
|
||||
|
|
@ -63,13 +59,10 @@ public class LookupExtractionFn extends FunctionalExtraction
|
|||
},
|
||||
retainMissingValue,
|
||||
replaceMissingValueWith,
|
||||
injective
|
||||
injective != null ? injective : lookup.isOneToOne()
|
||||
);
|
||||
this.lookup = lookup;
|
||||
this.optimize = optimize == null ? true : optimize;
|
||||
this.retainMissingValue = retainMissingValue;
|
||||
this.injective = injective;
|
||||
this.replaceMissingValueWith = replaceMissingValueWith;
|
||||
}
|
||||
|
||||
|
||||
|
|
@ -175,9 +168,9 @@ public class LookupExtractionFn extends FunctionalExtraction
|
|||
return "LookupExtractionFn{" +
|
||||
"lookup=" + lookup +
|
||||
", optimize=" + optimize +
|
||||
", retainMissingValue=" + retainMissingValue +
|
||||
", replaceMissingValueWith='" + replaceMissingValueWith + '\'' +
|
||||
", injective=" + injective +
|
||||
", retainMissingValue=" + isRetainMissingValue() +
|
||||
", replaceMissingValueWith='" + getReplaceMissingValueWith() + '\'' +
|
||||
", injective=" + isInjective() +
|
||||
'}';
|
||||
}
|
||||
}
|
||||
|
|
|
|||
|
|
@ -20,7 +20,6 @@
|
|||
package io.druid.query.lookup;
|
||||
|
||||
import com.fasterxml.jackson.databind.ObjectMapper;
|
||||
import com.google.common.base.Function;
|
||||
import com.google.common.base.Strings;
|
||||
import com.google.common.collect.ImmutableList;
|
||||
import com.google.common.collect.ImmutableMap;
|
||||
|
|
@ -36,11 +35,11 @@ import org.junit.Test;
|
|||
import org.junit.runner.RunWith;
|
||||
import org.junit.runners.Parameterized;
|
||||
|
||||
import javax.annotation.Nullable;
|
||||
import java.io.IOException;
|
||||
import java.util.Arrays;
|
||||
import java.util.List;
|
||||
import java.util.Map;
|
||||
import java.util.Optional;
|
||||
|
||||
@RunWith(Parameterized.class)
|
||||
public class LookupExtractionFnTest
|
||||
|
|
@ -53,30 +52,23 @@ public class LookupExtractionFnTest
|
|||
ImmutableList.of(
|
||||
ImmutableSet.of(true, false),
|
||||
ImmutableSet.of("", "MISSING VALUE"),
|
||||
ImmutableSet.of(true, false)
|
||||
ImmutableSet.of(Optional.of(true), Optional.of(false), Optional.empty())
|
||||
)
|
||||
), new Function<List<?>, Object[]>()
|
||||
{
|
||||
@Nullable
|
||||
@Override
|
||||
public Object[] apply(List<?> input)
|
||||
{
|
||||
return input.toArray();
|
||||
}
|
||||
}
|
||||
),
|
||||
List::toArray
|
||||
);
|
||||
}
|
||||
|
||||
private static final ObjectMapper OBJECT_MAPPER = new DefaultObjectMapper();
|
||||
private final boolean retainMissing;
|
||||
private final String replaceMissing;
|
||||
private final boolean injective;
|
||||
private final Boolean injective;
|
||||
|
||||
public LookupExtractionFnTest(boolean retainMissing, String replaceMissing, boolean injective)
|
||||
public LookupExtractionFnTest(boolean retainMissing, String replaceMissing, Optional<Boolean> injective)
|
||||
{
|
||||
this.replaceMissing = Strings.emptyToNull(replaceMissing);
|
||||
this.retainMissing = retainMissing;
|
||||
this.injective = injective;
|
||||
this.injective = injective.orElse(null);
|
||||
}
|
||||
|
||||
@Test
|
||||
|
|
@ -136,7 +128,12 @@ public class LookupExtractionFnTest
|
|||
|
||||
Assert.assertEquals(retainMissing, lookupExtractionFn2.isRetainMissingValue());
|
||||
Assert.assertEquals(replaceMissing, lookupExtractionFn2.getReplaceMissingValueWith());
|
||||
|
||||
if (injective == null) {
|
||||
Assert.assertEquals(lookupExtractionFn2.getLookup().isOneToOne(), lookupExtractionFn2.isInjective());
|
||||
} else {
|
||||
Assert.assertEquals(injective, lookupExtractionFn2.isInjective());
|
||||
}
|
||||
|
||||
Assert.assertArrayEquals(lookupExtractionFn.getCacheKey(), lookupExtractionFn2.getCacheKey());
|
||||
|
||||
|
|
|
|||
|
|
@ -28,6 +28,7 @@ import io.druid.query.extraction.ExtractionFn;
|
|||
|
||||
import javax.annotation.Nullable;
|
||||
import java.nio.ByteBuffer;
|
||||
import java.util.Objects;
|
||||
|
||||
public class RegisteredLookupExtractionFn implements ExtractionFn
|
||||
{
|
||||
|
|
@ -38,7 +39,7 @@ public class RegisteredLookupExtractionFn implements ExtractionFn
|
|||
private final String lookup;
|
||||
private final boolean retainMissingValue;
|
||||
private final String replaceMissingValueWith;
|
||||
private final boolean injective;
|
||||
private final Boolean injective;
|
||||
private final boolean optimize;
|
||||
|
||||
@JsonCreator
|
||||
|
|
@ -47,7 +48,7 @@ public class RegisteredLookupExtractionFn implements ExtractionFn
|
|||
@JsonProperty("lookup") String lookup,
|
||||
@JsonProperty("retainMissingValue") final boolean retainMissingValue,
|
||||
@Nullable @JsonProperty("replaceMissingValueWith") final String replaceMissingValueWith,
|
||||
@JsonProperty("injective") final boolean injective,
|
||||
@JsonProperty("injective") final Boolean injective,
|
||||
@JsonProperty("optimize") Boolean optimize
|
||||
)
|
||||
{
|
||||
|
|
@ -79,7 +80,7 @@ public class RegisteredLookupExtractionFn implements ExtractionFn
|
|||
}
|
||||
|
||||
@JsonProperty("injective")
|
||||
public boolean isInjective()
|
||||
public Boolean isInjective()
|
||||
{
|
||||
return injective;
|
||||
}
|
||||
|
|
@ -142,13 +143,17 @@ public class RegisteredLookupExtractionFn implements ExtractionFn
|
|||
// http://www.javamex.com/tutorials/double_checked_locking.shtml
|
||||
synchronized (delegateLock) {
|
||||
if (null == delegate) {
|
||||
final LookupExtractor factory = Preconditions.checkNotNull(
|
||||
manager.get(getLookup()),
|
||||
"Lookup [%s] not found",
|
||||
getLookup()
|
||||
).getLookupExtractorFactory().get();
|
||||
|
||||
delegate = new LookupExtractionFn(
|
||||
Preconditions.checkNotNull(manager.get(getLookup()), "Lookup [%s] not found", getLookup())
|
||||
.getLookupExtractorFactory()
|
||||
.get(),
|
||||
factory,
|
||||
isRetainMissingValue(),
|
||||
getReplaceMissingValueWith(),
|
||||
isInjective(),
|
||||
injective == null ? factory.isOneToOne() : injective,
|
||||
isOptimize()
|
||||
);
|
||||
}
|
||||
|
|
@ -158,7 +163,7 @@ public class RegisteredLookupExtractionFn implements ExtractionFn
|
|||
}
|
||||
|
||||
@Override
|
||||
public boolean equals(Object o)
|
||||
public boolean equals(final Object o)
|
||||
{
|
||||
if (this == o) {
|
||||
return true;
|
||||
|
|
@ -166,35 +171,19 @@ public class RegisteredLookupExtractionFn implements ExtractionFn
|
|||
if (o == null || getClass() != o.getClass()) {
|
||||
return false;
|
||||
}
|
||||
|
||||
RegisteredLookupExtractionFn that = (RegisteredLookupExtractionFn) o;
|
||||
|
||||
if (isRetainMissingValue() != that.isRetainMissingValue()) {
|
||||
return false;
|
||||
}
|
||||
if (isInjective() != that.isInjective()) {
|
||||
return false;
|
||||
}
|
||||
if (isOptimize() != that.isOptimize()) {
|
||||
return false;
|
||||
}
|
||||
if (!getLookup().equals(that.getLookup())) {
|
||||
return false;
|
||||
}
|
||||
return getReplaceMissingValueWith() != null
|
||||
? getReplaceMissingValueWith().equals(that.getReplaceMissingValueWith())
|
||||
: that.getReplaceMissingValueWith() == null;
|
||||
final RegisteredLookupExtractionFn that = (RegisteredLookupExtractionFn) o;
|
||||
return retainMissingValue == that.retainMissingValue &&
|
||||
optimize == that.optimize &&
|
||||
Objects.equals(lookup, that.lookup) &&
|
||||
Objects.equals(replaceMissingValueWith, that.replaceMissingValueWith) &&
|
||||
Objects.equals(injective, that.injective);
|
||||
}
|
||||
|
||||
@Override
|
||||
public int hashCode()
|
||||
{
|
||||
int result = getLookup().hashCode();
|
||||
result = 31 * result + (isRetainMissingValue() ? 1 : 0);
|
||||
result = 31 * result + (getReplaceMissingValueWith() != null ? getReplaceMissingValueWith().hashCode() : 0);
|
||||
result = 31 * result + (isInjective() ? 1 : 0);
|
||||
result = 31 * result + (isOptimize() ? 1 : 0);
|
||||
return result;
|
||||
|
||||
return Objects.hash(lookup, retainMissingValue, replaceMissingValueWith, injective, optimize);
|
||||
}
|
||||
|
||||
@Override
|
||||
|
|
|
|||
|
|
@ -23,6 +23,7 @@ import com.fasterxml.jackson.databind.ObjectMapper;
|
|||
import com.google.common.collect.ImmutableMap;
|
||||
import io.druid.jackson.DefaultObjectMapper;
|
||||
import io.druid.java.util.common.jackson.JacksonUtils;
|
||||
import io.druid.query.extraction.ExtractionFn;
|
||||
import io.druid.query.extraction.MapLookupExtractor;
|
||||
import org.easymock.EasyMock;
|
||||
import org.junit.Assert;
|
||||
|
|
@ -57,16 +58,39 @@ public class RegisteredLookupExtractionFnTest
|
|||
LOOKUP_NAME,
|
||||
true,
|
||||
null,
|
||||
true,
|
||||
false,
|
||||
false
|
||||
);
|
||||
EasyMock.verify(manager);
|
||||
|
||||
Assert.assertEquals(false, fn.isInjective());
|
||||
Assert.assertEquals(ExtractionFn.ExtractionType.MANY_TO_ONE, fn.getExtractionType());
|
||||
|
||||
for (String orig : Arrays.asList("", "foo", "bat")) {
|
||||
Assert.assertEquals(LOOKUP_EXTRACTOR.apply(orig), fn.apply(orig));
|
||||
}
|
||||
Assert.assertEquals("not in the map", fn.apply("not in the map"));
|
||||
}
|
||||
|
||||
@Test
|
||||
public void testInheritInjective()
|
||||
{
|
||||
final LookupReferencesManager manager = EasyMock.createStrictMock(LookupReferencesManager.class);
|
||||
managerReturnsMap(manager);
|
||||
EasyMock.replay(manager);
|
||||
final RegisteredLookupExtractionFn fn = new RegisteredLookupExtractionFn(
|
||||
manager,
|
||||
LOOKUP_NAME,
|
||||
true,
|
||||
null,
|
||||
null,
|
||||
false
|
||||
);
|
||||
EasyMock.verify(manager);
|
||||
|
||||
Assert.assertNull(fn.isInjective());
|
||||
Assert.assertEquals(ExtractionFn.ExtractionType.ONE_TO_ONE, fn.getExtractionType());
|
||||
}
|
||||
|
||||
@Test
|
||||
public void testMissingDelegation()
|
||||
|
|
|
|||
Loading…
Reference in New Issue