honeymoose/OpenSearch
1
0
Fork 0
mirror of https://github.com/honeymoose/OpenSearch.git synced 2025-04-04 14:28:32 +00:00
Code Issues Packages Projects Releases Wiki Activity
OpenSearch/src/test/java/org/elasticsearch/index/fielddata/AbstractNumericFieldDataTests.java

478 lines
24 KiB
Java
Raw Normal View History

initial commit
2013-01-18 12:00:49 +01:00
/*
* Licensed to ElasticSearch and Shay Banon under one
* or more contributor license agreements. See the NOTICE file
* distributed with this work for additional information
* regarding copyright ownership. ElasticSearch licenses this
* file to you under the Apache License, Version 2.0 (the
* "License"); you may not use this file except in compliance
* with the License. You may obtain a copy of the License at
*
* http://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing,
* software distributed under the License is distributed on an
* "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
* KIND, either express or implied. See the License for the
* specific language governing permissions and limitations
* under the License.
*/
Collapse test.{unit,integration} into org.elasticsearch.
2013-09-11 12:49:49 -05:00
package org.elasticsearch.index.fielddata;
initial commit
2013-01-18 12:00:49 +01:00
added predefined empty implementation for all atomic field datas
2013-01-22 14:45:09 +01:00
import org.apache.lucene.document.Document;
import org.apache.lucene.document.Field;
import org.apache.lucene.document.StringField;
Added sort by field that have multiple values per document. Closes #2634
2013-02-01 13:48:34 +01:00
import org.apache.lucene.index.DirectoryReader;
Cut over to JUnit & Randomized Runner from TestNG For better integration with the Lucene Test Framework and the availabilty of RandomizedRunner / Randommized Testing this commit moves over from TestNG to JUnit. This commit also moves relevant places over to RandomzedRunner for reproduceability and removes copied classes from the Lucene Test Framework.
2013-07-19 14:38:59 +02:00
import org.apache.lucene.search.*;
Added `sort_mode` option that defines what value to pick in the case the sort field is multi-valued. The `min` and `max` sort modes are supported for all field types. Either the lowest value or the highest value is picked. In addition to that number based fields also support `sum` and `avg` as sort mode. If `sum` sort mode is used then all the values for a field and belonging to a document are added together and the result of that is used as sort value. If the `avg` sort mode is used then the average of all values for the sort field belonging to that document is used as sort value. Relates to #2634
2013-02-12 20:34:26 +01:00
import org.elasticsearch.index.fielddata.fieldcomparator.SortMode;
Cut over to JUnit & Randomized Runner from TestNG For better integration with the Lucene Test Framework and the availabilty of RandomizedRunner / Randommized Testing this commit moves over from TestNG to JUnit. This commit also moves relevant places over to RandomzedRunner for reproduceability and removes copied classes from the Lucene Test Framework.
2013-07-19 14:38:59 +02:00
import org.junit.Test;
import static org.hamcrest.Matchers.equalTo;
initial commit
2013-01-18 12:00:49 +01:00
/**
*/
Configurable sort order for missing string values. This commit allows for configuring the sort order of missing values in BytesRef comparators (used for strings) with the following options: - _first: missing values will appear in the first positions, - _last: missing values will appear in the last positions (default), - <any value>: documents with missing sort value will use the given value when sorting. Since the default is _last, sorting by string value will have a different behavior than in previous versions of elasticsearch which used to insert missing value in the first positions when sorting in ascending order. Implementation notes: - Nested sorting is supported through the implementation of NestedWrappableComparator, - BytesRefValComparator was mostly broken since no field data implementation was using it, it is now tested through NoOrdinalsStringFieldDataTests, - Specialized BytesRefOrdValComparators have been removed now that the ordinals rely on packed arrays instead of raw arrays, - Field data tests hierarchy has been changed so that the numeric tests don't inherit from the string tests anymore, - When _first or _last is used, internally the comparators are told to use null or BytesRefFieldComparatorSource.MAX_TERM to replace missing values (depending on the sort order), - BytesRefValComparator just replaces missing values with the provided value and uses them for comparisons, - BytesRefOrdValComparator multiplies ordinals by 4 so that it can find ordinals for the missing value and the bottom value which are directly comparable with the segment ordinals. For example, if the segment values and ordinals are (a,1) and (b,2), they will be stored internally as (a,4) and (b,8) and if the missing value is 'ab', it will be assigned 6 as an ordinal, since it is between 'a' and 'b'. Then if the bottom value is 'abc', it will be assigned 7 as an ordinal since if it between 'ab' and 'b'. Closes #896
2013-08-02 13:06:13 +02:00
public abstract class AbstractNumericFieldDataTests extends AbstractFieldDataImplTests {
initial commit
2013-01-18 12:00:49 +01:00
protected abstract FieldDataType getFieldDataType();
@Test
public void testSingleValueAllSetNumber() throws Exception {
fillSingleValueAllSet();
IndexNumericFieldData indexFieldData = getForField("value");
AtomicNumericFieldData fieldData = indexFieldData.load(refreshReader());
assertThat(fieldData.getNumDocs(), equalTo(3));
LongValues longValues = fieldData.getLongValues();
assertThat(longValues.isMultiValued(), equalTo(false));
Refactor FieldData iterations This commit primarily folds [Double|Bytes|Long|GeoPoint]Values.Iter into [Double|Bytes|Long|GeoPoint]Values. Iterations now don't require a auxillary class (Iter) but instead driven by native for loops. All [Double|Bytes|Long|GeoPoint]Values are stateful and provide `setDocId` and `nextValue` methods to iterate over all values in a document. This has several advantage: * The amout of specialized classes is reduced * Iteration is clearly stateful ie. Iters can't be confused to be local. * All iterations are size bounded which prevents runtime checks and allows JIT optimizations / loop un-rolling and most iterations are branch free. * Due to the bounded iteration the need for a `hasNext` method call is removed. * Value iterations feels more native. This commit also adds consistent documentation and unifies the calcualtion if SortMode is involved. This commit also changes the runtime behavior of BytesValues#getValue() such that it will never return `null` anymore. If a document has no value in a field this method still returns a `BytesRef` with a `length` of 0. To identify documents with no values #hasValue() or #setDocument(int) should be used. The latter should be preferred if the value will be consumed in the case the document has a value.
2013-10-16 14:54:46 +02:00
assertThat(longValues.setDocument(0), equalTo(1));
assertThat(longValues.nextValue(), equalTo(2l));
initial commit
2013-01-18 12:00:49 +01:00
Refactor FieldData iterations This commit primarily folds [Double|Bytes|Long|GeoPoint]Values.Iter into [Double|Bytes|Long|GeoPoint]Values. Iterations now don't require a auxillary class (Iter) but instead driven by native for loops. All [Double|Bytes|Long|GeoPoint]Values are stateful and provide `setDocId` and `nextValue` methods to iterate over all values in a document. This has several advantage: * The amout of specialized classes is reduced * Iteration is clearly stateful ie. Iters can't be confused to be local. * All iterations are size bounded which prevents runtime checks and allows JIT optimizations / loop un-rolling and most iterations are branch free. * Due to the bounded iteration the need for a `hasNext` method call is removed. * Value iterations feels more native. This commit also adds consistent documentation and unifies the calcualtion if SortMode is involved. This commit also changes the runtime behavior of BytesValues#getValue() such that it will never return `null` anymore. If a document has no value in a field this method still returns a `BytesRef` with a `length` of 0. To identify documents with no values #hasValue() or #setDocument(int) should be used. The latter should be preferred if the value will be consumed in the case the document has a value.
2013-10-16 14:54:46 +02:00
assertThat(longValues.setDocument(1), equalTo(1));
assertThat(longValues.nextValue(), equalTo(1l));
initial commit
2013-01-18 12:00:49 +01:00
Refactor FieldData iterations This commit primarily folds [Double|Bytes|Long|GeoPoint]Values.Iter into [Double|Bytes|Long|GeoPoint]Values. Iterations now don't require a auxillary class (Iter) but instead driven by native for loops. All [Double|Bytes|Long|GeoPoint]Values are stateful and provide `setDocId` and `nextValue` methods to iterate over all values in a document. This has several advantage: * The amout of specialized classes is reduced * Iteration is clearly stateful ie. Iters can't be confused to be local. * All iterations are size bounded which prevents runtime checks and allows JIT optimizations / loop un-rolling and most iterations are branch free. * Due to the bounded iteration the need for a `hasNext` method call is removed. * Value iterations feels more native. This commit also adds consistent documentation and unifies the calcualtion if SortMode is involved. This commit also changes the runtime behavior of BytesValues#getValue() such that it will never return `null` anymore. If a document has no value in a field this method still returns a `BytesRef` with a `length` of 0. To identify documents with no values #hasValue() or #setDocument(int) should be used. The latter should be preferred if the value will be consumed in the case the document has a value.
2013-10-16 14:54:46 +02:00
assertThat(longValues.setDocument(2), equalTo(1));
assertThat(longValues.nextValue(), equalTo(3l));
initial commit
2013-01-18 12:00:49 +01:00
DoubleValues doubleValues = fieldData.getDoubleValues();
assertThat(doubleValues.isMultiValued(), equalTo(false));
Refactor FieldData iterations This commit primarily folds [Double|Bytes|Long|GeoPoint]Values.Iter into [Double|Bytes|Long|GeoPoint]Values. Iterations now don't require a auxillary class (Iter) but instead driven by native for loops. All [Double|Bytes|Long|GeoPoint]Values are stateful and provide `setDocId` and `nextValue` methods to iterate over all values in a document. This has several advantage: * The amout of specialized classes is reduced * Iteration is clearly stateful ie. Iters can't be confused to be local. * All iterations are size bounded which prevents runtime checks and allows JIT optimizations / loop un-rolling and most iterations are branch free. * Due to the bounded iteration the need for a `hasNext` method call is removed. * Value iterations feels more native. This commit also adds consistent documentation and unifies the calcualtion if SortMode is involved. This commit also changes the runtime behavior of BytesValues#getValue() such that it will never return `null` anymore. If a document has no value in a field this method still returns a `BytesRef` with a `length` of 0. To identify documents with no values #hasValue() or #setDocument(int) should be used. The latter should be preferred if the value will be consumed in the case the document has a value.
2013-10-16 14:54:46 +02:00
assertThat(1, equalTo(doubleValues.setDocument(0)));
assertThat(doubleValues.nextValue(), equalTo(2d));
initial commit
2013-01-18 12:00:49 +01:00
Refactor FieldData iterations This commit primarily folds [Double|Bytes|Long|GeoPoint]Values.Iter into [Double|Bytes|Long|GeoPoint]Values. Iterations now don't require a auxillary class (Iter) but instead driven by native for loops. All [Double|Bytes|Long|GeoPoint]Values are stateful and provide `setDocId` and `nextValue` methods to iterate over all values in a document. This has several advantage: * The amout of specialized classes is reduced * Iteration is clearly stateful ie. Iters can't be confused to be local. * All iterations are size bounded which prevents runtime checks and allows JIT optimizations / loop un-rolling and most iterations are branch free. * Due to the bounded iteration the need for a `hasNext` method call is removed. * Value iterations feels more native. This commit also adds consistent documentation and unifies the calcualtion if SortMode is involved. This commit also changes the runtime behavior of BytesValues#getValue() such that it will never return `null` anymore. If a document has no value in a field this method still returns a `BytesRef` with a `length` of 0. To identify documents with no values #hasValue() or #setDocument(int) should be used. The latter should be preferred if the value will be consumed in the case the document has a value.
2013-10-16 14:54:46 +02:00
assertThat(1, equalTo(doubleValues.setDocument(1)));
assertThat(doubleValues.nextValue(), equalTo(1d));
initial commit
2013-01-18 12:00:49 +01:00
Refactor FieldData iterations This commit primarily folds [Double|Bytes|Long|GeoPoint]Values.Iter into [Double|Bytes|Long|GeoPoint]Values. Iterations now don't require a auxillary class (Iter) but instead driven by native for loops. All [Double|Bytes|Long|GeoPoint]Values are stateful and provide `setDocId` and `nextValue` methods to iterate over all values in a document. This has several advantage: * The amout of specialized classes is reduced * Iteration is clearly stateful ie. Iters can't be confused to be local. * All iterations are size bounded which prevents runtime checks and allows JIT optimizations / loop un-rolling and most iterations are branch free. * Due to the bounded iteration the need for a `hasNext` method call is removed. * Value iterations feels more native. This commit also adds consistent documentation and unifies the calcualtion if SortMode is involved. This commit also changes the runtime behavior of BytesValues#getValue() such that it will never return `null` anymore. If a document has no value in a field this method still returns a `BytesRef` with a `length` of 0. To identify documents with no values #hasValue() or #setDocument(int) should be used. The latter should be preferred if the value will be consumed in the case the document has a value.
2013-10-16 14:54:46 +02:00
assertThat(1, equalTo(doubleValues.setDocument(2)));
assertThat(doubleValues.nextValue(), equalTo(3d));
initial commit
2013-01-18 12:00:49 +01:00
IndexSearcher searcher = new IndexSearcher(readerContext.reader());
TopFieldDocs topDocs;
topDocs = searcher.search(new MatchAllDocsQuery(), 10,
Added `sort_mode` option that defines what value to pick in the case the sort field is multi-valued. The `min` and `max` sort modes are supported for all field types. Either the lowest value or the highest value is picked. In addition to that number based fields also support `sum` and `avg` as sort mode. If `sum` sort mode is used then all the values for a field and belonging to a document are added together and the result of that is used as sort value. If the `avg` sort mode is used then the average of all values for the sort field belonging to that document is used as sort value. Relates to #2634
2013-02-12 20:34:26 +01:00
new Sort(new SortField("value", indexFieldData.comparatorSource(null, SortMode.MIN))));
initial commit
2013-01-18 12:00:49 +01:00
assertThat(topDocs.totalHits, equalTo(3));
assertThat(topDocs.scoreDocs[0].doc, equalTo(1));
assertThat(topDocs.scoreDocs[1].doc, equalTo(0));
assertThat(topDocs.scoreDocs[2].doc, equalTo(2));
topDocs = searcher.search(new MatchAllDocsQuery(), 10,
Added `sort_mode` option that defines what value to pick in the case the sort field is multi-valued. The `min` and `max` sort modes are supported for all field types. Either the lowest value or the highest value is picked. In addition to that number based fields also support `sum` and `avg` as sort mode. If `sum` sort mode is used then all the values for a field and belonging to a document are added together and the result of that is used as sort value. If the `avg` sort mode is used then the average of all values for the sort field belonging to that document is used as sort value. Relates to #2634
2013-02-12 20:34:26 +01:00
new Sort(new SortField("value", indexFieldData.comparatorSource(null, SortMode.MAX), true)));
initial commit
2013-01-18 12:00:49 +01:00
assertThat(topDocs.totalHits, equalTo(3));
assertThat(topDocs.scoreDocs[0].doc, equalTo(2));
assertThat(topDocs.scoreDocs[1].doc, equalTo(0));
assertThat(topDocs.scoreDocs[2].doc, equalTo(1));
}
@Test
public void testSingleValueWithMissingNumber() throws Exception {
fillSingleValueWithMissing();
IndexNumericFieldData indexFieldData = getForField("value");
AtomicNumericFieldData fieldData = indexFieldData.load(refreshReader());
assertThat(fieldData.getNumDocs(), equalTo(3));
LongValues longValues = fieldData.getLongValues();
assertThat(longValues.isMultiValued(), equalTo(false));
Refactor FieldData iterations This commit primarily folds [Double|Bytes|Long|GeoPoint]Values.Iter into [Double|Bytes|Long|GeoPoint]Values. Iterations now don't require a auxillary class (Iter) but instead driven by native for loops. All [Double|Bytes|Long|GeoPoint]Values are stateful and provide `setDocId` and `nextValue` methods to iterate over all values in a document. This has several advantage: * The amout of specialized classes is reduced * Iteration is clearly stateful ie. Iters can't be confused to be local. * All iterations are size bounded which prevents runtime checks and allows JIT optimizations / loop un-rolling and most iterations are branch free. * Due to the bounded iteration the need for a `hasNext` method call is removed. * Value iterations feels more native. This commit also adds consistent documentation and unifies the calcualtion if SortMode is involved. This commit also changes the runtime behavior of BytesValues#getValue() such that it will never return `null` anymore. If a document has no value in a field this method still returns a `BytesRef` with a `length` of 0. To identify documents with no values #hasValue() or #setDocument(int) should be used. The latter should be preferred if the value will be consumed in the case the document has a value.
2013-10-16 14:54:46 +02:00
assertThat(longValues.setDocument(0), equalTo(1));
assertThat(longValues.nextValue(), equalTo(2l));
initial commit
2013-01-18 12:00:49 +01:00
Refactor FieldData iterations This commit primarily folds [Double|Bytes|Long|GeoPoint]Values.Iter into [Double|Bytes|Long|GeoPoint]Values. Iterations now don't require a auxillary class (Iter) but instead driven by native for loops. All [Double|Bytes|Long|GeoPoint]Values are stateful and provide `setDocId` and `nextValue` methods to iterate over all values in a document. This has several advantage: * The amout of specialized classes is reduced * Iteration is clearly stateful ie. Iters can't be confused to be local. * All iterations are size bounded which prevents runtime checks and allows JIT optimizations / loop un-rolling and most iterations are branch free. * Due to the bounded iteration the need for a `hasNext` method call is removed. * Value iterations feels more native. This commit also adds consistent documentation and unifies the calcualtion if SortMode is involved. This commit also changes the runtime behavior of BytesValues#getValue() such that it will never return `null` anymore. If a document has no value in a field this method still returns a `BytesRef` with a `length` of 0. To identify documents with no values #hasValue() or #setDocument(int) should be used. The latter should be preferred if the value will be consumed in the case the document has a value.
2013-10-16 14:54:46 +02:00
assertThat(longValues.setDocument(1), equalTo(0));
Simplify FieldData API Removing #getValue and #hasValue to have a simple and consistent API for multiple values.
2013-10-30 15:17:50 +01:00
Refactor FieldData iterations This commit primarily folds [Double|Bytes|Long|GeoPoint]Values.Iter into [Double|Bytes|Long|GeoPoint]Values. Iterations now don't require a auxillary class (Iter) but instead driven by native for loops. All [Double|Bytes|Long|GeoPoint]Values are stateful and provide `setDocId` and `nextValue` methods to iterate over all values in a document. This has several advantage: * The amout of specialized classes is reduced * Iteration is clearly stateful ie. Iters can't be confused to be local. * All iterations are size bounded which prevents runtime checks and allows JIT optimizations / loop un-rolling and most iterations are branch free. * Due to the bounded iteration the need for a `hasNext` method call is removed. * Value iterations feels more native. This commit also adds consistent documentation and unifies the calcualtion if SortMode is involved. This commit also changes the runtime behavior of BytesValues#getValue() such that it will never return `null` anymore. If a document has no value in a field this method still returns a `BytesRef` with a `length` of 0. To identify documents with no values #hasValue() or #setDocument(int) should be used. The latter should be preferred if the value will be consumed in the case the document has a value.
2013-10-16 14:54:46 +02:00
assertThat(longValues.setDocument(2), equalTo(1));
assertThat(longValues.nextValue(), equalTo(3l));
initial commit
2013-01-18 12:00:49 +01:00
DoubleValues doubleValues = fieldData.getDoubleValues();
assertThat(doubleValues.isMultiValued(), equalTo(false));
Refactor FieldData iterations This commit primarily folds [Double|Bytes|Long|GeoPoint]Values.Iter into [Double|Bytes|Long|GeoPoint]Values. Iterations now don't require a auxillary class (Iter) but instead driven by native for loops. All [Double|Bytes|Long|GeoPoint]Values are stateful and provide `setDocId` and `nextValue` methods to iterate over all values in a document. This has several advantage: * The amout of specialized classes is reduced * Iteration is clearly stateful ie. Iters can't be confused to be local. * All iterations are size bounded which prevents runtime checks and allows JIT optimizations / loop un-rolling and most iterations are branch free. * Due to the bounded iteration the need for a `hasNext` method call is removed. * Value iterations feels more native. This commit also adds consistent documentation and unifies the calcualtion if SortMode is involved. This commit also changes the runtime behavior of BytesValues#getValue() such that it will never return `null` anymore. If a document has no value in a field this method still returns a `BytesRef` with a `length` of 0. To identify documents with no values #hasValue() or #setDocument(int) should be used. The latter should be preferred if the value will be consumed in the case the document has a value.
2013-10-16 14:54:46 +02:00
assertThat(1, equalTo(doubleValues.setDocument(0)));
assertThat(doubleValues.nextValue(), equalTo(2d));
initial commit
2013-01-18 12:00:49 +01:00
Refactor FieldData iterations This commit primarily folds [Double|Bytes|Long|GeoPoint]Values.Iter into [Double|Bytes|Long|GeoPoint]Values. Iterations now don't require a auxillary class (Iter) but instead driven by native for loops. All [Double|Bytes|Long|GeoPoint]Values are stateful and provide `setDocId` and `nextValue` methods to iterate over all values in a document. This has several advantage: * The amout of specialized classes is reduced * Iteration is clearly stateful ie. Iters can't be confused to be local. * All iterations are size bounded which prevents runtime checks and allows JIT optimizations / loop un-rolling and most iterations are branch free. * Due to the bounded iteration the need for a `hasNext` method call is removed. * Value iterations feels more native. This commit also adds consistent documentation and unifies the calcualtion if SortMode is involved. This commit also changes the runtime behavior of BytesValues#getValue() such that it will never return `null` anymore. If a document has no value in a field this method still returns a `BytesRef` with a `length` of 0. To identify documents with no values #hasValue() or #setDocument(int) should be used. The latter should be preferred if the value will be consumed in the case the document has a value.
2013-10-16 14:54:46 +02:00
assertThat(0, equalTo(doubleValues.setDocument(1)));
initial commit
2013-01-18 12:00:49 +01:00
Refactor FieldData iterations This commit primarily folds [Double|Bytes|Long|GeoPoint]Values.Iter into [Double|Bytes|Long|GeoPoint]Values. Iterations now don't require a auxillary class (Iter) but instead driven by native for loops. All [Double|Bytes|Long|GeoPoint]Values are stateful and provide `setDocId` and `nextValue` methods to iterate over all values in a document. This has several advantage: * The amout of specialized classes is reduced * Iteration is clearly stateful ie. Iters can't be confused to be local. * All iterations are size bounded which prevents runtime checks and allows JIT optimizations / loop un-rolling and most iterations are branch free. * Due to the bounded iteration the need for a `hasNext` method call is removed. * Value iterations feels more native. This commit also adds consistent documentation and unifies the calcualtion if SortMode is involved. This commit also changes the runtime behavior of BytesValues#getValue() such that it will never return `null` anymore. If a document has no value in a field this method still returns a `BytesRef` with a `length` of 0. To identify documents with no values #hasValue() or #setDocument(int) should be used. The latter should be preferred if the value will be consumed in the case the document has a value.
2013-10-16 14:54:46 +02:00
assertThat(1, equalTo(doubleValues.setDocument(2)));
assertThat(doubleValues.nextValue(), equalTo(3d));
initial commit
2013-01-18 12:00:49 +01:00
IndexSearcher searcher = new IndexSearcher(readerContext.reader());
TopFieldDocs topDocs;
topDocs = searcher.search(new MatchAllDocsQuery(), 10,
Added `sort_mode` option that defines what value to pick in the case the sort field is multi-valued. The `min` and `max` sort modes are supported for all field types. Either the lowest value or the highest value is picked. In addition to that number based fields also support `sum` and `avg` as sort mode. If `sum` sort mode is used then all the values for a field and belonging to a document are added together and the result of that is used as sort value. If the `avg` sort mode is used then the average of all values for the sort field belonging to that document is used as sort value. Relates to #2634
2013-02-12 20:34:26 +01:00
new Sort(new SortField("value", indexFieldData.comparatorSource(null, SortMode.MIN)))); // defaults to _last
initial commit
2013-01-18 12:00:49 +01:00
assertThat(topDocs.totalHits, equalTo(3));
assertThat(topDocs.scoreDocs[0].doc, equalTo(0));
assertThat(topDocs.scoreDocs[1].doc, equalTo(2));
assertThat(topDocs.scoreDocs[2].doc, equalTo(1));
topDocs = searcher.search(new MatchAllDocsQuery(), 10,
Added `sort_mode` option that defines what value to pick in the case the sort field is multi-valued. The `min` and `max` sort modes are supported for all field types. Either the lowest value or the highest value is picked. In addition to that number based fields also support `sum` and `avg` as sort mode. If `sum` sort mode is used then all the values for a field and belonging to a document are added together and the result of that is used as sort value. If the `avg` sort mode is used then the average of all values for the sort field belonging to that document is used as sort value. Relates to #2634
2013-02-12 20:34:26 +01:00
new Sort(new SortField("value", indexFieldData.comparatorSource(null, SortMode.MAX), true))); // defaults to _last
initial commit
2013-01-18 12:00:49 +01:00
assertThat(topDocs.totalHits, equalTo(3));
assertThat(topDocs.scoreDocs[0].doc, equalTo(2));
assertThat(topDocs.scoreDocs[1].doc, equalTo(0));
assertThat(topDocs.scoreDocs[2].doc, equalTo(1));
topDocs = searcher.search(new MatchAllDocsQuery(), 10,
Added `sort_mode` option that defines what value to pick in the case the sort field is multi-valued. The `min` and `max` sort modes are supported for all field types. Either the lowest value or the highest value is picked. In addition to that number based fields also support `sum` and `avg` as sort mode. If `sum` sort mode is used then all the values for a field and belonging to a document are added together and the result of that is used as sort value. If the `avg` sort mode is used then the average of all values for the sort field belonging to that document is used as sort value. Relates to #2634
2013-02-12 20:34:26 +01:00
new Sort(new SortField("value", indexFieldData.comparatorSource("_first", SortMode.MIN))));
initial commit
2013-01-18 12:00:49 +01:00
assertThat(topDocs.totalHits, equalTo(3));
assertThat(topDocs.scoreDocs[0].doc, equalTo(1));
assertThat(topDocs.scoreDocs[1].doc, equalTo(0));
assertThat(topDocs.scoreDocs[2].doc, equalTo(2));
topDocs = searcher.search(new MatchAllDocsQuery(), 10,
Added `sort_mode` option that defines what value to pick in the case the sort field is multi-valued. The `min` and `max` sort modes are supported for all field types. Either the lowest value or the highest value is picked. In addition to that number based fields also support `sum` and `avg` as sort mode. If `sum` sort mode is used then all the values for a field and belonging to a document are added together and the result of that is used as sort value. If the `avg` sort mode is used then the average of all values for the sort field belonging to that document is used as sort value. Relates to #2634
2013-02-12 20:34:26 +01:00
new Sort(new SortField("value", indexFieldData.comparatorSource("_first", SortMode.MAX), true)));
initial commit
2013-01-18 12:00:49 +01:00
assertThat(topDocs.totalHits, equalTo(3));
assertThat(topDocs.scoreDocs[0].doc, equalTo(1));
assertThat(topDocs.scoreDocs[1].doc, equalTo(2));
assertThat(topDocs.scoreDocs[2].doc, equalTo(0));
topDocs = searcher.search(new MatchAllDocsQuery(), 10,
Added `sort_mode` option that defines what value to pick in the case the sort field is multi-valued. The `min` and `max` sort modes are supported for all field types. Either the lowest value or the highest value is picked. In addition to that number based fields also support `sum` and `avg` as sort mode. If `sum` sort mode is used then all the values for a field and belonging to a document are added together and the result of that is used as sort value. If the `avg` sort mode is used then the average of all values for the sort field belonging to that document is used as sort value. Relates to #2634
2013-02-12 20:34:26 +01:00
new Sort(new SortField("value", indexFieldData.comparatorSource("1", SortMode.MIN))));
initial commit
2013-01-18 12:00:49 +01:00
assertThat(topDocs.totalHits, equalTo(3));
assertThat(topDocs.scoreDocs[0].doc, equalTo(1));
assertThat(topDocs.scoreDocs[1].doc, equalTo(0));
assertThat(topDocs.scoreDocs[2].doc, equalTo(2));
topDocs = searcher.search(new MatchAllDocsQuery(), 10,
Added `sort_mode` option that defines what value to pick in the case the sort field is multi-valued. The `min` and `max` sort modes are supported for all field types. Either the lowest value or the highest value is picked. In addition to that number based fields also support `sum` and `avg` as sort mode. If `sum` sort mode is used then all the values for a field and belonging to a document are added together and the result of that is used as sort value. If the `avg` sort mode is used then the average of all values for the sort field belonging to that document is used as sort value. Relates to #2634
2013-02-12 20:34:26 +01:00
new Sort(new SortField("value", indexFieldData.comparatorSource("1", SortMode.MAX), true)));
initial commit
2013-01-18 12:00:49 +01:00
assertThat(topDocs.totalHits, equalTo(3));
assertThat(topDocs.scoreDocs[0].doc, equalTo(2));
assertThat(topDocs.scoreDocs[1].doc, equalTo(0));
assertThat(topDocs.scoreDocs[2].doc, equalTo(1));
}
@Test
public void testMultiValueAllSetNumber() throws Exception {
fillMultiValueAllSet();
IndexNumericFieldData indexFieldData = getForField("value");
AtomicNumericFieldData fieldData = indexFieldData.load(refreshReader());
assertThat(fieldData.getNumDocs(), equalTo(3));
LongValues longValues = fieldData.getLongValues();
assertThat(longValues.isMultiValued(), equalTo(true));
Refactor FieldData iterations This commit primarily folds [Double|Bytes|Long|GeoPoint]Values.Iter into [Double|Bytes|Long|GeoPoint]Values. Iterations now don't require a auxillary class (Iter) but instead driven by native for loops. All [Double|Bytes|Long|GeoPoint]Values are stateful and provide `setDocId` and `nextValue` methods to iterate over all values in a document. This has several advantage: * The amout of specialized classes is reduced * Iteration is clearly stateful ie. Iters can't be confused to be local. * All iterations are size bounded which prevents runtime checks and allows JIT optimizations / loop un-rolling and most iterations are branch free. * Due to the bounded iteration the need for a `hasNext` method call is removed. * Value iterations feels more native. This commit also adds consistent documentation and unifies the calcualtion if SortMode is involved. This commit also changes the runtime behavior of BytesValues#getValue() such that it will never return `null` anymore. If a document has no value in a field this method still returns a `BytesRef` with a `length` of 0. To identify documents with no values #hasValue() or #setDocument(int) should be used. The latter should be preferred if the value will be consumed in the case the document has a value.
2013-10-16 14:54:46 +02:00
assertThat(longValues.setDocument(0), equalTo(2));
assertThat(longValues.nextValue(), equalTo(2l));
assertThat(longValues.nextValue(), equalTo(4l));
initial commit
2013-01-18 12:00:49 +01:00
Refactor FieldData iterations This commit primarily folds [Double|Bytes|Long|GeoPoint]Values.Iter into [Double|Bytes|Long|GeoPoint]Values. Iterations now don't require a auxillary class (Iter) but instead driven by native for loops. All [Double|Bytes|Long|GeoPoint]Values are stateful and provide `setDocId` and `nextValue` methods to iterate over all values in a document. This has several advantage: * The amout of specialized classes is reduced * Iteration is clearly stateful ie. Iters can't be confused to be local. * All iterations are size bounded which prevents runtime checks and allows JIT optimizations / loop un-rolling and most iterations are branch free. * Due to the bounded iteration the need for a `hasNext` method call is removed. * Value iterations feels more native. This commit also adds consistent documentation and unifies the calcualtion if SortMode is involved. This commit also changes the runtime behavior of BytesValues#getValue() such that it will never return `null` anymore. If a document has no value in a field this method still returns a `BytesRef` with a `length` of 0. To identify documents with no values #hasValue() or #setDocument(int) should be used. The latter should be preferred if the value will be consumed in the case the document has a value.
2013-10-16 14:54:46 +02:00
assertThat(longValues.setDocument(1), equalTo(1));
assertThat(longValues.nextValue(), equalTo(1l));
initial commit
2013-01-18 12:00:49 +01:00
Refactor FieldData iterations This commit primarily folds [Double|Bytes|Long|GeoPoint]Values.Iter into [Double|Bytes|Long|GeoPoint]Values. Iterations now don't require a auxillary class (Iter) but instead driven by native for loops. All [Double|Bytes|Long|GeoPoint]Values are stateful and provide `setDocId` and `nextValue` methods to iterate over all values in a document. This has several advantage: * The amout of specialized classes is reduced * Iteration is clearly stateful ie. Iters can't be confused to be local. * All iterations are size bounded which prevents runtime checks and allows JIT optimizations / loop un-rolling and most iterations are branch free. * Due to the bounded iteration the need for a `hasNext` method call is removed. * Value iterations feels more native. This commit also adds consistent documentation and unifies the calcualtion if SortMode is involved. This commit also changes the runtime behavior of BytesValues#getValue() such that it will never return `null` anymore. If a document has no value in a field this method still returns a `BytesRef` with a `length` of 0. To identify documents with no values #hasValue() or #setDocument(int) should be used. The latter should be preferred if the value will be consumed in the case the document has a value.
2013-10-16 14:54:46 +02:00
assertThat(longValues.setDocument(2), equalTo(1));
assertThat(longValues.nextValue(), equalTo(3l));
initial commit
2013-01-18 12:00:49 +01:00
DoubleValues doubleValues = fieldData.getDoubleValues();
assertThat(doubleValues.isMultiValued(), equalTo(true));
Refactor FieldData iterations This commit primarily folds [Double|Bytes|Long|GeoPoint]Values.Iter into [Double|Bytes|Long|GeoPoint]Values. Iterations now don't require a auxillary class (Iter) but instead driven by native for loops. All [Double|Bytes|Long|GeoPoint]Values are stateful and provide `setDocId` and `nextValue` methods to iterate over all values in a document. This has several advantage: * The amout of specialized classes is reduced * Iteration is clearly stateful ie. Iters can't be confused to be local. * All iterations are size bounded which prevents runtime checks and allows JIT optimizations / loop un-rolling and most iterations are branch free. * Due to the bounded iteration the need for a `hasNext` method call is removed. * Value iterations feels more native. This commit also adds consistent documentation and unifies the calcualtion if SortMode is involved. This commit also changes the runtime behavior of BytesValues#getValue() such that it will never return `null` anymore. If a document has no value in a field this method still returns a `BytesRef` with a `length` of 0. To identify documents with no values #hasValue() or #setDocument(int) should be used. The latter should be preferred if the value will be consumed in the case the document has a value.
2013-10-16 14:54:46 +02:00
assertThat(2, equalTo(doubleValues.setDocument(0)));
assertThat(doubleValues.nextValue(), equalTo(2d));
assertThat(doubleValues.nextValue(), equalTo(4d));
assertThat(1, equalTo(doubleValues.setDocument(1)));
assertThat(doubleValues.nextValue(), equalTo(1d));
assertThat(1, equalTo(doubleValues.setDocument(2)));
assertThat(doubleValues.nextValue(), equalTo(3d));
initial commit
2013-01-18 12:00:49 +01:00
}
@Test
public void testMultiValueWithMissingNumber() throws Exception {
fillMultiValueWithMissing();
IndexNumericFieldData indexFieldData = getForField("value");
AtomicNumericFieldData fieldData = indexFieldData.load(refreshReader());
assertThat(fieldData.getNumDocs(), equalTo(3));
LongValues longValues = fieldData.getLongValues();
assertThat(longValues.isMultiValued(), equalTo(true));
Refactor FieldData iterations This commit primarily folds [Double|Bytes|Long|GeoPoint]Values.Iter into [Double|Bytes|Long|GeoPoint]Values. Iterations now don't require a auxillary class (Iter) but instead driven by native for loops. All [Double|Bytes|Long|GeoPoint]Values are stateful and provide `setDocId` and `nextValue` methods to iterate over all values in a document. This has several advantage: * The amout of specialized classes is reduced * Iteration is clearly stateful ie. Iters can't be confused to be local. * All iterations are size bounded which prevents runtime checks and allows JIT optimizations / loop un-rolling and most iterations are branch free. * Due to the bounded iteration the need for a `hasNext` method call is removed. * Value iterations feels more native. This commit also adds consistent documentation and unifies the calcualtion if SortMode is involved. This commit also changes the runtime behavior of BytesValues#getValue() such that it will never return `null` anymore. If a document has no value in a field this method still returns a `BytesRef` with a `length` of 0. To identify documents with no values #hasValue() or #setDocument(int) should be used. The latter should be preferred if the value will be consumed in the case the document has a value.
2013-10-16 14:54:46 +02:00
assertThat(longValues.setDocument(0), equalTo(2));
assertThat(longValues.nextValue(), equalTo(2l));
assertThat(longValues.nextValue(), equalTo(4l));
initial commit
2013-01-18 12:00:49 +01:00
Refactor FieldData iterations This commit primarily folds [Double|Bytes|Long|GeoPoint]Values.Iter into [Double|Bytes|Long|GeoPoint]Values. Iterations now don't require a auxillary class (Iter) but instead driven by native for loops. All [Double|Bytes|Long|GeoPoint]Values are stateful and provide `setDocId` and `nextValue` methods to iterate over all values in a document. This has several advantage: * The amout of specialized classes is reduced * Iteration is clearly stateful ie. Iters can't be confused to be local. * All iterations are size bounded which prevents runtime checks and allows JIT optimizations / loop un-rolling and most iterations are branch free. * Due to the bounded iteration the need for a `hasNext` method call is removed. * Value iterations feels more native. This commit also adds consistent documentation and unifies the calcualtion if SortMode is involved. This commit also changes the runtime behavior of BytesValues#getValue() such that it will never return `null` anymore. If a document has no value in a field this method still returns a `BytesRef` with a `length` of 0. To identify documents with no values #hasValue() or #setDocument(int) should be used. The latter should be preferred if the value will be consumed in the case the document has a value.
2013-10-16 14:54:46 +02:00
assertThat(longValues.setDocument(1), equalTo(0));
initial commit
2013-01-18 12:00:49 +01:00
Refactor FieldData iterations This commit primarily folds [Double|Bytes|Long|GeoPoint]Values.Iter into [Double|Bytes|Long|GeoPoint]Values. Iterations now don't require a auxillary class (Iter) but instead driven by native for loops. All [Double|Bytes|Long|GeoPoint]Values are stateful and provide `setDocId` and `nextValue` methods to iterate over all values in a document. This has several advantage: * The amout of specialized classes is reduced * Iteration is clearly stateful ie. Iters can't be confused to be local. * All iterations are size bounded which prevents runtime checks and allows JIT optimizations / loop un-rolling and most iterations are branch free. * Due to the bounded iteration the need for a `hasNext` method call is removed. * Value iterations feels more native. This commit also adds consistent documentation and unifies the calcualtion if SortMode is involved. This commit also changes the runtime behavior of BytesValues#getValue() such that it will never return `null` anymore. If a document has no value in a field this method still returns a `BytesRef` with a `length` of 0. To identify documents with no values #hasValue() or #setDocument(int) should be used. The latter should be preferred if the value will be consumed in the case the document has a value.
2013-10-16 14:54:46 +02:00
assertThat(longValues.setDocument(2), equalTo(1));
assertThat(longValues.nextValue(), equalTo(3l));
initial commit
2013-01-18 12:00:49 +01:00
DoubleValues doubleValues = fieldData.getDoubleValues();
assertThat(doubleValues.isMultiValued(), equalTo(true));
Refactor FieldData iterations This commit primarily folds [Double|Bytes|Long|GeoPoint]Values.Iter into [Double|Bytes|Long|GeoPoint]Values. Iterations now don't require a auxillary class (Iter) but instead driven by native for loops. All [Double|Bytes|Long|GeoPoint]Values are stateful and provide `setDocId` and `nextValue` methods to iterate over all values in a document. This has several advantage: * The amout of specialized classes is reduced * Iteration is clearly stateful ie. Iters can't be confused to be local. * All iterations are size bounded which prevents runtime checks and allows JIT optimizations / loop un-rolling and most iterations are branch free. * Due to the bounded iteration the need for a `hasNext` method call is removed. * Value iterations feels more native. This commit also adds consistent documentation and unifies the calcualtion if SortMode is involved. This commit also changes the runtime behavior of BytesValues#getValue() such that it will never return `null` anymore. If a document has no value in a field this method still returns a `BytesRef` with a `length` of 0. To identify documents with no values #hasValue() or #setDocument(int) should be used. The latter should be preferred if the value will be consumed in the case the document has a value.
2013-10-16 14:54:46 +02:00
assertThat(2, equalTo(doubleValues.setDocument(0)));
assertThat(doubleValues.nextValue(), equalTo(2d));
assertThat(doubleValues.nextValue(), equalTo(4d));
initial commit
2013-01-18 12:00:49 +01:00
Refactor FieldData iterations This commit primarily folds [Double|Bytes|Long|GeoPoint]Values.Iter into [Double|Bytes|Long|GeoPoint]Values. Iterations now don't require a auxillary class (Iter) but instead driven by native for loops. All [Double|Bytes|Long|GeoPoint]Values are stateful and provide `setDocId` and `nextValue` methods to iterate over all values in a document. This has several advantage: * The amout of specialized classes is reduced * Iteration is clearly stateful ie. Iters can't be confused to be local. * All iterations are size bounded which prevents runtime checks and allows JIT optimizations / loop un-rolling and most iterations are branch free. * Due to the bounded iteration the need for a `hasNext` method call is removed. * Value iterations feels more native. This commit also adds consistent documentation and unifies the calcualtion if SortMode is involved. This commit also changes the runtime behavior of BytesValues#getValue() such that it will never return `null` anymore. If a document has no value in a field this method still returns a `BytesRef` with a `length` of 0. To identify documents with no values #hasValue() or #setDocument(int) should be used. The latter should be preferred if the value will be consumed in the case the document has a value.
2013-10-16 14:54:46 +02:00
assertThat(0, equalTo(doubleValues.setDocument(1)));
initial commit
2013-01-18 12:00:49 +01:00
Refactor FieldData iterations This commit primarily folds [Double|Bytes|Long|GeoPoint]Values.Iter into [Double|Bytes|Long|GeoPoint]Values. Iterations now don't require a auxillary class (Iter) but instead driven by native for loops. All [Double|Bytes|Long|GeoPoint]Values are stateful and provide `setDocId` and `nextValue` methods to iterate over all values in a document. This has several advantage: * The amout of specialized classes is reduced * Iteration is clearly stateful ie. Iters can't be confused to be local. * All iterations are size bounded which prevents runtime checks and allows JIT optimizations / loop un-rolling and most iterations are branch free. * Due to the bounded iteration the need for a `hasNext` method call is removed. * Value iterations feels more native. This commit also adds consistent documentation and unifies the calcualtion if SortMode is involved. This commit also changes the runtime behavior of BytesValues#getValue() such that it will never return `null` anymore. If a document has no value in a field this method still returns a `BytesRef` with a `length` of 0. To identify documents with no values #hasValue() or #setDocument(int) should be used. The latter should be preferred if the value will be consumed in the case the document has a value.
2013-10-16 14:54:46 +02:00
assertThat(1, equalTo(doubleValues.setDocument(2)));
assertThat(doubleValues.nextValue(), equalTo(3d));
initial commit
2013-01-18 12:00:49 +01:00
}
added predefined empty implementation for all atomic field datas
2013-01-22 14:45:09 +01:00
@Test
public void testMissingValueForAll() throws Exception {
fillAllMissing();
IndexNumericFieldData indexFieldData = getForField("value");
AtomicNumericFieldData fieldData = indexFieldData.load(refreshReader());
empty numeric field data should retain the correct num docs the fact that there are no values in the numeric field data, doesn't mean there are no docs, behavior should be to the bytes variant
2013-08-13 18:52:35 +02:00
assertThat(fieldData.getNumDocs(), equalTo(3));
added predefined empty implementation for all atomic field datas
2013-01-22 14:45:09 +01:00
// long values
LongValues longValues = fieldData.getLongValues();
assertThat(longValues.isMultiValued(), equalTo(false));
Refactor FieldData iterations This commit primarily folds [Double|Bytes|Long|GeoPoint]Values.Iter into [Double|Bytes|Long|GeoPoint]Values. Iterations now don't require a auxillary class (Iter) but instead driven by native for loops. All [Double|Bytes|Long|GeoPoint]Values are stateful and provide `setDocId` and `nextValue` methods to iterate over all values in a document. This has several advantage: * The amout of specialized classes is reduced * Iteration is clearly stateful ie. Iters can't be confused to be local. * All iterations are size bounded which prevents runtime checks and allows JIT optimizations / loop un-rolling and most iterations are branch free. * Due to the bounded iteration the need for a `hasNext` method call is removed. * Value iterations feels more native. This commit also adds consistent documentation and unifies the calcualtion if SortMode is involved. This commit also changes the runtime behavior of BytesValues#getValue() such that it will never return `null` anymore. If a document has no value in a field this method still returns a `BytesRef` with a `length` of 0. To identify documents with no values #hasValue() or #setDocument(int) should be used. The latter should be preferred if the value will be consumed in the case the document has a value.
2013-10-16 14:54:46 +02:00
assertThat(longValues.setDocument(0), equalTo(0));
assertThat(longValues.setDocument(1), equalTo(0));
assertThat(longValues.setDocument(2), equalTo(0));
added predefined empty implementation for all atomic field datas
2013-01-22 14:45:09 +01:00
// double values
DoubleValues doubleValues = fieldData.getDoubleValues();
assertThat(doubleValues.isMultiValued(), equalTo(false));
Refactor FieldData iterations This commit primarily folds [Double|Bytes|Long|GeoPoint]Values.Iter into [Double|Bytes|Long|GeoPoint]Values. Iterations now don't require a auxillary class (Iter) but instead driven by native for loops. All [Double|Bytes|Long|GeoPoint]Values are stateful and provide `setDocId` and `nextValue` methods to iterate over all values in a document. This has several advantage: * The amout of specialized classes is reduced * Iteration is clearly stateful ie. Iters can't be confused to be local. * All iterations are size bounded which prevents runtime checks and allows JIT optimizations / loop un-rolling and most iterations are branch free. * Due to the bounded iteration the need for a `hasNext` method call is removed. * Value iterations feels more native. This commit also adds consistent documentation and unifies the calcualtion if SortMode is involved. This commit also changes the runtime behavior of BytesValues#getValue() such that it will never return `null` anymore. If a document has no value in a field this method still returns a `BytesRef` with a `length` of 0. To identify documents with no values #hasValue() or #setDocument(int) should be used. The latter should be preferred if the value will be consumed in the case the document has a value.
2013-10-16 14:54:46 +02:00
assertThat(0, equalTo(doubleValues.setDocument(0)));
added predefined empty implementation for all atomic field datas
2013-01-22 14:45:09 +01:00
Refactor FieldData iterations This commit primarily folds [Double|Bytes|Long|GeoPoint]Values.Iter into [Double|Bytes|Long|GeoPoint]Values. Iterations now don't require a auxillary class (Iter) but instead driven by native for loops. All [Double|Bytes|Long|GeoPoint]Values are stateful and provide `setDocId` and `nextValue` methods to iterate over all values in a document. This has several advantage: * The amout of specialized classes is reduced * Iteration is clearly stateful ie. Iters can't be confused to be local. * All iterations are size bounded which prevents runtime checks and allows JIT optimizations / loop un-rolling and most iterations are branch free. * Due to the bounded iteration the need for a `hasNext` method call is removed. * Value iterations feels more native. This commit also adds consistent documentation and unifies the calcualtion if SortMode is involved. This commit also changes the runtime behavior of BytesValues#getValue() such that it will never return `null` anymore. If a document has no value in a field this method still returns a `BytesRef` with a `length` of 0. To identify documents with no values #hasValue() or #setDocument(int) should be used. The latter should be preferred if the value will be consumed in the case the document has a value.
2013-10-16 14:54:46 +02:00
assertThat(0, equalTo(doubleValues.setDocument(1)));
assertThat(0, equalTo(doubleValues.setDocument(2)));
}
added predefined empty implementation for all atomic field datas
2013-01-22 14:45:09 +01:00
protected void fillAllMissing() throws Exception {
Document d = new Document();
d.add(new StringField("_id", "1", Field.Store.NO));
writer.addDocument(d);
d = new Document();
d.add(new StringField("_id", "2", Field.Store.NO));
writer.addDocument(d);
d = new Document();
d.add(new StringField("_id", "3", Field.Store.NO));
writer.addDocument(d);
}
Added sort by field that have multiple values per document. Closes #2634
2013-02-01 13:48:34 +01:00
@Test
public void testSortMultiValuesFields() throws Exception {
fillExtendedMvSet();
IndexFieldData indexFieldData = getForField("value");
IndexSearcher searcher = new IndexSearcher(DirectoryReader.open(writer, true));
TopFieldDocs topDocs = searcher.search(new MatchAllDocsQuery(), 10,
Added `sort_mode` option that defines what value to pick in the case the sort field is multi-valued. The `min` and `max` sort modes are supported for all field types. Either the lowest value or the highest value is picked. In addition to that number based fields also support `sum` and `avg` as sort mode. If `sum` sort mode is used then all the values for a field and belonging to a document are added together and the result of that is used as sort value. If the `avg` sort mode is used then the average of all values for the sort field belonging to that document is used as sort value. Relates to #2634
2013-02-12 20:34:26 +01:00
new Sort(new SortField("value", indexFieldData.comparatorSource(null, SortMode.MIN)))); // defaults to _last
Added sort by field that have multiple values per document. Closes #2634
2013-02-01 13:48:34 +01:00
assertThat(topDocs.totalHits, equalTo(8));
assertThat(topDocs.scoreDocs.length, equalTo(8));
assertThat(topDocs.scoreDocs[0].doc, equalTo(7));
assertThat(((Number) ((FieldDoc) topDocs.scoreDocs[0]).fields[0]).intValue(), equalTo(-10));
assertThat(topDocs.scoreDocs[1].doc, equalTo(0));
assertThat(((Number) ((FieldDoc) topDocs.scoreDocs[1]).fields[0]).intValue(), equalTo(2));
assertThat(topDocs.scoreDocs[2].doc, equalTo(2));
assertThat(((Number) ((FieldDoc) topDocs.scoreDocs[2]).fields[0]).intValue(), equalTo(3));
assertThat(topDocs.scoreDocs[3].doc, equalTo(3));
assertThat(((Number) ((FieldDoc) topDocs.scoreDocs[3]).fields[0]).intValue(), equalTo(4));
assertThat(topDocs.scoreDocs[4].doc, equalTo(4));
assertThat(((Number) ((FieldDoc) topDocs.scoreDocs[4]).fields[0]).intValue(), equalTo(6));
assertThat(topDocs.scoreDocs[5].doc, equalTo(6));
assertThat(((Number) ((FieldDoc) topDocs.scoreDocs[5]).fields[0]).intValue(), equalTo(8));
assertThat(topDocs.scoreDocs[6].doc, equalTo(1));
// assertThat(((FieldDoc) topDocs.scoreDocs[6]).fields[0], equalTo(null));
assertThat(topDocs.scoreDocs[7].doc, equalTo(5));
// assertThat(((FieldDoc) topDocs.scoreDocs[7]).fields[0], equalTo(null));
topDocs = searcher.search(new MatchAllDocsQuery(), 10,
Added `sort_mode` option that defines what value to pick in the case the sort field is multi-valued. The `min` and `max` sort modes are supported for all field types. Either the lowest value or the highest value is picked. In addition to that number based fields also support `sum` and `avg` as sort mode. If `sum` sort mode is used then all the values for a field and belonging to a document are added together and the result of that is used as sort value. If the `avg` sort mode is used then the average of all values for the sort field belonging to that document is used as sort value. Relates to #2634
2013-02-12 20:34:26 +01:00
new Sort(new SortField("value", indexFieldData.comparatorSource(null, SortMode.MAX), true))); // defaults to _last
Added sort by field that have multiple values per document. Closes #2634
2013-02-01 13:48:34 +01:00
assertThat(topDocs.totalHits, equalTo(8));
assertThat(topDocs.scoreDocs.length, equalTo(8));
assertThat(topDocs.scoreDocs[0].doc, equalTo(6));
assertThat(((Number) ((FieldDoc) topDocs.scoreDocs[0]).fields[0]).intValue(), equalTo(10));
assertThat(topDocs.scoreDocs[1].doc, equalTo(4));
assertThat(((Number) ((FieldDoc) topDocs.scoreDocs[1]).fields[0]).intValue(), equalTo(8));
assertThat(topDocs.scoreDocs[2].doc, equalTo(3));
assertThat(((Number) ((FieldDoc) topDocs.scoreDocs[2]).fields[0]).intValue(), equalTo(6));
assertThat(topDocs.scoreDocs[3].doc, equalTo(0));
assertThat(((Number) ((FieldDoc) topDocs.scoreDocs[3]).fields[0]).intValue(), equalTo(4));
assertThat(topDocs.scoreDocs[4].doc, equalTo(2));
assertThat(((Number) ((FieldDoc) topDocs.scoreDocs[4]).fields[0]).intValue(), equalTo(3));
assertThat(topDocs.scoreDocs[5].doc, equalTo(7));
assertThat(((Number) ((FieldDoc) topDocs.scoreDocs[5]).fields[0]).intValue(), equalTo(-8));
assertThat(topDocs.scoreDocs[6].doc, equalTo(1));
// assertThat(((FieldDoc) topDocs.scoreDocs[6]).fields[0], equalTo(null));
assertThat(topDocs.scoreDocs[7].doc, equalTo(5));
// assertThat(((FieldDoc) topDocs.scoreDocs[7]).fields[0], equalTo(null));
Added `sort_mode` option that defines what value to pick in the case the sort field is multi-valued. The `min` and `max` sort modes are supported for all field types. Either the lowest value or the highest value is picked. In addition to that number based fields also support `sum` and `avg` as sort mode. If `sum` sort mode is used then all the values for a field and belonging to a document are added together and the result of that is used as sort value. If the `avg` sort mode is used then the average of all values for the sort field belonging to that document is used as sort value. Relates to #2634
2013-02-12 20:34:26 +01:00
searcher = new IndexSearcher(DirectoryReader.open(writer, true));
Added sort by field that have multiple values per document. Closes #2634
2013-02-01 13:48:34 +01:00
topDocs = searcher.search(new MatchAllDocsQuery(), 10,
Added `sort_mode` option that defines what value to pick in the case the sort field is multi-valued. The `min` and `max` sort modes are supported for all field types. Either the lowest value or the highest value is picked. In addition to that number based fields also support `sum` and `avg` as sort mode. If `sum` sort mode is used then all the values for a field and belonging to a document are added together and the result of that is used as sort value. If the `avg` sort mode is used then the average of all values for the sort field belonging to that document is used as sort value. Relates to #2634
2013-02-12 20:34:26 +01:00
new Sort(new SortField("value", indexFieldData.comparatorSource(null, SortMode.SUM)))); // defaults to _last
assertThat(topDocs.totalHits, equalTo(8));
assertThat(topDocs.scoreDocs.length, equalTo(8));
assertThat(topDocs.scoreDocs[0].doc, equalTo(7));
assertThat(((Number) ((FieldDoc) topDocs.scoreDocs[0]).fields[0]).intValue(), equalTo(-27));
assertThat(topDocs.scoreDocs[1].doc, equalTo(2));
assertThat(((Number) ((FieldDoc) topDocs.scoreDocs[1]).fields[0]).intValue(), equalTo(3));
assertThat(topDocs.scoreDocs[2].doc, equalTo(0));
assertThat(((Number) ((FieldDoc) topDocs.scoreDocs[2]).fields[0]).intValue(), equalTo(6));
assertThat(topDocs.scoreDocs[3].doc, equalTo(3));
assertThat(((Number) ((FieldDoc) topDocs.scoreDocs[3]).fields[0]).intValue(), equalTo(15));
assertThat(topDocs.scoreDocs[4].doc, equalTo(4));
assertThat(((Number) ((FieldDoc) topDocs.scoreDocs[4]).fields[0]).intValue(), equalTo(21));
assertThat(topDocs.scoreDocs[5].doc, equalTo(6));
assertThat(((Number) ((FieldDoc) topDocs.scoreDocs[5]).fields[0]).intValue(), equalTo(27));
assertThat(topDocs.scoreDocs[6].doc, equalTo(1));
// assertThat(((FieldDoc) topDocs.scoreDocs[6]).fields[0], equalTo(null));
assertThat(topDocs.scoreDocs[7].doc, equalTo(5));
// assertThat(((FieldDoc) topDocs.scoreDocs[7]).fields[0], equalTo(null));
searcher = new IndexSearcher(DirectoryReader.open(writer, true));
topDocs = searcher.search(new MatchAllDocsQuery(), 10,
new Sort(new SortField("value", indexFieldData.comparatorSource(null, SortMode.SUM), true))); // defaults to _last
assertThat(topDocs.totalHits, equalTo(8));
assertThat(topDocs.scoreDocs.length, equalTo(8));
assertThat(topDocs.scoreDocs[0].doc, equalTo(6));
assertThat(((Number) ((FieldDoc) topDocs.scoreDocs[0]).fields[0]).intValue(), equalTo(27));
assertThat(topDocs.scoreDocs[1].doc, equalTo(4));
assertThat(((Number) ((FieldDoc) topDocs.scoreDocs[1]).fields[0]).intValue(), equalTo(21));
assertThat(topDocs.scoreDocs[2].doc, equalTo(3));
assertThat(((Number) ((FieldDoc) topDocs.scoreDocs[2]).fields[0]).intValue(), equalTo(15));
assertThat(topDocs.scoreDocs[3].doc, equalTo(0));
assertThat(((Number) ((FieldDoc) topDocs.scoreDocs[3]).fields[0]).intValue(), equalTo(6));
assertThat(topDocs.scoreDocs[4].doc, equalTo(2));
assertThat(((Number) ((FieldDoc) topDocs.scoreDocs[4]).fields[0]).intValue(), equalTo(3));
assertThat(topDocs.scoreDocs[5].doc, equalTo(7));
assertThat(((Number) ((FieldDoc) topDocs.scoreDocs[5]).fields[0]).intValue(), equalTo(-27));
assertThat(topDocs.scoreDocs[6].doc, equalTo(1));
// assertThat(((FieldDoc) topDocs.scoreDocs[6]).fields[0], equalTo(null));
assertThat(topDocs.scoreDocs[7].doc, equalTo(5));
// assertThat(((FieldDoc) topDocs.scoreDocs[7]).fields[0], equalTo(null));
searcher = new IndexSearcher(DirectoryReader.open(writer, true));
topDocs = searcher.search(new MatchAllDocsQuery(), 10,
new Sort(new SortField("value", indexFieldData.comparatorSource(null, SortMode.AVG)))); // defaults to _last
assertThat(topDocs.totalHits, equalTo(8));
assertThat(topDocs.scoreDocs.length, equalTo(8));
assertThat(topDocs.scoreDocs[0].doc, equalTo(7));
assertThat(((Number) ((FieldDoc) topDocs.scoreDocs[0]).fields[0]).intValue(), equalTo(-9));
assertThat(topDocs.scoreDocs[1].doc, equalTo(0));
assertThat(((Number) ((FieldDoc) topDocs.scoreDocs[1]).fields[0]).intValue(), equalTo(3));
assertThat(topDocs.scoreDocs[2].doc, equalTo(2));
assertThat(((Number) ((FieldDoc) topDocs.scoreDocs[2]).fields[0]).intValue(), equalTo(3));
assertThat(topDocs.scoreDocs[3].doc, equalTo(3));
assertThat(((Number) ((FieldDoc) topDocs.scoreDocs[3]).fields[0]).intValue(), equalTo(5));
assertThat(topDocs.scoreDocs[4].doc, equalTo(4));
assertThat(((Number) ((FieldDoc) topDocs.scoreDocs[4]).fields[0]).intValue(), equalTo(7));
assertThat(topDocs.scoreDocs[5].doc, equalTo(6));
assertThat(((Number) ((FieldDoc) topDocs.scoreDocs[5]).fields[0]).intValue(), equalTo(9));
assertThat(topDocs.scoreDocs[6].doc, equalTo(1));
// assertThat(((FieldDoc) topDocs.scoreDocs[6]).fields[0], equalTo(null));
assertThat(topDocs.scoreDocs[7].doc, equalTo(5));
// assertThat(((FieldDoc) topDocs.scoreDocs[7]).fields[0], equalTo(null));
searcher = new IndexSearcher(DirectoryReader.open(writer, true));
topDocs = searcher.search(new MatchAllDocsQuery(), 10,
new Sort(new SortField("value", indexFieldData.comparatorSource(null, SortMode.AVG), true))); // defaults to _last
assertThat(topDocs.totalHits, equalTo(8));
assertThat(topDocs.scoreDocs.length, equalTo(8));
assertThat(topDocs.scoreDocs[0].doc, equalTo(6));
assertThat(((Number) ((FieldDoc) topDocs.scoreDocs[0]).fields[0]).intValue(), equalTo(9));
assertThat(topDocs.scoreDocs[1].doc, equalTo(4));
assertThat(((Number) ((FieldDoc) topDocs.scoreDocs[1]).fields[0]).intValue(), equalTo(7));
assertThat(topDocs.scoreDocs[2].doc, equalTo(3));
assertThat(((Number) ((FieldDoc) topDocs.scoreDocs[2]).fields[0]).intValue(), equalTo(5));
assertThat(topDocs.scoreDocs[3].doc, equalTo(0));
assertThat(((Number) ((FieldDoc) topDocs.scoreDocs[3]).fields[0]).intValue(), equalTo(3));
assertThat(topDocs.scoreDocs[4].doc, equalTo(2));
assertThat(((Number) ((FieldDoc) topDocs.scoreDocs[4]).fields[0]).intValue(), equalTo(3));
assertThat(topDocs.scoreDocs[5].doc, equalTo(7));
assertThat(((Number) ((FieldDoc) topDocs.scoreDocs[5]).fields[0]).intValue(), equalTo(-9));
assertThat(topDocs.scoreDocs[6].doc, equalTo(1));
// assertThat(((FieldDoc) topDocs.scoreDocs[6]).fields[0], equalTo(null));
assertThat(topDocs.scoreDocs[7].doc, equalTo(5));
// assertThat(((FieldDoc) topDocs.scoreDocs[7]).fields[0], equalTo(null));
topDocs = searcher.search(new MatchAllDocsQuery(), 10,
new Sort(new SortField("value", indexFieldData.comparatorSource("_first", SortMode.MIN))));
Added sort by field that have multiple values per document. Closes #2634
2013-02-01 13:48:34 +01:00
assertThat(topDocs.totalHits, equalTo(8));
assertThat(topDocs.scoreDocs.length, equalTo(8));
assertThat(topDocs.scoreDocs[0].doc, equalTo(1));
assertThat(topDocs.scoreDocs[1].doc, equalTo(5));
assertThat(topDocs.scoreDocs[2].doc, equalTo(7));
assertThat(topDocs.scoreDocs[3].doc, equalTo(0));
assertThat(topDocs.scoreDocs[4].doc, equalTo(2));
assertThat(topDocs.scoreDocs[5].doc, equalTo(3));
assertThat(topDocs.scoreDocs[6].doc, equalTo(4));
assertThat(topDocs.scoreDocs[7].doc, equalTo(6));
topDocs = searcher.search(new MatchAllDocsQuery(), 10,
Added `sort_mode` option that defines what value to pick in the case the sort field is multi-valued. The `min` and `max` sort modes are supported for all field types. Either the lowest value or the highest value is picked. In addition to that number based fields also support `sum` and `avg` as sort mode. If `sum` sort mode is used then all the values for a field and belonging to a document are added together and the result of that is used as sort value. If the `avg` sort mode is used then the average of all values for the sort field belonging to that document is used as sort value. Relates to #2634
2013-02-12 20:34:26 +01:00
new Sort(new SortField("value", indexFieldData.comparatorSource("_first", SortMode.MAX), true)));
Added sort by field that have multiple values per document. Closes #2634
2013-02-01 13:48:34 +01:00
assertThat(topDocs.totalHits, equalTo(8));
assertThat(topDocs.scoreDocs.length, equalTo(8));
assertThat(topDocs.scoreDocs[0].doc, equalTo(1));
assertThat(topDocs.scoreDocs[1].doc, equalTo(5));
assertThat(topDocs.scoreDocs[2].doc, equalTo(6));
assertThat(topDocs.scoreDocs[3].doc, equalTo(4));
assertThat(topDocs.scoreDocs[4].doc, equalTo(3));
assertThat(topDocs.scoreDocs[5].doc, equalTo(0));
assertThat(topDocs.scoreDocs[6].doc, equalTo(2));
assertThat(topDocs.scoreDocs[7].doc, equalTo(7));
topDocs = searcher.search(new MatchAllDocsQuery(), 10,
Added `sort_mode` option that defines what value to pick in the case the sort field is multi-valued. The `min` and `max` sort modes are supported for all field types. Either the lowest value or the highest value is picked. In addition to that number based fields also support `sum` and `avg` as sort mode. If `sum` sort mode is used then all the values for a field and belonging to a document are added together and the result of that is used as sort value. If the `avg` sort mode is used then the average of all values for the sort field belonging to that document is used as sort value. Relates to #2634
2013-02-12 20:34:26 +01:00
new Sort(new SortField("value", indexFieldData.comparatorSource("-9", SortMode.MIN))));
Added sort by field that have multiple values per document. Closes #2634
2013-02-01 13:48:34 +01:00
assertThat(topDocs.totalHits, equalTo(8));
assertThat(topDocs.scoreDocs.length, equalTo(8));
assertThat(topDocs.scoreDocs[0].doc, equalTo(7));
assertThat(topDocs.scoreDocs[1].doc, equalTo(1));
assertThat(topDocs.scoreDocs[2].doc, equalTo(5));
assertThat(topDocs.scoreDocs[3].doc, equalTo(0));
assertThat(topDocs.scoreDocs[4].doc, equalTo(2));
assertThat(topDocs.scoreDocs[5].doc, equalTo(3));
assertThat(topDocs.scoreDocs[6].doc, equalTo(4));
assertThat(topDocs.scoreDocs[7].doc, equalTo(6));
topDocs = searcher.search(new MatchAllDocsQuery(), 10,
Added `sort_mode` option that defines what value to pick in the case the sort field is multi-valued. The `min` and `max` sort modes are supported for all field types. Either the lowest value or the highest value is picked. In addition to that number based fields also support `sum` and `avg` as sort mode. If `sum` sort mode is used then all the values for a field and belonging to a document are added together and the result of that is used as sort value. If the `avg` sort mode is used then the average of all values for the sort field belonging to that document is used as sort value. Relates to #2634
2013-02-12 20:34:26 +01:00
new Sort(new SortField("value", indexFieldData.comparatorSource("9", SortMode.MAX), true)));
Added sort by field that have multiple values per document. Closes #2634
2013-02-01 13:48:34 +01:00
assertThat(topDocs.totalHits, equalTo(8));
assertThat(topDocs.scoreDocs.length, equalTo(8));
assertThat(topDocs.scoreDocs[0].doc, equalTo(6));
assertThat(topDocs.scoreDocs[1].doc, equalTo(1));
assertThat(topDocs.scoreDocs[2].doc, equalTo(5));
assertThat(topDocs.scoreDocs[3].doc, equalTo(4));
assertThat(topDocs.scoreDocs[4].doc, equalTo(3));
assertThat(topDocs.scoreDocs[5].doc, equalTo(0));
assertThat(topDocs.scoreDocs[6].doc, equalTo(2));
assertThat(topDocs.scoreDocs[7].doc, equalTo(7));
}
initial commit
2013-01-18 12:00:49 +01:00
}
Reference in New Issue Copy Permalink