Commit Graph

28726 Commits

Author SHA1 Message Date
Jim Ferenczi 3435c9f4e2 #26496: Fix sporadic failure of ContextCompletionSuggestSearchIT#testGeoBoosting
This test should not rely on strict ordering for same score suggestions.
The Lucene completion suggester uses the doc id in case of a tie and documents are indexed randomly.
2017-09-08 11:30:40 +02:00
Jason Tedor e3b0cc9867 Remove norelease regarding destroying history
This commit removes a norelease from the codebase now that there is a CI
job that fails on the norelease pattern being present. Instead, a new
issue has been opened to track this one.

Relates #26544
2017-09-07 21:57:08 -04:00
Andy Bristol 33faf5ec70 forbid ICU Collator creation with default locale (#26476) 2017-09-07 14:47:52 -07:00
Jim Ferenczi e684c5e0a5 #26496: handle `shard_size` correctly in the completion suggester and tests.
The completion suggester has a `shard_size` option that sets the size of the suggestions to retrieve per shard but it is ignored
 by the builder. This commit restores the handling of this option and fixes a test that can randomly fail without it.
2017-09-07 18:22:28 +02:00
Lee Hinman cff904bf97 Enable adaptive replica selection by default (#26522)
Relates to #24915
2017-09-07 09:25:05 -06:00
Jim Ferenczi d68d8c9cef Expose duplicate removal in the completion suggester (#26496)
This change exposes the duplicate removal option added in Lucene for the completion suggester
with a new option called `skip_duplicates` (defaults to false).
This commit also adapts the custom suggest collector to handle deduplication when multiple contexts match the input.

Closes #23364
2017-09-07 17:11:01 +02:00
Jim Ferenczi abe83c4fac Fail query when a sort is provided in conjunction with rescorers (#26510)
This change fixes a regression introduced in 6 that removes the skipping of the rescore phase
when a sort other than _score is used.
We now fail the request when a sort is provided in conjunction with rescore instead of just skipping the rescore phase
This commit also adds an assert that checks if the topdocs are sorted by _score after the rescoring.
This is the responsibility of the rescorer to make sure that topdocs are sorted after rescore so we
just check that this condition is met in the rescore phase.
2017-09-07 14:17:37 +02:00
Jim Ferenczi b41c44370e #26448: Fix rest test not cleaning the settings 2017-09-07 14:11:04 +02:00
Christoph Büscher ba02485541 Make sure SortBuilders rewrite inner nested sorts (#26532)
The three SortBuilders that can have inner NestedSortBuilders currently don't
rewrite any of the filters contained in them. This change adds a rewrite method
to NestedSortBuilder and changes rewriting in FieldSortBuilder,
ScriptSortBuilder and GeoDistanceSortBuilder to make sure inner nested sorts get
rewritten if they need to.
2017-09-07 14:04:50 +02:00
Jim Ferenczi 6f69b25f61 Restore bwc skip test because backport is missing 2017-09-07 12:38:41 +02:00
Jim Ferenczi 1b3b1c0647 Set bwc versions for scroll keep alive rest tests (backported to 6.1) 2017-09-07 12:34:33 +02:00
Christoph Büscher 47ffa17efb Extend testing of build method in ScriptSortBuilder (#26520)
Improve testing around the ScriptSortBuilder#build method, adding checks for
correct transfers of the sort mode and nested sorts.

Also changing the behaviour around the nested_path, nested_filter vs. nested
parameter in a similar way as in #26490 and deprecating the setters/getters for
the old syntax.

Closes #17286
2017-09-07 10:37:50 +02:00
Ryan Ernst c9964d17bf Internal: Add versionless alias for rest client codebase in policy files (#26521)
Security manager policy files contains grants for specific codebases,
where a codebase is a jar file. We use a system property containing the
name of the jar file to resolve the jar file location when parsing the
policy file. However, this means the version of the jars must be
modified when versions of dependencies change. This is particularly
messy for elasticsearch, where we now have a dependency on the rest
client, and need to support both a snapshot version for testing and non
snapshot for release.

This commit adds an alias for the elasticsearch rest client without a
version to be used in policy files. That allows the policy files to not care whether
the rest client is a snapshot or release.
2017-09-06 18:57:10 -07:00
Lee Hinman fe02350e73 With too many incoming tasks, reset measurements to 1ns instead of 0ns
Resoves #26332 where too many tasks occurred while adjustment was happening, the
measurements were reset to 0, and then an assert failed due to tasks executing
in 0 nanoseconds
2017-09-06 15:34:51 -06:00
Jason Tedor 9c795bd838 Fix cache compute if absent for expired entries
When a cache entry expires, it remains in the cache (both the segment
that it belongs to, and the LRU list) until an eviction occurs. The
problem here is that the compute if absent implementation relies on
there not being an association to a key that we are trying to put
because it internally uses put if absent on the underlying segment. If
we try to put an association for a key that has expired but not been
evicted, then compute if absent will return as if there is nothing in
the cache for the given key, yet no call to compute if absent will
succeed in putting a new association for the key. To remedy this, we
modify the internal get method for the cache to let the caller take
action if the entry they are retrieving is expired. This allows the
compute if absent method to take the action of evicting the entry from
the cache, thus allowing the put if absent method used by compute if
absent to succeed for one of the callers trying to compute if absent a
new association.

Relates #26516
2017-09-06 13:44:20 -04:00
Tanguy Leroux ecf39bc0c1 [Docs] Add "Using Java Builders" section (#26517)
The current "Building Queries" and "Building Aggregations" pages are
located under the "Supported Apis" section because they are linked to
the "Search API" page.

It should instead be in a dedicated section: this commit adds a new
"Using Java Builders" section and renames few filenames in favor of
more meaningful names.
2017-09-06 14:06:41 +02:00
Jim Ferenczi 0c799eedc5 Add upper limit for scroll expiry (#26448)
This change adds a dynamic cluster setting named `search.max_keep_alive`.
It is used as an upper limit for scroll expiry time in scroll queries and defaults to 1 hour.
This change also ensures that the existing setting `search.default_keep_alive` is always smaller than `search.max_keep_alive`.

Relates #11511

* check style

* add skip for bwc

* iter

* Add a maxium throttle wait time of 1h for reindex

* review

* remove empty line
2017-09-06 10:06:48 +02:00
Tanguy Leroux ad355f3e0b Forbid direct usage of ContentType.create() methods (#26457)
It's easy to create a wrong Content-Type header when converting a 
XContentType to a Apache HTTP ContentType instance.

This commit the direct usages of ContentType.create() methods in favor of a Request.createContentType(XContentType) method that does the right thing.

Closes #26438
2017-09-06 09:58:46 +02:00
Martijn van Groningen 6bdf591193
removed unused import 2017-09-06 07:01:58 +02:00
Martijn van Groningen 77bbe99102
Fix two unreleased percolator query analyze bugs
* If in a range query upper is smaller than lower then ignore the range query
* If two empty range extractions are compared don't fail with NoSuchElementException
2017-09-06 06:47:01 +02:00
Martijn van Groningen 2ad3608245
percolator: handle point queries with 2 or more dimensions correctly 2017-09-06 06:36:47 +02:00
Jason Tedor f6a489f323 Add Log4j to SLF4J binding for repository-hdfs
This commit adds the Log4j to SLF4J binding JAR to the repository-hdfs
plugin so that SLF4J can detect Log4j at runtime and therefore use the
server Log4j implementation for logging (and the usual Elasticsearch
APIs can be used for setting logging levels).

Relates #26514
2017-09-05 19:38:17 -04:00
Christoph Büscher 1b49bf3079 Remove deprecated parameters from `ids_query` (#26508)
The `_type` and `types` version of the current `type` parameter have been
deprecated since 5.0. We can remove support for them in 7.0 and also in 6.x and
6.0.
2017-09-05 18:12:31 +02:00
Tim Brooks c1a20f7e48 Merge tsa with ts (#26369)
We currently have a weird relationship between Transport,
TransportService, and TransportServiceAdaptor. At some point I think
that we would like to collapse these all into one concept as we only
support TCP transports.

This commit moves in that direction by eliminating the adaptor and just
passing the transport service to the transport.
2017-09-05 09:15:56 -06:00
Christoph Büscher 760bd6c568 Extend testing of build method in GeoDistanceSortBuilder (#26498)
Improve testing around the GeoDistanceSortBuilder#build method, adding checks for correct
transfers of the sort order, mode, nested sorts and points validation and coercion.

Also changing the behaviour around the nested_path, nested_filter vs. nested parameter in
a similar way as in #26490 and deprecating the setters/getters for the old syntax.

Relates to #17286
2017-09-05 14:38:10 +02:00
marcocova eeded72b19 [Docs] Fix wrong indent in gateway documentation (#26501)
This changeset fixes a spurious indent that causes a code block to be generated instead of a regular paragraph.
2017-09-05 10:42:58 +02:00
Martijn van Groningen 78e9c96d7f
Added a limit to from + size in top_hits and inner hits.
Relates to #11511
2017-09-05 08:44:45 +02:00
Christoph Büscher 8f0369296f Prohibit using `nested_filter`, `nested_path` and new `nested` Option at the same time in FieldSortBuilder (#26490)
Currently we allow both "old" and "new" way of setting nested sorts on the
FieldSortBuilder at the same time. This should throw an error, instead the user
should choose one of the two possible options.

Also adding testing for the now deprecated nestedPath/nestedFilter parameters,
inlcuding checks that they emmit warnings on parsing and that the new
NestetedSortBuilder overwrites the deprecated parameters when building the
SortField.

Relates to #17286
2017-09-04 17:19:52 +02:00
Boaz Leskes 2fd4af82e4 Move `UNASSIGNED_SEQ_NO` and `NO_OPS_PERFORMED` to SequenceNumbers (#26494)
Where they better belong.
2017-09-04 16:31:00 +02:00
Martijn van Groningen a4d5c6418e
percolator: Rename map_unmapped_fields_as_string setting to map_unmapped_fields_as_text
The `index.percolator.map_unmapped_fields_as_text` is a more better name, because unmapped fields are mapped to a text field with default settings
and string is no longer a field type (it is either keyword or text).
2017-09-04 14:12:44 +02:00
Alexander Reelsen 3706a16baf Docs: Update broken link to flake ids in uuid generators 2017-09-04 10:48:50 +02:00
shaulzorea 666cf4b872
fixing typo in nested-aggregation.asciidoc (#26481) 2017-09-04 06:42:44 +02:00
Jason Tedor 279be13a00 Clarify development vs. production mode
The definition of development vs. production mode has evolved slightly
over time (with the introduction of single-node) discovery. This commit
clarifies the documentation to better account for this adjustment.

Relates #26460
2017-09-02 09:47:39 -04:00
Christoph Büscher f8fc0f3ebe [Tests] Check that quoteAnalyzer overrides analyzer in `query_string` query (#26473)
Adding a check to QueryStringQueryBuilderTests that checks the override
behaviour of `quote_analyzer`, also adding documentation explaining the use of
this parameter in `query_string` query.

Closes #25417
2017-09-02 11:53:02 +02:00
Jason Tedor 1757bd8d92 Prettify primary response in assertion message
We are getting the default Object#toString implementation here, we need
more than this. This commit instead formats the primary response to JSON
so we can see into its soul.
2017-09-01 19:25:06 -04:00
Tal Levy 9735e7d706 migrate some MasterNodeRequest subclasses to Writeable Readers (#26463)
migrate some MasterNodeRequest subclasses to Writeable Readers
2017-09-01 15:27:45 -07:00
Boaz Leskes 2d0997be16 Add version 6.0.0-rc1 2017-09-01 17:48:24 -04:00
Jason Tedor 19a2156d18 Skip some logging tests on JDK 9
There is a bug in Log4j on JDK 9 for walking the stack to find where a
log line is coming from. This bug is impacting some of our testing, so
this commit marks these tests as skippable only on JDK 9 until the bug
is fixed upstream.

Relates #26467
2017-09-01 12:38:22 -04:00
Christoph Büscher c2853c8281 Remove old norelease comment, the test is okay as it is 2017-09-01 18:25:27 +02:00
Lee Hinman 4157eead22 [DOCS] Add documentation for adaptive replica selection
This adds a blurb for adaptive replica selection since it was previously
undocumented.

Relates to #24915
2017-09-01 09:53:22 -06:00
Michael Basnight af7cc0f2cf Add a shading example to the rest client docs
This adds a maven and gradle example for shading as well as the caveat
about shading and commons logging.

Ref #26449
2017-09-01 08:57:16 -05:00
Alexander Reelsen 563ff3cbce Tests: Use script.max_compilations_rate for BWC tests
As the setting is now backported to 6.0, the cluster
formation task can use only the new setting, no need
to use the old one anymore.
2017-09-01 14:43:00 +02:00
Christoph Büscher 2d342c0830 [Tests] Add unit tests for NestedSortBuilder (#26458)
The new NestedSortBuilder currently is only tested via its use in the other
SortBuilder implementations it can be used in. This adds its own simple unit
test class that at first checks our usual fromXContent parsing, serialization
and hashCode/equals checks. It also adds tests for cases where NestedSortBuilder
is nested in itself and reuses the code for creating randomized instances in the
other SortBuilder tests.

In addition to the tests, this changes the `path` parameter in NestedSortBuilder
to be mandatory and removes the `read` method since it is not really needed.
2017-09-01 10:53:51 +02:00
Alexander Reelsen 80d0a32f8e ScriptService: Replace max compilation per minute setting with max compilation rate (#26399)
The current script service has a script compilation limit for a one
minute window. This is set to a small default value of 15. Instead of
increasing that default value, this commit introduces a new setting 
that allows to configure a rate per time unit, so that the script service can deal with bursts better.

The new setting is named `script.max_compilations_rate`,
requires a nonnegative number and a positive time value.

The default is `75/5m`, which is equivalent to the existing 15 per minute.
2017-09-01 10:15:27 +02:00
Jason Tedor 53848877c8 Reenable BWC tests after backport
This commit reenables the BWC tests after they were disabled for
backporting a change that had broad BWC implications.
2017-08-31 22:13:50 -04:00
Ryan Ernst 6ffbb9dfc6 Test: Quiet failing java 9 test due to log4j upgrade
See https://github.com/elastic/elasticsearch/issues/26464
2017-08-31 16:04:18 -07:00
Jason Tedor 111defdfe1 Allow double aborts on bulk item requests
In some cases a request can already be aborted and retried. This means
the condition that aborting a request should only happen when an item
has not been processed yet is too strict. This commit allows for a
double abort. If we attempt to abort an operation that was previously
processed but not aborted, we treat that as a hard failure.

Relates #26434
2017-08-31 14:37:02 -04:00
Christoph Büscher 294d167973 Revert accidental deletion of cast needed for Java 9 2017-08-31 16:13:12 +02:00
Jason Tedor 697bc266ce Upgrade to Log4j 2.9.0
This commit upgrades the Log4j dependency from version 2.8.2 to version
2.9.0.

Relates #26450
2017-08-31 09:54:35 -04:00
Tim Vernum eb87df9ff9 Allow abort of bulk items before processing (#26434)
Adds support for bulk items to be aborted before they are processed by the TransportShardBulkAction.
This can be used by an ActionFilter to reject a subset of the items in a bulk action without rejecting the whole action (or all the items for a shard).
2017-08-31 21:23:14 +10:00