Commit Graph

28320 Commits

Author SHA1 Message Date
Boaz Leskes 17a587e709 bwc checkout should fetch from all remotes
This is a regression introduced in #25510, which removed the explicit fetching of upstream. Sadly this doesn't work if you don't have any local branch referring to `upstream` as an upstream branch.
2017-07-07 13:39:24 +02:00
Boaz Leskes 006e8cd1ea IndexingIT should check for global checkpoints regardless of master version
In the past global checkpoint syncing was done in the background based an interval set by an index setting. In order to set that setting something reasonable for a test, the master needed to know about the setting. Therefore the test didn't check global checkpoints if the master was old. These days the global checkpoint sync is inlined with indexing operations and that restriction is not needed.
2017-07-07 12:55:42 +02:00
Christoph Büscher 870d63d0cd [Tests] Add tests for PhraseSuggestionBuilder#build() (#25571)
This adds a unit test that checks the PhraseSuggestionContext output 
of PhraseSuggestionBuilder#build.
2017-07-07 12:53:06 +02:00
Christoph Büscher abe80b9ccb Remove unused class MinimalMap (#25590) 2017-07-07 12:51:38 +02:00
Tanguy Leroux b06a744b05 [Docs] Document Scroll API for Java High Level REST Client (#25554)
This commit adds documentation for _search/scroll and clear scroll methods of the high level Java REST client
2017-07-07 12:19:33 +02:00
Yu 2e5e45161e Disable date field mapping changing (#25285)
Make date field mapping unchangeable.

Closes #25271
2017-07-07 11:49:09 +02:00
Boaz Leskes f79c2cb8c0 Allow BWC Testing against a specific branch (#25510)
Some times we need a fix / change to have two parts in two different branches (corresponding to two different ES releases). In order to be able to test these cases you need to run the BWC tests against a local branch rather than then using a branch from `github.com/elastic/elasticsearch`.

This commit adds a system property called `tests.bwc.refspec` that allows you to do it. Note that I've chosen to go with the simplest code change for now, at the expense of some user friendliness.
2017-07-07 11:18:03 +02:00
Simon Willnauer d368d7cb9f [TEST] Remove test trace logging 2017-07-07 11:03:07 +02:00
Christoph Büscher 31f73cc06c
[Tests] Fixing test failure in CompletionSuggesterBuilderTests 2017-07-07 10:39:58 +02:00
Martijn van Groningen 00a9811f02
test: re-enable `Reindex from remote with parent join field` test no that join field type no longer always emits its fields inside search hits.
Closes #25363
2017-07-07 10:18:13 +02:00
Martijn van Groningen 6db708ef75
Move more token filters to analysis-common module
The following token filters were moved: common grams, limit token, pattern capture and pattern raplace.

Relates to #23658
2017-07-07 10:02:52 +02:00
Christoph Büscher d71feceb23 [Tests] Add tests for TermSuggestionBuilder#build() (#25558)
Adds a unit test that checks the TermSuggestionContext contents that is the result 
of TermSuggestionBuilder#build vs. the values the original builder contains.
2017-07-07 09:47:21 +02:00
Simon Willnauer 1f67d079b1 Validate `transport.profiles.*` settings (#25508)
Transport profiles unfortunately have never been validated. Yet, it's very
easy to make a mistake when configuring profiles which will most likely stay
undetected since we don't validate the settings but allow almost everything
based on the wildcard in `transport.profiles.*`. This change removes the
settings subset based parsing of profiles but rather uses concrete affix settings
for the profiles which makes it easier to fall back to higher level settings since
the fallback settings are present when the profile setting is parsed. Previously, it was
unclear in the code which setting is used ie. if the profiles settings (with removed
prefixes) or the global node setting. There is no distinction anymore since we don't pull
prefix based settings.
2017-07-07 09:40:59 +02:00
Jason Tedor c96257ca73 Upgrade to Netty 4.1.13.Final
This commit upgrades the Netty dependency from version 4.1.11.Final to
4.1.13.Final.

Relates #25581
2017-07-06 15:37:00 -04:00
Ali Beyad cc1f40ca18 Fix cluster health wait conditions in rolling restart tests
In the rolling upgrade tests, there is a test to create an index with
replica shards and ensure that in the mixed cluster environment, the
cluster health is green before any other tests are executed.  However,
there were two problems with this.  First, if the replica shard was
residing on the restarted node, then delayed allocation will kick in and
cause the cluster health request to timeout after 1m.  The fix to this
was to drastically lower the delayed allocation setting.  Second, if the
primary exists on the higher version node, then the replica cannot be
assigned to the lower version node because recovery cannot happen from
lower lucene versions.  The fix here was to wait for the cluster health
to be yellow instead of green in the mixed cluster environment.  In the
fully upgraded cluster, the cluster health check waits for a green
cluster as before.

Closes #25185
2017-07-06 14:35:07 -04:00
Simon Willnauer e9f6210dac Add cluster name validation to RemoteClusterConnection (#25568)
This change adds validation to the RemoteClusterConnection to ensure
we always use seed nodes from the same cluster. While we still allow to use
an arbitrary cluster alias we ensure that we, once we connected to a cluster the first time,
we always check against that initial cluster name when we execute a seed node handshake.
2017-07-06 19:18:10 +02:00
Ali Beyad dda68643b6 Removes deprecated usage of the FieldStats API in a test that verifies
sequence number data in Lucene commit points.  Instead, the test
retrieves the _seq_no value from the commit point directly and converts
it to a Long value.
2017-07-06 12:00:00 -04:00
Christoph Büscher 41d0ff32c8 [Tests] Check output of SuggestionBuilder#build method (#25549)
This change adds a basic unit test for the SuggestionSearchContext that is
created as output of SuggestionBuilder#build. The current test only adds checks
for the common fields (like text, prefix, fieldName etc...).

Relates to #17118
2017-07-06 17:32:34 +02:00
Tanguy Leroux d9bc0f48b4 [Docs] Document Bulk Processor for Java High Level REST Client (#25572) 2017-07-06 17:05:10 +02:00
Luca Cavanna bd6d7f1495 Update REST client deps license and notice files (#25573)
We previously grouped all the license and notice files for httpcore, httpcore-nio, httpclient and httpasyncclient under the same license and notice file. There were though subtle differences between those which we didn't keep track of. For instance the httpcore license file has slightly changed since 4.4 which we have missed to track.

This commit goes back to having one license and notice file for each jar, to be completely sure that each dependency is associated with exactly the right licene and notice file.

 Closes #25567
2017-07-06 16:52:17 +02:00
Martijn van Groningen d0f9f425bd
parent/child: Removed ParentJoinFieldSubFetchPhase 2017-07-06 13:15:02 +02:00
Jim Ferenczi 31614c3ddb Remove deprecated fielddata_fields from search request (#25566)
... and inner_hits
2017-07-06 13:02:28 +02:00
Clinton Gormley ca12b1f2a6 Tidied up the disk allocator docs 2017-07-06 12:16:53 +02:00
Martijn van Groningen 407273f81d
parent/child: Support parent id being specified as number in the _source 2017-07-06 11:48:57 +02:00
Luca Cavanna 26bc900058 [DOCS] Rewrite low-level REST client docs and verify snippets compile (#25559)
Using the infra that we now have in place, we can convert the low-level REST client docs so that they extract code snippets from real Java classes. This way we make sure that all the snippets properly compile. Compared to the high level REST client docs, in this case we don't run the tests themselves, as that would require depending on test-framework which requires java 8 while the low-level REST client is compatible with java 7. I think that compiling snippets is enough for now.
2017-07-06 10:05:50 +02:00
Simon Willnauer 38a1df7da1 Use a port range per JVM in MockTransportService (#25565)
Some tests use MockTransportService to do network based testing.
Yet, we run tests in multiple JVMs that means
concurrent tests could claim port that another JVM just released
and if that test tries to simulate a disconnect it might be smart
enough to re-connect depending on what is tested. To reduce the risk,
since this is very hard to debug we use a different default
port range per JVM unless the incoming settings overriding it.

Closes #25301
2017-07-06 09:14:52 +02:00
Lee Hinman 30b5ca7ab7 Refactor PathTrie and RestController to use a single trie for all methods (#25459)
* Refactor PathTrie and RestController to use a single trie for all methods

This changes `PathTrie` and `RestController` to use a single `PathTrie` for all
endpoints, it also allows retrieving the endpoints' supported HTTP methods more
easily.

This is a spin-off and prerequisite of #24437

* Use EnumSet instead of multiple if conditions

* Make MethodHandlers package-private and final

* Remove duplicate registerHandler method

* Remove public modifier
2017-07-05 17:28:10 -06:00
Simon Willnauer 6e5cc424a8 Switch indices read-only if a node runs out of disk space (#25541)
Today when we run out of disk all kinds of crazy things can happen
and nodes are becoming hard to maintain once out of disk is hit.
While we try to move shards away if we hit watermarks this might not
be possible in many situations. Based on the discussion in #24299
this change monitors disk utilization and adds a flood-stage watermark
that causes all indices that are allocated on a node hitting the flood-stage
mark to be switched read-only (with the option to be deleted). This allows users to react on the low disk
situation while subsequent write requests will be rejected. Users can switch
individual indices read-write once the situation is sorted out. There is no
automatic read-write switch once the node has enough space. This requires
user interaction.

The flood-stage watermark is set to `95%` utilization by default.

Closes #24299
2017-07-05 22:18:23 +02:00
Adrien Grand 26de905f1e Fix the documentation to state that the `_id` field is indexed. (#25540) 2017-07-05 16:09:31 +02:00
Jason Tedor 7dcd81b41b Throw back replica local checkpoint on new primary
This commit causes a replica to throwback its local checkpoint to the
global checkpoint when learning of a new primary through a replica
operation.

Relates #25452
2017-07-05 09:17:16 -04:00
Simon Willnauer 7c637a0bfe Ensure `index.mapping.single_type` can only be set on 5.x indices (#25375)
In 6.x we prevent multiple types and default to `index.mapping.single_type: false`
This change removes the registered setting and ensures that it's preserved for
5.x indices.

Relates to #24961
2017-07-05 15:16:40 +02:00
Clinton Gormley 0170e0e8d3 Remove usage of multi-types from the docs and added a page explaining type removal (#25543)
Closes #25401
2017-07-05 12:30:19 +02:00
Tanguy Leroux fefcae3d45 [Docs] Document Update API for Java High Level REST Client (#25536)
This commit adds documentation for Java High Level REST Client's Update API.
2017-07-05 12:16:42 +02:00
javanna 7899dbef3b [DOCS] High Level Client: fix index items order
Also group APIs under single document, multi-document and search
2017-07-05 12:07:09 +02:00
Luca Cavanna 6f8c0453bc [DOCS] add docs for high level client get method (#25538)
Document high level client get method
2017-07-05 11:57:57 +02:00
Russ Cam 83522ab4e5 Important note for MSI upgrade from command line (#25544)
Add an Important admonition for upgrading via the command line
using the Windows MSI Installer. This calls out the need to pass
the same command line options for an upgrade as were used for
the initial installation.
2017-07-05 11:38:21 +02:00
Luca Cavanna d91d96e06e [DOCS] update index, type and id for indexed documents in high level client docs (#25546)
Especially the type change is convenient as it becomes `doc`
2017-07-05 11:32:59 +02:00
Simon Willnauer ca351b60b7 [TEST] Enable transport tracer for RemoteClusterServiceTests#testCollectNodes #25301 2017-07-05 11:23:14 +02:00
Simon Willnauer 8e861b3896 [TEST] Add another valid exception that can occure with concurrent disconnects 2017-07-05 11:23:14 +02:00
Wesley Merkel 7994703903 [Docs] Fix some errors in the index aliases docs (#25449) 2017-07-05 11:16:17 +02:00
Christoph Büscher 3185eaece8 QueryBuilders should implement ToXContentObject (#25530)
All query builders written as self contained xContent objects, to we should mark
them accordingly using ToXContentObject. This also makes it possible to use
things like XContentHelper#toXContent to render query builders in tests.
2017-07-05 09:50:10 +02:00
Tanguy Leroux 3da3632021 [Docs] Document Bulk API for Java High Level REST Client (#25532)
This commit adds documentation for Java High Level REST Client's Bulk API.
2017-07-05 09:26:26 +02:00
Martijn van Groningen 141aa226f4
test: avoid NPE 2017-07-04 20:33:02 +02:00
Adrien Grand e7e5216382 Make totalHits a long in CollapseTopFieldDocs.
Relates to #25349.
2017-07-04 18:35:51 +02:00
Tanguy Leroux c52c3cea68 [Doc] Remove mention of generated fields in get/mget docs (#25492)
This option has been removed in #20102.
2017-07-04 18:06:00 +02:00
Colin Goodheart-Smithe 41abccf6c5 Adds rewrite phase to aggregations (#25495)
* Adds rewrite phase to aggregations

This change adds aggregations to the rewrite performed by the `SearchSourceBuilder`. This means that `AggregationBuilder`s are able to implement a `rewrite()` method where they can return a new `AggregationBuilder` which is functionally the same but in a more primitive form. This is exactly analogous to the rewrite done by the `QueryBuilder`s.

The first aggregation to implement the rewrite are the filter and filters aggregations so they can rewrite the filters they contain.

Closes #17676

* Removes rewrite from PipelineAggregationBuilder

Rewrite is based on shard level information. Since pipeline aggregation are run in the reduce phase it doesn’t make sense to rewrite them on the shards. In fact eventually we shouldn’t be transporting them to the shards at all and should be retaining them on the coordinating node for execution in the reduce phase

* Addresses review comments

* addresses more review comments

* Fixed imports
2017-07-04 16:47:48 +01:00
Simon Willnauer 1c4ef0d214 Upgrade randomizedrunner to 2.5.2 (#25533)
An issue causing confusing error messages during test execution
has been fixed randomizedtesting/randomizedtesting#250
2017-07-04 16:48:11 +02:00
Jun Ohtani 6894ef6057 [Analysis] Support normalizer in request param (#24767)
* [Analysis] Support normalizer in request param

Support normalizer param
Support custom normalizer with char_filter/filter param

Closes #23347
2017-07-04 19:16:56 +09:00
Christoph Büscher 5200665295 Remove deprecated IdsQueryBuilder constructor (#25529)
The constructor using `types` has been deprecated for a while now (starting with
ES 5.1.). It can be removed in the next mayor version. Since types are optional
they should be added with the #types() setter.
2017-07-04 11:59:48 +02:00
Colin Goodheart-Smithe 43efcffcc2 Adds check for negative search request size (#25397)
* Adds check for negative search request size

This change adds a check to `SearchSourceBuilder` to throw and exception if the size set on it is set to a negative value.

Closes #22530

* fix error in reindex

* update re-index tests

* Addresses review comment

* Fixed tests

* Added random negative size test

* Fixes test
2017-07-04 10:51:38 +01:00