Commit Graph

47212 Commits

Author SHA1 Message Date
Tal Levy 075a3f0e99
remove usage of ActionType#(String) (#44459) (#44526)
this commit removes usage of the deprecated
constructor with a single argument and no Writeable.Reader.

The purpose of this is to reduce the boilerplate necessary for
properly implementing a new action, as well as reducing the
chances of using the incorrect super constructor while classes
are being migrated to Writeable

relates #34389.
2019-07-17 20:28:11 -07:00
Nhat Nguyen 51180af91d Make peer recovery send file chunks async (#44468)
Relates #44040
Relates #36195
2019-07-17 22:25:43 -04:00
Nhat Nguyen 458f24c46a Reenable accounting circuit breaker (#44495)
We have a new Lucene 8.2 snapshot on master and 7.x; hence we can
re-enable the accounting on these branches.

Relates #30290
2019-07-17 22:25:43 -04:00
Julie Tibshirani 34c6067018
Convert several classes in 'server' to Writeable. (#44527)
* Convert FieldCapabilities*.
* Convert MultiTermVectors*.
* Convert SyncedFlush*.
* Convert SearchTemplateRequest.
* Convert MultiSearchTemplateRequest.
* Convert GrokProcessorGet*.
* Remove a stray reference to SearchTemplateRequest#readFrom.

Relates to #34389.
2019-07-17 19:04:21 -07:00
Ryan Ernst 2a2686e6e7
Convert remaining ActionTypes to writeable in xpack core (#44467) (#44525)
This commit converts all remaining ActionType response classes to
writeable in xpack core. It also converts a few from server which were
used by xpack core.

relates #34389
2019-07-17 18:01:45 -07:00
Ryan Ernst 17c4b2b839
Convert MasterNodeRequest to implement Writeable.Reader (#44452) (#44513)
This commit converts all MasterNodeRequest subclasses to fullfill
Writeable.Reader constructors.

relates #34389
2019-07-17 18:01:29 -07:00
Paul Sanwald 7114fe786b
Fix incorrect calculation of how many buckets will result from a merge operation. (#44461) (#44515) 2019-07-17 19:14:16 -04:00
Julie Tibshirani 8841779de8
Convert ClearScroll* to Writeable. (#44511)
This PR converts `ClearScrollRequest` and `ClearScrollResponse` to
`Writeable`.

Relates to #34389.
2019-07-17 15:49:38 -07:00
Jason Tedor 39c5f98de7
Introduce test issue logging (#44477)
Today we have an annotation for controlling logging levels in
tests. This annotation serves two purposes, one is to control the
logging level used in tests, when such control is needed to impact and
assert the behavior of loggers in tests. The other use is when a test is
failing and additional logging is needed. This commit separates these
two concerns into separate annotations.

The primary motivation for this is that we have a history of leaving
behind the annotation for the purpose of investigating test failures
long after the test failure is resolved. The accumulation of these stale
logging annotations has led to excessive disk consumption. Having
recently cleaned this up, we would like to avoid falling into this state
again. To do this, we are adding a link to the test failure under
investigation to the annotation when used for the purpose of
investigating test failures. We will add tooling to inspect these
annotations, in the same way that we have tooling on awaits fix
annotations. This will enable us to report on the use of these
annotations, and report when stale uses of the annotation exist.
2019-07-18 05:33:33 +09:00
Lisa Cawley 621ec7cb84 [DOCS] Removes out-dated info from Watcher limitations (#44252) 2019-07-17 13:24:02 -07:00
Ryan Ernst 0755a13c9f
Convert AcknowledgedRequest to Writeable.Reader (#44412) (#44454)
This commit adds constructors to AcknolwedgedRequest subclasses to
implement Writeable.Reader, and ensures all future subclasses implement
the same.

relates #34389
2019-07-17 11:17:36 -07:00
Armin Braun 65fcaecce1
Remove Minio Host Hack in S3 Repository Build (#44491) (#44497)
* Resolving the todo to clean this hackyness up
2019-07-17 19:59:00 +02:00
Yannick Welsch c8b66c549d Ignore failures to set socket options on Mac (#44355)
Brings some temporary relief for test failures until #41071 is addressed.
2019-07-17 18:51:25 +02:00
Yannick Welsch f78e64e3e2 Terminate linearizability check early on large histories (#44444)
Large histories can be problematic and have the linearizability checker occasionally run OOM. As it's
very difficult to bound the size of the histories just right, this PR will let it instead run for 10 seconds
on large histories and then abort.

Closes #44429
2019-07-17 18:51:25 +02:00
Igor Motov a19c7977ac Docs: fix WKTToSQL function example (#44377)
Fixes wrong example snippet in WKTToSQL documentation.

Closes #44299
2019-07-17 12:24:28 -04:00
Igor Motov d3cb7bbc8f Geo: fix GeoWKTShapeParserTests (#44448)
Changes in #44187 introduced some optimization in the way shapes are
generated. These changes were not captured in GeoWKTShapeParserTests.

Relates #44187
2019-07-17 12:09:38 -04:00
Igor Motov cd5a334864 Geo: extract dateline handling logic from ShapeBuilders (#44187)
Extracts dateline decomposition logic from ShapeBuilder into a separate
utility class that is used on the indexing side. The search side
will be handled as part of another PR at this time we will remove
the decomposition logic from ShapeBuilders as well. This PR also doesn't
change any existing logic including bugs.

Relates to #40908
2019-07-17 12:09:38 -04:00
Lisa Cawley 51d8e6e3de [DOCS] Adds sync to data frame transform API (#44254) 2019-07-17 08:56:58 -07:00
Alan Woodward b6a0f098e6 Don't use index_phrases on graph queries (#44340)
Due to https://issues.apache.org/jira/browse/LUCENE-8916, when you
try to use a synonym filter with the index_phrases option on a text field,
you can end up with null values in a Phrase query, leading to weird
exceptions further down the querying chain. As a workaround, this commit
disables the index_phrases optimization for queries that produce token
graphs.

Fixes #43976
2019-07-17 16:46:00 +01:00
Yannick Welsch ddd740162e
Do not use CancellableThreads for Zen1 (#44430)
Zen 1 stops pinging threads in ZenDiscovery by calling Thread.interrupt(). This is incompatible with
the CancellableThreads that only allow threads to be interrupted through cancellation. The use of
CancellableThreads was introduced in #42844 and added to UnicastZenPing as part of the
backport, as both Zen1 and Zen2 share the same SeedHostsResolver implementation. This commit
effectively undoes the change in the backport while still allowing to share same implementation.

Closes #44425
2019-07-17 17:32:47 +02:00
Yannick Welsch d98b3e4760
Move frozen indices to x-pack module (#44490)
Backport of #44408 and #44286.
2019-07-17 16:53:10 +02:00
Alpar Torok 3a199d044c Add a list of BWC versions for CI (#44418)
This PR adds a list of index compatible versions to the `.ci` directory
as well as a way to generate and verify it.

Unfortunetly there is no easy way in Jenkins to have the build generate then
consume this YML axis config.
I hate that this would need maintenance on versions bumps, but the
potential benefir here is reducing the bwc builds that can take more than
24 hours to less than 20 minutes.
This is possible because the CI setup would use a matrix job to run
something like:
```
./graldew 'v7.0.0#bwcTest'
```
For every index compatible version.
On top of that `--parallel` should be possible even without testclusters
due to the limited number of clusters being set up here.

The example command above runs in exactly 10 minutes on my laptop,
thus I'm proposing to accept this compromise while we work out the
infra to do this more dinamically.
2019-07-17 17:44:49 +03:00
Zachary Tong 103ba976fd Convert BucketScript to static parser (#44385)
BucketScript was using the old-style parser and could easily be
converted over to the newer static parser.

Also adds a test for GapPolicy enum serialization
2019-07-17 10:22:42 -04:00
James Rodewig 34725e20fb [DOCS] Move Elasticsearch APIs to REST APIs section. (#44238) (#44372)
Moves the following API sections under the REST APIs navigations:
- API Conventions
- Document APIs
- Search APIs
- Index APIs (previously named Indices APIs)
- cat APIs
- Cluster APIs

Other supporting changes:
- Removes the previous index APIs page under REST APIs. Adds a redirect for the removed page.
- Removes several [partintro] macros so the docs build correctly.
- Changes anchors for pages that become sections of a parent page.
- Adds several redirects for existing pages that become sections of a parent page.

This commit re-applies changes from #44238. Changes from that PR were reverted due to broken links in several repos. This commit adds redirects for those broken links.
2019-07-17 09:18:31 -04:00
David Turner 377a6a47ac Improve handshake failure messages (#44485)
Today we report an exception on a handshake failure (e.g. cluster name
mismatch) but the message does not include all the details of the mismatch. If
the mismatch is something subtle like `my-cluster` instead of `my_cluster` then
we cannot diagnose this from the message alone. This commit adds the details of
the local cluster to the message, along with the details of the remote cluster,
improving the utility of the exception message if reported in isolation.
2019-07-17 13:33:28 +01:00
Armin Braun 91673e373a
Fix Incorrect Uncompressed Error Handling in InboundMessage (#44317) (#44483)
* Fix Incorrect Uncompressed Error Handling in InboundMessage

* CompressorFactory.compressor does not throw uncompressed exception on uncompressed bytes, it merely returns `null` in this case if the bytes are at least XContent so the current catch and re-throw logic is dead code
* Made it work again by throwing on a `null` return so we get a real error message instead of an NPE
2019-07-17 14:31:46 +02:00
Ignacio Vera eb348d2593
Upgrade to lucene-8.2.0-snapshot-6413aae226 (#44480) 2019-07-17 13:28:28 +02:00
Armin Braun c8db0e9b7e
Remove blobExists Method from BlobContainer (#44472) (#44475)
* We only use this method in one place in production code and can replace that with a read -> remove it to simplify the interface
   * Keep it as an implementation detail in the Azure repository
2019-07-17 11:56:02 +02:00
Tanguy Leroux e423b7341a Log non-acknowledged close index response in ReplicaToPrimaryPromotionIT
Relates #44479
2019-07-17 10:32:44 +02:00
David Turner dca8a918f3 Use applied cluster state in cluster health (#44426)
In #44348 we changed the cluster health action so that it sometimes uses the
cluster state directly from the master service rather than from the cluster
applier. If the state is not recovered then this is inappropriate, because
prior to state recovery the state available to the cluster applier contains no
indices. This commit moves us back to using the state from the applier.

Fixes #44416.
2019-07-17 08:36:13 +01:00
David Turner 0fd33b089f Report shard state changes better (#44419)
Today when the cluster health changes the `AllocationService` reports at most
ten shards that were started or failed, and always ends its message with `...`
suggesting that the list is truncated. This commit adjusts these messages to be
clearer about whether the list is truncated or not. When debug logging is
enabled the list is not truncated; if the list is truncated then its length is
logged, and if it is not truncated then no `...` is included in the message.
2019-07-17 08:36:06 +01:00
Ryan Ernst 6e50bafa8f
Convert Broadcast request and response to use writeable.reader (#44386) (#44453)
This commit converts the request and response classes for broadcast
actions to implement ctors for Writeable.Reader and forces all future
implementations to implement the same.

relates #34389
2019-07-16 23:24:02 -07:00
Tim Brooks 6b1a769638
Move CORS Config into :server package (#43779)
This commit moves the config that stores Cors options into the server
package. Currently both nio and netty modules must have a copy of this
config. Moving it into server allows one copy and the tests to be in a
common location.
2019-07-16 17:50:42 -06:00
Tim Brooks 0a352486e8
Isolate nio channel registered from channel active (#44388)
Registering a channel with a selector is a required operation for the
channel to be handled properly. Currently, we mix the registeration with
other setup operations (ip filtering, SSL initiation, etc). However, a
fail to register is fatal. This PR modifies how registeration occurs to
immediately close the channel if it fails.

There are still two clear loopholes for how a user can interact with a
channel even if registration fails. 1. through the exception handler.
2. through the channel accepted callback. These can perhaps be improved
in the future. For now, this PR prevents writes from proceeding if the
channel is not registered.
2019-07-16 17:18:57 -06:00
Julie Tibshirani cc0ff3aa71 Ensure field caps doesn't error on rank feature fields. (#44370)
The contract for MappedFieldType#fielddataBuilder is to throw an
IllegalArgumentException if fielddata is not supported. The rank feature mappers
were instead throwing an UnsupportedOperationException, which caused
MappedFieldType#isAggregatable to fail.
2019-07-16 15:56:50 -07:00
Jason Tedor 100cb89f3e
Avoid stack overflow in auto-follow coordinator (#44421)
This commit avoids a situation where we might stack overflow in the
auto-follower coordinator. In the face of repeated failures to get the
remote cluster state, we would previously be called back on the same
thread and then recurse to try again. If this failure persists, the
repeated callbacks on the same thread would lead to a stack
overflow. The repeated failures can occur, for example, if the connect
queue is full when we attempt to make a connection to the remote
cluster. This commit avoids this by truncating the call stack if we are
called back on the same thread as the initial request was made on.
2019-07-17 07:39:11 +09:00
Lisa Cawley c18a6402cd [DOCS] Renames data frame APIs section (#44458) 2019-07-16 15:10:46 -07:00
Jason Tedor becbf450fa
Avoid NPE when checking for CCR index privileges (#44397)
This commit avoids an NPE when checking for privileges to follow
indices. The problem here is that in some cases we might not be able to
read the authentication info from the thread context. In that case, a
null user would be returned and we were not guarding against this.
2019-07-17 06:27:55 +09:00
Lisa Cawley 53514b0477 [DOCS] Separates data frame analytics APIs (#44451) 2019-07-16 13:33:23 -07:00
Mayya Sharipova 3220709b0a Add positions info into term_vector doc (#44379) 2019-07-16 16:24:50 -04:00
Ryan Ernst c26edb4c43
Ensure replication response/requests implement writeable (#44392) (#44446)
This commit cleans up replication response and request so that the base
class does not allow subclasses to implement Streamable.

relates #34389
2019-07-16 12:53:08 -07:00
Tal Levy 901310a826
[7.x] Migrate ML Actions to use writeable ActionType (#44302) (#44391)
* Migrate ML Actions to use writeable ActionType (#44302)

This commit converts all the StreamableResponseActionType
actions in the ML core module to be ActionType and leverage
the Writeable infrastructure.
2019-07-16 12:41:10 -07:00
István Zoltán Szabó 642e9019ff [DOCS] Adds max_page_search_size to data frame transform pivot properties (#44126) 2019-07-16 21:33:19 +02:00
Przemysław Witek 9613700a63
[7.x] Implement MlConfigIndexMappingsFullClusterRestartIT test which verifies that .ml-config index mappings are properly updated during cluster upgrade (#44341) (#44366) 2019-07-16 21:22:40 +02:00
James Rodewig ac07eef86c [DOCS] Remove :edit_url: overrides. (#44445)
These overrides do not work in Asciidoctor and are no longer needed.
2019-07-16 15:04:44 -04:00
Nhat Nguyen 301c8daf4c Revert "Make peer recovery send file chunks async (#44040)"
This reverts commit a2b4687d89.
2019-07-16 14:18:35 -04:00
James Baiera a3ba11cfc1
Improve CryptoService error message on missing secure file (#43623) (#44364)
This improves the error message when encrypting of sensitive watcher
data is configured, but no system file was specified in the keystore.
This error message is displayed on startup.

This also closes the input stream of the secure file properly.

Closes #43619
2019-07-16 13:58:57 -04:00
Christoph Büscher 67ec0a4e9b
Unmute SpecificMasterNodesIT test (#44436)
The underlying issue is closed and the fix in #42454 seems to have been
backported to 7.x and 7.3 so we can reactivate the test.
2019-07-16 19:41:59 +02:00
Yu 563a78829f Do not allow version in Rest Update API (#43516)
The versioning of Update API doesn't rely on version number anymore (and
rather on sequence number). But in rest api level we ignored the
"version" and "version_type" parameter, so that the server cannot raise
the exception when whey were set.

This PR restores "version" and "version_type" parsing in Update Rest API
so that we can get the appropriate errors.

Relates to #42497
2019-07-16 13:19:07 -04:00
Hendrik Muhs 5ea89a4e7b Remove PR 41491 from release notes (#44417)
PR 41491 hasn't been merged into 7.2 but got closed and superseeded by #41800 (merged via feature branch merge into 7.3)
2019-07-16 10:03:40 -07:00