Commit Graph

43703 Commits

Author SHA1 Message Date
Ryan Ernst 722b850efd
Build: Make assemble a noop in bwc projects (#37245)
This commit makes the assemble tasks in the bwc projects noops by
setting the dependsOn directly. While we can not remove things from
dependsOn, we can still completely override the dependencies.

closes #33581
2019-01-09 10:57:26 -08:00
Ryan Ernst 29c895b55c
Build: Quiet naming convention logging (#37244)
This commit moves log statements related to classification of naming
convention checks for tests to debug level. At info level they emit an
enormous amount of output in CI, while these are not generally useful
for debugging normal build failures.
2019-01-09 10:56:30 -08:00
Jack Conradson 95eef77ad4
[Style] Fix line length violations for threadpool, indexing, and script packages (#37205) 2019-01-09 10:55:52 -08:00
Tim Brooks cfa58a51af
Add TLS/SSL channel close timeouts (#37246)
Closing a channel using TLS/SSL requires reading and writing a
CLOSE_NOTIFY message (for pre-1.3 TLS versions). Many implementations do
not actually send the CLOSE_NOTIFY message, which means we are depending
on the TCP close from the other side to ensure channels are closed. In
case there is an issue with this, we need a timeout. This commit adds a
timeout to the channel close process for TLS secured channels.

As part of this change, we need a timer service. We could use the
generic Elasticsearch timeout threadpool. However, it would be nice to
have a local to the nio event loop timer service dedicated to network needs. In
the future this service could support read timeouts, connect timeouts,
request timeouts, etc. This commit adds a basic priority queue backed
service. Since our timeout volume (channel closes) is very low, this
should be fine. However, this can be updated to something more efficient
in the future if needed (timer wheel). Everything being local to the event loop
thread makes the logic simple as no locking or synchronization is necessary.
2019-01-09 11:46:24 -07:00
Morris Schreibman 13b8bad2b8 [DOCS] Clarify client settings (#31469) 2019-01-09 10:00:00 -08:00
Alpar Torok 6a5f3f05f4 Fix build on Fips
testing convetions need to be disabled if the test task is for fips.
2019-01-09 19:27:01 +02:00
Alpar Torok d68d6aa00c
Compatability mode for forbidden APIs for Java 12 (#37199) 2019-01-09 18:26:08 +02:00
Albert Zaharovits 4efbda1bc0
[DOCS] Rolling upgrade with old internal indices (#37184)
Upgrading the Elastic Stack perfectly documents the process to
upgrade ES from 5 to 6 when internal indices are present. However,
the rolling upgrade docs do not mention anything about internal indices.

This adds a warning in the rolling upgrade procedure, highlighting that
internal indices should be upgraded before the rolling upgrade procedure
can be started.
2019-01-09 18:14:22 +02:00
Evangelos Chatzikalymnios 85a603ee61 Use List instead of priority queue for stable sorting in bucket sort aggregator (#36748)
Update BucketSortPipelineAggregator to use a List and Collections.sort() for sorting instead of a priority queue. This preserves the order for equal values. Closes #36322.
2019-01-09 18:01:39 +02:00
Armin Braun eacc63b032
TESTS: Real Coordinator in SnapshotServiceTests (#37162)
* TESTS: Real Coordinator in SnapshotServiceTests

* Introduce real coordinator in SnapshotServiceTests to be able to test network disruptions realistically
  * Make adjustments to cluster applier service so that we can pass a mocked single threaded executor for tests
2019-01-09 16:53:49 +01:00
Alpar Torok ae086ebcc4 Muting SnapshotDisruptionIT
Tracked in #36779
2019-01-09 16:55:11 +02:00
Alpar Torok 7de4d2cb0f Mute failing test ChildQuerySearchIT
Tracked in #37266
2019-01-09 16:48:49 +02:00
Christoph Büscher c149bb8cc2
Support 'include_type_name' in RestGetIndicesAction (#37149)
This change adds support for the 'include_type_name' parameter for the
indices.get API. This parameter, which defaults to `false` starting in 7.0,
changes the response to not include the indices type names any longer.

If the parameter is set in the request, we additionally emit a deprecation
warning since using the parameter should be only temporarily necessary while
adapting to the new response format and we will remove it with the next major
version.
2019-01-09 14:17:17 +01:00
Jim Ferenczi 84d520b0e5 Throw an ISE rather than an hard assertion in SearchPhaseController#getTotalHits
This change turns an assertion into an IllegalStateException in SearchPhaseController#getTotalHits.
The goal is to help identify the cause of the failures in https://github.com/elastic/elasticsearch/issues/37179
which seems to fail only in CI.
The assertion will be restored when the issue is solved (NORELEASE).
2019-01-09 13:36:48 +01:00
Tanguy Leroux dc371ef593 [Tests] Fix ReopenWhileClosingIT with correct min num shards
The test intercepts TransportVerifyShardBeforeCloseAction shard
requests, so it needs a minimum of 2 primary shards on 2 different
 nodes to correctly intercepts requests.
2019-01-09 13:34:38 +01:00
Alpar Torok 7ff846ffb2 mute failing test: org.elasticsearch.index.shard.RefreshListenersTests.testConcurrentRefresh 2019-01-09 13:43:21 +02:00
Martijn van Groningen 9122585359
[CCR] Added more logging. 2019-01-09 12:17:47 +01:00
Tanguy Leroux f1f5d834c3 Merge branch 'close-index-api-refactoring' 2019-01-09 11:48:57 +01:00
David Roberts e0ce73713f
[ML] Stop datafeeds running when their jobs are stale (#37227)
We already had logic to stop datafeeds running against
jobs that were OPENING, but a job that relocates from
one node to another while OPENED stays OPENED, and this
could cause the datafeed to fail when it sent data to
the OPENED job on its new node before it had a
corresponding autodetect process.

This change extends the check to stop datafeeds running
when their job is OPENING _or_ stale (i.e. has not had
its status reset since relocating to a different node).

Relates #36810
2019-01-09 10:42:47 +00:00
Tanguy Leroux 096a83183e Merge branch 'master' into close-index-api-refactoring 2019-01-09 10:52:46 +01:00
David Roberts f14cff2102
[TEST] Ensure interrupted flag reset after test that sets it (#37230)
Test fix to stop a problem in one test leaking into a different
test and causing that other test to spuriously fail.
2019-01-09 08:51:00 +00:00
Armin Braun 224475e6aa
MINOR: Some Cleanups in Snapshots (#37175)
* Removed pre v6 Bwc logic + comments
* Removed some dead code
2019-01-09 09:45:37 +01:00
Yannick Welsch 9040a96daf
Zen2: Add test that checks that CoordinatorTests are reproducible (#37225)
Ensures reproducibility of the tests in CoordinatorTests.
2019-01-09 09:32:24 +01:00
Tanguy Leroux 7f6fe14b66 Merge branch 'master' into close-index-api-refactoring 2019-01-09 09:26:05 +01:00
Tanguy Leroux 0a40564068
[Tests] Change cluster scope in CorruptedFileIT and FlushIT (#37229)
These tests failed on CI multiple times in the past weeks because they use a 
test cluster with a SUITE scope that recreates nodes between tests. With such 
a scope, nodes can be recreated in between test executions and can inherit a 
node id from a previous test execution, while they are assigned a random data 
path. With the successive node recreations it is possible that a newly recreated 
node shares the same node id (but different data path) as a non recreated node.

This commit changes the cluster scope of the CorruptedFileIT and FlushIT 
tests which often fail.

The failure is reproducable with :

./gradlew :server:integTest -Dtests.seed=EF3A50C225CF377 
-Dtests.class=org.elasticsearch.index.store.CorruptedFileIT 
-Dtests.security.manager=true -Dtests.locale=th-TH-u-nu-thai-x-lvariant-TH -Dtests.timezone=America/Rio_Branco 
-Dcompiler.java=11 -Druntime.java=8
2019-01-09 09:24:56 +01:00
David Turner d9e2ebca67
Add more detail to recovery bandwidth limit docs (#37156) 2019-01-09 08:18:25 +00:00
Ioannis Kakavas 9049263c2c
Handle malformed license signatures (#37137)
This commit adds a more user friendly error message when a license
signature is malformed/truncated in a way that it cannot be
meaningfully parsed.
2019-01-09 07:29:22 +02:00
Ioannis Kakavas 2a79c468f8 Ensure that ActionListener is called exactly once
This bug was introduced in #36893 and had the effect that
execution would continue after calling onFailure on the the
listener in checkIfTokenIsValid in the case that the token is
expired. In a case of many consecutive requests this could lead to
the unwelcome side effect of an expired access token producing a
successful authentication response.
2019-01-09 07:23:35 +02:00
Jun Ohtani 38b698d455
[Analysis] Deprecate Standard Html Strip Analyzer in master (#26719)
* [Analysis] Deprecate Standard Html Strip Analyzer

Deprecate only Standard Html Strip Analyzer
If user create index with the analyzer since 7.0, es throws an exception.
If an index was created before 7.0, es issue deprecation log
We will remove it in 8.0

Related #4704
2019-01-09 12:42:00 +09:00
Mayya Sharipova ec32e66088 Deprecate reference to _type in lookup queries (#37016)
Relates to #35190
2019-01-08 18:46:41 -08:00
Marios Trivyzas 5f2fbedd8c
SQL: Replace String.format() with LoggerMessageFormat.format() (#37216)
Fixes: #36532
2019-01-08 23:56:00 +02:00
Nhat Nguyen 87ac3103f4
Do not mutate RecoveryResponse (#37204)
Today we create a global instance of RecoveryResponse then mutate it
when executing each recovery step. This is okay for the current
sequential recovery flow but  not suitable for an asynchronous recovery
which we are targeting. With this commit, we return the result of each
step separately, then construct a RecoveryResponse at the end.

Relates #37174
2019-01-08 16:12:18 -05:00
Ryan Ernst 86b71930f6
Remove more checkstyle suppressions (#36735)
This commit removes several more line length violations from checkstyle
suppression.

relates #34884
2019-01-08 11:04:54 -08:00
Martijn van Groningen d6608caf55
Muted rolling upgrade tests.
Relates to #37231
2019-01-08 16:52:22 +01:00
Jay Modi 1514bbcdde
Security: propagate auth result to listeners (#36900)
After #30794, our caching realms limit each principal to a single auth
attempt at a time. This prevents hammering of external servers but can
cause a significant performance hit when requests need to go through a
realm that takes a long time to attempt to authenticate in order to get
to the realm that actually authenticates. In order to address this,
this change will propagate failed results to listeners if they use the
same set of credentials that the authentication attempt used. This does
prevent these stalled requests from retrying the authentication attempt
but the implementation does allow for new requests to retry the
attempt.
2019-01-08 08:52:12 -07:00
Michael Basnight dd69553d4d
HLRC: Use nonblocking entity for requests (#32249)
Previously the HLRC used a blocking ByteArrayEntity, but the Request
class also allows to set a NByteArrayEntity, and defaults to nonblocking
when calling the createJsonEntity method. This commit cleans up all the
uses of ByteArrayEntity in the RequestConverters to use the nonblocking
entity.
2019-01-08 09:11:58 -06:00
Jim Ferenczi 054c3bb04f
Fix topDocs.totalHits assignment on scroll queries (#37180)
This change fixes an unreleased bug that assigns the wrong totalHits to scroll
queries.

Closes #37179
2019-01-08 13:31:53 +01:00
Alpar Torok 20c2c439e7
Upgrade to Gradle 5.1 (#36767) 2019-01-08 13:43:10 +02:00
Alpar Torok 6344e9a3ce
Testing conventions: add support for checking base classes (#36650) 2019-01-08 13:39:03 +02:00
Alpar Torok 0c68eead51 Mute failing test clusters test
Tracking issue: #37218
2019-01-08 13:00:55 +02:00
Tanguy Leroux 6e852dfa7c Merge branch 'master' into close-index-api-refactoring 2019-01-08 11:28:51 +01:00
Martijn van Groningen c980cc12df
Added CCR rolling upgrade tests (#36648)
Added CCR rolling upgrade tests.
2019-01-08 11:05:18 +01:00
Andrew Banchich 1d371a427f [Docs] Fix wrong math in overview.asciidoc (#37209) 2019-01-08 10:25:02 +01:00
Tanguy Leroux 6c78aff830 Merge branch 'master' into close-index-api-refactoring 2019-01-08 10:00:47 +01:00
Luca Cavanna 67f97e3686 Re-enable bwc tests
Relates to #37187
2019-01-08 09:53:58 +01:00
Tanguy Leroux d70ebfd1d6 Merge branch 'master' into close-index-api-refactoring 2019-01-08 09:17:48 +01:00
Andrei Stefan 3fad9d25f6
SQL: fix COUNT DISTINCT filtering (#37176)
* Use `_count` aggregation value only for not-DISTINCT COUNT function calls
* COUNT DISTINCT will use the _exact_ version of a field (the `keyword` sub-field for example), if there is one
2019-01-08 08:47:35 +02:00
Tim Brooks b0665963e8
Make remote ping and compress settings dynamic (#37200)
Traditionally remote clusters can be configured dynamically. However,
the compress and ping settings are not currently set to be configured
dynamically. This commit changes that.
2019-01-07 23:35:22 -07:00
Jason Tedor c8c596cead
Introduce retention lease expiration (#37195)
This commit implements a straightforward approach to retention lease
expiration. Namely, we inspect which leases are expired when obtaining
the current leases through the replication tracker. At that moment, we
clean the map that persists the retention leases in memory.
2019-01-07 22:03:52 -08:00
Julie Tibshirani c5aac4705d Revert "Stop automatically nesting mappings in index creation requests. (#36924)"
This reverts commit ac1c6940d2.
2019-01-07 17:56:40 -08:00