Commit Graph

43941 Commits

Author SHA1 Message Date
David Kyle 1abe5df09c Mute IndexShardRetentionLeaseTests.testCommit #37420 2019-01-14 14:17:11 +00:00
Martijn van Groningen de852765d6
unmuted test
Relates to #37014
2019-01-14 14:27:42 +01:00
Daniel Mitterdorfer abe35fb99b
Remove unused index store in directory service
With this commit we remove the unused field `indexStore` from all
implementations of `FsDirectoryService`.

Relates #37097
2019-01-14 13:44:32 +01:00
Tanguy Leroux 07dc8c7eee
Improve CloseWhileRelocatingShardsIT (#37348) 2019-01-14 13:14:36 +01:00
Tanguy Leroux 6ca076bf74
Fix ClusterBlock serialization and Close Index API logic after backport to 6.x (#37360)
This commit changes the versions in the serialization logic of ClusterBlock 
after the backport to 6.x of the Close Index API refactoring (#37359).
2019-01-14 13:13:15 +01:00
Georgi Ivanov 87f9148580 Update the scroll example in the docs (#37394)
Update the scroll example ascii and Java docs, so it is more clear when to 
consume the scroll documents. Before this change the user could loose 
the first results if one uses copy & paste.
2019-01-14 13:03:00 +01:00
Jiyu-Zhang-Zendesk 4b1368183d Update analysis.asciidoc (#37404)
STConvert plugin is made by Medcl to convert between Simplified Chinese and Traditional Chinese. It's widely used by the Search Community for Chinese
2019-01-14 11:22:03 +01:00
Christoph Büscher 89b45f1fc6
Remove deprecated pipeline request contructors (#37366)
The constructors in PutPipelineRequest and SimulatePipelineRequest that guess
the xContent type from the provided source are deprecated since 6.0 and each
have a counterpart that takes the xContent type as an explicit argument.
Removing these ctors together with the builders and methods in
ClusterAdminClient that don't have the xContent type as argument.
2019-01-14 11:14:38 +01:00
Nhat Nguyen d44a6f9fbc
Simplify SyncedFlushService flow with StepListener (#37383)
Today the SyncedFlushService flow is written with multiple nested 
callbacks which are hard to read. This commit replaces them with 
sequential step listeners.
2019-01-14 03:54:34 -05:00
Ioannis Kakavas 374e24c7fd Mute SslNullCipherTests on JDK12
JDK12 doesn't support NULL cipher for TLS by default. This commit
mutes these tests on JDK12 until we decide whether we need to keep
or remove them
2019-01-14 10:50:24 +02:00
Luca Cavanna d54f88f62c
Remove unused empty constructors from suggestions classes (#37295)
We recently migrated suggestions to `Writeable`. That allows us to also
clean up empty constructors and methods that called them as they are no
longer needed. They are replaced by constructors that accept a
`StreamInput` instance.
2019-01-14 08:32:45 +01:00
Alpar Torok a566bacbc8 Upgrade ASM for java 12 compatability (#37385)
Closes #37371
2019-01-13 09:33:39 -08:00
Albert Zaharovits 6fd57d90da
Security Audit includes HTTP method for requests (#37322)
Adds another field, named "request.method", to the structured logfile audit.
This field is present for all events associated with a REST request (not a
transport request) and the value is one of GET, POST, PUT, DELETE, OPTIONS,
HEAD, PATCH, TRACE and CONNECT.
2019-01-13 15:26:23 +02:00
Peter Dyson 96cfa000a5
[DOCS] copy_to only works one level deep, not recursively (#37249) 2019-01-13 16:24:34 +10:00
Costin Leau a4339ec7e9
SQL: Use declared source for error messages (#37161)
Improve error messages by returning the original SQL statement
declaration instead of trying to reproduce it as the casing and
whitespaces are not preserved accurately leading to small 
differences.

Close #37161
2019-01-13 01:40:22 +02:00
Jason Tedor 03be4dbaca
Introduce retention lease persistence (#37375)
This commit introduces the persistence of retention leases by persisting
them in index commits and recovering them when recovering a shard from
store.
2019-01-12 14:43:19 -08:00
Nhat Nguyen 44a1071018
Make recovery source partially non-blocking (#37291)
Today a peer-recovery may run into a deadlock if the value of
node_concurrent_recoveries is too high. This happens because the
peer-recovery is executed in a blocking fashion. This commit attempts
to make the recovery source partially non-blocking. I will make three
follow-ups to make it fully non-blocking: (1) send translog operations,
(2) primary relocation, (3) send commit files.

Relates #36195
2019-01-12 12:49:48 -05:00
Armin Braun 63fe3c6ed6
Fix PrimaryAllocationIT Race Condition (#37355)
* Fix PrimaryAllocationIT Race Condition

* Forcing a stale primary allocation on a green index was tripping the assertion that was removed
   * Added a test that this case still errors out correctly
* Made the ability to wipe stopped datanode's data public on the internal test cluster and used it to ensure correct behaviour on the fixed test
   * Previously it simply passed because the test finished before the index went green and would NPE when the index was green at the time of the shard store status request, that would then come up empty
* Closes #37345
2019-01-11 23:26:04 +01:00
Marios Trivyzas 359222c55c
SQL: Make `FULL` non-reserved keyword in the grammar (#37377)
Since `full` can be common as a field name or part of a field name
(e.g.: `full.name` or `name.full`), it's nice if it's not a reserved
keyword of the grammar so a user can use it without resorting to quotes.

Fixes: #37376
2019-01-11 23:08:00 +02:00
Marios Trivyzas 85531f0285
SQL: [Tests] Fix and enable internalClusterTests (#37300)
SqlPlugin cannot have more than one public constructor, so for the testing
purposes the `getLicenseState()` should be overriden.

Fixes: #37191

Co-authored-by: Michael Basnight <mbasnight@gmail.com>
2019-01-11 22:43:17 +02:00
Benjamin Trent 5101e51891
ML: Fix testMigrateConfigs (#37373)
* ML: :s/execute/get

* Fixing other broken tests

* unmuting test
2019-01-11 13:29:30 -06:00
Zachary Tong de52ba1f78 Fix RollupDocumentation test to wait for job to stop
Also adds some extra state debug information to various log messages
2019-01-11 14:14:58 -05:00
Gordon Brown 827ece73c8 Mute MlConfigMigratorIT.testMigrateConfigs (#37374) 2019-01-11 11:11:58 -07:00
Nhat Nguyen 70cee18e56
Introduce StepListener (#37327)
This commit introduces StepListener which provides a simple way to write
a flow consisting of multiple asynchronous steps without having nested
callbacks.

Relates #37291
2019-01-11 13:06:17 -05:00
Gordon Brown 955d3aea19 Mute testRoundRobinWithFailures (#32190) 2019-01-11 09:38:40 -07:00
David Roberts 953fb9352f
[ML] Update error message for process update (#37363)
When this message was first added the model debug config was
the only thing that could be updated, but now more aspects of
the config can be updated so the message needs to be more
general.
2019-01-11 16:31:55 +00:00
Martijn van Groningen e4391afd98
Test fix, wait for auto follower to have stopped in the background
Relates to #36761
2019-01-11 17:26:17 +01:00
Christoph Büscher bb6d8784e7
Switch indices.get rest after backport of `include_type_name` (#37351)
With the `include_type_name` available now for indices.get on 6.x after the
backport, the corresponsing yaml test can include anything from 6.7 on.
Also changing the RestGetIndicesActionTests base test class.
2019-01-11 17:24:12 +01:00
Boaz Leskes 093d4a371c followup to #37284 with additional feedback 2019-01-11 08:07:35 -08:00
Boaz Leskes cae71cddfe
Document Seq No powered optimistic concurrency control (#37284)
Add documentation to describe the new sequence number powered optimistic concurrency control

Relates #36148
Relates #10708
2019-01-11 07:59:15 -08:00
Armin Braun 1eba1d1df9
Fix SnapshotDisruptionIT Race Condition (#37358)
* Due to a race between retrying the snapshot creation and the failed snapshot create trying to delete the snapshot there is no guarantee that the snapshot is eventually created by retries
   * Adjusted the assertion accordingly
* Closes #36779
2019-01-11 16:09:26 +01:00
Yannick Welsch f4abf9628a
Mock connections more accurately in DisruptableMockTransport (#37296)
This commit moves DisruptableMockTransport to use a more accurate representation of connection
management, which allows to use the full connection manager and does not require mocking out
any behavior. With this, we can implement restarting nodes in CoordinatorTests.
2019-01-11 16:06:48 +01:00
Jake Landis fad6329808
add link to freeze from cold phase (#37332) 2019-01-11 08:58:41 -06:00
Boaz Leskes d21df2a17a
Use Sequence number powered OCC for processing updates (#37308)
Updates perform realtime get, perform the requested update and then index the document again
using optimistic concurrency control. This PR changes the logic to use sequence numbers instead
of versioning. 

Note that the current versioning logic isn't suffering from the same problem as external OCC
requests because the get and indexing is always done on the same primary.

Relates #36148 
Relates #10708
2019-01-11 06:23:55 -08:00
Benjamin Trent 19a7e0f4eb
ML: update .ml-state actions to support > 1 index (#37307)
* ML: Updating .ml-state calls to be able to support > 1 index

* Matching bulk delete behavior with dbq

* Adjusting state name

* refreshing indices before search

* fixing line length

* adjusting index expansion options
2019-01-11 08:03:41 -06:00
David Roberts 1da59db3fb
[ML] Wait for autodetect to be ready in the datafeed (#37349)
This is a reinforcement of #37227.  It turns out that
persistent tasks are not made stale if the node they
were running on is restarted and the master node does
not notice this.  The main scenario where this happens
is when minimum master nodes is the same as the number
of nodes in the cluster, so the cluster cannot elect a
master node when any node is restarted.

When an ML node restarts we need the datafeeds for any
jobs that were running on that node to not just wait
until the jobs are allocated, but to wait for the
autodetect process of the job to start up.  In the case
of reassignment of the job persistent task this was
dealt with by the stale status test.  But in the case
where a node restarts but its persistent tasks are not
reassigned we need a deeper test.

Fixes #36810
2019-01-11 13:22:35 +00:00
Yannick Welsch 4d3928d444 Increase timeouts in UnicastZenPingTests
Relates to #37268
2019-01-11 11:33:55 +01:00
Andrei Stefan ed2b5e80eb
Update discovery-ec2.asciidoc (#28925)
* Be more clear about the need to defining tags in EC2 to be able to filter
ES instances.
2019-01-11 11:24:31 +02:00
Alpar Torok 100ef2cc0e
Fix artifactId in plugin poms (#37315) 2019-01-11 11:23:54 +02:00
Yannick Welsch 3a929c7aea Increase assertBusy timeouts for RefreshListenersTests 2019-01-11 10:15:39 +01:00
Alpar Torok 82ca2d62de Mute CloseWhileRelocatingShardsIT.testCloseWhileRelocatingShards
Tracked by #37274
2019-01-11 10:54:53 +02:00
Jim Ferenczi b24dc2c541
[TEST] Awaits tasks termination in the RestHighLevelClient tests (#37302)
This change ensures that TasksIT#testGetValidTask and ReindexIT#testReindexTask
don't leave a non-completed task on the cluster when they finish.

Closes #35644
2019-01-11 09:47:46 +01:00
Alpar Torok 3e73911cbe Mute PrimaryAllocationIT.testForceStaleReplicaToBePromotedToPrimaryOnWrongNode
Tracking issue: #37345
2019-01-11 10:49:25 +02:00
Ignacio Vera 0a50821bb2
Geo: Do not normalize the longitude with value -180 for Lucene shapes (#37299)
Lucene based shapes should not normalize the longitude value -180 to 180.
2019-01-11 09:37:18 +01:00
Nhat Nguyen 360c430ad7
Add runAfter and notifyOnce wrapper to ActionListener (#37331)
Relates #37291
2019-01-11 03:33:06 -05:00
Alexander Reelsen 9f3da013d8
Date/Time parsing: Use java time API instead of exception handling (#37222)
* Add benchmark

* Use java time API instead of exception handling

when several formatters are used, the existing way of parsing those is
to throw an exception catch it, and try the next one. This is is
considerably slower than the approach taken in joda time, so that
indexing is reduced when a date format like `x||y` is used and y is the
date format being used.

This commit now uses the java API to parse the date by appending the
date time formatters to each other and does not rely on exception
handling.

* fix benchmark

* fix tests by changing formatter, also expose printer

* restore optional printing logic to fix tests

* fix tests

* incorporate review comments
2019-01-11 09:25:05 +01:00
Alexander Reelsen bbd093059f
Add whitelist to watcher HttpClient (#36817)
This adds a configurable whitelist to the HTTP client in watcher. By
default every URL is allowed to retain BWC. A dynamically configurable
setting named "xpack.http.whitelist" was added that allows to
configure an array of URLs, which can also contain simple regexes.

Closes #29937
2019-01-11 09:22:47 +01:00
Martijn van Groningen 37493c204d
Unmuted test now that #37239 has been merged and backported.
Relates to #37231
2019-01-11 09:02:46 +01:00
Ioannis Kakavas 80084138dd [DOCS] Fix link to role mapping doc 2019-01-11 09:22:40 +02:00
Armin Braun 860a8a7b23
Improve Precision for scaled_float (#37169)
* Use `toString` and `Bigdecimal` parsing to get intuitive behaviour for `scaled_float` as discussed in #32570
* Closes #32570
2019-01-11 08:07:55 +01:00