Commit Graph

42773 Commits

Author SHA1 Message Date
Martijn van Groningen fba811fa3a
[TEST] increased the number of index and delete ops to make it less likely that all ops exist as soft delete docs. 2018-11-09 15:31:51 +01:00
Martijn van Groningen 83152b3835
[CCR] Get all auto follow patterns and no auto follow metadata (#35381)
Return empty response when querying all auto follow patterns,
but there is no auto follow metadata.
2018-11-09 14:24:27 +01:00
Marios Trivyzas 36da6e1671
SQL: Introduce IsNull node to simplify expressions (#35206)
Add `IsNull` node in parser to simplify expressions so that `<value> IS NULL` is
no longer translated internally to `NOT(<value> IS NOT NULL)`

Replace `IsNotNullProcessor` with `CheckNullProcessor` to encapsulate both
isNull and isNotNull functionality.

Closes: #34876
Fixes: #35171
2018-11-09 11:32:38 +01:00
Armin Braun 529910a43c
DISCOVERY: Fix RollingUpgradeTests (#35375)
* DISCOVERY: Fix RollingUpgradeTests

* Don't manually manage min master nodes if not necessary
* Remove some dead code
* Allow for manually supplying list of seed nodes
* Closes #35178
2018-11-09 10:47:30 +01:00
Albert Zaharovits 617f91bb0f
DOCS Auditing search queries (#35301)
This documents how to include the search queries in the audit log.

There is a catch, that even if enabling `emit_request_body`, which should
output queries included in request bodies, search queries were not output
because, implicitly, no REST layer audit event type was included.

This folk knowledge is herein imprinted.
2018-11-09 11:38:45 +02:00
Jim Ferenczi 7054e289fa
Add trace log of the request for the query and fetch phases (#34479)
This change adds a logger for the query and fetch phases that prints all requests
before their execution at the trace level. This will help debugging cases where an issue
occurs during the execution since only completed queries are logged by the slow logs.
2018-11-09 09:41:51 +01:00
Tim Vernum a9641960e1
Refactor realm tests to use single settings (#35362)
Many realm tests were written to use separate setting objects for
"global settings" and "realm settings".
Since #30241 there is no distinction between these settings, so these
tests can be cleaned up to use a single Settings object.
2018-11-09 18:13:24 +11:00
Alpar Torok 66e3ca9724 Revert "Mute test for #35361"
This reverts commit f9154758e6.
2018-11-09 08:04:56 +02:00
Tim Vernum b0acd709d7
DOCS: Add SecurityExtension breaking change (#35358)
We changed the way realm settings are defined, and this affects custom
realms in SecurityExtensions. This change adds those details to the
breaking changes docs.

Relates: #30241
2018-11-09 16:57:49 +11:00
Armin Braun 02b4e28534
#31608 Add S3 Setting to Force Path Type Access (#34721)
* SNAPSHOTS: Use Path Style Access in S3

* Use path style access pattern to fix #31608
* closes #31608
2018-11-09 05:07:26 +01:00
Tim Brooks bccc99c2be
Fix TcpTransport compression test (#35396)
This commit fixes an assertion in the TcpTransportTests compresssion
test.
2018-11-08 18:04:48 -07:00
debadair b485e69a5f
ILM API ref edits (#35112)
* [DOCS] ILM API Ref edits

* [DOCS] Fixed endpoint for DELETE policy.

* [DOCS] Removed comparison to setting index.lifecycle.name to null.

* [DOCS] Fixed xrefs to explain API.
2018-11-08 15:26:27 -08:00
Marios Trivyzas c9a84fe361
SQL: Handle null literal for AND and OR in `WHERE` (#35236)
Change `nullable()` logic of AND and OR to false since in the Optimizer
we cannot fold to null as we might be handling and expression in the
SELECT clause.

Introduce folding of null for AND and OR expressions in PruneFilter()
since we now know that we are in HAVING or WHERE clause and we
can fold `null` to `false`

Fixes: #35088

Co-authored-by: Costin Leau <costin.leau@gmail.com>
2018-11-08 23:32:59 +01:00
Lisa Cawley c88b2d7e5a
[DOCS] Cleans up monitoring configuration steps (#35111) 2018-11-08 13:53:09 -08:00
Mayya Sharipova 7e144b318a
Painless Context Doc: Add field context example (#35130)
* Painless Context Doc: Add field context example

relates to #34829
2018-11-08 14:54:20 -05:00
Tim Brooks 93c2c604e5
Move compression config to ConnectionProfile (#35357)
This is related to #34483. It introduces a namespaced setting for
compression that allows users to configure compression on a per remote
cluster basis. The transport.tcp.compress remains as a fallback
setting. If transport.tcp.compress is set to true, then all requests
and responses are compressed. If it is set to false, only requests to
clusters based on the cluster.remote.cluster_name.transport.compress
setting are compressed. However, after this change regardless of any
local settings, responses will be compressed if the request that is
received was compressed.
2018-11-08 10:37:59 -07:00
Jason Tedor 5c2a5f2e37
Adjust BWC version on OS pretty name
This commit adjusts the BWC version the OS pretty name field on OsInfo
now that this field has been backported to the 6.x development branch.
2018-11-08 12:24:10 -05:00
Jason Tedor 730ec1ddfb
Add more detailed OS name on Linux (#35352)
Today our OS information returned in node stats only returns a
high-level name of the OS (e.g., "Linux"). Yet, for some uses this is
too high-level and knowing at a finer level of granularity the
underlying OS can be useful. This commit extracts the pretty name on
Linux from /etc/os-release. This pretty name usually includes the Linux
vendor and the Linux vendor version number (e.g., Fedora 28).
2018-11-08 12:16:58 -05:00
Albert Zaharovits 85a8b517bd
HLRest: refactor put_user to utilize `User` object (#35188)
This follows #33552 , when the `_authenticate` API added a new
`User` object for the API's response. This changes the `put_user`
API to also employ a `User` object in the request.
The User object changed slightly.
A bug with put_user only putting/updating enabled (but not disabled)
users has been fixed.
2018-11-08 18:42:36 +02:00
Christoph Büscher 113af7996c
Make limit on number of expanded fields configurable (#35284)
Currently we introduced a hard limit of 1024 to the number of fields a query can
be expanded to in #26541. Instead of using a hard limit, we should make this
configurable. This change removes the hard limit check and uses the existing
`max_clause_count` setting instead.

Closes #34778
2018-11-08 17:04:40 +01:00
Zachary Tong 55c6481b6b
[Docs] Add painless context details for bucket_selector (#35162)
Adds docs for the bucket_selector context, an example
and corresponding doc test
2018-11-08 11:02:30 -05:00
Zachary Tong ac1d8da283 [Test] Remove unused imports
Checkstyle
2018-11-08 10:31:56 -05:00
Hendrik Muhs 433469d4d0
[Rollup] improve handling of failures on first search (#35269)
Improve error handling in the Indexer if an exception occurs during the very 1st retrieval (query execution)
2018-11-08 16:28:48 +01:00
Martijn van Groningen 07a69a528b
[CCR] Rename leaderClient variables and parameters to remoteClient (#35368) 2018-11-08 16:26:14 +01:00
Zachary Tong 54b445d74b [Test] Remove obsolete job/cluster cleanup code
Also makes sure the awaitBusy for job stoppage is checked, so
that we can fail if we timed out waiting for a job to stop.

Closes #35295
2018-11-08 10:23:23 -05:00
Jim Ferenczi d96202a282 [DOCS] Fix missing callouts 2018-11-08 15:40:01 +01:00
Daniel Mitterdorfer 6980feddd2
Remove unused class MemoryCircuitBreaker
The class `MemoryCircuitBreaker` is unused so we remove all its traces
from the code base.

Relates #35367
2018-11-08 15:33:24 +01:00
Daniel Mitterdorfer 20cf13d10e
Use Gradle wrapper in client benchmark README
With this commit we use the Gradle wrapper in documentation examples for the
client benchmarks instead of Gradle itself.

Relates #35366
2018-11-08 15:32:47 +01:00
Alpar Torok ca12de5841 Mute test for #35295 2018-11-08 15:49:40 +02:00
Dominik Stadler d351422215 Add parent-aggregation to parent-join module (#34210)
Add `parent` aggregation, a special single bucket aggregation that joins children documents to their parent.
2018-11-08 14:13:00 +01:00
Albert Zaharovits 025a0c82e5
Remove deprecated audit settings (#35205)
Removes `.prefix` deprecated settings for the logfile
auditing and also documents it in the migrate asciidoc.
2018-11-08 14:06:47 +02:00
Russ Cam 848847d8c7 [Docs] Section header preceded by blank line (#34340) 2018-11-08 12:44:13 +01:00
Alpar Torok 518e0de078 Mute test #35365 2018-11-08 12:27:40 +02:00
Jeff Hajewski d00b23c8b1 Fixes fast vector highlighter docs per issue 24318. (#34190)
The `fvh` highlighter does not support span queries. This fix updates
the docs to add a warning stating the lack of span query support for
`fvh`.
2018-11-08 11:09:03 +01:00
David Kyle 9494e046e7
[ML] Prevent notifications on deletion of a non existent job (#35337) 2018-11-08 09:57:07 +00:00
Martijn van Groningen 8a85251da0
[CCR] Auto follow Coordinator fetch cluster state in system context (#35120)
Auto follow Coordinator should fetch the leader cluster state using system context.
2018-11-08 10:48:27 +01:00
Alpar Torok f9154758e6 Mute test for #35361 2018-11-08 11:31:51 +02:00
Christoph Büscher 14b811446f
Preserve `date_histogram` format when aggregating on unmapped fields (#35254)
Currently when aggregating on an unmapped date field (e.g. using a
date_histogram) we don't preserve the aggregations `format` setting but instead
use the default format. This can lead to loosing the aggregations `format` when
aggregating over several indices where some of them contain unmapped date fields
and are encountered first in the reduce phase.

Related to #31760
2018-11-08 10:22:25 +01:00
Tim Vernum 0b24c4f8e7 Test: Mute failing SSL test
Relates: https://github.com/elastic/elasticsearch/issues/35360
2018-11-08 19:37:29 +11:00
Jim Ferenczi 891fdda68e
Allow unmapped fields in composite aggregations (#35331)
Today the `composite` aggregation throws an error if a source targets an
unmapped field and `missing_bucket` is set to false. Documents without a
value for a source cannot produce any bucket if `missing_bucket` is not
activated so the error is a shortcut to say that the response will be empty.
However this is not consistent with the `terms` aggregation which accepts
unmapped field by default even if the response is also guaranteed to be empty.
This commit removes this restriction, if a source contains an unmapped field
we now return an empty response (no buckets).

Closes #35317
2018-11-08 09:30:52 +01:00
Tanguy Leroux 1703a61fec
[RCI] Add IndexShardOperationPermits.asyncBlockOperations(ActionListener<Releasable>) (#34902)
The current implementation of asyncBlockOperations() can be used to
execute some code once all indexing operations permits have been acquired,
 then releases all permits immediately after the code execution. This
 immediate release is not suitable for treatments that need to keep all
 permits over multiple execution steps.

This commit adds a new asyncBlockOperations() that exposes a Releasable,
 making it possible to acquire all permits and only release them all
 when needed by closing the Releasable. The existing blockOperations() 
method has been modified to delegate permit acquisition/releasing to this new
method.

Relates to #33888
2018-11-08 09:23:33 +01:00
Przemyslaw Gomulka a90ef6bd6e
HLRC: reindex API with wait_for_completion false (#35202)
Extend High Level Rest Client Reindex API to support requests with
wait_for_completion=false. This method will return a TaskSubmissionResult with task identifier as string and results can be queried with Task API

refers: #27205
2018-11-08 08:19:27 +01:00
Jason Tedor a6073f5130
Add docs on JNA temp directory not being noexec (#35355)
If the underlying mount point for the JNA temporary directory is mounted
noexec on Linux, then the JVM will not be able to map the native code in
as executable. This will prevent JNA from executing and will prevent
Elasticsearch from being able to execute some functions that rely on
native code (e.g., memory locking, and installing system call
filters). We do not want to get into the business of catching exceptions
and parsing messages towards this because these exception messages can
change on us. We also do not want to jump through a lot of hoops to
check the underlying mount point for noexec. Instead, we will rely on
documentation to address this problem. This commit adds to the important
system configuration section of the docs that the JNA temporary
directory is not on a mount point with the noexec mount option.
2018-11-07 22:25:37 -05:00
Martijn van Groningen 2f2090f562 [CCR] Adjust list of dynamic index settings that should be replicated (#35195)
Adjust list of dynamic index settings that should be replicated
and added a test that verifies whether builtin dynamic index settings
are classified as replicated or non replicated (whitelisted).
2018-11-07 21:59:58 -05:00
Jason Tedor 4f4fc3b8f8
Replicate index settings to followers (#35089)
This commit uses the index settings version so that a follower can
replicate index settings changes as needed from the leader.

Co-authored-by: Martijn van Groningen <martijn.v.groningen@gmail.com>
2018-11-07 21:20:51 -05:00
Tim Vernum 7d05257896
Rename RealmConfig.globalSettings() to settings() (#35330)
There is no longer a concept of non-global "realm settings". All realm
settings should be loaded from the node's settings using standard
Setting classes.

This change renames the "globalSettings" field and method to simply be
"settings".
2018-11-08 12:57:42 +11:00
Tim Vernum 566979cc31
[TEST] Cleanup FileUserPasswdStoreTests (#35329)
The file realm has not supported custom filenames/locations since at
least 5.0, but this test still tried to configure them.

Remove all configuration of file locations, and cleaned up a few other
warnings and deprecations
2018-11-08 12:57:22 +11:00
Ryan Ernst a4d979cfc8 Scripting: Add back lookup vars in score script (#34833)
The lookup vars under params (namely _fields and _source) were
inadvertently removed when scoring scripts were converted to using
script contexts. This commit adds them back, along with deprecation
warnings for those that should not be used.
2018-11-07 15:09:09 -08:00
Jake Landis d7a4cef483
watcher: Fix integration tests to ensure correct start/stop of Watcher (#35271)
Ensure that Watcher is correctly started and stopped between tests for
SmokeTestWatcherWithSecurityIT,
SmokeTestWatcherWithSecurityClientYamlTestSuiteIT,
SmokeTestWatcherTestSuiteIT, WatcherRestIT,
XDocsClientYamlTestSuiteIT, and XPackRestIT

The change here is to throw an `AssertionError` instead of `break;` to
allow the `assertBusy()` to continue to busy wait until the desired
state is reached.

closes #33291, closes #29877, closes #34462, closes #30705, closes #34448
2018-11-07 15:08:03 -06:00
Lee Hinman b1655c468a
Remove ALL shard check in CheckShrinkReadyStep (#35346)
Since it's still possible to shrink an index when replicas are unassigned, we
should not check that all copies are available when performing the shrink, since
we set the allocation requirement for a single node.

Resolves #35321
2018-11-07 13:40:19 -07:00