Commit Graph

43192 Commits

Author SHA1 Message Date
Alpar Torok 541869a96a
Add missing entries to conffiles (#35810) 2018-11-27 16:38:56 +02:00
Marios Trivyzas 1da9c6faa0
SQL: Fix translation of math functions to painless (#35910)
`SIGN` and `RADIANS` where wrongly overriding `mathFunction()`.
Converted `mathFunction()` to private in `MathFunction` since it
shouldn't be overriden, as it uses the assigned `MathOperation`
to get the funtion name for painless scripts.

Fixes: #35654
2018-11-27 15:33:43 +01:00
Tim Vernum 8d9e21ffaa Fix mutator in GetRolesRequestTests (#35932)
There was no guarantee that the mutated object would not be the same
as the original object (and was for seed 6A8C4CBF63B5AA63)

Resolves #35947
2018-11-27 16:10:14 +02:00
Nik Everett cb8646ac0e
Tests: If dpkg fails try and log failure (#35551)
If `dpkg` fails, try and look for who has `/var/lib/dpkg/lock` open. If
it exists and is open then return a failure with information about who
has file open. This should help us debug #33762.

Closes #34309
2018-11-27 09:06:10 -05:00
Andrey Ershov 0e283f9670
[Zen2] PersistedState interface implementation (#35819)
Today GatewayMetaState is capable of atomically storing MetaData to
disk. We've also moved fields that are needed to be persisted in Zen2
from ClusterState to ClusterState.MetaData.CoordinationMetaData.

This commit implements PersistedState interface.

version and currentTerm are persisted as a part of Manifest.
GatewayMetaState now implements both ClusterStateApplier and
PersistedState interfaces. We started with two descendants
Zen1GatewayMetaState and Zen2GatewayMetaState, but it turned
out to be not easy to glue it.
GatewayMetaState now constructs previousClusterState (including
MetaData) and previousManifest inside the constructor so that all
PersistedState methods are usable as soon as GatewayMetaState
instance is constructed. Also, loadMetaData is renamed to
getMetaData, because it just returns
previousClusterState.metaData().
Sadly, we don't have access to localNode (obtained from 
TransportService in the constructor, so getLastAcceptedState
should be called, after setLocalNode method is invoked.
Currently, when deciding whether to write IndexMetaData to disk,
we're comparing current IndexMetaData version and received
IndexMetaData version. This is not safe in Zen2 if the term has changed.
So updateClusterState now accepts incremental write
method parameter. When it's set to false, we always write
IndexMetaData to disk.
Things that are not covered by GatewayMetaStateTests are covered
by GatewayMetaStatePersistedStateTests.
This commit also adds an option to use GatewayMetaState instead of
InMemoryPersistedState in TestZenDiscovery. However, by default
InMemoryPersistedState is used and only one test in PersistedStateIT
used GatewayMetaState. In order to use it for other tests, proper
state recovery should be implemented.
2018-11-27 15:04:52 +01:00
Marios Trivyzas c91ef1105d
SQL: Implement data type verification for conditionals (#35916)
Add special verifier rule to check that the arguments of conditional
functions are of the same or compatible types. This way the user gets
a descriptive error message with line number and column indicating
where is the offending argument.

Closes: #35907
2018-11-27 14:46:11 +01:00
David Roberts 110c4fdd65
[ML] Adjust file structure finder parser config (#35935) 2018-11-27 12:52:52 +00:00
Tim Vernum b6eb73a835
Update BWC checks in HasPrivilegesResponse (#35929)
This serialization has been backported to 6.x
2018-11-27 21:05:12 +11:00
Martijn van Groningen 4a3d66c668
[TEST] wrong bugUrl...
Relates to #35937
2018-11-27 10:53:04 +01:00
Martijn van Groningen fabffb89b8
Muted test. This test expose an issue inside the auto follower coordinator,
it has nothing to do with the hlrc support for put auto follow pattern api,
this test was added for.

Relates to #35480
2018-11-27 10:49:03 +01:00
Yogesh Gaikwad 31fdb76973
[DOCS] Document authorization_realms for Kerberos realm (#35927)
This commit adds documentation for authorization_realms
setting for the Kerberos realm and also corrects a typo in
existing documentation.

Co-authored-by: @A-Hall
2018-11-27 19:09:17 +11:00
Martijn van Groningen df1e02d0d5
[HLRC] Added support for CCR Put Auto Follow Pattern API (#35780)
This change also adds documentation for the Put Auto Follow Pattern API.

Relates to #33824
2018-11-27 08:53:22 +01:00
Martijn van Groningen 447e5d212a
Changed versions in serialization code after backporting #35535 2018-11-27 08:00:06 +01:00
Ioannis Kakavas 998c663f0d
Add Tests for findSamlRealm (#35905)
This commit adds a test for handling correctly all they possible 
`SamlPrepareAuthenticationRequest` parameter combinations that 
we might get from Kibana or a custom web application talking to the
SAML APIs. 
We can match the correct SAML realm based either on the realm name
or the ACS URL. If both are included in the request then both need to 
match the realm configuration.
2018-11-27 08:18:18 +02:00
Tim Vernum 5b427d415e
Add "request.id" to file audit logs (#35536)
This generates a synthesized "id" for each incoming request that is
included in the audit logs (file only).
This id can be used to correlate events for the same request (e.g.
authentication success with access granted).

This request.id is specific to the audit logs and is not used for any
other purpose

The request.id is consistent across nodes if a single request requires
execution on multiple nodes (e.g. search acros multiple shards).
2018-11-27 15:19:47 +11:00
Tim Vernum a18b219f79
Allow noop PutUser updates (#35843)
When assertions are enabled, a Put User action that have no effect (a
noop update) would trigger an assertion failure and shutdown the node.

This change accepts "noop" as an update result, and adds more
diagnostics to the assertion failure message.
2018-11-27 15:08:53 +11:00
Tim Vernum 3435fc4613
HLRC: Add ability to put user with a password hash (#35844)
Update PutUserRequest to support password_hash (see: #35242)

This also updates the documentation to bring it in line with our more
recent approach to HLRC docs.
2018-11-27 15:07:24 +11:00
Ryan Ernst 72d7f236a5
Scripting: Add back joda to whitelist (#35915)
Watcher still exposes some dates as joda DateTime objects. This commit
adds back joda to the painless whitelist so they can still be accessed.

closes #35913
2018-11-26 16:33:05 -08:00
Ryan Ernst 10447dd962
Build: Fix jdbc jar to include deps (#35602)
This commit adds back bundling of all deps of the sql jdbc jar. This was
lost in a refactoring of how the shadow plugin is handled for the entire
elasticsearch project.
2018-11-26 16:18:35 -08:00
Gordon Brown 119835decd
Always enforce cluster-wide shard limit (#34892)
This removes the option to run a cluster without enforcing the
cluster-wide shard limit, making strict enforcement the default and only
behavior.  The limit can still be adjusted as desired using the cluster
settings API.
2018-11-26 17:05:12 -07:00
Christoph Büscher 4d525e3e33
[Docs] Align callouts in search.asciidoc (#35897) 2018-11-27 00:05:15 +01:00
Gordon Brown 5c11b8612a
Add HLRC docs for Explain Lifecycle (#35803)
Adds HLRC documentation for the Explain Lifecycle API.
2018-11-26 15:25:56 -07:00
Sachin Araballi 5f656e4eab Fields are declared at the top of the class. (#35891)
Fields should be declared at the top of the class, before any method declarations, constructors, initializers or inner classes.
2018-11-26 22:05:28 +01:00
Tal Levy 8fe5f66637
[ILM] fix rest-api-spec doc links (#35478) 2018-11-26 12:26:36 -08:00
Igor Motov 663563f64b
Geo: better handling of malformed geo_points (#35554)
Improves handling of malformed geo_points when `ignore_malformed` is
set to true

Closes #35419
2018-11-26 09:44:42 -10:00
Jim Ferenczi 900caa20ef
Handles exists query in composite aggs (#35758)
This commit adds the support for exists query in the sorted execution mode
of the `composite` aggregation. We'll execute the aggregation from the sorted
points and use early termination if the main query is an `exists` query over the
first source of the `composite` aggregation.
2018-11-26 19:08:14 +01:00
Marios Trivyzas 3f7cae3f0d
SQL: Implement GREATEST and LEAST functions (#35879)
Add GREATEST(expr1, expr2, ... exprN) and LEAST(expr1, expr2, exprN)
functions which are in the family of CONDITIONAL functions.

Implementation follows PostgreSQL behaviour, so the functions return
`NULL` when all of their arguments evaluate to `NULL`.

Renamed `CoalescePipe` and `CoalesceProcessor` to `ConditionalPipe` and
`ConditionalProcessor` respectively, to be able to reuse them for
`Greatest` and `Least` evaluations. To achieve that `ConditionalOperation`
has been added to differentiate between the functionalities at execution
time.

Closes: #35878
2018-11-26 18:21:36 +01:00
Christophe Bismuth b95a4db6e6 Throw a parsing exception when boost is set in span_or query (#28390) (#34112) 2018-11-26 12:15:59 -05:00
John 0baffda390 ingest: grok remove duplicated patterns (#35886)
This commit removes the redundant (and incorrect) JAVACLASS
and JAVAFILE grok patterns. This helps to keep parity with 
Logstash's patterns. 

See also: https://github.com/logstash-plugins/logstash-patterns-core/pull/237
 
closes #35699
2018-11-26 11:13:46 -06:00
Jake Landis 4b99a663c1
ingest: fix broken doc link 2018-11-26 10:34:42 -06:00
Ed Savage 13e11966ca
[HLRC][ML] Add delete expired data API (#35906)
Relates to #29827
2018-11-26 16:15:54 +00:00
Jim Ferenczi 3ed3aab127
Fix rest test when numShards is greater than 1 (#35909) 2018-11-26 16:45:56 +01:00
Jake Landis 7f7b31723e
ingest: extended `if` documentation (#35044)
part of #33188
2018-11-26 09:35:45 -06:00
Costin Leau d291b080de
SQL: SYS COLUMNS returns ODBC specific schema (#35870)
Due to some unresolvable type conflict between the expected definition
 in JDBC vs ODBC, the driver mode is now passed over so that certain
 command can change their results accordingly (in this case SYS COLUMNS)

Fix #35376
2018-11-26 16:59:09 +02:00
Simon Willnauer ca9b2b9931
Repsect indices options on _msearch (#35887)
Today we don't respect the indices options when they are passed
as request parameters to the `_msearch` endpoint. This is unintuitive
and doesn't cause any errors. This changes uses the top-level indices
options as the defaults for each sub search-request.

Closes #35851
2018-11-26 14:26:39 +01:00
Marios Trivyzas b078e2970c
SQL: Implement null safe equality operator `<=>` (#35873)
This operator handles nulls in different way than the normal `=`.
If one of the operants is `null` and the other not it returns `false`.
If both operants are `null` it returns `true`. Therefore in contrary to
`=`, which returns `null` if at least one of the operants is `null`, this one
never returns `null` as a result.

Closes: #35871
2018-11-26 14:02:02 +01:00
Christophe Bismuth 04ebc63e34 RoutingMissingException in more like this (#33974)
More like this query allows to provide identifiers of documents to be retrieved as like/unlike items. 
It can happen that at retrieval time an error is thrown, for instance caused by missing routing value when `_routing` is set required in the mapping. 
Instead of ignoring such error and returning no documents for the query, the error should be re-thrown and returned to users. As part of this 
change also mget and mtermvectors are unified in the way they throw such exception like it happens in other places, so that a `RoutingMissingException` is raised.

Closes #29678
2018-11-26 13:57:57 +01:00
Tanguy Leroux 9bdbba23f8 [Tests] Fix IndexShardTests.testAcquirePrimaryAllOperationsPermits()
This test fails on CI because of an inappropriate assertion, which is
I think a leftover and has no real value.
2018-11-26 13:44:12 +01:00
Luca Cavanna e44390ac20
InitialSearchPhase minor cleanups (#35864)
This commit simplifies  the throttling logic in InitialSearchPhase and removes some asserts from it. Also, a few formatting changes are applied to its code and surrounding classes.
2018-11-26 13:42:41 +01:00
David Turner a68a46450b
[Zen2] Add lag detector (#35685)
A publication can succeed and complete before all nodes have applied the
published state and acknowledged it, thanks to the publication timeout; however
we need every node eventually either to apply the published state (or a later
state) or be removed from the cluster. This change introduces the LagDetector
which achieves this liveness property by removing any lagging nodes from the
cluster.
2018-11-26 10:52:49 +00:00
Costin Leau cd822b7ca8 SQL: Fix incorrect merging of #35852 2018-11-26 12:41:43 +02:00
Costin Leau 633a562208 SQL: Fix incorrect merging of #35852 2018-11-26 12:35:46 +02:00
Ioannis Kakavas acb65dd004 [DOCS] Fix missing include for get-roles 2018-11-26 11:45:39 +02:00
Costin Leau 971299baf4
SQL: Add filtering to SYS TYPES (#35852)
Fix #35342
2018-11-26 11:26:24 +02:00
Ioannis Kakavas 8daa854f90
[HLRC] Add support for get roles API (#35787)
This commits adds support for the Get Roles API to the HLRC

Relates: #29827
2018-11-26 11:25:07 +02:00
Yannick Welsch dfd93deabe Remove superfluous comment 2018-11-26 09:57:40 +01:00
iverase 401b814d1a [CI] Muting method testOperationPermitOnReplicaShards in IndexShardTests
Relates to #35850
2018-11-26 09:33:23 +01:00
Martijn van Groningen 7624734f14
Added wait_for_metadata_version parameter to cluster state api. (#35535)
The `wait_for_metadata_version` parameter will instruct the cluster state
api to only return a cluster state until the metadata's version is equal or
greater than the version specified in `wait_for_metadata_version`. If  
the specified `wait_for_timeout` has expired then a timed out response 
is returned. (a response with no cluster state and wait for timed out flag set to true)
In  the case metadata's version is equal or higher than  `wait_for_metadata_version`
then the api will immediately return.

This feature is useful to avoid external components from constantly
polling the cluster state to whether somethings have changed in the
cluster state's metadata.
2018-11-26 08:50:08 +01:00
Andrei Stefan 00e6fec718
SQL: XPack FeatureSet functionality (#35725)
* Introduced "client.id" parameter for REST requests
* Bug that made the Verifier run twice, fixed in the Analyzer
* Single node IT and unit testing
2018-11-26 07:11:43 +02:00
Tal Levy a0f8a22d53
[TEST] ILM skip checking _ilm/explain response (#35830)
relates #35562

Closes #35824.
2018-11-25 17:19:17 -08:00