Commit Graph

42728 Commits

Author SHA1 Message Date
John 0baffda390 ingest: grok remove duplicated patterns (#35886)
This commit removes the redundant (and incorrect) JAVACLASS
and JAVAFILE grok patterns. This helps to keep parity with 
Logstash's patterns. 

See also: https://github.com/logstash-plugins/logstash-patterns-core/pull/237
 
closes #35699
2018-11-26 11:13:46 -06:00
Jake Landis 4b99a663c1
ingest: fix broken doc link 2018-11-26 10:34:42 -06:00
Ed Savage 13e11966ca
[HLRC][ML] Add delete expired data API (#35906)
Relates to #29827
2018-11-26 16:15:54 +00:00
Jim Ferenczi 3ed3aab127
Fix rest test when numShards is greater than 1 (#35909) 2018-11-26 16:45:56 +01:00
Jake Landis 7f7b31723e
ingest: extended `if` documentation (#35044)
part of #33188
2018-11-26 09:35:45 -06:00
Costin Leau d291b080de
SQL: SYS COLUMNS returns ODBC specific schema (#35870)
Due to some unresolvable type conflict between the expected definition
 in JDBC vs ODBC, the driver mode is now passed over so that certain
 command can change their results accordingly (in this case SYS COLUMNS)

Fix #35376
2018-11-26 16:59:09 +02:00
Simon Willnauer ca9b2b9931
Repsect indices options on _msearch (#35887)
Today we don't respect the indices options when they are passed
as request parameters to the `_msearch` endpoint. This is unintuitive
and doesn't cause any errors. This changes uses the top-level indices
options as the defaults for each sub search-request.

Closes #35851
2018-11-26 14:26:39 +01:00
Marios Trivyzas b078e2970c
SQL: Implement null safe equality operator `<=>` (#35873)
This operator handles nulls in different way than the normal `=`.
If one of the operants is `null` and the other not it returns `false`.
If both operants are `null` it returns `true`. Therefore in contrary to
`=`, which returns `null` if at least one of the operants is `null`, this one
never returns `null` as a result.

Closes: #35871
2018-11-26 14:02:02 +01:00
Christophe Bismuth 04ebc63e34 RoutingMissingException in more like this (#33974)
More like this query allows to provide identifiers of documents to be retrieved as like/unlike items. 
It can happen that at retrieval time an error is thrown, for instance caused by missing routing value when `_routing` is set required in the mapping. 
Instead of ignoring such error and returning no documents for the query, the error should be re-thrown and returned to users. As part of this 
change also mget and mtermvectors are unified in the way they throw such exception like it happens in other places, so that a `RoutingMissingException` is raised.

Closes #29678
2018-11-26 13:57:57 +01:00
Tanguy Leroux 9bdbba23f8 [Tests] Fix IndexShardTests.testAcquirePrimaryAllOperationsPermits()
This test fails on CI because of an inappropriate assertion, which is
I think a leftover and has no real value.
2018-11-26 13:44:12 +01:00
Luca Cavanna e44390ac20
InitialSearchPhase minor cleanups (#35864)
This commit simplifies  the throttling logic in InitialSearchPhase and removes some asserts from it. Also, a few formatting changes are applied to its code and surrounding classes.
2018-11-26 13:42:41 +01:00
Costin Leau cd822b7ca8 SQL: Fix incorrect merging of #35852 2018-11-26 12:41:43 +02:00
Costin Leau 633a562208 SQL: Fix incorrect merging of #35852 2018-11-26 12:35:46 +02:00
Ioannis Kakavas acb65dd004 [DOCS] Fix missing include for get-roles 2018-11-26 11:45:39 +02:00
Costin Leau 971299baf4
SQL: Add filtering to SYS TYPES (#35852)
Fix #35342
2018-11-26 11:26:24 +02:00
Ioannis Kakavas 8daa854f90
[HLRC] Add support for get roles API (#35787)
This commits adds support for the Get Roles API to the HLRC

Relates: #29827
2018-11-26 11:25:07 +02:00
Yannick Welsch dfd93deabe Remove superfluous comment 2018-11-26 09:57:40 +01:00
iverase 401b814d1a [CI] Muting method testOperationPermitOnReplicaShards in IndexShardTests
Relates to #35850
2018-11-26 09:33:23 +01:00
Martijn van Groningen 7624734f14
Added wait_for_metadata_version parameter to cluster state api. (#35535)
The `wait_for_metadata_version` parameter will instruct the cluster state
api to only return a cluster state until the metadata's version is equal or
greater than the version specified in `wait_for_metadata_version`. If  
the specified `wait_for_timeout` has expired then a timed out response 
is returned. (a response with no cluster state and wait for timed out flag set to true)
In  the case metadata's version is equal or higher than  `wait_for_metadata_version`
then the api will immediately return.

This feature is useful to avoid external components from constantly
polling the cluster state to whether somethings have changed in the
cluster state's metadata.
2018-11-26 08:50:08 +01:00
Andrei Stefan 00e6fec718
SQL: XPack FeatureSet functionality (#35725)
* Introduced "client.id" parameter for REST requests
* Bug that made the Verifier run twice, fixed in the Analyzer
* Single node IT and unit testing
2018-11-26 07:11:43 +02:00
Tal Levy a0f8a22d53
[TEST] ILM skip checking _ilm/explain response (#35830)
relates #35562

Closes #35824.
2018-11-25 17:19:17 -08:00
Armin Braun 04a48430aa
NETWORKING: Upgrade Netty 4.3.31.Final (#35877) 2018-11-25 10:17:54 +01:00
Armin Braun 05620ccaaf
TESTS: Disable Snapshot Repo Verify in BwC Test (#35876)
* Disable snapshot repo verification temporarily because it runs into HTTP-500 in 6.6.0-SNAPSHOT
* Relates #35874
2018-11-24 19:24:59 +01:00
Simon Willnauer 4711c5cdf3
Always return false from `refreshNeeded` on ReadOnlyEngine (#35837)
Acquiring a searcher is unnecessary to determine if a refresh is
necessary since read-only engines never refresh.

Closes #35785
2018-11-24 09:25:42 +01:00
Simon Willnauer e46e44ce38
Wrap can_match reader with ElasticsearchDirectoryReader (#35857)
Code that operates on-top of the engine requires all readers returned to be
unwrapped into ElasticsearchDirectoryReader. The special reader
the FrozenEngine uses wasn't wrapped.
2018-11-24 09:23:53 +01:00
Marios Trivyzas 410f570d5f
SQL: Implement NULLIF(expr1, expr2) function (#35826)
NULLIF returns null if the 2 expressions are equal or the
expr1 otherwise.

Closes: #35818
2018-11-23 22:19:27 +01:00
Yannick Welsch 51d2e986c5 Remove BWC conditions after backport of #35731
This PR was backported to 6.x, so the extra BWC conditions are not needed anymore
2018-11-23 17:11:06 +01:00
Costin Leau e22a83554e
SQL: Polish grammar for intervals (#35853) 2018-11-23 17:14:52 +02:00
William Desportes a204d1cdff [Docs] Fix typo in datehistogram-aggregation.asciidoc (#35855) 2018-11-23 15:16:53 +01:00
Adrien Grand 5b370316a6
Remove some legacy code from when indices could have multiple types. (#35815)
This code is only necessary up to indices created with version 5.x while 7.0
only supports indices created with 6.x or 7.0.
2018-11-23 15:15:26 +01:00
Yannick Welsch 2970abfce9
Add read-only repository verification (#35731)
Adds a verification mode for read-only repositories. It also makes the extra bucket check on
repository creation obsolete, which fixes #35703.
2018-11-23 14:45:05 +01:00
Christoph Büscher 88d862e69f [CI] Muting two methods in IndexShardTests
Relates to #35850
2018-11-23 14:29:26 +01:00
David Turner d01436de3c
Copy checkpoint atomically when rolling generation (#35407)
Today when rolling a transog generation we copy the checkpoint from
`translog.ckp` to `translog-nnnn.ckp` using a simple `Files.copy()` followed by
appropriate `fsync()` calls. The copy operation is not atomic, so if we crash
at the wrong moment we can leave an incomplete checkpoint file on disk. In
practice the checkpoint is so small that it's either empty or fully written.
However, we do not correctly handle the case where it's empty when the node
restarts.

In contrast, in `recoverFromFiles()` we _do_ copy the checkpoint atomically.
This commit extracts the atomic copy operation from `recoverFromFiles()` and
re-uses it in `rollGeneration()`.
2018-11-23 08:43:34 +00:00
Jim Ferenczi be69a774df
Fix analyzed prefix query in query_string (#35756)
This change fixes analyzed prefix queries in `query_string` to be ignored
if all terms are removed during the analysis.

Closes #31702
2018-11-23 09:42:23 +01:00
Tanguy Leroux 2e37f17a7d
Expose all permits acquisition in IndexShard and TransportReplicationAction (#35540)
This pull request exposes two new methods in the IndexShard and 
TransportReplicationAction classes in order to allow transport replication 
actions to acquire all index shard operation permits for their execution.

It first adds the acquireAllPrimaryOperationPermits() and the 
acquireAllReplicaOperationsPermits() methods to the IndexShard class 
which allow to acquire all operations permits on a shard while exposing 
a Releasable. It also refactors the TransportReplicationAction class to 
expose two protected methods (acquirePrimaryOperationPermit() and 
acquireReplicaOperationPermit()) that can be overridden when a transport 
replication action requires the acquisition of all permits on primary and/or 
replica shard during execution.

Finally, it adds a TransportReplicationAllPermitsAcquisitionTests which
 illustrates how a transport replication action can grab all permits before 
adding a cluster block in the cluster state, making subsequent operations 
that requires a single permit to fail).

Related to elastic #33888
2018-11-23 09:26:38 +01:00
David Roberts 3c059ee057
[HLRC][ML] Add ML find file structure API (#35833)
Relates to #29827
2018-11-23 06:58:05 +00:00
Martijn van Groningen 813e053f62
Fixed response classes in hlrc docs 2018-11-23 07:52:45 +01:00
Stéphane Campinas fb80d298de [Docs] Correct template example description #35829 2018-11-23 05:29:20 +01:00
Christoph Büscher 6fb7ce66e6
[Tests] Fix creating ExplainLifecycleRequest with no indices (#35828)
We didn't check that the ExplainLifecycleRequest was constructed with at least
one index before, now that we do we must also make sure the tests
mutateInstance() method used in equals/hashCode checks doesn't accidentally
create an empty index array.

Closes #35822
2018-11-23 05:03:19 +01:00
David Roberts 6f46584380
[ML] Add docs for ML info endpoint (#35783)
This endpoint was not previously documented as it was not
particularly useful to end users.  However, since the HLRC
will support the endpoint we need some documentation to
link to.

The purpose of the endpoint is to provide defaults and
limits used by ML.  These are needed to fully understand
configurations that have missing values because the missing
value means the default should be used.

Relates #35777
2018-11-22 16:23:31 +00:00
Dimitris Athanasiou 43d6ec8bcd
Remove unnecessary throws IOException in CompressedXContent.string() (#35821) 2018-11-22 15:08:46 +00:00
Jim Ferenczi e37a0ef844
Upgrade to lucene-8.0.0-snapshot-67cdd21996 (#35816) 2018-11-22 15:42:59 +01:00
Martijn van Groningen 96a741f2c2
[TEST] escape brackets
Relates to #35496
2018-11-22 14:58:42 +01:00
Albert Zaharovits 4fc911a129 Mute test InternalEngineTests
Relates #35823
2018-11-22 15:34:53 +02:00
Albert Zaharovits 7daccaa422 Mute test
Relates #35822
2018-11-22 15:12:16 +02:00
Tanguy Leroux f9f7261d60 Revert "Revert "[RCI] Check blocks while having index shard permit in TransportReplicationAction (#35332)""
This reverts commit d3d7c01
2018-11-22 12:13:19 +01:00
Mayya Sharipova b6014d971c
Forbid negative scores in functon_score query (#35709)
* Forbid negative scores in functon_score query

- Throw an exception when scores are negative in field_value_factor
function
- Throw an exception when scores are negative in script_score
function

Relates to #33309
2018-11-22 06:08:48 -05:00
Marios Trivyzas 92acf47c16
SQL: Implement NVL(expr1, expr2) (#35794)
Add NVL as alias to IFNULL as they have the same
behaviour. Add basic tests and docs.

Closes: #35782
2018-11-22 11:41:00 +01:00
Tanguy Leroux 11052b75c7
TransportResyncReplicationAction should not honour blocks (#35795)
After #35332 has been merged, we noticed some test failures like #35597 
in which one or more replica shards failed to be promoted as primaries 
because the primary replica re-synchronization never succeed.

After some digging it appeared that the execution of the resync action was 
blocked because of the presence of a global cluster block in the cluster state 
(in this case, the "no master" block), making the resync action to fail when 
executed on the primary.

Until #35332 such failures never happened because the 
TransportResyncReplicationAction is skipping the reroute phase, the only 
place where blocks were checked. Now with #35332 blocks are checked 
during reroute and also during the execution of the transport replication 
action on the primary. After some internal discussion, we decided that the TransportResyncReplicationAction should never be blocked. This action is 
part of the replica to primary promotion and makes sure that replicas are in 
sync and should not be blocked when the cluster state has no master or 
when the index is read only.

This commit changes the TransportResyncReplicationAction to make obvious 
that it does not honor blocks. It also adds a simple test that fails if the resync 
action is blocked during the primary action execution.

Closes #35597
2018-11-22 10:50:12 +01:00
Alan Woodward be8097f9ce
Improve docs for index_prefixes option (#35778)
This commit moves the documentation and examples for the `index_prefixes`
option on text fields to its own file, to bring it in line with other mapping 
parameters, and expands a bit on both.
2018-11-22 09:20:46 +00:00