Commit Graph

39577 Commits

Author SHA1 Message Date
Sohaib Iftikhar c4f8df3ad6 REST high-level client: add validate query API (#31077)
Adds the validate query API to the high level rest client.
2018-06-18 09:59:29 -04:00
Martijn van Groningen 47095357bc
Move language analyzers from server to analysis-common module. (#31300)
The following analyzers were moved from server module to analysis-common module:
`greek`, `hindi`, `hungarian`, `indonesian`, `irish`, `italian`, `latvian`,
`lithuanian`, `norwegian`, `persian`, `portuguese`, `romanian`, `russian`,
`sorani`, `spanish`, `swedish`, `turkish` and `thai`.

Relates to #23658
2018-06-18 11:24:43 +02:00
Tanguy Leroux f513505052 [Test] Fix :example-plugins:rest-handler on Windows 2018-06-18 10:49:22 +02:00
Alan Woodward 8c0ec05a12
Expose lucene's RemoveDuplicatesTokenFilter (#31275) 2018-06-18 09:46:12 +01:00
Albert Zaharovits 3378240b29
Reload secure settings for plugins (#31383)
Adds the ability to reread and decrypt the local node keystore.
Commonly, the contents of the keystore, backing the `SecureSettings`,
are not retrievable except during node initialization. This changes that
by adding a new API which broadcasts a password to every node. The
password is used to decrypt the local keystore and use it to populate
a `Settings` object that is passes to all the plugins implementing the
`ReloadablePlugin` interface. The plugin is then responsible to do
whatever "reload" means in his case. When the `reload`handler returns,
the keystore is closed and its contents are no longer retrievable.
Password is never stored persistently on any node.
Plugins that have been moded in this commit are: `repository-azure`,
`repository-s3`, `repository-gcs` and `discovery-ec2`.
2018-06-18 09:42:11 +03:00
Julie Tibshirani 16fa6b270f
Remove some cases in FieldTypeLookupTests that are no longer relevant. (#31381) 2018-06-17 21:42:42 -07:00
Simon Willnauer 3d5f113ada
Ensure we don't use a remote profile if cluster name matches (#31331)
If we are running into a race condition between a node being configured
to be a remote node for cross cluster search etc. and that node joining
the cluster we might connect to that node with a remote profile. If that
node now joins the cluster it connected to it as a CCS remote node we use
the wrong profile and can't use bulk connections etc. anymore. This change
uses the remote profile only if we connect to a node that has a different cluster
name than the local cluster. This is not a perfect fix for this situation but
is the safe option while potentially only loose a small optimization of using
less connections per node which is small anyways since we only connect to a
small set of nodes.

Closes #29321
2018-06-17 13:32:53 +02:00
Albert Zaharovits 5b94afd309 [TEST] Double write alias fault (#30942) 2018-06-17 12:17:28 +03:00
Costin Leau bbcfcd1ca5 [DOCS] Fix version in SQL JDBC Maven template 2018-06-17 00:40:01 +03:00
Costin Leau 215c5f292a [DOCS] Improve install and setup section for SQL JDBC 2018-06-17 00:33:55 +03:00
Igor Motov c3084a332b
SQL: Fix rest endpoint names in node stats (#31371)
Fixes wrong name for the sql translate endpoint and makes rest endpoint
names in stats more consistent.
2018-06-15 19:07:47 -04:00
Vladimir Dolzhenko babb16d90c Support for remote path in reindex api - post backport fix
Closes #22913
2018-06-15 22:24:47 +02:00
Dimitris Athanasiou c6a5a6d924
[ML] Put ML filter API response should contain the filter (#31362) 2018-06-15 21:15:35 +01:00
Vladimir Dolzhenko dbc9d60260
Support for remote path in reindex api (#31290)
Support for remote path in reindex api
Closes #22913
2018-06-15 22:14:28 +02:00
Tim Brooks a705e1a9e3
Add byte array pooling to nio http transport (#31349)
This is related to #28898. This PR implements pooling of bytes arrays
when reading from the wire in the http server transport. In order to do
this, we must integrate with netty reference counting. That manner in
which this PR implements this is making Pages in InboundChannelBuffer
reference counted. When we accessing the underlying page to pass to
netty, we retain the page. When netty releases its bytebuf, it releases
the underlying pages we have passed to it.
2018-06-15 14:01:03 -06:00
Tim Brooks 605dbbeabd
Remove trial status info from start trial doc (#31365)
This is related to #31325. There is currently information about the
get-trial-status api on the start-trial api documentation page. It also
has the incorrect route for that api. This commit removes that
information as the start-trial page properly links to a page providing
documenation about get-trial-status.
2018-06-15 12:39:25 -06:00
lcawl 4918ae6fe4 [DOCS] Adds links to release notes and highlights 2018-06-15 11:04:11 -07:00
Tal Levy 3b70e943eb
add is-write-index flag to aliases (#30942)
This commit adds the is-write-index flag for aliases.
It allows requests to set the flag, and responses to display the flag.
It does not validate and/or affect any indexing/getting/updating behavior
of Elasticsearch -- this will be done in a follow-up PR.
2018-06-15 08:45:29 -07:00
Tal Levy eda4964f64
Add rollover-creation-date setting to rolled over index (#31144)
This commit introduces a new property to IndexMetaData called
RolloverInfo. This object contains a map containing the aliases
that were used to rollover the related index, which conditions
were met, and at what time the rollover took place.

much like the `index.creation_date`, it captures the approximate time
that the index was rolled over to a new one.
2018-06-15 08:44:29 -07:00
Dimitris Athanasiou da5bfda5f3
[ML] Hold ML filter items in sorted set (#31338)
Filter items should be unique. They should also
be sorted to make them easier to read plus save
sorting in the autodetect process.
2018-06-15 16:29:09 +01:00
Christoph Büscher fec7860edc
[Tests] Fix edge case in ScriptedMetricAggregatorTests (#31357)
An expected exception is only thrown when there are documents in the index
created in the test setup. Fixed the test by making sure there is at least one.

Closes #31307
2018-06-15 17:12:42 +02:00
Nhat Nguyen 8453ca638d
Upgrade to Lucene-7.4.0-snapshot-518d303506 (#31360) 2018-06-15 10:58:21 -04:00
Christoph Büscher 02346c20a2
Rankeval: Fold template test project into main module (#31203)
This change moves tests in `smoke-test-rank-eval-with-mustache` into the main
ranking evaluation module by declaring that the integration testing cluster
requires the `lang-mustache` plugin. This avoids having to maintain the qa
project for only one basic test suite.
2018-06-15 15:55:39 +02:00
Tanguy Leroux afc91e92fb
Add QA project and fixture based test for discovery-ec2 plugin (#31107)
This commit adds a new QA sub project to the discovery-ec2 plugin. 
This project uses a fixture to test the plugin using a multi-node cluster. 
Once all nodes are started, the nodes transport addresses are written 
in a file that is later read by the fixture.
2018-06-15 15:49:14 +02:00
Tanguy Leroux 3274e7fd1a
[Docs] Remove reference to repository-s3 plugin creating an S3 bucket (#31359)
Closes #30910
2018-06-15 14:55:14 +02:00
Nik Everett 856936c286
REST Client: NodeSelector for node attributes (#31296)
Add a `NodeSelector` so that users can filter the nodes that receive
requests based on node attributes.

I believe we'll need this to backport #30523 and we want it anyway.

I also added a bash script to help with rebuilding the sniffer parsing
test documents.
2018-06-15 08:04:54 -04:00
Nik Everett 045f76d67f LLClient: Fix assertion on windows
In windows the exception message is ever so slightly differant than in
Linux and OSX. That is fine. We'll just catch either.
2018-06-15 08:03:53 -04:00
Christoph Büscher a0d6c19e75
Add details section for dcg ranking metric (#31177)
While the other two ranking evaluation metrics (precicion and reciprocal rank)
already provide a more detailed output for how their score is calculated, the
discounted cumulative gain metric (dcg) and its normalized variant are lacking
this until now. Its not really clear which level of detail might be useful for
debugging and understanding the final metric calculation, but this change adds a
`metric_details` section to REST output that contains some information about the
evaluation details.
2018-06-15 11:56:16 +02:00
David Kyle ca00deb8ad [ML] Re-enable tests muted in #30982 2018-06-15 10:54:13 +01:00
Tanguy Leroux 992c7889ee
Uncouple persistent task state and status (#31031)
This pull request removes the relationship between the state 
of persistent task (as stored in the cluster state) and the status 
of the task (as reported by the Task APIs and used in various 
places) that have been confusing for some time (#29608).

In order to do that, a new PersistentTaskState interface is added. 
This interface represents the persisted state of a persistent task. 
The methods used to update the state of persistent tasks are 
renamed: updatePersistentStatus() becomes updatePersistentTaskState() 
and now takes a PersistentTaskState as a parameter. The 
Task.Status type as been changed to PersistentTaskState in all 
places were it make sense (in persistent task customs in cluster 
state and all other methods that deal with the state of an allocated 
persistent task).
2018-06-15 09:26:47 +02:00
David Pilato 8c6ee7db54
Describe how to add a plugin in Dockerfile (#31340)
When installing a plugin, people need to add the `--batch` option.
It's better to document it as it could be a common use case.
2018-06-15 07:32:49 +02:00
Tim Brooks 1c5cec0ac7
Remove http status code maps (#31350)
Currently we maintain a compatibility map of http status codes in both
the netty4 and nio modules. These maps convert a RestStatus to a netty
HttpResponseStatus. However, as these fundamentally represent integers,
we can just use the netty valueOf method to convert a RestStatus to a
HttpResponseStatus.
2018-06-14 20:16:40 -06:00
Ben Abrams 87a676e4d5 Do not set vm.max_map_count when unnecessary (#31285)
This commit modifies the Sys V init startup scripts to only modify
vm.max_map_count if needed. In this case, needed means that the current
value is less than our default value of 262144 maps.
2018-06-14 21:41:02 -04:00
Nhat Nguyen e5b7137508
TEST: getCapturedRequestsAndClear should be atomic (#31312)
We might lose messages between getCapturedRequestsAndClear calls.
This commit makes sure that both getCapturedRequestsAndClear and
getCapturedRequestsByTargetNodeAndClear are atomic.
2018-06-14 21:32:07 -04:00
Jack Conradson 0324103737
Painless: Fix bug for static method calls on interfaces (#31348)
Static method calls on interfaces were not being called correctly
which was causing JVM crashes.  This change fixes the issue.
2018-06-14 18:30:37 -07:00
Nik Everett d6d0727aac
QA: Fix resolution of default distribution (#31351)
If you run `./gradlew -p qa bwcTest -Dtests.distribution=zip` then we
need to resolve older versions of the default distribution. Since those
aren't available in maven central, we need add the elastic maven repo to
the project.
2018-06-14 18:25:47 -04:00
Tim Brooks fcf1e41e42
Extract common http logic to server (#31311)
This is related to #28898. With the addition of the http nio transport,
we now have two different modules that provide http transports.
Currently most of the http logic lives at the module level. However,
some of this logic can live in server. In particular, some of the
setting of headers, cors, and pipelining. This commit begins this moving
in that direction by introducing lower level abstraction (HttpChannel,
HttpRequest, and HttpResonse) that is implemented by the modules. The
higher level rest request and rest channel work can live entirely in
server.
2018-06-14 15:10:02 -06:00
Nik Everett 6dd81ead74
Build: Fix the license in the pom zip and tar (#31336)
For 6.3 we renamed the `tar` and `zip` distributions to `oss-tar` and
`oss-zip`. Then we added new `tar` and `zip` distributions that contain
x-pack and are licensed under the Elastic License. Unfortunately we
accidentally generated POM files along side the new `tar` and `zip`
distributions that incorrectly claimed that they were Apache 2 licensed.
Oooops.

This fixes the license on the POMs generated for the `tar` and `zip`
distributions.
2018-06-14 16:22:00 -04:00
Yannick Welsch 8f886cd4be
Treat ack timeout more like a publish timeout (#31303)
This commit changes the ack timeout mechanism so that its behavior is closer to the publish
timeout, i.e., it only comes into play after committing a cluster state. This ensures for example that
an index creation request with a low (ack) timeout value does not return before the cluster state
that contains information about the newly created index is even committed.
2018-06-14 18:32:35 +02:00
Dimitris Athanasiou 9b293275af
[ML] Add description to ML filters (#31330)
This adds a `description` to ML filters in order
to allow users to describe their filters in a human
readable form which is also editable (filter updates
to be added shortly).
2018-06-14 16:52:32 +01:00
Costin Leau f7a0cafe55 SQL: Fix build on Java 10
Due to a runtime classpath clash, featureAware task was failing on JVMs
higher than 1.8 (since the ASM version from Painless was used instead
which does not recognized Java 9 or 10 bytecode) causing the task to
fail.
This commit excludes the ASM dependency (since it's not used by SQL
itself).
2018-06-14 18:30:27 +03:00
Simon Willnauer 375d09c588 [TEST] Fix RemoteClusterClientTests#testEnsureWeReconnect
Closes #29547
2018-06-14 16:21:35 +02:00
David Turner 4877cec3e8
More detailed tracing when writing metadata (#31319)
Packaging tests are occasionally failing (#30295) because of very slow index
template creation. It looks like the slow part is updating the on-disk cluster
state, and this change will help to confirm this.
2018-06-14 13:41:25 +01:00
Tanguy Leroux bbfe1eccc7
[Tests] Mutualize fixtures code in BaseHttpFixture (#31210)
Many fixtures have similar code for writing the pid & ports files or
for handling HTTP requests. This commit adds an AbstractHttpFixture 
class in the test framework that can be extended for specific testing purposes.
2018-06-14 14:09:56 +02:00
Luca Cavanna ce245a7320
Remove RestGetAllAliasesAction (#31308)
We currently have a specific REST action to retrieve all aliaes, which
uses internally the get index API. This doesn't seem to be required
anymore though as the existing RestGetAliaesAction could as well take
the requests with no indices and aliases specified.

This commit removes the RestGetAllAliasesAction in favour of using
RestGetAliasesAction also for requests that don't specify indices nor
aliases. Similar to #31129.
2018-06-14 11:21:16 +02:00
Tim Vernum 89a24698b1 Temporary fix for broken build
x-pack/sql depends on lang-painless which depends on ASM 5.1
FeatureAwareCheck needs ASM 6
This is a hack to strip ASM5 from the classpath for FeatureAwareCheck
2018-06-14 18:22:24 +10:00
Tanguy Leroux 4d7447cb5e
Reenable Checkstyle's unused import rule (#31270) 2018-06-14 09:52:46 +02:00
Tanguy Leroux 2d4c9ce08c Remove remaining unused imports before merging #31270 2018-06-14 09:52:03 +02:00
Tim Vernum 74c6f18fea Fix non-REST doc snippet 2018-06-14 16:26:17 +10:00
Costin Leau 870a913217 [DOC] Extend SQL docs
Add overview section
Add data type section
Improve function section
2018-06-14 00:38:53 +03:00