Commit Graph

5437 Commits

Author SHA1 Message Date
Jonathan Little 8e4768890a Migrate scripted metric aggregation scripts to ScriptContext design (#30111)
* Migrate scripted metric aggregation scripts to ScriptContext design #29328

* Rename new script context container class and add clarifying comments to remaining references to params._agg(s)

* Misc cleanup: make mock metric agg script inner classes static

* Move _score to an accessor rather than an arg for scripted metric agg scripts

This causes the score to be evaluated only when it's used.

* Documentation changes for params._agg -> agg

* Migration doc addition for scripted metric aggs _agg object change

* Rename "agg" Scripted Metric Aggregation script context variable to "state"

* Rename a private base class from ...Agg to ...State that I missed in my last commit

* Clean up imports after merge
2018-06-25 12:01:33 +01:00
Vladimir Dolzhenko 27bd68862a
Add get field mappings to High Level REST API Client (#31423)
Add get field mappings to High Level REST API Client
Relates to #27205
2018-06-25 09:41:23 +01:00
Lisa Cawley efffa0226d
[DOCS] Move sql to docs (#31474) 2018-06-25 09:41:22 +01:00
Lisa Cawley baa9b8575c
[DOCS] Move monitoring to docs folder (#31477) 2018-06-25 09:41:22 +01:00
Vladimir Dolzhenko b7ef75fed6
Add get field mappings to High Level REST API Client (#31423)
Add get field mappings to High Level REST API Client
Relates to #27205
2018-06-23 09:39:17 +02:00
Lisa Cawley 638b9fd88c
[DOCS] Move sql to docs (#31474) 2018-06-22 15:40:25 -07:00
Lisa Cawley eb81a305ae
[DOCS] Move monitoring to docs folder (#31477) 2018-06-22 15:39:34 -07:00
Tal Levy 1d11fdaad8 Merge branch 'master' into index-lifecycle 2018-06-22 10:11:15 -07:00
Luca Cavanna 16e4e7a7cf
Node selector per client rather than per request (#31471)
We have made node selectors configurable per request, but all 
of other language clients don't allow for that.
A good reason not to do so, is that having a different node selector 
per request breaks round-robin. This commit makes NodeSelector 
configurable only at client initialization. It also improves the docs 
on this matter, important given that a single node selector can still 
affect round-robin.
2018-06-22 17:15:29 +02:00
Colin Goodheart-Smithe d062556754
Merge branch 'master' into index-lifecycle 2018-06-22 10:00:35 +01:00
Sohaib Iftikhar eade161894 REST high-level client: add simulate pipeline API (#31158)
relates to #27205
2018-06-22 09:59:04 +02:00
Tal Levy c9eec7eadc Merge branch 'master' into index-lifecycle 2018-06-21 09:56:45 -07:00
Lisa Cawley 68ec958873
[DOCS] Move migration APIs to docs (#31473) 2018-06-21 08:19:23 -07:00
Tal Levy 8d9b98569f Merge branch 'master' into index-lifecycle 2018-06-20 08:30:49 -07:00
Lisa Cawley f012de0f00
[DOCS] Move licensing APIs to docs (#31445) 2018-06-20 08:17:11 -07:00
Daniel Mitterdorfer 90d62e6e4d
Add Delete Snapshot High Level REST API
With this commit we add the delete snapshot API to the Java high level
REST client.

Relates #27205
Relates #31393
2018-06-20 16:42:50 +02:00
Jonathan Pool 297e99c4c2 [Docs] Extend Homebrew installation instructions (#28902)
Adding a note about proceeding after a successful homebrew installation.
2018-06-20 14:20:51 +02:00
Peter Dyson e7a7b9689d [Docs] Mention ip_range datatypes on ip type page (#31416)
A link to the ip_range datatype page provides a way for newer users to know
it exists if they land directly on the ip datatype page first via a search.
2018-06-20 13:04:03 +02:00
Colin Goodheart-Smithe c5b69c87c6
Merge branch 'master' into index-lifecycle 2018-06-20 10:31:45 +01:00
Alan Woodward 5683bc60a6
Multiplexing token filter (#31208)
The `multiplexer` filter emits multiple tokens at the same position, each 
version of the token haivng been passed through a different filter chain.
Identical tokens at the same position are removed.

This allows users to, for example, index lowercase and original-case tokens,
or stemmed and unstemmed versions, in the same field, so that they can search
for a stemmed term within x positions of an unstemmed term.
2018-06-20 10:16:26 +01:00
Sue Gallagher b44e1c1978
[DOCS] Removed and params from MLT. Closes #28128 (#31370) 2018-06-19 13:48:13 -07:00
Tal Levy eee1a451fc Merge branch 'master' into index-lifecycle 2018-06-19 11:37:47 -07:00
Lisa Cawley 8fd1f5fbed
[DOCS] Moves the info API to docs (#31121) 2018-06-19 10:33:57 -07:00
Nik Everett 5236d0291e
Docs: Advice for reindexing many indices (#31279)
Folks tend to want to be able to make a single `_reindex` call to
migrate many indices. You *can* do that and we even have an example of
how to do that in the docs but it isn't always a good idea. This change
adds some advice to the docs: generally you want to make one reindex
call per index.

Closes #22920
2018-06-19 11:15:50 -04:00
Ryan Ernst c0961b79be
Docs: Add note about removing prepareExecute from the java client (#31401)
relates #30966
2018-06-19 07:21:58 -07:00
Vladimir Dolzhenko 04e4e44409
Add get stored script and delete stored script to high level REST API (#31355)
Add get stored script and delete stored script to high level REST API

Relates to #27205
2018-06-19 14:21:11 +02:00
Lee Hinman 4ad334f8e0
Fix reference to XContentBuilder.string() (#31337)
In 6.3 this was moved to `Strings.toString(XContentBuilder)` as part of the
XContent extraction. This commit fixes the docs to reference the new method.

Resolves #31326
2018-06-18 15:04:26 -06:00
Ryan Ernst f3297ed23a
Packaging: Remove windows bin files from the tar distribution (#30596)
This commit removes windows specific files from the tar distribution.
Windows users use the zip, linux users use the tar.
2018-06-18 19:02:51 +02:00
Nik Everett 73549281e8
Docs: Use the default distribution to test docs (#31251)
This switches the docs tests from the `oss-zip` distribution to the
`zip` distribution so they have xpack installed and configured with the
default basic license. The goal is to be able to merge the
`x-pack/docs` directory into the `docs` directory, marking the x-pack
docs with some kind of marker. This is the first step in that process.

This also enables `-Dtests.distribution` support for the `docs`
directory so you can run the tests against the `oss-zip` distribution
with something like
```
./gradlew -p docs check -Dtests.distribution=oss-zip
```

We can set up Jenkins to run both.

Relates to #30665
2018-06-18 12:06:42 -04:00
Julie Tibshirani 3f5ebb862d
Clarify that IP range data can be specified in CIDR notation. (#31374) 2018-06-18 08:21:41 -07:00
Zachary Tong 1502812c1a
Percentile/Ranks should return null instead of NaN when empty (#30460)
The other metric aggregations (min/max/etc) return `null` as their XContent value and string when nothing was computed (due to empty/missing fields).  Percentiles and Percentile Ranks, however, return `NaN `which is inconsistent and confusing for the user.  This fixes the inconsistency by making the aggs return `null`.  This applies to both the numeric value and the "as string" value.  

Note: like the metric aggs, this does not change the value if fetched directly from the percentiles object, which will return as `NaN`/`"NaN"`. This only changes the XContent output.

While this is a bugfix, it still breaks bwc in a minor way as the response changes from prior version.

Closes #29066
2018-06-18 10:01:28 -04:00
Sohaib Iftikhar c4f8df3ad6 REST high-level client: add validate query API (#31077)
Adds the validate query API to the high level rest client.
2018-06-18 09:59:29 -04:00
Alan Woodward 8c0ec05a12
Expose lucene's RemoveDuplicatesTokenFilter (#31275) 2018-06-18 09:46:12 +01:00
Vladimir Dolzhenko dbc9d60260
Support for remote path in reindex api (#31290)
Support for remote path in reindex api
Closes #22913
2018-06-15 22:14:28 +02:00
lcawl 4918ae6fe4 [DOCS] Adds links to release notes and highlights 2018-06-15 11:04:11 -07:00
Tal Levy 3b70e943eb
add is-write-index flag to aliases (#30942)
This commit adds the is-write-index flag for aliases.
It allows requests to set the flag, and responses to display the flag.
It does not validate and/or affect any indexing/getting/updating behavior
of Elasticsearch -- this will be done in a follow-up PR.
2018-06-15 08:45:29 -07:00
Tanguy Leroux 3274e7fd1a
[Docs] Remove reference to repository-s3 plugin creating an S3 bucket (#31359)
Closes #30910
2018-06-15 14:55:14 +02:00
Nik Everett 856936c286
REST Client: NodeSelector for node attributes (#31296)
Add a `NodeSelector` so that users can filter the nodes that receive
requests based on node attributes.

I believe we'll need this to backport #30523 and we want it anyway.

I also added a bash script to help with rebuilding the sniffer parsing
test documents.
2018-06-15 08:04:54 -04:00
David Pilato 8c6ee7db54
Describe how to add a plugin in Dockerfile (#31340)
When installing a plugin, people need to add the `--batch` option.
It's better to document it as it could be a common use case.
2018-06-15 07:32:49 +02:00
Colin Goodheart-Smithe 58e9446e00
Removes experimental tag from scripted_metric aggregation (#31298) 2018-06-13 17:24:32 +01:00
Luca Cavanna 24163d10b7
REST hl client: cluster health to default to cluster level (#31268)
With #29331 we added support for the cluster health API to the
high-level REST client. The transport client does not support the level
parameter, and it always returns all the info needed for shards level
rendering. We have maintained that behaviour when adding support for
cluster health to the high-level REST client, to ease migration, but the
correct thing to do is to default the high-level REST client to
`cluster` level, which is the same default as when going through the
Elasticsearch REST layer.
2018-06-13 15:06:13 +02:00
Luca Cavanna 92eb324776
REST high-level Client: remove deprecated API methods (#31200)
This commit removes all the API methods that accept a `Header` varargs
argument, in favour of the newly introduced API methods that accept a
`RequestOptions` argument.

Relates to #31069
2018-06-12 21:00:06 +02:00
Albert Zaharovits e988ace5f7
[DOCS] Clarify audit index settings when remote indexing (#30923) 2018-06-12 16:19:43 +03:00
Van0SS d5e8a5cd69 REST high-level client: add Cluster Health API (#29331)
Relates to #27205
2018-06-12 13:34:06 +02:00
olcbean 7d7ead95b2 Add Get Aliases API to the high-level REST client (#28799)
Given the weirdness of the response returned by the get alias API, we went for a client specific response, which allows us to hold the error message, exception and status returned as part of the response together with aliases. See #30536 .

Relates to #27205
2018-06-12 10:26:17 +02:00
Martijn van Groningen 6030d4be1e
[INGEST] Interrupt the current thread if evaluation grok expressions take too long (#31024)
This adds a thread interrupter that allows us to encapsulate calls to org.joni.Matcher#search()
This method can hang forever if the regex expression is too complex.

The thread interrupter in the background checks every 3 seconds whether there are threads
execution the org.joni.Matcher#search() method for longer than 5 seconds and
if so interrupts these threads.

Joni has checks that that for every 30k iterations it checks if the current thread is interrupted and
if so returns org.joni.Matcher#INTERRUPTED

Closes #28731
2018-06-12 07:49:03 +02:00
Jason Tedor 32cbbc2ecf
Revert "[DOCS] Added 6.3 info & updated the upgrade table. (#30940)"
This reverts commit 40d31d2ae0.
2018-06-11 22:04:36 -04:00
Jason Tedor 0e6589f489
Revert "Fix snippets in upgrade docs"
This reverts commit 955542a50a.
2018-06-11 22:04:30 -04:00
Jason Tedor 955542a50a
Fix snippets in upgrade docs
The upgrade docs were using the json language tag but console wants
these to be js for proper formatting. This commit addresses that.
2018-06-11 21:32:39 -04:00
debadair 40d31d2ae0 [DOCS] Added 6.3 info & updated the upgrade table. (#30940) 2018-06-11 15:55:58 -07:00
Nik Everett 0d9b78834f
LLClient: Support host selection (#30523)
Allows users of the Low Level REST client to specify which hosts a
request should be run on. They implement the  `NodeSelector` interface
or reuse a built in selector like `NOT_MASTER_ONLY` to chose which nodes
are valid. Using it looks like:
```
Request request = new Request("POST", "/foo/_search");
RequestOptions options = request.getOptions().toBuilder();
options.setNodeSelector(NodeSelector.NOT_MASTER_ONLY);
request.setOptions(options);
...
```

This introduces a new `Node` object which contains a `HttpHost` and the
metadata about the host. At this point that metadata is just `version`
and `roles` but I plan to add node attributes in a followup. The
canonical way to **get** this metadata is to use the `Sniffer` to pull
the information from the Elasticsearch cluster.

I've marked this as "breaking-java" because it breaks custom
implementations of `HostsSniffer` by renaming the interface to
`NodesSniffer` and by changing it from returning a `List<HttpHost>` to a
`List<Node>`. It *shouldn't* break anyone else though.

Because we expect to find it useful, this also implements `host_selector`
support to `do` statements in the yaml tests. Using it looks a little
like:

```
---
"example test":
  - skip:
      features: host_selector
  - do:
      host_selector:
        version: " - 7.0.0" # same syntax as skip
      apiname:
        something: true
```

The `do` section parses the `version` string into a host selector that
uses the same version comparison logic as the `skip` section. When the
`do` section is executed it passed the off to the `RestClient`, using
the `ElasticsearchHostsSniffer` to sniff the required metadata.

The idea is to use this in mixed version tests to target a specific
version of Elasticsearch so we can be sure about the deprecation
logging though we don't currently have any examples that need it. We do,
however, have at least one open pull request that requires something
like this to properly test it.

Closes #21888
2018-06-11 17:07:27 -04:00
Nhat Nguyen 99e04582de
HLRest: Add get index templates API (#31161)
Relates #27205
2018-06-11 11:06:28 -04:00
Tanguy Leroux 42608881b0
[Docs] Remove mention pattern files in Grok processor (#31170)
Pattern files have been removed in 
16fa3e546e
2018-06-11 09:32:12 +02:00
Simon Willnauer f825a530b8
Limit the number of concurrent requests per node (#31206)
With `max_concurrent_shard_requests` we used to throttle / limit
the number of concurrent shard requests a high level search request
can execute per node. This had several problems since it limited the
number on a global level based on the number of nodes. This change
now throttles the number of concurrent requests per node while still
allowing concurrency across multiple nodes.

Closes #31192
2018-06-11 08:49:18 +02:00
Sue Gallagher cdb486ae70
[DOCS] Added 'fail_on_unsupported_field' param to MLT. Closes #28008 (#31160)
* [DOCS] Added 'fail_on_unsupported_field' param to MLT. Closes 28008

* [DOCS] Added 'fail_on_unsupported_field' param to MLT. Closes #28008

* [DOCS] Added 'fail_on_unsupported_field' param to MLT. Closes #28008

* [DOCS] Added 'fail_on_unsupported_field' param to MLT. Closes #28008
2018-06-08 14:41:01 -07:00
David Turner 8d4f09f7f2
[DOCS] Add note about long-lived idle connections (#30990)
Clarify that we expect to have idle inter-node connections within the cluster,
and that the network needs to be configured not to disrupt these.
2018-06-08 13:36:19 +01:00
Jack Conradson d6a4c14e1b
Painless: Restructure/Clean Up of Spec Documentation (#31013)
Full restructure of the spec into new sections for operators, statements, scripts, functions, lambdas, and regexes.  Split of operators into 6 sections, a table, reference, array, numeric, boolean, and general.  Clean up of all operators sections.  Sporadic clean up else where.
2018-06-07 17:11:56 -07:00
Paul Sanwald e82e5cc2e8
high level REST api: cancel task (#30745)
* Initial commit of rest high level exposure of cancel task

* fix javadocs

* address some code review comments

* update branch to use tasks namespace instead of cluster

* High-level client: list tasks failure to not lose nodeId

This commit reworks testing for `ListTasksResponse` so that random
fields insertion can be tested and xcontent equivalence can be checked
too. Proper exclusions need to be configured, and failures need to be
tested separately. This helped finding a little problem, whenever there
is a node failure returned, the nodeId was lost as it was never printed
out as part of the exception toXContent.

* added comment

* merge from master

* re-work CancelTasksResponseTests to separate XContent failure cases from non-failure cases

* remove duplication of logic in parser creation

* code review changes

* refactor TasksClient to support RequestOptions

* add tests for parent task id

* address final PR review comments, mostly formatting and such
2018-06-07 14:02:23 -07:00
lcawl 5dc9e87bad [DOCS] Fixes broken link in auditing settings 2018-06-07 10:49:22 -07:00
Igor Motov 7a9d9b0abf
Add support for ignore_unmapped to geo sort (#31153)
Adds support for `ignore_unmapped` parameter in geo distance sorting,
which is functionally equivalent to specifying an `unmapped_type` in
the field sort.

Closes #28152
2018-06-07 11:11:13 -04:00
David Turner 6ad7217656
Remove reference to multiple fields with one name (#31127)
If there is only one type per index then each field's name is unique.
2018-06-07 12:38:57 +01:00
Rafał Bigaj 749d39061a [Docs] Correct minor typos in templates.asciidoc (#31167) 2018-06-07 10:44:57 +02:00
Adrien Grand 458bca11bc
Add a `feature_vector` field. (#31102)
This field is similar to the `feature` field but is better suited to index
sparse feature vectors. A use-case for this field could be to record topics
associated with every documents alongside a metric that quantifies how well
the topic is connected to this document, and then boost queries based on the
topics that the logged user is interested in.

Relates #27552
2018-06-07 10:05:37 +02:00
Nirmal Chidambaram 75a676c70b Fail `span_multi` queries that exceeds boolean max clause limit (#30913)
By default span_multi query will limit term expansions = boolean max clause.
This will limit high heap usage in case of high cardinality term
expansions. This applies only if top_terms_N is not used in inner multi
query.
2018-06-07 09:34:39 +02:00
Lisa Cawley 7f0c2e89c2
[DOCS] Moves X-Pack setup to docs (#31145) 2018-06-06 14:46:20 -07:00
Lisa Cawley b4514d3cc1
[DOCS] Moves ML node info to docs (#31142) 2018-06-06 12:39:24 -07:00
Lisa Cawley 45537c59e5
[DOCS] Moves X-Pack settings to docs folder (#31120) 2018-06-06 10:05:32 -07:00
Lisa Cawley 6fd4eb52b8
[DOCS] Moves commands to docs folder (#31114) 2018-06-06 07:49:15 -07:00
Colin Goodheart-Smithe d09d60858a
[DOCS] Clarify nested datatype introduction (#31055) 2018-06-06 09:32:45 +01:00
Christoph Büscher 1cee45e768
[Docs] Delete superfluous callouts (#31111)
Those callout create rendering problems on the subsequent page.

Closes #30532
2018-06-06 09:53:14 +02:00
lcawl 7c05f69c39 [DOCS] Creates rest-api folder in docs 2018-06-05 16:43:55 -07:00
Adrien Grand 1af6d20efe Fix docs build. 2018-06-05 14:55:40 +02:00
Adrien Grand 984523dda9
Clarify docs about boolean operator precedence. (#30808)
Unfortunately, the classic queryparser does not honor the usual precedence
rules of boolean operators. See
https://issues.apache.org/jira/browse/LUCENE-3674.
2018-06-05 08:59:17 +02:00
Adrien Grand 21fe6159d4
Docs: remove notes on sparsity. (#30905)
Sparsity is less of a concern since 6.0.

Closes #30833
2018-06-05 08:58:52 +02:00
Adrien Grand 500094f5c8
Improve documentation of dynamic mappings. (#30952)
Closes #30939
2018-06-05 08:51:52 +02:00
Adrien Grand f5073813ef
Docs: Clarify constraints on scripted similarities. (#31076)
Scripted similarities provide a lot of flexibility but they still need to obey
some rules to not confuse Lucene.
2018-06-05 08:51:00 +02:00
Dmitri Moore c7c0acc2c7 Update get.asciidoc (#31084)
Fix typo

(cherry picked from commit 51fbf8a3d3486fcf8a66dd6a6fe0940482b0f2d8)
2018-06-04 15:44:00 -07:00
Lee Hinman b22a055bcf
Add get mappings support to high-level rest client (#30889)
This adds support for the get mappings API to the high level rest client.

Relates to #27205
2018-06-04 14:31:08 -06:00
Jim Ferenczi f94a75778c
Fix index prefixes to work with span_multi (#31066)
* Fix index prefixes to work with span_multi

Text fields that use `index_prefixes` can rewrite `prefix` queries into
`term` queries internally. This commit fix the handling of this rewriting
in the `span_multi` query.
This change also copies the index options of the text field into the
prefix field in order to be able to run positional queries. This is mandatory
for `span_multi` to work but this could also be useful to optimize `match_phrase_prefix`
queries in a follow up. Note that this change can only be done on indices created
after 6.3 since we set the index options to doc only in this version.

Fixes #31056
2018-06-04 21:48:56 +02:00
Alan Woodward 852df128a5
Match phrase queries against non-indexed fields should throw an exception (#31060)
When `lenient=false`, attempts to create match phrase queries with custom analyzers against non-text fields will throw an IllegalArgumentException.

Also changes `*Match*QueryBuilderTests` so that it avoids this scenario

Fixes #31061
2018-06-04 19:12:45 +01:00
Itamar Syn-Hershko 526f906927 Remove usage of explicit type in docs (#29667) 2018-06-04 09:42:29 -04:00
Jim Ferenczi fa6b7266eb Remove wrong link in index phrases doc
Relates #30450
2018-06-04 12:13:55 +02:00
Sohaib Iftikhar 3809450bf3 Move pipeline APIs to ingest namespace (#31027) 2018-06-04 10:34:55 +02:00
Colin Goodheart-Smithe 360b09f148
[DOCS] Fixes accounting setting names (#30863)
The documentation for the account circuit breaker listed the settings for it's limit and overhead to be `network.breaker.accounting.limit` and `network.breaker.accounting.overhead` when in `HieratchyCircuitBreakerService` it seems the settings are actually `indices.breaker.accounting.limit` and `indices.breaker.accounting.overhead`.
2018-06-04 09:20:54 +01:00
Colin Goodheart-Smithe 1efb1aae28
[DOCS] Rewords _field_names documentation (#31029)
* [DOCS] Rewords _field_names documentation

Corrects the language around when we write to `_field_names` and when you might want to disable it given that n recent versions it does not carry the indexing overhead it once did.

Relates to #30862

* Update wording following review
2018-06-04 09:17:11 +01:00
Alan Woodward 0427339ab0
Index phrases (#30450)
Specifying `index_phrases: true` on a text field mapping will add a subsidiary
[field]._index_phrase field, indexing two-term shingles from the parent field.
The parent analysis chain is re-used, wrapped with a FixedShingleFilter.

At query time, if a phrase match query is executed, the mapping will redirect it
to run against the subsidiary field.

This should trade faster phrase querying for a larger index and longer indexing
times.

Relates to #27049
2018-06-04 08:50:35 +01:00
Igor Motov 7376c35960
[DOCS] Make geoshape docs less memory hungry (#31014)
Reduces shape size and precision in geo shape mapper examples to reduce
amount of memory required to check docs.

Fixes #23836
2018-06-01 15:05:37 -04:00
lipsill a8c643e833 [Docs] Fix a typo in Create Index naming limitation (#30891) 2018-06-01 18:15:24 +02:00
Sohaib Iftikhar 11887fa54a REST high-level client: add delete ingest pipeline API (#30865)
Relates to #27205
2018-06-01 14:13:41 +02:00
Jim Ferenczi 0791f93dbd
Add an option to split keyword field on whitespace at query time (#30691)
This change adds an option named `split_queries_on_whitespace` to the `keyword`
field type. When set to true full text queries (`match`, `multi_match`, `query_string`, ...) that target the field will split the input on whitespace to build the query terms. Defaults to `false`.
Closes #30393
2018-06-01 09:47:03 +02:00
Sohaib Iftikhar 80d20a9010 REST high-level client: add get ingest pipeline API (#30847)
Relates to #27205
2018-06-01 08:55:43 +02:00
Lisa Cawley 3960a7a310
[DOCS] Update readme for testing x-pack code snippets (#30696) 2018-05-31 09:32:22 -07:00
Jim Ferenczi 0f5e570184
Deprecates indexing and querying a context completion field without context (#30712)
This change deprecates completion queries and documents without context that target a
context enabled completion field. Querying without context degrades the search
performance considerably (even when the number of indexed contexts is low).
This commit targets master but the deprecation will take place in 6.x and the functionality
will be removed in 7 in a follow up.

Closes #29222
2018-05-31 16:09:48 +02:00
Christoph Büscher 4777d8a2df
[Docs] Fix typo in Min Aggregation reference (#30899) 2018-05-31 15:05:03 +02:00
Michael Basnight b716b08197
Add Verify Repository High Level REST API (#30934)
This commit adds Verify Repository, the associated docs and tests for
the high level REST API client. A few small changes to the Verify
Repository Response went into the commit as well.

Relates #27205
2018-05-30 11:10:00 -05:00
Christoph Büscher 1ea9f11b03
Change ScriptException status to 400 (bad request) (#30861)
Currently failures to compile a script usually lead to a ScriptException, which
inherits the 500 INTERNAL_SERVER_ERROR from ElasticsearchException if it does
not contain another root cause. Instead, this should be a 400 Bad Request error.
This PR changes this more generally for script compilation errors by changing 
ScriptException to return 400 (bad request) as status code.

Closes #12315
2018-05-30 14:00:07 +02:00
Nik Everett 7c5abc0a6b Fix synced flush docs
They had some copy and paste errors that failed the docs build.
2018-05-30 07:51:12 -04:00
Sohaib Iftikhar 2c27c58718 REST high-level client: add synced flush API (2) (#30650)
Adds the synced flush API to the high level REST client.

Relates to #27205.
2018-05-30 07:32:52 -04:00
Jim Ferenczi f582418ada Fix missing option serialization after backport
Relates #29465
2018-05-30 12:55:31 +02:00
Jim Ferenczi e33d107f84
Add missing_bucket option in the composite agg (#29465)
This change adds a new option to the composite aggregation named `missing_bucket`.
This option can be set by source and dictates whether documents without a value for the
source should be ignored. When set to true, documents without a value for a field emits
an explicit `null` value which is then added in the composite bucket.
The `missing` option that allows to set an explicit value (instead of `null`) is deprecated in this change and will be removed in a follow up (only in 7.x).
This commit also changes how the big arrays are allocated, instead of reserving
the provided `size` for all sources they are created with a small intial size and they grow
depending on the number of buckets created by the aggregation:
Closes #29380
2018-05-30 09:48:40 +02:00
Alan Woodward 67905c85a5
Rename index_prefix to index_prefixes (#30932)
This commit also adds index_prefixes tests to TextFieldMapperTests to ensure that cloning and wire-serialization work correctly
2018-05-30 08:32:31 +01:00
Lisa Cawley 6ce86a8d7f
[DOCS] Reset edit links (#30909) 2018-05-29 08:16:53 -07:00
Luca Cavanna eaee530778
Move list tasks under Tasks namespace (#30906)
Our API spec define the tasks API as e.g. tasks.list, meaning that they belong to their own namespace. This commit moves them from the cluster namespace to their own namespace.

Relates to #29546
2018-05-29 10:54:41 +02:00
David Turner 89869a2d0d
Improve allocation-disabling instructions (#30248)
Clarify the “one minute” in the instructions to disable the shard allocation
when doing maintenance to say that it is configurable.
2018-05-29 08:34:20 +01:00
Vladimir Dolzhenko b55b079a90
Include size of snapshot in snapshot metadata #18543, bwc clean up (#30890) 2018-05-26 21:20:44 +02:00
Vladimir Dolzhenko 81eb8ba0f0
Include size of snapshot in snapshot metadata (#29602)
Include size of snapshot in snapshot metadata

Adds difference of number of files (and file sizes) between prev and current snapshot. Total number/size reflects total number/size of files in snapshot.

Closes #18543
2018-05-25 21:04:50 +02:00
Zachary Tong 6909a05f3d
[DOCS] Document index name limitations (#30826)
Also tidy up the docs a bit, there's no yaml example anymore, etc
2018-05-25 10:21:09 -04:00
Peter Dyson adc2d408d3 [Docs] Add reindex.remote.whitelist example (#30828) 2018-05-25 11:17:55 +02:00
Sohaib Iftikhar 5a97423b7a REST high-level client: add put ingest pipeline API (#30793)
REST high-level client: add put ingest pipeline API

Adds the put ingest pipeline API to the high level rest client.
2018-05-24 19:02:26 -04:00
Igor Motov cf0e0606af
Use geohash cell instead of just a corner in geo_bounding_box (#30698)
Treats geohashes as grid cells instead of just points when the
geohashes are used to specify the edges in the geo_bounding_box
query. For example, if a geohash is used to specify the top_left
corner, the top left corner of the geohash cell will be used as the
corner of the bounding box.

Closes #25154
2018-05-24 14:46:15 -04:00
Julie Tibshirani 638a719370
Ensure that ip_range aggregations always return bucket keys. (#30701) 2018-05-24 08:55:14 -07:00
Christoph Büscher 3f78b3f5e1
[Docs] Explain incomplete dates in range queries (#30689)
The current documentation isn't very clear about how incomplete dates are
treated when specifying custom formats in a `range` query. This change adds a
note explaining how missing month or year coordinates translate to dates that
have the missings slots filled with unix time start date (1970-01-01)

Closes #30634
2018-05-24 11:20:00 +02:00
Tim Brooks d7040ad7b4
Reintroduce mandatory http pipelining support (#30820)
This commit reintroduces 31251c9 and 63a5799. These commits introduced a
memory leak and were reverted. This commit brings those commits back
and fixes the memory leak by removing unnecessary retain method calls.
2018-05-23 14:38:52 -06:00
Jack Conradson a96a45c6ae
Painless: Types Section Clean Up (#30283)
Clean up of types section, casting section, and a large number of examples.
2018-05-23 13:36:58 -07:00
Igor Motov 4b6915976c
Add support for indexed shape routing in geo_shape query (#30760)
Adds ability to specify the routing value for the indexed shape in the
geo_shape query.

Closes #7663
2018-05-23 15:15:19 -04:00
Colin Goodheart-Smithe 4fd0a3e492 Revert "Make http pipelining support mandatory (#30695)" (#30813)
This reverts commit 31251c9 introduced in #30695.

We suspect this commit is causing the OOME's reported in #30811 and we will use this PR to test this assertion.
2018-05-23 10:54:46 -06:00
Adrien Grand a19df4ab3b
Add a `format` option to `docvalue_fields`. (#29639)
This commit adds the ability to configure how a docvalue field should be
formatted, so that it would be possible eg. to return a date field
formatted as the number of milliseconds since Epoch.

Closes #27740
2018-05-23 14:39:04 +02:00
Adrien Grand 886db84ad2
Expose Lucene's FeatureField. (#30618)
Lucene has a new `FeatureField` which gives the ability to record numeric
features as term frequencies. Its main benefit is that it allows to boost
queries with the values of these features and efficiently skip non-competitive
documents at the same time using block-max WAND and indexed impacts.
2018-05-23 08:55:21 +02:00
Fernando Medina Corey 739bb4f0ec Fix a grammatical error in the 'search types' documentation.
Simple grammatical fix.
2018-05-22 22:09:04 -07:00
Luca Cavanna a17d6cab98
Replace Request#setHeaders with addHeader (#30588)
Adding headers rather than setting them all at once seems more
user-friendly and we already do it in a similar way for parameters
(see Request#addParameter).
2018-05-22 20:32:30 +02:00
Christoph Büscher f7b5986682
[Docs] Fix script-fields snippet execution (#30693)
Currently the first snippet in the documentation test in script-fields.asciidoc
isn't executed, although it has the CONSOLE annotation. Adding a test setup
annotation to it seems to fix the problem.
2018-05-22 20:22:42 +02:00
Tim Brooks 31251c9a6d
Make http pipelining support mandatory (#30695)
This is related to #29500 and #28898. This commit removes the abilitiy
to disable http pipelining. After this commit, any elasticsearch node
will support pipelined requests from a client. Additionally, it extracts
some of the http pipelining work to the server module. This extracted
work is used to implement pipelining for the nio plugin.
2018-05-22 09:29:31 -06:00
Lee Jones 37f67d9e21 [Docs] Fix typo in circuit breaker docs (#29659)
The previous description had a part that didn't fit and was probably
from a copy/paste of the in flight requests description above.
2018-05-22 16:43:45 +02:00
Itamar Syn-Hershko 5f172b6795 [Feature] Adding a char_group tokenizer (#24186)
=== Char Group Tokenizer

The `char_group` tokenizer breaks text into terms whenever it encounters
a
character which is in a defined set. It is mostly useful for cases where
a simple
custom tokenization is desired, and the overhead of use of the
<<analysis-pattern-tokenizer, `pattern` tokenizer>>
is not acceptable.

=== Configuration

The `char_group` tokenizer accepts one parameter:

`tokenize_on_chars`::
    A string containing a list of characters to tokenize the string on.
Whenever a character
    from this list is encountered, a new token is started. Also supports
escaped values like `\\n` and `\\f`,
    and in addition `\\s` to represent whitespace, `\\d` to represent
digits and `\\w` to represent letters.
    Defaults to an empty list.

=== Example output

```The 2 QUICK Brown-Foxes jumped over the lazy dog's bone for $2```

When the configuration `\\s-:<>` is used for `tokenize_on_chars`, the
above sentence would produce the following terms:

```[ The, 2, QUICK, Brown, Foxes, jumped, over, the, lazy, dog's, bone,
for, $2 ]```
2018-05-22 16:26:31 +02:00
Tanguy Leroux 74474e99d6 [Docs] Fix broken cross link in documentation 2018-05-22 16:03:33 +02:00
Martijn van Groningen 59fc6a478e
[DOCS] fixed incorrect default 2018-05-22 10:57:59 +02:00
Jim Ferenczi bdb79d021a
Fix docs failure on language analyzers (#30722)
This commit fixes docs failure on language analyzers when compared to the built in analyzers.
The `elision` filters used by the rebuilt language analyzers should be case insensitive to match
the definition of the prebuilt analyzers.

Closes #30557
2018-05-22 09:58:12 +02:00
Tanguy Leroux c351b51ac4
[Docs] Fix inconsistencies in snapshot/restore doc (#30480)
Closes #30444
2018-05-22 09:19:07 +02:00
Michael Basnight c6be3b4e5a
Add Delete Repository High Level REST API (#30666)
This commit adds Delete Repository, the associated docs and tests for
the high level REST API client. It also cleans up a seemingly innocuous
line in the RestDeleteRepositoryAction and some naming in SnapshotIT.

Relates #27205
2018-05-21 19:52:21 -05:00
Martijn van Groningen a722445fa3
[DOCS] Mark painless execute api as experimental (#30710) 2018-05-21 09:49:25 +02:00
Adam Chalkley 7cc38ab45a Fix default shards count in create index docs (#30747)
Update the default number of primary shards to match doc update work
done in #30539.
2018-05-20 14:59:28 -04:00
Ryan Ernst 34180f2285
Scripting: Remove getDate methods from ScriptDocValues (#30690)
The getDate() and getDates() existed prior to 5.x on long fields in
scripting. In 5.x, a new Date type for ScriptDocValues was added. The
getDate() and getDates() methods were left on long fields and added to date
fields to ease the transition. This commit removes those methods for
7.0.
2018-05-18 21:26:26 -07:00
Christoph Büscher 994405a768
[Docs] Fix single page :docs:check invocation (#30725)
The docs/README shows how to run the :docs:check goal on a subset of pages,
however using the current example fails. Removing the masking character before
the first wildcard fixes the problem.
2018-05-18 23:33:41 +02:00
William Dearden 13c56ae444 Docs: Add uptasticsearch to list of clients (#30738)
It is a client for r.
2018-05-18 17:13:12 -04:00
Lisa Cawley 6846d2c94a
[DOCS] Removes redundant index.asciidoc files (#30707) 2018-05-18 11:05:40 -07:00
Ryan Ernst b3f3a4312b
Plugins: Remove meta plugins (#30670)
Meta plugins existed only for a short time, in order to enable breaking
up x-pack into multiple plugins. However, now that x-pack is no longer
installed as a plugin, the need for them has disappeared. This commit
removes the meta plugins infrastructure.
2018-05-18 10:56:08 -07:00
Lisa Cawley e750462e0c
[DOCS] Moves X-Pack configurationg pages in table of contents (#30702) 2018-05-18 10:26:03 -07:00
Jason Tedor d68c44b76c
Default copy settings to true and deprecate on the REST layer (#30598)
This commit defaults the copy_settings REST parameter to the shrink and
split APIs to true, and deprecates the parameter.
2018-05-18 10:12:08 -04:00
lcawl 663295d635 [DOCS] Replace X-Pack terms with attributes 2018-05-17 09:57:11 -07:00
Piotr Prądzyński a0a8c4f186 filters agg docs duplicated 'bucket' word removal (#30677)
In one place word 'bucket' was duplicated.
2018-05-17 15:21:50 +01:00
Piotr Prądzyński cefbd29db3 top_hits doc example description update (#30676)
Example description does not fit example code.
2018-05-17 15:21:25 +01:00
Christoph Büscher 712473b558
[Docs] Replace InetSocketTransportAddress with TransportAdress (#30673)
The former class has been removed in 6.0, the documentation code
snippets should be updated accordingly.
2018-05-17 14:23:08 +02:00
Zachary Tong df853c49c0
Add a MovingFunction pipeline aggregation, deprecate MovingAvg agg (#29594)
This pipeline aggregation gives the user the ability to script functions that "move" across a window
of data, instead of single data points.  It is the scripted version of MovingAvg pipeline agg.

Through custom script contexts, we expose a number of convenience methods:

 - MovingFunctions.max()
 - MovingFunctions.min()
 - MovingFunctions.sum()
 - MovingFunctions.unweightedAvg()
 - MovingFunctions.linearWeightedAvg()
 - MovingFunctions.ewma()
 - MovingFunctions.holt()
 - MovingFunctions.holtWinters()
 - MovingFunctions.stdDev()

The user can also define any arbitrary logic via their own scripting, or combine with the above methods.
2018-05-16 10:57:00 -04:00
Van0SS 4478f10a2a Rest High Level client: Add List Tasks (#29546)
This change adds a `listTasks` method to the high level java
ClusterClient which allows listing running tasks through the 
task management API.

Related to #27205
2018-05-16 13:31:37 +02:00
lukens 9434f25ee3 [Docs] Update code snippet in has-child-query.asciidoc (#30510)
Changed `InetSocketTransportAddress` to `TransportAddress`, as that 
seems to be  the thing now.
2018-05-16 09:25:48 +02:00
Vladimir Dolzhenko fe3e0257ae
Allow date math for naming newly-created snapshots (#7939) (#30479)
Allow date math for naming newly-created snapshots (#7939)
2018-05-16 07:23:25 +02:00
Michael Basnight b94bc70aee
Add Create Repository High Level REST API (#30501)
This commit adds Create Repository, the associated docs and tests
for the high level REST API client. A few small changes to the
PutRepository Request and Response went into the commit as well.
2018-05-15 21:21:11 -05:00
Jason Tedor abc06d5b79
Expose master version in REST test context (#30623)
This commit exposes the master version to the REST test context. This
will be needed in a follow-up where the master version will be used to
determine whether or not a certain warning header is expected.
2018-05-15 17:26:43 -04:00
Julie Tibshirani 4f9dd37169
Add support for search templates to the high-level REST client. (#30473) 2018-05-15 13:07:58 -07:00
lcawl 5894e3574f [DOCS] Restores 7.0.0 release notes and highlights 2018-05-15 08:48:41 -07:00
Julie Tibshirani ab0be394e9
Remove assert statements from field caps documentation. (#30601)
Reorganize the test in `SearchDocumentationIT` so the assertions aren't shown in the generated documentation.
2018-05-15 08:37:50 -07:00
Albert Zaharovits 801973fa9f
Repository GCS plugin new client library (#30168)
This does away with the deprecated `com.google.api-client:google-api-client:1.23`
and replaces it with `com.google.cloud:google-cloud-storage:1.28.0`.
It also changes security permissions for the repository-gcs plugin.
2018-05-15 18:22:58 +03:00
javanna e1d675c690 [DOCS] Remove references to changelog and to highlights
highlights reference the changelog and it currently breaks the docs.

This aligns changes in master with the ones made in other branches.
2018-05-15 12:42:15 +02:00
javanna 098b3b7fb4 [DOCS] Remove references to removed changelog 2018-05-15 11:47:56 +02:00
srini-raman 0592b685b9 [Docs] Improve section detailing translog usage (#30573) 2018-05-15 10:43:57 +02:00
Jason Tedor 0f85c6429c
Remove the changelog (#30593)
We are starting over on the changelog with a different approach. This
commit removes the existing incarnation of the changelog to remove
confusion that we need to continue adding entries to it.
2018-05-14 21:57:08 -04:00
Lisa Cawley 21d67d1bd7
[DOCS] Adds release highlight pages (#30590) 2018-05-14 15:49:00 -07:00
Nik Everett 9881bfaea5
Docs: Document how to rebuild analyzers (#30498)
Adds documentation for how to rebuild all the built in analyzers and
tests for that documentation using the mechanism added in #29535.

Closes #29499
2018-05-14 18:40:54 -04:00
Igor Motov b30f2913cf
Docs: document precision limitations of geo_bounding_box (#30540)
The geo_bounding_box query might produce false positives alongside
the right and upper edges and false negatives alongside left and
bottom edges. This commit documents the behavior and defines the
maximum error.

Closes #29196
2018-05-14 15:54:42 -04:00
Jason Tedor 4a4e3d70d5
Default to one shard (#30539)
This commit changes the default out-of-the-box configuration for the
number of shards from five to one. We think this will help address a
common problem of oversharding. For users with time-based indices that
need a different default, this can be managed with index templates. For
users with non-time-based indices that find they need to re-shard with
the split API in place they no longer need to resort only to
reindexing.

Since this has the impact of changing the default number of shards used
in REST tests, we want to ensure that we still have coverage for issues
that could arise from multiple shards. As such, we randomize (rarely)
the default number of shards in REST tests to two. This is managed via a
global index template. However, some tests check the templates that are
in the cluster state during the test. Since this template is randomly
there, we need a way for tests to skip adding the template used to set
the number of shards to two. For this we add the default_shards feature
skip. To avoid having to write our docs in a complicated way because
sometimes they might be behind one shard, and sometimes they might be
behind two shards we apply the default_shards feature skip to all docs
tests. That is, these tests will always run with the default number of
shards (one).
2018-05-14 12:22:35 -04:00
Nik Everett 41148e4bb1
Docs: Update HighLevelRestClient migration docs (#30544)
The High Level REST Client's documentation suggested that users should
use the Low Level REST Client for index management activities. This
change removes that suggestion because the high level REST client
supports those APIs now.

This also changes the examples in the migration docs to that still use
the Low Level REST Client to use the non-deprecated varieats of
`performRequest`.
2018-05-14 11:11:27 -04:00
Yannick Welsch c96f2d7bf7
Document woes between auto-expand-replicas and allocation filtering (#30531)
Relates to #2869
2018-05-14 12:14:37 +02:00
Jason Tedor 901436148b Adjust copy settings versions
This commit adjusts the versions on the copy settings behavior now
that the default behavior is configured in 7.0.0.
2018-05-13 22:23:13 -04:00
Jason Tedor 593fdd40ed
Deprecate not copy settings and explicitly disallow (#30404)
We want copying settings to be the default behavior. This commit
deprecates not copying settings, and disallows explicitly not copying
settings. This gives users a transition path to the future default
behavior.
2018-05-13 10:30:05 -04:00
Costin Leau 52580b5ca8
SQL: Fix parsing of dates with milliseconds (#30419)
Dates internally contain milliseconds (which appear when converting them
to Strings) however parsing does not accept them (and is being strict).
The parser has been changed so that Date is mandatory but the time
(including its fractions such as millis) are optional.

Fix #30002
2018-05-10 20:14:54 +03:00
Nik Everett b4502dbf74
LLClient: Add setJsonEntity (#30447)
Adds `Request#setJsonEntity(String)` which short circuits the process of
sending a json string which is super common.
2018-05-09 18:33:03 -04:00
Mueed Chaudhry bf141a3fd1 [docs] add warning for read-write indices in force merge documentation (#28869) 2018-05-09 18:53:55 +02:00
Nik Everett f9dc86836d
Docs: Test examples that recreate lang analyzers (#29535)
We have a pile of documentation describing how to rebuild the built in
language analyzers and, previously, our documentation testing framework
made sure that the examples successfully built *an* analyzer but they
didn't assert that the analyzer built by the documentation matches the
built in anlayzer. Unsuprisingly, some of the examples aren't quite
right.

This adds a mechanism that tests that the analyzers built by the docs.
The mechanism is fairly simple and brutal but it seems to be working:
build a hundred random unicode sequences and send them through the
`_analyze` API with the rebuilt analyzer and then again through the
built in analyzer. Then make sure both APIs return the same results.
Each of these calls to `_anlayze` takes about 20ms on my laptop which
seems fine.
2018-05-09 09:23:10 -04:00
Michael Basnight 3b9c8204a6
Add GET Repository High Level REST API (#30362)
This commit adds the Snapshot Client with a first API call within it,
the get repositories call in snapshot/restore module. This also creates
a snapshot namespace for the docs, as well as get repositories docs.

Relates #27205
2018-05-09 07:25:23 -05:00
Yu 106bed90c7 Add `coordinating_only` node selector (#30313)
Today we can execute cluster API actions on only master, data or ingest nodes
using the `master:true`, `data:true` and `ingest:true` filters, but it is not
so easy to select coordinating-only nodes (i.e. those nodes that are neither
master nor data nor ingest nodes). This change fixes this by adding support for
a `coordinating_only` filter such that `coordinating_only:true` adds all
coordinating-only nodes to the set of selected nodes, and 
`coordinating_only:false` deletes them.

Resolves #28831.
2018-05-09 12:14:07 +01:00
Ke Li 0c6789bc72 Use date format in `date_range` mapping before fallback to default (#29310)
If the date format is not forced in query, use the format in mapping before 
fallback to the default format.

Closes #29282
2018-05-09 09:41:44 +02:00
Alexander Reelsen f00890ee38
Watcher: Increase HttpClient parallel sent requests (#30130)
The HTTPClient used in watcher is based on the apache http client. The
current client is using a lot of defaults - which are not always
optimal. Two of those defaults are the maximum number of total
connections and the maximum number of connections to a single route.

If one of those limits is reached, the HTTPClient waits for a connection
to be finished thus acting in a blocking fashion. In order to prevent
this when many requests are being executed, we increase the limit of
total connections as well as the connections per route (a route is
basically an endpoint, which also contains proxy information, not
containing an URL, just hosts).

On top of that an additional option has been set to evict
long running connections, which can potentially be reused after some
time. As this requires an additional background thread, this required
some changes to ensure that the httpclient is closed properly. Also the
timeout for this can be configured.
2018-05-09 09:37:47 +02:00
Nik Everett b062ce5634
Client: Deprecate many argument performRequest (#30315)
Deprecate the many arguments versions of `performRequest` and
`performRequestAsync` in favor of the `Request` object flavored variants
introduced in #29623. We'll be dropping the many arguments variants in
7.0 because they make it difficult to add new features in a backwards
compatible way and they create a *ton* of intellisense noise.
2018-05-08 14:38:55 -04:00
Nik Everett d20e8e2bb4
Docs: Use task_id in examples of tasks (#30436)
We had been using `task_id:1` or `taskId:1` because it is parses as a
valid task identifier but the `:1` part is confusing. This replaces
those examples with `task_id` which matches the response from the list
tasks API.

Closes #28314
2018-05-08 14:23:32 -04:00
Karim Frenn 3acca0b35c [Docs] Fix typo in cardinality-aggregation.asciidoc (#30434) 2018-05-08 16:12:36 +02:00
aditya-agrawal 27ddb4ffea Avoid NPE in `more_like_this` when field has zero tokens (#30365)
Fixes and edge case when using `more_like_this` where TermVectorsWriter
could throw an NPE when a field produced zero tokens after analysis. This
changes the implementation to use an empty list of tokens in this case.

Closes #30148
2018-05-08 15:13:07 +02:00
Yannick Welsch 82b251adcf
Auto-expand replicas when adding or removing nodes (#30423)
Auto-expands replicas in the same cluster state update (instead of a follow-up reroute) where nodes are added or removed.

Closes #1873, fixing an issue where nodes drop their copy of auto-expanded data when coming up, only to sync it again later.
2018-05-07 22:26:31 +02:00
Igor Motov 44c6dcf5be Docs: fix changelog merge 2018-05-07 14:25:03 -04:00
Igor Motov 6fb189ce47
Add stricter geohash parsing (#30376)
Adds verification that geohashes are not empty and contain only
valid characters. It fixes the issue when en empty geohash is
treated as [-180, -90] and geohashes with non-geohash character
are getting resolved into invalid coordinates.

Closes #23579
2018-05-07 13:56:39 -04:00
javanna c9f5a7893b [DOCS] convert forcemerge snippet
Relates to #30113
2018-05-07 16:09:03 +02:00
Matija Bruncic e5653e635d Update forcemerge.asciidoc (#30113) 2018-05-07 14:56:12 +02:00
Dave Moore 391bcbcbe1 Added zentity to the list of API extension plugins (#29143) 2018-05-07 14:46:47 +02:00
Ke Li d373e1b49c Fix the search request default operation behavior doc (#29302) (#29405) 2018-05-07 14:43:45 +02:00
Tanguy Leroux 1987d6261f
Do not fail snapshot when deleting a missing snapshotted file (#30332)
When deleting or creating a snapshot for a given shard, elasticsearch 
usually starts by listing all the existing snapshotted files in the repository. 
Then it computes a diff and deletes the snapshotted files that are not 
needed anymore. During this deletion, an exception is thrown if the file 
to be deleted does not exist anymore.

This behavior is challenging with cloud based repository implementations 
like S3 where a file that has been deleted can still appear in the bucket for 
few seconds/minutes (because the deletion can take some time to be fully 
replicated on S3). If the deleted file appears in the listing of files, then the 
following deletion will fail with a NoSuchFileException and the snapshot 
will be partially created/deleted.

This pull request makes the deletion of these files a bit less strict, ie not 
failing if the file we want to delete does not exist anymore. It introduces a 
new BlobContainer.deleteIgnoringIfNotExists() method that can be used 
at some specific places where not failing when deleting a file is 
considered harmless.

Closes #28322
2018-05-07 09:35:55 +02:00
Nhat Nguyen 3e58463256 DOCS: Correct mapping tags in put-template api
The mapping tags were not named consistently and not linked correctly.

Relates #30400
2018-05-06 15:56:49 -04:00
Nhat Nguyen eed8a3b585
Add put index template api to high level rest client (#30400)
Relates #27205
2018-05-06 09:47:36 -04:00
Jim Ferenczi d3ee35ef18 [Docs] Add snippets for POS stop tags default value
relates #30397
2018-05-05 07:53:50 +02:00
Jason Tedor 10fcf30ce1 Move respect accept header on no handler to 6.3.1
This commit moves the changelog entry for the change to respect the
accept header on no handler from the 7.0.0 section of the docs to 6.3.1.
2018-05-04 20:50:30 -04:00
Jason Tedor beee5fe004
Respect accept header on no handler (#30383)
Today when processing a request for a URL path for which we can not find
a handler we send back a plain-text response. Yet, we have the accept
header in our hand and can respect the accepted media type of the
request. This commit addresses this.
2018-05-04 18:13:50 -04:00
Jim Ferenczi ec187ed3be [Docs] Fix bad link
relates #30397
2018-05-04 22:07:12 +02:00
Jim Ferenczi d7c2a99347 [Docs] Fix end of section in the korean plugin docs
relates #30397
2018-05-04 21:41:50 +02:00
Jim Ferenczi 891d3bd9c3
Expose the Lucene Korean analyzer module in a plugin (#30397)
This change adds a new plugin called `analysis-nori` that exposes
Korean text analysis in es using the new Lucene Korean analyzer module named (`nori`).
The plugin adds:
* a Korean analyzer: `nori`
* a Korean tokenizer: `nori_tokenizer`
* a part of speech stop filter: `nori_part_of_speech`
* a filter that can replace Hanja characters with their Hangul transcription: `nori_readingform`
2018-05-04 20:46:13 +02:00
Zachary Tong 1c0d339904
[Rollup] Validate timezone in range queries (#30338)
When validating the search request, we make sure any date_histogram
aggregations have timezones that match the jobs.  But we didn't
do any such validation on range queries.

While it wouldn't produce incorrect results, it would be confusing
to the user as no documents would match the aggregation (because we
add a filter clause on the timezone for the agg).

Now the user gets an exception up front, and some helpful text about
why the range query didnt match, and which timezones are acceptable
2018-05-04 10:45:16 -07:00
tomcallahan 0a93956194
Add Get Settings API support to java high-level rest client (#29229)
This PR adds support for the Get Settings API to the java high-level rest client.
Furthermore, logic related to the retrieval of default settings has been moved from the rest layer into the transport layer and now default settings may be retrieved consistency via both the rest API and the transport API.
2018-05-04 11:14:28 -04:00
Jim Ferenczi dbd857341f
Upgrade to 7.4.0-snapshot-1ed95c097b (#30357)
Upgrade to lucene-7.4.0-snapshot-1ed95c097b

This version contains:
* An Analyzer for Korean
* An IntervalQuery and IntervalsSource that retrieve minimum intervals of positional queries.
* A new API to retrieve matches (offsets and positions) of a query for a single document.
* Support for soft deletes in the index writer.
* A fixed shingle filter that handles index time synonyms.
* Support for emoji sequence in ICUTokenizer (with an upgrade to icu 61.1)
2018-05-04 11:44:22 +02:00
lcawley 137ce702a4 [DOCS] Added coming qualifiers in changelog 2018-05-03 19:38:19 -07:00
debadair 19624466e8
[DOCS] Commented out empty sections in the changelog to fix the doc build. (#30372) 2018-05-03 16:31:32 -07:00
Jay Modi aa0d7c73f8
Security: reduce garbage during index resolution (#30180)
The IndexAndAliasesResolver resolves the indices and aliases for each
request and also handles local and remote indices. The current
implementation uses the ResolvedIndices class to hold the resolved
indices and aliases. While evaluating the indices and aliases against
the user's permissions, the final value for ResolvedIndices is
constructed. Prior to this change, this was done by creating a
ResolvedIndices for the first set of indices and for each additional
addition, a new ResolvedIndices object is created and merged with
the existing one. With a small number of indices and aliases this does
not pose a large problem; however as the number of indices/aliases
grows more list allocations and array copies are needed resulting in a
large amount of garbage and severely impacted performance.

This change introduces a builder for ResolvedIndices that appends to
mutable lists until the final value has been constructed, which will
ultimately reduce the amount of garbage generated by this code.
2018-05-03 12:48:23 -06:00
Sue Gallagher 09a6ba4fea
Change quad tree max levels to 29. Closes #21191 (#29663)
* [DOCS] Changed quad tree max levels to 29. Clears 21191

* Changed QuadPrefixTree max levels to 29 and added defaults. Closes #21191
2018-05-03 09:48:21 -07:00
Dimitris Athanasiou 3b260dcfc1
[ML] Account for gaps in data counts after job is reopened (#30294)
This commit fixes an issue with the data diagnostics were
empty buckets are not reported even though they should. Once
a job is reopened, the diagnostics do not get initialized from
the current data counts (especially the latest record timestamp).
The result is that if the data that is sent have a time gap compared
to the previous ones, that gap is not accounted for in the empty bucket
count.

This commit fixes that by initializing the diagnostics with the current
data counts.

Closes #30080
2018-05-03 15:08:24 +01:00