Commit Graph

7681 Commits

Author SHA1 Message Date
Lisa Cawley 75f1f612c2 [DOCS] Merges duplicate pages for Active Directory realms (#49205) 2019-11-19 13:18:01 -08:00
Lisa Cawley c4c8a7a43c [DOCS] Merges duplicate pages for PKI realms (#49206) 2019-11-19 10:51:09 -08:00
Lisa Cawley 62bbe419d3 [DOCS] Removes Beats security page (#49276) 2019-11-19 09:15:30 -08:00
Lisa Cawley 97cdfd2848 [DOCS] Clarify ML job closure prerequisites (#49265) 2019-11-19 08:36:50 -08:00
James Rodewig a26916cc23 [DOCS] Reformat elision token filter docs (#49262) 2019-11-19 10:55:22 -05:00
James Rodewig 8639ddab5e [DOCS] Reformat fingerprint token filter docs (#49311) 2019-11-19 10:55:21 -05:00
jimczi cb5169ae37 update release notes for 7.5.0 after respin 2019-11-19 16:24:04 +01:00
Marios Trivyzas fd1bb4a33a SQL: Fix issue with mins & hours for DATEDIFF (#49252)
Previously, DATEDIFF for minutes and hours was doing a
rounding calculation using all the time fields (secs, msecs/micros/nanos).
Instead it should first truncate the 2 dates to the respective field (mins or hours)
zeroing out all the more detailed time fields and then make the subtraction.

(cherry picked from commit 124cd18e20429e19d52fd8dc383827ea5132d428)
2019-11-19 14:25:28 +01:00
Lisa Cawley abd4a70b10 [DOCS] Merges duplicate pages for Kerberos realms (#49207) 2019-11-18 15:23:06 -08:00
Lisa Cawley b4f82c9cdb [DOCS] Merges duplicate pages for LDAP realms (#49203) 2019-11-18 14:09:24 -08:00
Julie Tibshirani 81a9d98a47 Remove the 'experimental' marking from vector fields. (#49120)
We wrapped up the API changes we wanted to make, and vector fields can now be
considered GA.
2019-11-18 12:42:46 -08:00
Lisa Cawley b0054eecd6 [DOCS] Merges duplicate pages for file realms (#49200) 2019-11-18 12:02:18 -08:00
Lisa Cawley 48f53efd9a [DOCS] Merges duplicate pages for SAML realms (#49209) 2019-11-18 10:09:29 -08:00
Lisa Cawley b0b5fcc4f6
[DOCS] Removes closed security PRs from release notes (#49256) 2019-11-18 09:19:11 -08:00
gpaimla 7d20b50f45 Implement Lucene EstonianAnalyzer, Stemmer (#49149)
This PR adds a new analyzer and stemmer for the Estonian language.

Closes #48895
2019-11-18 17:24:21 +01:00
Antoine Garcia 288217e82b [Docs] Specify field types not supporting doc values (#49041)
The `string` type (with option `analyzed`) has been replaced by `text` after `6.0`, 
also the `annonated_text` field do not support doc values and should be mentioned.
2019-11-18 16:38:31 +01:00
Yannick Welsch af797a77a1 Auto-expand indices according to allocation filtering rules (#48974)
Honours allocation filtering rules when auto-expanding indices.
2019-11-18 12:01:56 +01:00
Rory Hunter e84e21174b
Support `_FILE` suffixed env vars in Docker entrypoint (#49182)
Backport of #47573.

Closes #43603. Allow environment variables to be passed to ES in a Docker
container via a file, by setting an environment variable with the `_FILE`
suffix that points to the file with the intended value of the env var.
2019-11-18 08:22:35 +00:00
Lisa Cawley 09a9ec4d23 [DOCS] Merges duplicate pages for native realms (#49198) 2019-11-15 15:35:53 -08:00
Lisa Cawley de8107e350
[DOCS] Adds ml-cpp PRs to release notes (#49185) 2019-11-15 09:36:39 -08:00
Lisa Cawley eca93fcc5f [DOCS] Adds machine learning node type and filters (#49121) 2019-11-15 08:31:59 -08:00
Christos Soulios d9f0245b10
[7.x] Implement stats aggregation for string terms (#49097)
Backport of #47468 to 7.x

This PR adds a new metric aggregation called string_stats that operates on string terms of a document and returns the following:

min_length: The length of the shortest term
max_length: The length of the longest term
avg_length: The average length of all terms
distribution: The probability distribution of all characters appearing in all terms
entropy: The total Shannon entropy value calculated for all terms

This aggregation has been implemented as an analytics plugin.
2019-11-15 14:36:21 +02:00
SylvainJuge e8f49cdee0 [DOCS] minor fix to documentation: http.host can't default to itself (#48135)
fix minor typos on http.host and transport.host default values.

7.x backport of https://github.com/elastic/elasticsearch/pull/48135
2019-11-14 18:16:38 +01:00
James Rodewig e1726fff56 [DOCS] Reformat update license API docs (#48967)
Makes a few changes to better align the update license API docs with
the [API reference template][0].

Changes:

* Replaces POST with PUT in several snippet examples.
  While both are valid, PUT is a bit more RESTful.

* Removes leading slashes (/) from all snippets.

* Relocates and retitles the 'Authorization' section to 'Prerequisites'.

* Replaces explicit titles with the appropriate API reference template
  attributes.

* Replaces unneeded `[float]` tags with explicit anchors.

Closes #35341

[0]: https://github.com/elastic/docs/blob/master/shared/api-ref-ex.asciidoc
2019-11-14 08:00:42 -05:00
Rory Hunter c46a0e8708
Apply 2-space indent to all gradle scripts (#49071)
Backport of #48849. Update `.editorconfig` to make the Java settings the
default for all files, and then apply a 2-space indent to all `*.gradle`
files. Then reformat all the files.
2019-11-14 11:01:23 +00:00
James Rodewig 095c34359f [DOCS] Note limitations of `max_gram` parm in `edge_ngram` tokenizer for index analyzers (#49007)
The `edge_ngram` tokenizer limits tokens to the `max_gram` character
length. Autocomplete searches for terms longer than this limit return
no results.

To prevent this, you can use the `truncate` token filter to truncate
tokens to the `max_gram` character length. However, this could return irrelevant results.

This commit adds some advisory text to make users aware of this limitation and outline the tradeoffs for each approach.

Closes #48956.
2019-11-13 14:28:12 -05:00
James Rodewig 838af15d29 [DOCS] Reformat compound word token filters (#49006)
* Separates the compound token filters doc pages into separate token
  filter pages:
  * Dictionary decompounder token filter
  * Hyphenation decompounder token filter

* Adds analyze API examples for each compound token filter

* Adds a redirect for the removed compound token filters page

Co-Authored-By: debadair <debadair@elastic.co>
2019-11-13 09:36:52 -05:00
István Zoltán Szabó b55022b59f [DOCS] Adds test clause to the code snippets in the cluster restart page (#49023) 2019-11-13 14:36:44 +01:00
Julie Tibshirani 37fa3fb4ff
Ensure parameters are updated when merging flattened mappings. (#48971) (#49014)
This PR makes the following two fixes around updating flattened fields:

* Make sure that the new value for ignore_above is immediately taken into
  affect. Previously we recorded the new value but did not use it when parsing
  documents.
* Allow depth_limit to be updated dynamically. It seems plausible that a user
  might want to tweak this setting as they encounter more data.
2019-11-12 21:50:39 -05:00
David Roberts 698ebd3d0a [TEST] Mute docs snippet test in close-job.asciidoc (#49000)
Due to https://github.com/elastic/elasticsearch/pull/48583#issuecomment-552991325
2019-11-12 17:34:27 +00:00
Michael Basnight bc23bc5146 Add delete alias to the HLRC (#48819)
The delete alias call is a rest only API call, but should still be added
to the rest client. This commit adds it as well as relevant tests.

Ref #47678
2019-11-12 11:02:53 -06:00
Orhan Toy 561351d2fc [Docs] Fix _count HTTP method (#48979) 2019-11-12 15:45:26 +01:00
István Zoltán Szabó fc145575c4 [DOCS] Creates a cluster restart documentation page (#48583)
Co-Authored-By: James Rodewig <james.rodewig@elastic.co>
2019-11-12 14:50:53 +01:00
James Rodewig 42e92616f6 [DOCS] Document indices response parameters for node stats API (#47525) 2019-11-12 08:35:35 -05:00
jimczi 0e82b5f59b add release notes for 7.5.0 2019-11-12 09:59:14 +01:00
Benjamin Trent 46ab1db54f
[7.x] [ML] Add new geo_results.(actual_point|typical_point) fields for `lat_long` results (#47050) (#48958)
* [ML] Add new geo_results.(actual_point|typical_point) fields for `lat_long` results (#47050)

[ML] Add new geo_results.(actual_point|typical_point) fields for `lat_long` results (#47050)

Related PR: https://github.com/elastic/ml-cpp/pull/809

* adjusting bwc version
2019-11-11 15:43:03 -05:00
István Zoltán Szabó c2f52015d3 [DOCS] Removes best practice about fields that are highly correlated to the dependent variable. (#48935) 2019-11-11 16:01:21 +01:00
István Zoltán Szabó 91888959e8 [DOCS] Extends analyzed_fields description in PUT DFA API docs. (#48307) 2019-11-11 15:55:12 +01:00
Patrick Maynard 4b85498617 [DOCS] Fix typo in search type docs (#48868) 2019-11-11 09:38:48 -05:00
James Rodewig dd92830801 [DOCS] Reformat condition token filter (#48775) 2019-11-11 08:49:44 -05:00
Arne Welzel f642baa9fb [DOCS] Remove extra "when" (#48926) 2019-11-11 10:11:02 +01:00
Yannick Welsch 87862868c6 Allow realtime get to read from translog (#48843)
The realtime GET API currently has erratic performance in case where a document is accessed
that has just been indexed but not refreshed yet, as the implementation will currently force an
internal refresh in that case. Refreshing can be an expensive operation, and also will block the
thread that executes the GET operation, blocking other GETs to be processed. In case of
frequent access of recently indexed documents, this can lead to a refresh storm and terrible
GET performance.

While older versions of Elasticsearch (2.x and older) did not trigger refreshes and instead opted
to read from the translog in case of realtime GET API or update API, this was removed in 5.0
(#20102) to avoid inconsistencies between values that were returned from the translog and
those returned by the index. This was partially reverted in 6.3 (#29264) to allow _update and
upsert to read from the translog again as it was easier to guarantee consistency for these, and
also brought back more predictable performance characteristics of this API. Calls to the realtime
GET API, however, would still always do a refresh if necessary to return consistent results. This
means that users that were calling realtime GET APIs to coordinate updates on client side
(realtime GET + CAS for conditional index of updated doc) would still see very erratic
performance.

This PR (together with #48707) resolves the inconsistencies between reading from translog and
index. In particular it fixes the inconsistencies that happen when requesting stored fields, which
were not available when reading from translog. In case where stored fields are requested, this
PR will reparse the _source from the translog and derive the stored fields to be returned. With
this, it changes the realtime GET API to allow reading from the translog again, avoid refresh
storms and blocking the GET threadpool, and provide overall much better and predictable
performance for this API.
2019-11-09 17:47:50 +01:00
Julian Simioni 5e4501eb3f [Docs] Consolidate single example into a single line (#48904)
The first example of splitting rules for the `word_delimiter` token filter was spread across two bullet points. This makes it look like they are two separate splitting rules.
2019-11-08 15:12:45 -05:00
Yannick Welsch af887be3e5 Hide orphaned tasks from follower stats (#48901)
CCR follower stats can return information for persistent tasks that are in the process of being cleaned up. This is problematic for tests where CCR follower indices have been deleted, but their persistent follower task is only cleaned up asynchronously afterwards. If one of the following tests then accesses the follower stats, it might still get the stats for that follower task.

In addition, some tests were not cleaning up their auto-follow patterns, leaving orphaned patterns behind. Other tests cleaned up their auto-follow patterns. As always the same name was used, it just depended on the test execution order whether this led to a failure or not. This commit fixes the offensive tests, and will also automatically remove auto-follow-patterns at the end of tests, like we do for many other features.

Closes #48700
2019-11-08 13:56:53 +01:00
bellengao bdc7057d58 [DOCS] Correct typo in split index API docs (#48894) 2019-11-07 15:27:27 -05:00
bellengao 293902c6a5 [DOCS] Fix shard type in CCR overview doc (#48882)
Closes #48875
2019-11-07 10:09:45 -05:00
Tanguy Leroux 552381d7f9 Add mention to Pause Auto-Follower API in Upgrade Clusters docs (#48764)
Relates #46665
2019-11-06 09:48:44 -05:00
István Zoltán Szabó 3c9bd13dca [DOCS] Adds classification type DFA API docs and ml-shared.asciidoc (#48241) 2019-11-06 07:41:38 -05:00
István Zoltán Szabó 70765dfb05 [DOCS] Adds classification type evaluation docs to the DFA evaluation API (#47657) 2019-11-06 07:38:33 -05:00
glerb baabc21a04 [DOCS] Correct typo in Discovery docs (#48494) 2019-11-05 08:48:43 -05:00
James Rodewig 700a316bb3 [DOCS] Reformat decimal digit token filter docs (#48722) 2019-11-01 12:38:14 -04:00
James Rodewig 680999f246 [DOCS] List `indices.lifecycle.poll_interval` as cluster-level (#48813)
Lists `indices.lifecycle.poll_interval` with other cluster-level ILM
settings.

Previously, it was included under index-level settings.
2019-11-01 11:54:46 -04:00
pulysak 9a0a7ab95a [DOCS] Fix typo in Index API reference docs (#48760) 2019-11-01 09:16:11 -04:00
Alexander Reelsen 80cde68af7 [DOCS] Remove unneeded // CONSOLE comments from snippets (#48763)
Updates the docs README file to remove outdated `// CONSOLE` instructions
2019-11-01 09:04:24 -04:00
debadair b9f4b32892 [DOCS] Fix cross-doc link. (#48783)
* [DOCS] Fix cross-doc link.

* Fixed xref
2019-10-31 18:59:17 -07:00
Lisa Cawley 40834c229f
[7.x][DOCS] Copies ESMS monitoring details to Elasticsearch Reference (#48780) 2019-10-31 18:22:08 -07:00
debadair 457379e74e [DOCS] Edited Docker install & tweaked Docker compose file. (#47715)
* [DOCS] Edited Docker install & tweaked Docker compose file.

* Synced with Docker GS in SO

* Incorporated review comments
2019-10-31 18:12:39 -07:00
Tal Levy 4be54402de
[7.x] Add ingest info to Cluster Stats (#48485) (#48661)
* Add ingest info to Cluster Stats (#48485)

This commit enhances the ClusterStatsNodes response to include global
processor usage stats on a per-processor basis.

example output:

```
...
    "processor_stats": {
      "gsub": {
        "count": 0,
        "failed": 0
        "current": 0
        "time_in_millis": 0
      },
      "script": {
        "count": 0,
        "failed": 0
        "current": 0,
        "time_in_millis": 0
      }
    }
...
```

The purpose for this enhancement is to make it easier to collect stats on how specific processors are being used across the cluster beyond the current per-node usage statistics that currently exist in node stats.

Closes #46146.

* fix BWC of ingest stats

The introduction of processor types into IngestStats had a bug.
It was set to `null` and set as the key to the map. This would
throw a NPE. This commit resolves this by setting all the processor
types from previous versions that are not serializing it out to
`_NOT_AVAILABLE`.
2019-10-31 14:36:54 -07:00
Deb Adair 6412d0f528 [DOCS] Remove coming tag from 7.4.2 RN backport. 2019-10-31 09:43:26 -07:00
Lisa Cawley b7559f23cc [DOCS] Fixes PR#48055 in release notes (#48726) 2019-10-31 07:37:44 -07:00
Peter Johnson 3f7aafa421 [DOCS] Fix typo in synonym token filter docs (#48691) 2019-10-31 09:12:24 -04:00
James Rodewig 3d5b1725a9 [DOCS] Remove unneeded filter from common grams analyze ex (#48748) 2019-10-31 09:08:14 -04:00
Brandon Morelli aa02174d53 [DOCS] Fix typo in ILM policy definition docs (#48723)
Removes an extra "by".
2019-10-31 08:30:54 -04:00
Andrei Dan ffe5d5417f
ILM Make the `check-rollover-ready` step retryable (#48256) (#48740)
This adds the infrastructure to be able to retry the execution of retryable
steps and makes the `check-rollover-ready` retryable as an initial step to
make the rollover action more resilient to transient errors.

(cherry picked from commit 454020ac8acb147eae97acb4ccd6fb470d1e5f48)
Signed-off-by: Andrei Dan <andrei.dan@elastic.co>
2019-10-31 11:28:55 +00:00
debadair a876760848 [DOCS] Add placeholder for 7.4.2 release notes (#48724) 2019-10-30 16:09:29 -07:00
Jason Tedor 13043219ac
Fix specification for cluster.remote.connect (#48690)
The docs specify that cluster.remote.connect disables cross-cluster
search. This is correct, but not fully accurate as it disables any
functionality that relies on remote cluster connections: cross-cluster
search, remote data feeds, and cross-cluster replication. This commit
updates the docs to reflect this.
2019-10-30 11:26:15 -04:00
James Rodewig 0b062bbc82 [DOCS] Correct required file ext for user agent ingest processor (#48688)
For the user agent ingest processor, custom regex files must end
with the `.yml` file extension.

This corrects the docs which said the `.yaml` extension was required.
2019-10-30 11:11:29 -04:00
Dan Hermann dbc05cd808
Add option to split processor for preserving trailing empty fields (#48685) 2019-10-30 08:25:03 -05:00
James Rodewig 77acbc4fa9 [DOCS] Reformat common grams token filter (#48426) 2019-10-30 08:40:56 -04:00
Yannick Welsch 356066ce6a Revert "Mute get-ccr-stats doctest (#48375)"
This reverts commit f861927e8b8fc987949ce996a131a2d272b9646e.
2019-10-30 11:13:29 +01:00
Julie Tibshirani 89c65752dc
Update the signature of vector script functions. (#48653)
Previously the functions accepted a doc values reference, whereas they now
accept the name of the vector field. Here's an example of how a vector function
was called before and after the change.

```
Before: cosineSimilarity(params.query_vector, doc['field'])
After:  cosineSimilarity(params.query_vector, 'field')
```

This seems more intuitive, since we don't allow direct access to vector doc
values and the the meaning of `doc['field']` is unclear.

The PR makes the following changes (broken into distinct commits):
* Add new function signatures of the form `function(params.query_vector,
'field')` and deprecates the old ones. Because Painless doesn't allow two
methods with the same name and number of arguments, we allow a generic `Object`
to be passed in to the function and decide on the behavior through an
`instanceof` check.
* Refactor the class bindings so that the document field is passed to the
constructor instead of the instance method. This allows us to avoid retrieving
the vector doc values on every function invocation, which gives a tiny speed-up
in benchmarks.

Note that this PR adds new signatures for the sparse vector functions too, even
though sparse vectors are deprecated. It seemed simplest to understand (for both
us and users) to keep everything symmetric between dense and sparse vectors.
2019-10-29 15:46:05 -07:00
James Rodewig 7002ce1e9c [DOCS] Replace `_uid` refs in reindex slicing docs (#48649)
PR #25543 removed the `_uid` field in favor of the `_id` field,
including for use in slicing.

This removes an outdated reference to `_uid` in our reindex docs.
2019-10-29 16:41:53 -04:00
Christoph Büscher 1de49d8a70 Remove Ranking Evaluation API experimental status (#48603)
The API has been released long enough to remove the experimental status.
2019-10-29 20:57:39 +01:00
Lisa Cawley c6f4662038
[DOCS] Updates ML PRs in 7.4.1 release notes (#48600) 2019-10-29 09:35:11 -07:00
Daniel Andion d0cbbf9d58
SQL: [Docs] Typo in HAVING section (#48609)
`HAVING` section code states `GROUP BY`  instead of the appropriate keyword.

(cherry picked from commit 9d505dc3db51e250fdf1b44e4d952dcd97bf1bc1)
2019-10-29 16:37:39 +01:00
lgypro abddf51672 [Docs] Fix syntax error leading to wrong doc ID (#48554)
In order to index a document with id 2, the "&" should be replaced by "?"
2019-10-29 10:27:23 +01:00
Ian Danforth 82e25c4ac7 [Docs] Fix typo in suggesters search API doc (#48477) 2019-10-29 09:58:05 +01:00
Ian Danforth 4a076f5e92 [Doc] Fix typo in indices module docs (#48598) 2019-10-28 21:40:09 +01:00
Julie Tibshirani 605500df7e
Add sparse vector deprecation to 7.6 migration docs. (#48435)
This note was accidentally omitted from the deprecation PR.
2019-10-28 11:57:20 -07:00
Benjamin Trent 6ea59dd428
[ML][Transforms] add wait_for_checkpoint flag to stop (#47935) (#48591)
Adds `wait_for_checkpoint` for `_stop` API.
2019-10-28 13:02:57 -04:00
Lisa Cawley 13ce179706
[DOCS] Re-enable code snippet testing in close anomaly detection job API (#48259) (#48585) 2019-10-28 08:42:09 -07:00
Michael Basnight 1ba57dbe08 [Docs] add missing snapshot restore reference (#45256) 2019-10-28 09:55:10 -05:00
Shaunak Kashyap d27a307379 [DOCS] Remove extraneous comma in Enrich Stats API's JSON response (#48539) 2019-10-25 12:35:50 -04:00
James Rodewig e9c8e4f6d1 [DOCS] Fix note format in index suggestion docs (#48536) 2019-10-25 11:31:47 -04:00
Christoph Büscher 055a0800eb [Docs] Mention reserved completion suggestion characters (#48445)
We currently don't mention the three reserved characters anywhere. This change
adds a short note mentioning them

Closes #48341
2019-10-25 16:58:23 +02:00
Julie Tibshirani b2974e3816 Correct outdated information in _index docs. (#48436)
This PR makes the following updates:
* Update the supported query types to include `prefix` and `wildcard`.
* Specify that queries accept index aliases.
* Clarify that when querying on a remote index name, the separator `:` must be
  present.
2019-10-24 11:02:25 -07:00
Hendrik Muhs 5ecfcdb162 update warning about index names after transform rename (#48457)
update warning about index names after transform rename
2019-10-24 15:17:20 +02:00
Julie Tibshirani 4375316b9d Make sure to list the 7.5 migration docs. 2019-10-23 18:52:22 -07:00
Julie Tibshirani 2664cbd20b
Deprecate the sparse_vector field type. (#48368)
We have not seen much adoption of this experimental field type, and don't see a
clear use case as it's currently designed. This PR deprecates the field type in
7.x. It will be removed from 8.0 in a follow-up PR.
2019-10-23 16:35:03 -07:00
James Rodewig 06dc1fbd96 [DOCS] Reformat ASCII folding token filter docs (#48143) 2019-10-23 15:06:55 -05:00
Jim Ferenczi 96556d72cc
Add a known issue to the release notes of 7.4.0 (#48373)
A [bug](https://github.com/elastic/elasticsearch/issues/48358) in 7.4.0 prevents
the activation of the search slow log. This change adds an entry in the release
notes to warn users to not activate it in this version.

Relates #48358
2019-10-23 19:57:37 +02:00
James Rodewig 640d7416b1
[DOCS] Change prev version to 7.5 in upgrade docs (#48415) 2019-10-23 12:09:26 -05:00
James Rodewig 19afe3f84c [DOCS] Remove duplicate links for ingest processor overview (#48394) 2019-10-23 10:55:49 -05:00
James Rodewig 9c75f14a9f [DOCS] Reformat classic token filter docs (#48314) 2019-10-23 10:14:25 -05:00
James Rodewig 852622d970 [DOCS] Remove binary gendered language (#48362) 2019-10-23 09:37:12 -05:00
Alan Woodward 3be6135ceb Mute get-ccr-stats doctest (#48375)
This test is failing frequently, due to #47718
2019-10-23 15:20:35 +01:00
Tom Callahan f573cd6a2c [DOCS] Add 7.4.1 Release Notes (#48359) 2019-10-23 08:22:00 -05:00
James Rodewig e45b0cd7e3 [DOCS] Sort cluster API docs alphabetically (#48198) 2019-10-22 12:28:39 -05:00
Brandon DeVault 2f162a98a0 [DOCS] Added the voting node.role value (#48162)
* [DOCS] Added the voting node.role value 

With the elasticsearch.yml option "node.voting_only: true", if you perform a "_cat/nodes" there is a new option for `v` that will display if the node is a voting node.

* [DOCS] Added the voting node.role value

Updated "voting node" to read "voting-only node"
2019-10-22 15:26:05 +02:00
James Rodewig e931fcd331 [DOCS] Add placeholder for 7.4.1 release notes (#48316) 2019-10-22 07:53:28 -05:00
Alexandre Fonseca c41951c6b3 [Docs] Fix opType options in IndexRequest API example. (#48290) 2019-10-22 13:49:19 +02:00
Hendrik Muhs 0656207e9a [DOCS][Transform] document limitation regarding rolling upgrade with 7.2, 7.3 (#48118)
adds a limitation about rolling upgrade from 7.2 or 7.3. and fixes a problem with renamed preferences
2019-10-22 09:01:45 +02:00
Martijn van Groningen c09b62d5bf
Backport: also validate source index at put enrich policy time (#48311)
Backport of: #48254

This changes tests to create a valid
source index prior to creating the enrich policy.
2019-10-22 07:38:16 +02:00
James Rodewig f4fa61b2f2 [DOCS] Add 'Selecting gateway and seed nodes' section to CCS docs (#48297) 2019-10-21 11:14:23 -05:00
James Rodewig a66bb2c7ed [DOCS] Reformat CJK bigram and CJK width token filter docs (#48210) 2019-10-21 08:44:49 -05:00
Ignacio Vera b1224fca8c
upgrade to Lucene-8.3.0-snapshot-25968e3b75e (#48227) 2019-10-21 08:21:09 +02:00
Christoph Büscher 3ea666d694 Clarify mapping types that support ignore_malformed (#48206)
The `ignore_malformed` setting only works on selected mapping types, otherwise
we throw an mapper_parsing_exception. We should add a list of all the mapping
types that support it, since the number of types not supporting it seems larger.

Closes #47166
2019-10-18 20:39:38 +02:00
Guido Lena Cota 1beeb3a83e [DOCS] Fix typos in Painless execute API docs(#47518) 2019-10-18 10:31:04 -04:00
Alexander Reelsen 66581d8158
update ingest-user-agent regexes.yml (#47807)
This new regexes are from:
154eba17f5/regexes.yaml
2019-10-18 16:26:48 +02:00
James Rodewig 1d06be2f05 [DOCS] Note clause limit in `index.mapping.total_fields.limit` docs (#48153)
Notes that you should increase the `indices.query.bool.max_clause_count`
setting if you increase the `index.mapping.total_fields.limit`
setting.

Closes #46928.
2019-10-18 10:20:49 -04:00
markharwood fbd43196a8 [DOCS] Add TermVectors API reference to MLT query docs (#37228)
Adds a reference the use of the TermVectors API following an issue raised about lack of flexibility in the MLT query: https://github.com/elastic/elasticsearch/issues/35509
2019-10-18 10:08:57 -04:00
Przemysław Witek 28f68fa221
Make num_top_classes parameter's default value equal to 2 (#48119) (#48201) 2019-10-17 18:43:15 +02:00
James Rodewig 4c6d56bef0 [DOCS] Expand put mapping API docs examples (#47462) 2019-10-17 09:31:52 -04:00
James Rodewig b4bf47cce4 [DOCS] Document ingest pipeline and processor node stats (#47884) 2019-10-17 09:19:45 -04:00
Anton b54d817f1d [DOCS] Fix typo in intervals query docs (#48180) 2019-10-17 09:15:55 -04:00
Hendrik Muhs 06f54428c2 update internal index name in documentation warning (#48125) 2019-10-17 15:09:47 +02:00
István Zoltán Szabó ecd62934f6 [DOCS] Applies rename of transform related roles and privileges (#48123) 2019-10-17 10:24:56 +02:00
Martijn van Groningen a5fe69c344
Include enrich into the info api as feature (#48157)
This commit also fixes a bug, the enrich enabled setting
was not included in the list of settings.

Backport of #48109
2019-10-17 09:51:32 +02:00
Lee Hinman 5af66d79ef
Add SLM support to xpack usage and info APIs (#48149)
* Add SLM support to xpack usage and info APIs

This is a backport of #48096

This adds the missing xpack usage and info information into the
`/_xpack` and `/_xpack/usage` APIs. The output now looks like:

```
GET /_xpack/usage
{
  ...
  "slm" : {
    "available" : true,
    "enabled" : true,
    "policy_count" : 1,
    "policy_stats" : {
      "retention_runs" : 0,
      ...
    }
  }
```

and

```
GET /_xpack
{
  ...
  "features" : {
    ...
    "slm" : {
      "available" : true,
      "enabled" : true
    },
    ...
  }
}
```

Relates to #43663

* Fix missing license
2019-10-16 21:06:27 -06:00
David Woods 73f5fef678 Add a note to query_string docs around spaces in field names. (#47326) 2019-10-16 13:28:55 -07:00
James Rodewig 8677653c5b [DOCS] Reformat apostrophe token filter docs (#48076) 2019-10-16 08:51:14 -04:00
François-Clément Brossard f501a4b2b5 Clarify low watermark documentation (#48112)
Today the docs say that the low watermark has no effect on any shards that have
never been allocated, but this is confusing. Here "shard" means "replication
group" not "shard copy" but this conflicts with the "never been allocated"
qualifier since one allocates shard copies and not replication groups.

This commit removes the misleading words. A newly-created replication group
remains newly-created until one of its copies is assigned, which might be quite
some time later, but it seems better to leave this implicit.
2019-10-16 12:27:49 +01:00
Ian Danforth 28c1677341 [DOCS] Fix typo in percentile rank aggregation docs (#47247) 2019-10-15 15:56:45 -04:00
Wilder Pereira 8c73e215b2 [DOCS] Remove unneeded spaces from custom analyzer snippet (#47332) 2019-10-15 15:53:16 -04:00
James Rodewig 601a88bede [DOCS] Sort analyzers, tokenizers, and token filters alphabetically (#48068) 2019-10-15 15:47:25 -04:00
James Rodewig 3a7c2a4d17 [DOCS] Add `wait_for_completion` parm to execute enrich policy API docs (#48077) 2019-10-15 13:47:30 -04:00
Martijn van Groningen aff0c9babc
This commits merges (#48040) the enrich-7.x feature branch,
which is backport merge and adds a new ingest processor, named enrich processor,
that allows document being ingested to be enriched with data from other indices.

Besides a new enrich processor, this PR adds several APIs to manage an enrich policy.
An enrich policy is in charge of making the data from other indices available to the enrich processor in an efficient manner.

Related to #32789
2019-10-15 17:31:45 +02:00
James Rodewig 05b3de0110 [DOCS] Expand `updated` response parm in reindex API docs (#48008) 2019-10-15 11:17:09 -04:00
Marios Trivyzas 7fddf198b7 SQL: Implement DATEDIFF function (#47920)
Implement DATEDIFF/TIMESTAMPDIFF function as per the MS-SQL spec:
https://docs.microsoft.com/en-us/sql/t-sql/functions/datediff-transact-sql?view=sql-server-2017
which allows a user to substract two date/datetime fields and return the
difference in the date/time unit specified.

Closes: #47919
(cherry picked from commit 745699f38dc8222670ffd65b66df33b5da39040b)
2019-10-15 15:12:11 +02:00
James Rodewig a61d12afea [DOCS] Add #46860 to 7.4 release notes (#48018) 2019-10-15 08:48:27 -04:00
Marios Trivyzas 6589617a51
SQL: Fix arg verification for DateAddProcessor (#48041)
Previously, the safety check for the 2nd argument of the DateAddProcessor was
restricting it to Integer which was wrong since we allow all non-rational
numbers, so it's changed to a Number check as it's done in other cases.

Enhanced some tests regarding the check for an integer (non-rational
argument).

(cherry picked from commit 0516b6eaf5eb98fa5bd087c3fece80139a6b118e)
2019-10-15 12:52:11 +02:00
Martijn van Groningen 31e41d4ac2
fixed invalid reference 2019-10-15 10:45:35 +02:00
Tanguy Leroux fe15d9f784
Add documentation for Pause/Resume Auto-Follower APIs (#47985) (#48034)
Relates #47510

Co-Authored-By: James Rodewig <james.rodewig@elastic.co>
2019-10-15 08:38:49 +02:00
David Roberts 984323783e
[ML][7.x] Add lazy assignment job config option (#47993)
This change adds:

- A new option, allow_lazy_open, to anomaly detection jobs
- A new option, allow_lazy_start, to data frame analytics jobs

Both work in the same way: they allow a job to be
opened/started even if no ML node exists that can
accommodate the job immediately. In this situation
the job waits in the opening/starting state until ML
node capacity is available. (The starting state for data
frame analytics jobs is new in this change.)

Additionally, the ML nightly maintenance tasks now
creates audit warnings for ML jobs that are unassigned.
This means that jobs that cannot be assigned to an ML
node for a very long time will show a yellow warning
triangle in the UI.

A final change is that it is now possible to close a job
that is not assigned to a node without using force.
This is because previously jobs that were open but
not assigned to a node were an aberration, whereas
after this change they'll be relatively common.
2019-10-15 06:55:11 +01:00
Martijn van Groningen cc4b6c43b3
Merge remote-tracking branch 'es/7.x' into enrich-7.x 2019-10-15 07:23:47 +02:00
Gordon Brown 300ddfa3c1
SLM Start/Stop HLRC and docs (#47966)
This commit adds HLRC support and documentation for the SLM Start and
Stop APIs, as well as updating existing documentation where appropriate.

This commit also ensures that the SLM APIs are properly included in the
HLRC documentation.
2019-10-14 16:56:31 -06:00
Martijn van Groningen 7fc9198d46
Change how `max_matches` affects `target_field` option. (#47982)
Prior to this change the `target_field` would always be a json array
field in the document being ingested. This to take into account that
multiple enrich documents could be inserted into the `target_field`.

However the default `max_matches` is `1`. Meaning that by default
only a single enrich document would be added to `target_field` json
array field.

This commit changes this; if `max_matches` is set to `1` then the single
document would be added as a json object to the `target_field` and
if it is configured to a higher value then the enrich documents will be
added as a json array (even if a single enrich document happens to be
enriched).
2019-10-14 21:09:48 +02:00
Martijn van Groningen 7cc73f6193
Add HLRC support for enrich execute policy API (#47991)
This PR also includes HLRC docs for the enrich stats api.

Relates to #32789
2019-10-14 19:55:48 +02:00
James Rodewig 170266765b [DOCS] Reformat docs for several snapshot lifecycle policy APIs (#47998) 2019-10-14 12:30:30 -04:00
Tanguy Leroux e4ea8b46b6
Add Pause/Resume Auto-Follower APIs to High Level REST Client (#48004)
This commit adds support for Pause/Resume Auto-Follower APIs 
to the HLRC, with the documentation.

Relates #47510
2019-10-14 18:25:53 +02:00
David Roberts 1ca25bed38
[ML][7.x] Add option to stop datafeed that finds no data (#47995)
Adds a new datafeed config option, max_empty_searches,
that tells a datafeed that has never found any data to stop
itself and close its associated job after a certain number
of real-time searches have returned no data.

Backport of #47922
2019-10-14 17:19:13 +01:00
James Rodewig 5f3ef2e09c [DOCS] Remove unsupported `local` and `master_timeout` parms from cat API docs (#47933) 2019-10-14 11:39:31 -04:00
Martijn van Groningen d4901a71d7
Merge remote-tracking branch 'es/7.x' into enrich-7.x 2019-10-14 10:27:17 +02:00
Tanguy Leroux 742fa818b8
Add Pause/Resume Auto Follower APIs (#47510) (#47904)
This commit adds two APIs that allow to pause and resume
CCR auto-follower patterns:

// pause auto-follower
POST /_ccr/auto_follow/my_pattern/pause

// resume auto-follower
POST /_ccr/auto_follow/my_pattern/resume

The ability to pause and resume auto-follow patterns can be
useful in some situations, including the rolling upgrades of
cluster using a bi-directional cross-cluster replication scheme
(see #46665).

This commit adds a new active flag to the AutoFollowPattern
and adapts the AutoCoordinator and AutoFollower classes so
that it stops to fetch remote's cluster state when all auto-follow
patterns associate to the remote cluster are paused.

When an auto-follower is paused, remote indices that match the
pattern are just ignored: they are not added to the pattern's
followed indices uids list that is maintained in the local cluster
state. This way, when the auto-follow pattern is resumed the
indices created in the remote cluster in the meantime will be
picked up again and added as new following indices. Indices
created and then deleted in the remote cluster will be ignored
as they won't be seen at all by the auto-follower pattern at
resume time.

Backport of #47510 for 7.x
2019-10-13 09:22:51 +02:00
Yogesh Gaikwad ac209c142c
Remove uniqueness constraint for API key name and make it optional (#47549) (#47959)
Since we cannot guarantee the uniqueness of the API key `name` this commit removes the constraint and makes this field optional.

Closes #46646
2019-10-12 22:22:16 +11:00
James Rodewig 256102042b
[DOCS] Reformat get snapshot lifecycle policy API docs (#47827) (#47930) 2019-10-11 13:00:48 -04:00
James Rodewig aa5689f5b3
[DOCS] Reformat put snapshot lifecycle policy API docs (#47811) (#47925) 2019-10-11 11:28:38 -04:00
David Turner ba62eb3dce Allow truncation of clean translog (#47866)
Today the `elasticsearch-shard remove-corrupted-data` tool will only truncate a
translog it determines to be corrupt. However there may be other cases in which
it is desirable to truncate the translog, for instance if an operation in the
translog cannot be replayed for some reason other than corruption. This commit
adds a `--truncate-clean-translog` option to skip the corruption check on the
translog and blindly truncate it.
2019-10-11 15:48:12 +01:00
James Rodewig 58289a857d
[DOCS] Reformat reload search analyzers API docs (#47824) (#47907) 2019-10-11 09:40:49 -04:00
Przemysław Witek d210bfa888
[7.x] Add MlClientDocumentationIT tests for classification. (#47569) (#47896) 2019-10-11 10:19:55 +02:00