Commit Graph

8745 Commits

Author SHA1 Message Date
Benjamin Trent bf8641aa15
[7.x] [ML] calculate cache misses for inference and return in stats (#58252) (#58363)
When a local model is constructed, the cache hit miss count is incremented.

When a user calls _stats, we will include the sum cache hit miss count across ALL nodes. This statistic is important to in comparing against the inference_count. If the cache hit miss count is near the inference_count it indicates that the cache is overburdened, or inappropriately configured.
2020-06-19 09:46:51 -04:00
James Rodewig d8dc638a67
[DOCS] Document get data stream API response body (#58344) (#58360) 2020-06-18 16:42:05 -04:00
James Rodewig b8fa90198b
[DOCS] Prohibit deletion of a data stream's write index (#58341) (#58358) 2020-06-18 16:00:10 -04:00
Lisa Cawley 6680271691 [DOCS] Updates pull and issue release attributes (#58348) 2020-06-18 12:55:02 -07:00
Tal Levy 11086d5c7d
add geo_shape documentation for supported aggregations (#58284) (#58354)
This commit adds documentation for geo_shape fields in aggregations

Closes #55495.
2020-06-18 12:36:24 -07:00
Stuart Tettemer 20abba8433
Scripting: Deprecate general cache settings (#55753) (#58283)
Backport: ef543b0
2020-06-18 11:54:23 -06:00
Jason Tedor be08268562
Allow follower indices to override leader settings (#58103)
Today when creating a follower index via the put follow API, or via an
auto-follow pattern, it is not possible to specify settings overrides
for the follower index. Instead, we copy all of the leader index
settings to the follower. Yet, there are cases where a user would want
some different settings on the follower index such as the number of
replicas, or allocation settings. This commit addresses this by allowing
the user to specify settings overrides when creating follower index via
manual put follower calls, or via auto-follow patterns. Note that not
all settings can be overrode (e.g., index.number_of_shards) so we also
have detection that prevents attempting to override settings that must
be equal between the leader and follow index. Note that we do not even
allow specifying such settings in the overrides, even if they are
specified to be equal between the leader and the follower
index. Instead, the must be implicitly copied from the leader index, not
explicitly set by the user.
2020-06-18 11:56:06 -04:00
James Rodewig 9ba1b1d067
[DOCS] Reformat data stream API docs (#58322) (#58334) 2020-06-18 10:59:12 -04:00
Marios Trivyzas 50b391e91b
SQL: [Docs] Fix TIME_PARSE documentation (#58182) (#58317)
TIME_PARSE works correctly if both date and time parts are specified,
and a TIME object (that contains only time is returned).

Adjust docs and add a unit test that validates the behavior.

Follows: #55223
(cherry picked from commit 9d6b679a5da88f3c131b9bdba49aa92c6c272abe)
2020-06-18 16:09:13 +02:00
Dan Hermann 3b511fd829
[DOCS] Add data stream APIs to main API page (#58204) (#58325) 2020-06-18 08:41:43 -05:00
Dan Hermann a2837097ff
[DOCS] Move some docs about data streams from the create page to the intro page 2020-06-18 08:24:06 -05:00
James Rodewig 64fb326637
[DOCS] Add data streams to search docs (#58278) (#58320)
Changes:

* Adds additional examples to the `Search a data stream` section of
  `Use a data stream`
* Updates existing search docs to make them aware of data streams
2020-06-18 08:59:00 -04:00
Jim Ferenczi 82db0b575c
Allow index filtering in field capabilities API (#57276) (#58299)
This change allows to use an `index_filter` in the
field capabilities API. Indices are filtered from
the response if the provided query rewrites to `match_none`
on every shard:

````
GET metrics-*
{
  "index_filter": {
    "bool": {
      "must": [
        "range": {
          "@timestamp": {
            "gt": "2019"
          }
        }
      }
  }
}
````

The filtering is done on a best-effort basis, it uses the can match phase
to rewrite queries to `match_none` instead of fully executing the request.
The first shard that can match the filter is used to create the field
capabilities response for the entire index.

Closes #56195
2020-06-18 10:23:26 +02:00
Yannick Welsch ffeff4090e Add new flag to check whether alias exists on remove (#58100)
This allows doing true CAS operations on aliases, making sure that an alias is actually properly
moved from a given source index onto a given target index. This is useful to ensure that an
alias is actually moved from a given index to another one, and not just added to another index.
2020-06-18 10:15:26 +02:00
James Rodewig 8b99a891a8 [DOCS] Fix typo in create data stream API docs 2020-06-17 17:15:50 -04:00
James Rodewig 4ab9aea965 [DOCS] Remove redundant links in data stream docs 2020-06-17 17:08:19 -04:00
James Rodewig 5e0b00f022
[DOCS] Fix routing param in search API docs (#58267) (#58288) 2020-06-17 15:19:53 -04:00
James Rodewig 01043eb8aa
[7.x] [DOCS] Add 'update/delete docs in a data stream' tutorial (#58194) (#58264)
Adds a tutorial for updating and deleting documents in the backing
indices of a data stream.
2020-06-17 12:41:24 -04:00
Dan Hermann 4962a91157
Document that data stream write indices cannot be closed 2020-06-17 10:39:58 -05:00
Przemyslaw Gomulka 9894d90e0b
[doc] known issues - week based patterns not working in 7.6 (#58099) (#58227)
relates #57128
# Conflicts:
#	docs/reference/release-notes/7.6.asciidoc
2020-06-17 10:54:22 +02:00
Lisa Cawley 46d797b1d9 [DOCS] Fixes license management links (#58213) 2020-06-16 16:49:48 -07:00
debadair cfef2b2bec
[DOCS] Removed unused pages (#58209) 2020-06-16 15:55:56 -07:00
Stuart Tettemer 01795d1925
Revert "Scripting: Deprecate general cache settings (#55753)" (#58201)
This reverts commit 88e8b34fc2.
2020-06-16 14:58:18 -06:00
James Rodewig ce22e951f8
[DOCS] Add resolve index API check to DS setup tutorial (#58167) (#58197)
Updates the set up a data stream tutorial to include a name check
using the resolve index API.
2020-06-16 16:28:42 -04:00
James Rodewig c548a87673
[DOCS] Add 'Change DS mappings and settings' tutorial (#58148) (#58195)
Adds a tutorial for updating the mappings and
index settings of a data stream's backing indices.
2020-06-16 16:20:32 -04:00
Stuart Tettemer 88e8b34fc2
Scripting: Deprecate general cache settings (#55753)
Backport: ef543b0
2020-06-16 13:06:59 -06:00
debadair 276a4898ba
[DOCS] Fixes problematic terminology (#58184)
* [DOCS] Fixes problematic terminology (#58178)

* [DOCS] Fixes problematic terminology.

* Update docs/reference/snapshot-restore/register-repository.asciidoc

Co-authored-by: James Rodewig <james.rodewig@elastic.co>
2020-06-16 11:43:22 -07:00
debadair 2edcd064fe
[DOCS] Fix bad xref (#58150) 2020-06-15 15:50:49 -07:00
Jake Landis dc7ffb154a
Update hh to HH in date processor example (#58089) (#58144)
Co-authored-by: Leaf-Lin <39002973+Leaf-Lin@users.noreply.github.com>
2020-06-15 17:04:14 -05:00
Adam Locke ad0364dc06
[DOCS] Add documentation for near real-time search (#57560) (#58138)
* Adding documentation for near real-time search.

* Adding link to NRT topic and clarifying some text.

* Adding diagrams and incorporating changes from David T.
2020-06-15 16:42:57 -04:00
debadair 80524098fc
[DOCS] Reformat release highlights as What's new. (#58073) 2020-06-15 13:26:03 -07:00
James Rodewig e268a89ef2 [DOCS] Fix typo in data stream docs 2020-06-15 12:59:36 -04:00
Andrei Dan 3635bd741c
[DOCS] Make ILM documentation data stream aware (#58035) (#58110)
Co-authored-by: James Rodewig <james.rodewig@elastic.co>
Co-authored-by: Lee Hinman <dakrone@users.noreply.github.com>
(cherry picked from commit 25cbbe56dd29fbee2efe8040e9c8b92d168cb670)
Signed-off-by: Andrei Dan <andrei.dan@elastic.co>
2020-06-15 15:16:14 +01:00
James Rodewig 0bc7c4f69e [DOCS] Fix xref in data stream docs 2020-06-15 09:49:18 -04:00
Dan Hermann 8a910443c4
Add ignore_empty_value parameter in set ingest processor (#57030) (#58108) 2020-06-15 08:35:08 -05:00
István Zoltán Szabó c3e6aa65dc [DOCS] Adds web session details example to painless transform examples (#57942) 2020-06-15 15:19:02 +02:00
István Zoltán Szabó 3a5ee4476d Merge branch '7.x' of github.com:elastic/elasticsearch into 7.x 2020-06-15 15:18:39 +02:00
James Rodewig 66c33c8c96 [DOCS] Update prohibited ops for ds write index
Adds 'clone' and 'split' to a list of actions that are prohibited on a data
stream's write index
2020-06-15 08:57:33 -04:00
James Rodewig 9392210cb5
[DOCS] Add get and delete steps to data stream setup tutorial (#58068) (#58107)
Adds corresponding steps for the get and delete data stream APIs to the
data stream setup tutorial. Also provides some guidance on how to
determine the current write index for a data stream.
2020-06-15 08:52:43 -04:00
Gordon Brown a33b2f9956
Clarify `expand_wildcard` option in snapshot docs (#58016)
This commit clarifies that the `expand_wildcards` option (as well as other
`IndicesOptions` parameters) can be used with the Create Snapshot API, but that
they must be in the body of the request.

Also clarifies the connection between `expand_wildcards` and hidden indices as
it relates to snapshots.
2020-06-12 16:38:52 -06:00
James Rodewig 0906709598 [DOCS] Reword prohibited ops for data stream write index 2020-06-12 14:00:24 -04:00
Matthias Gärtner 65ca575632 Off-by-ones on data sizes in documentation (#57735)
Fixes exponent off-by-ones in Painless documentation for int and long.
2020-06-12 09:59:15 -07:00
James Rodewig d0d8ef7270
[DOCS] Document reindex for data streams (#58013) (#58051)
Changes:

* Adds new 'Reindex with a data stream' section to 'Use a data stream'

* Makes the existing reindex API docs aware of data streams

* Rewrites the reindex glossary definition to include data streams
2020-06-12 11:21:31 -04:00
James Rodewig 82493615ce
[DOCS] Add data streams glossary def (#58014) (#58049) 2020-06-12 10:39:51 -04:00
James Rodewig 4539f15b90
[DOCS] Remove 7.8.0 coming tags (#58047) 2020-06-12 10:17:40 -04:00
Rory Hunter e840ffa300 Add release notes for 7.8.0 (#56340)
Co-authored-by: James Rodewig <james.rodewig@elastic.co>
Co-authored-by: István Zoltán Szabó <istvan.szabo@elastic.co>
Co-authored-by: Tim Vernum <tim@adjective.org>
Co-authored-by: lcawl <lcawley@elastic.co>
2020-06-12 10:03:41 -04:00
Martijn van Groningen 01d8bb8cfa
Enforce valid field mapping exists for timestamp_field in templates. (#58036)
Backport of #57741 to 7.x branch.

Relates to #53100
2020-06-12 15:24:42 +02:00
James Rodewig 922423c52c [DOCS] Remove unneeded comma in data stream docs 2020-06-12 09:14:56 -04:00
David Kyle 39020f3900
HLRC for delete expired data by job Id (#57722) (#57975)
High level rest client changes for #57337
2020-06-12 09:44:17 +01:00
Martijn van Groningen c8031c6f99
Add data stream support to the reindex api. (#57970)
Backport of #57870 to 7.x branch.

This change now also copies the op_type from the reindex request's destination index request to the actual index request being used in the bulk request.

For ensuring no document exists, the op_type create doesn't need to be copied, since Versions.MATCH_DELETED will copied from the 'mainRequest.getDestination().version()'.
The `version()` method on IndexRequest only returns Versions.MATCH_DELETED if op_type=create and no specific version has been specified.

However in order to be able to index into a data stream, the op_type must be create. So in order to support that the op_type must be copied from the reindex request's destination index request to the actual index request being used in the bulk request.

Relates to #53100 and #57788
2020-06-12 09:54:37 +02:00
James Rodewig bf90b6f221 [DOCS] Remove extra word from data stream docs 2020-06-11 17:44:59 -04:00
James Rodewig 1814b66a69 [DOCS] Fix typos in data stream docs 2020-06-11 16:21:09 -04:00
Lisa Cawley 7442808869 [DOCS] Rename monitoring collection from internal to legacy (#56395) 2020-06-11 10:21:01 -07:00
James Rodewig c36df27730
[DOCS] Reformat `pattern_replace` token filter (#57699) (#57995)
Changes:

* Rewrites description and adds Lucene link
* Adds analyze example
* Adds parameter definitions
* Adds custom analyzer example
2020-06-11 12:19:38 -04:00
James Rodewig 6fc8317f07
[DOCS] Reformat data streams intro and overview (#57954) (#57993)
Changes:

* Updates 'Data streams' intro page to focus on problem solution and
  benefits.

* Adds 'Data streams overview' page to cover conceptual information,
  based on existing content in the 'Data streams' intro.

* Adds diagrams for data streams and search/indexing request examples.

* Moves API jump list and API docs to a new 'Data streams APIs' section.
  Links to these APIs will be available through tutorials.

* Add xrefs to existing docs for concepts like generation, write index,
  and append-only.
2020-06-11 11:32:09 -04:00
James Rodewig 4e738f60f8 [DOCS] Fix typo in data stream docs 2020-06-11 11:30:00 -04:00
James Rodewig d534862d41
[DOCS] Move search API's `docvalue_fields` examples (#57760) (#57989)
Changes:

* Condenses and relocates the `docvalue_fields` example to the 'Run a search'
   page.
* Adds docs for the `docvalue_fields` request body parameter.
* Updates several related xrefs.

Co-authored-by: debadair <debadair@elastic.co>
2020-06-11 11:25:04 -04:00
David Turner 9b52a250f8 Add admonition to cluster state instability note (#57985)
We document that the cluster state API is an internal representation which may
change, but apparently not emphatically enough. This commit adds a `NOTE:`
admonition to this paragraph.
2020-06-11 15:32:18 +01:00
Igor Motov 947573f309
Added standard deviation / variance sampling to extended stats (#49782) (#57947)
Per 49554 I added standard deviation sampling and variance sampling to the extended stats interface.
 
Closes #49554

Co-authored-by: Igor Motov <igor@motovs.org>

Co-authored-by: andrewjohnson2 <aj114114@gmail.com>
2020-06-11 09:19:44 -04:00
James Rodewig 9eb8085ac0
[DOCS] Reformat data stream tutorial docs (#57883) (#57946)
Creates a new page for a 'Set up a data stream' tutorial, based on
existing content in 'Data streams'.

Also adds tutorials for:

* Configuring an ILM policy for a data stream
* Indexing documents to a data stream
* Searching a data stream
* Manually rolling over a data stream
2020-06-10 14:03:46 -04:00
James Rodewig 44c3bb29e2 [DOCS] EQL: Correct EQL search API's `size` param def
The `size` parameter can be used to limit matching events or sequences.
2020-06-10 10:12:54 -04:00
Hendrik Muhs 95bd7b63b0 [Transform] fix page size return in cat transform, add dps (#57871)
fixes the page size reported after moving page size to settings(#56007) and
adds documents per second(throttling) to the output.

fixes #56498
2020-06-10 08:10:25 +02:00
Simon 18fc4395c6 [DOCS] Fix incorrect AD realm setting (#57520) 2020-06-09 16:56:19 -07:00
Jake Landis a370d5eead
[7.x] Ensure Joni warning are logged at debug (#57302) (#57897)
When Joni, the regex engine that powers grok emits a warning it
does so by default to System.err. System.err logs are all bucketed
together in the server log at WARN level. When Joni emits a warning,
it can be extremely verbose, logging a message for each execution
again that pattern. For ingest node that means for every document
that is run that through Grok. Fortunately, Joni provides a call
back hook to push these warnings to a custom location.

This commit implements Joni's callback hook to push the Joni warning
to the Elasticsearch server logger (logger.org.elasticsearch.ingest.common.GrokProcessor)
at debug level. Generally these warning indicate a possible issue with
the regular expression and upon creation of the Grok processor will
do a "test run" of the expression and log the result (if any) at WARN 
level. This WARN level log should only occur on pipeline creation which 
is a much lower frequency then every document. 

Additionally, the documentation is updated with instructions for how
to set the logger to debug level.
2020-06-09 17:06:29 -05:00
James Rodewig 9bcb41a39e
[DOCS] Fix source-related search API params (#57691) (#57889)
Cleans up the reference documentation for the following
search API parameters:

* `_source` query parameter
* `_source_excludes` query parameter
* `_source_includes` query parameter
* `_source` request body parameter
* `hits._source` response property
2020-06-09 13:01:17 -04:00
James Rodewig 24a50eb3af
[DOCS] Reformat `mapping` charfilter (#57818) (#57885)
Changes:

* Adds title abbreviation
* Adds Lucene link to description
* Adds standard headings
* Simplifies analyze example
* Simplifies analyzer example and adds contextual text
2020-06-09 12:43:01 -04:00
Andrew Wilkins c39a8d20cb
[DOCS] Fix typo in ILM rollover docs (#57856) 2020-06-09 12:15:36 -04:00
Dan Hermann b501b282f8
Change default backing index naming scheme 2020-06-09 09:31:34 -05:00
Dan Hermann 60db6c7c45
[7.x] Update rollover index API docs for data streams 2020-06-09 06:57:16 -05:00
bellengao 19042ddf3b [DOCS] Fix typo in remote-clusters doc (#57820) 2020-06-09 11:28:30 +02:00
Benjamin Trent d5522c2747
[ML] add new circuit breaker for inference model caching (#57731) (#57830)
This adds new plugin level circuit breaker for the ML plugin.

`model_inference` is the circuit breaker qualified name.

Right now it simply adds to the breaker when the model is loaded (and possibly breaking) and removing from the breaker when the model is unloaded.
2020-06-08 16:02:48 -04:00
Lee Hinman 6e8cf0973f
[7.x] Disallow merging existing mapping field definitions in templates (#57701) (#57822)
Backports the following commits to 7.x:

    Disallow merging existing mapping field definitions in templates (#57701)
2020-06-08 12:56:09 -06:00
Przemysław Witek 7a1300a09e
[7.x] Make ModelPlotConfig.annotations_enabled default to ModelPlotConfig.enabled if unset (#57808) (#57815) 2020-06-08 17:41:12 +02:00
James Rodewig 63e962c99a [DOCS] Fix typo in `html_strip` char filter docs 2020-06-08 10:37:35 -04:00
James Rodewig 2ef30217e0 [DOCS] Fix typo in data stream docs 2020-06-08 10:19:39 -04:00
István Zoltán Szabó 536dadc4e0 [DOCS] Extends geo_shape query docs to explain behavior when data is indexed as array of shapes (#57806)
Co-authored-by: James Rodewig <james.rodewig@elastic.co>
2020-06-08 15:06:10 +02:00
David Turner 663702e609 Small improvements to resilience design docs (#57791)
A follow-up to #47233 to clarify a few points.
2020-06-08 13:50:41 +01:00
James Rodewig 2121eb528c
[DOCS] Reformat `html_strip` charfilter (#57764) (#57810)
Changes:

* Converts title to sentence case
* Adds a title abbreviation
* Adds Lucene link to description
* Reformat sections
2020-06-08 08:44:43 -04:00
István Zoltán Szabó 37293a9cf6 [DOCS] Adds security privileges to _scripts endpoint docs. (#57795) 2020-06-08 14:35:56 +02:00
David Kyle 08d1286de7
[7.x] Delete expired data by job (#57337) (#57796)
Deleting expired data can take a long time leading to timeouts if there
are many jobs. Often the problem is due to a few large jobs which 
prevent the regular maintenance of the remaining jobs. This change adds
a job_id parameter to the delete expired data endpoint to help clean up
those problematic jobs.
2020-06-08 13:00:23 +01:00
James Rodewig 6c93fed204
[DOCS] Document `doc_as_upsert` does not support ingest pipelines (#57649) (#57783)
Co-authored-by: James Rodewig <james.rodewig@elastic.co>

Co-authored-by: Asfaloth <asfalots@users.noreply.github.com>
2020-06-06 16:10:42 -04:00
David Roberts 1d64d55a86
[7.x][ML] Add per-partition categorization option (#57723)
This PR adds the initial Java side changes to enable
use of the per-partition categorization functionality
added in elastic/ml-cpp#1293.

There will be a followup change to complete the work,
as there cannot be any end-to-end integration tests
until elastic/ml-cpp#1293 is merged, and also
elastic/ml-cpp#1293 does not implement some of the
more peripheral functionality, like stop_on_warn and
per-partition stats documents.

The changes so far cover REST APIs, results object
formats, HLRC and docs.

Backport of #57683
2020-06-06 08:15:17 +01:00
debadair 100d2bd063
[DOCS] Editorial ILM cleanup (#57565) (#57776)
* [DOCS] Editorial cleanup

* Moved example of applying a template to multiple indices.

* Combine existing indices topics

* Fixed test

* Add skip rollover file.

* Revert rename.

* Update include.

* Revert rename

* Apply suggestions from code review

Co-authored-by: Adam Locke <adam.locke@elastic.co>
Co-authored-by: Lee Hinman <dakrone@users.noreply.github.com>

* Apply suggestions from code review

* Fixed callout

* Update docs/reference/ilm/ilm-with-existing-indices.asciidoc

Co-authored-by: Lee Hinman <dakrone@users.noreply.github.com>

* Update docs/reference/ilm/ilm-with-existing-indices.asciidoc

Co-authored-by: Lee Hinman <dakrone@users.noreply.github.com>

* Apply suggestions from code review

* Restored policy to template example.

* Fixed JSON parse error

Co-authored-by: Adam Locke <adam.locke@elastic.co>
Co-authored-by: Lee Hinman <dakrone@users.noreply.github.com>

Co-authored-by: Adam Locke <adam.locke@elastic.co>
Co-authored-by: Lee Hinman <dakrone@users.noreply.github.com>
2020-06-05 18:55:51 -07:00
James Rodewig 26d2b4f871
[DOCS] Fix typo in CCR overview docs (#57709) (#57773)
Co-authored-by: bellengao <gbl_long@163.com>
2020-06-05 17:42:49 -04:00
James Rodewig bc921ea17c
[DOCS] Add docs for designing resilient clusters (#47233) (#57743)
Adds some guidance for designing clusters to be resilient to
failures, including example architectures.

Co-authored-by: James Rodewig <james.rodewig@elastic.co>

Co-authored-by: David Turner <david.turner@elastic.co>
2020-06-05 12:08:45 -04:00
Adam Locke fe558f6373
[DOCS] Clarifying env variable substitution (#57370) (#57737)
* Clarifying environment variable substitution in the ES configuration YAML
* Update code snippet
* Remove extraneous quotes from string example
* Incorporating review feedback
2020-06-05 11:08:03 -04:00
Dimitris Athanasiou f49a14ce6f
[7.x][ML] Fix race condition when force stopping DF analytics job (#57680) (#57717)
When we force delete a DF analytics job, we currently first force
stop it and then we proceed with deleting the job config.
This may result in logging errors if the job config is deleted
before it is retrieved while the job is starting.

Instead of force stopping the job, it would make more sense to
try to stop the job gracefully first. So we now try that out first.
If normal stop fails, then we resort to force stopping the job to
ensure we can go through with the delete.

In addition, this commit introduces `timeout` for the delete action
and makes use of it in the child requests.

Backport of #57680
2020-06-05 17:50:01 +03:00
Martijn van Groningen c407b0f40d
[DOCS] Add data stream overview and intro (#57724)
Backporting #57596 to 7.x branch.

Added data streams overview page and
an introduction to data streams.

Relates to #53100

Co-authored-by: Dan Hermann danhermann@users.noreply.github.com
Co-authored-by: James Rodewig james.rodewig@elastic.co
2020-06-05 16:10:03 +02:00
Jun Ohtani c75c8b6e9d Expose discard_compound_token option to kuromoji_tokenizer (#57421)
This commit exposes the new Lucene option `discard_compound_token` to the Elasticsearch Kuromoji plugin.
2020-06-05 15:41:01 +02:00
James Rodewig b03a83a69d
[DOCS] Fix source filtering xrefs (#57720) (#57725) 2020-06-05 09:05:30 -04:00
DU-ds 925c01f1d7 add jvm clarification (#57460)
Emphasise in the Docker documentation that although the default heap size is
1GB, the docker-compose.yml example specifies 512MB.
2020-06-05 11:48:15 +01:00
William Brafford dfb6def3da Revert "Restore xpack.ilm.enabled and xpack.slm.enabled settings (#57383)"
This reverts commit 7a67fb2d04.
2020-06-04 16:25:05 -04:00
James Rodewig 9c7a5c7b83
[DOCS] Move source filtering examples (#57689) (#57695)
Moves the source filtering example snippets form the "Request body
search" API docs page to the "Return fields in a search" section of the
"Run a search" page.
2020-06-04 15:34:10 -04:00
Ioannis Kakavas af9f9d7f03
[7.x] Add http proxy support for OIDC realm (#57039) (#57584)
This change introduces support for using an http proxy for egress
communication of the OpenID Connect realm.
2020-06-04 20:51:00 +03:00
William Brafford 7a67fb2d04
Restore xpack.ilm.enabled and xpack.slm.enabled settings (#57383)
In #55592 and #55416, we deprecated the settings for enabling and disabling
basic license features and turned those settings into no-ops. Since doing so,
we've had feedback that this change may not give users enough time to cleanly
switch from non-ILM index management tools to ILM. If two index managers
operate simultaneously, results could be strange and difficult to
reconstruct. We don't know of any cases where SLM will cause a problem, but we
are restoring that setting as well, to be on the safe side.

This PR is not a strict commit reversion. First, we are keeping the new
xpack.watcher.use_ilm_index_management setting, introduced when
xpack.ilm.enabled was made a no-op, so that users can begin migrating to using
it. Second, the SLM setting was modified in the same commit as a group of other
settings, so I have taken just the changes relating to SLM.
2020-06-04 13:38:22 -04:00
James Rodewig 2104b0503c
[DOCS] Reformat whitespace in search API docs (#57667) (#57669)
Changes the search API docs to use:

* Consistent indentation in param definitions
* Two-space indentation in JSON snippets
2020-06-04 10:02:39 -04:00
Przemysław Witek 6b5f49d097
[7.x] Introduce ModelPlotConfig. annotations_enabled setting (#57539) (#57641) 2020-06-04 15:15:35 +02:00
Martijn van Groningen 4b5c4b7966
[DOCS] Add 7.8 release notes entry for auto create index change (#57582)
The create index action name (`indices:admin/create`) can no longer be used to grant privileges to auto create indices and instead the `create_index` builtin privilege should be used.

Relates to #55858

Co-authored-by: Jake Landis <jake.landis@elastic.co>
2020-06-04 07:36:07 +02:00
debadair f616c7c177
[DOCS] Fix chunking in template API docs (#57632) (#57636)
* [DOCS] Fix chunking in template API docs

* Fixed typo in xref

* Added anchor for beats xref

* Fixed example
2020-06-03 18:15:39 -07:00
Igor Motov 8d7f389f3a
Increase search.max_buckets to 65,535 (#57042)
Increases the default search.max_buckets limit to 65,535, and only counts
buckets during reduce phase.

Closes #51731
2020-06-03 15:35:41 -04:00
Julie Tibshirani 4f4c4a8713 Add a reference on returning fields during a search. (#57500)
This PR adds a section to the new 'run a search' reference that explains
the options for returning fields. Previously each option was only listed as a
separate request parameter and it was hard to know what was available.
2020-06-03 09:41:48 -07:00
James Rodewig 80faafdfc7
[DOCS] Add clear scroll API reference docs (#57367) (#57611) 2020-06-03 11:58:16 -04:00
Marios Trivyzas a674844893
SQL: Implement TRIM function (#57518) (#57593)
Add `TRIM` function which combines the functionality of both
`LTRIM` and `RTRIM` by stripping both leading and trailing
whitespaces.

Refers to #41195

(cherry picked from commit 6c86c919e12f0c4cb5e39d129aa65ab3e274268f)
2020-06-03 15:19:48 +02:00
William Brafford 6e67f1b3dc
[DOCS] Add release notes for 7.7.1 (#57566)
Co-authored-by: James Rodewig <james.rodewig@elastic.co>
2020-06-03 08:54:32 -04:00
Tal Levy d638ecda68
fix missing _doc type from geo-shape doc test (#57556) 2020-06-02 13:44:08 -07:00
Jess dc12a687be
[DOCS] Edit validation section of dynamic templates docs (#57510) 2020-06-02 16:28:06 -04:00
Tal Levy f360020048 Update geo_point docs for geo_shape queries (#57487)
This commit highlights the ability for geo_point fields to be
used in geo_shape queries. It also adds an explicit geo_point
example in the geo_shape query documentation

Closes #56927.
2020-06-02 12:56:54 -07:00
James Rodewig a2e44a0c76
[DOCS] Refactor admons for multi-parameter options (#57491) (#57540)
Several APIs support options that can be specified as a query parameter or a
request body parameter.

Currently, this is documented using notes, which can get rather lengthy. This
replaces those multiple notes with a single note and a footnote.
2020-06-02 12:12:29 -04:00
James Rodewig 5950d1aec8 [DOCS] Fix heading capitalization in CCS docs 2020-06-02 11:13:13 -04:00
James Rodewig 641ed484d8
[DOCS] EQL: Add `dev` admonition to EQL pages (#57531) (#57533)
Adds the `dev` admonition to EQL features, which are in development
under a feature flag.
2020-06-02 11:03:12 -04:00
James Rodewig 808835ac1c
[DOCS] Add scroll API reference docs (#57153) (#57528)
Changes:

* Adds API reference docs for the scroll API
* Documents several related parameters in the search API docs
2020-06-02 10:11:12 -04:00
James Rodewig fd6dabf158
[DOCS] EQL: Fix hits param for sequences (#57410) (#57524) 2020-06-02 09:38:00 -04:00
Marios Trivyzas b8a13de20f
SQL: Implement TOP as an alternative to LIMIT (#57428) (#57507)
Add basic support for `TOP X` as a synonym to LIMIT X which is used
by [MS-SQL server](https://docs.microsoft.com/en-us/sql/t-sql/queries/top-transact-sql?view=sql-server-ver15),
e.g.:

```
SELECT TOP 5 a, b, c FROM test
```

TOP in SQL server also supports the `PERCENTAGE` and `WITH TIES`
keywords which this implementation doesn't.

Don't allow usage of both TOP and LIMIT in the same query.

Refers to #41195

(cherry picked from commit 2f5ab81b9ad884434d1faa60f4391f966ede73e8)
2020-06-02 10:53:42 +02:00
Julie Tibshirani e434c481dc Avoid unnecessary use of stored_fields in our docs. (#57488)
Generally we don't advocate for using `stored_fields`, and we're interested in
eventually removing the need for this parameter. So it's best to avoid using
stored fields in our docs examples when it's not actually necessary.

Individual changes:
* Avoid using 'stored_fields' in our docs.
* When defining script fields in top-hits, de-emphasize stored fields.
2020-06-01 17:31:42 -07:00
Lisa Cawley db5bf92acf
[7.x][DOCS] Replace docdir attribute with es-repo-dir (#57489) (#57494) 2020-06-01 16:42:53 -07:00
James Rodewig 6592c3856d [DOCS] Fix deep paging recommendations
Corrects recommendation to reference the `search_after` parameter,
not API.

Also corrects a typo and whitespace inconsistencies in the search docs.
2020-06-01 18:04:04 -04:00
Lisa Cawley a1514c9ffe
[DOCS] Replaces docdir attributes in ML APIs (#57390) (#57467) 2020-06-01 13:46:15 -07:00
James Rodewig 994781ff36
[DOCS] Add search pagination docs (#56785) (#57477)
Reworks the `from / size` content to `Paginate search results`.

Moves those docs from the request body search API page (slated for
deletion) to the `Run a search` tutorial docs.

Also adds some notes to the `from` and `size` param docs.

Co-authored-by: debadair <debadair@elastic.co>
2020-06-01 16:43:06 -04:00
James Rodewig ab8ae7cf25
[DOCS] Combine search API and URI search API reference docs (#55884) (#57469)
The search API and URI search pages document the same `_search` API.
This combines the documentation from each page under the search API
docs.
2020-06-01 15:53:40 -04:00
Benjamin Trent 1aea9d5f49
Adding transform docs for geotile_grid (#57000) (#57474)
transforms and composite aggs support geotile_grid as a source. This adds documentation explaining that support.
2020-06-01 15:46:37 -04:00
Zachary Tong 4dc12633cf
Better description of real-memory breaker changes to aggs (#57306)
The old description mentions a setting that we ended up not merging.
The periodic real-memory checks are automatic and do not require
the user to configure any setting.
2020-06-01 14:58:20 -04:00
James Rodewig 50a8779c94
[DOCS] Create top-level "Search your data" page (#56058) (#57463)
**Goal**

Create a top-level search section. This will let us clean up our search
API reference docs, particularly content from [`Request body search`][0].

**Changes**

* Creates a top-level `Search your data` page. This page is designed to
  house concept and tutorial docs related to search.

* Creates a `Run a search` page under `Search your data`. For now, This
  contains a basic search tutorial. The goal is to add content from
  [`Request body search`][0] to this in the future.

* Relocates `Long-running searches` and `Search across clusters` under
  `Search your data`. Increments several headings in that content.

* Reorders the top-level TOC to move `Search your data` higher. Also
  moves the `Query DSL`, `EQL`, and `SQL access` chapters immediately
  after.

Relates to #48194

[0]: https://www.elastic.co/guide/en/elasticsearch/reference/master/search-request-body.html
2020-06-01 14:55:26 -04:00
James Rodewig 12813d5918
[DOCS] Document `dynamic` and `static` setting types (#56919) (#57457) 2020-06-01 14:19:52 -04:00
James Rodewig cde5b7d2b3
[DOCS] Relocate discovery module content (#56611) (#57454)
* Moves `Discovery and cluster formation` content from `Modules` to
`Set up Elasticsearch`.

* Combines `Adding and removing nodes` with `Adding nodes to your
  cluster`. Adds related redirect.

* Removes and redirects the `Modules` page.

* Rewrites parts of `Discovery and cluster formation` to remove `module`
  references and meta references to the section.
2020-06-01 14:13:13 -04:00
James Rodewig 6934264162
[DOCS] Relocate `shard allocation` module content (#56535) (#57448) 2020-06-01 13:15:08 -04:00
Lisa Cawley b5542e0480
[DOCS] Adds ml-cpp PRs to release notes (#57444) 2020-06-01 09:56:32 -07:00
James Rodewig 5c12a1f063
[DOCS] Relocate `local gateway` setting content (#56448) (#57443)
Moves `local gateway` setting content to `Configuring Elasticsearch`
from `Modules`.

Relates to #53303.
2020-06-01 12:40:15 -04:00
James Rodewig 2ea5372eb7
[DOCS] Relocate `HTTP` module content (#56386) (#57439)
Moves `HTTP` content from the [Modules][0] section to the
[Configuring Elasticsearch][1] section.

Supporting changes:
* Replaces `http` with `HTTP` throughout
* Replaces `HTTP module` with `HTTP layer` throughout
* Increments several headings
* Adds explicit anchors to several headings
* Removes several unneeded `[float]` attributes

Closes #53306

[0]: https://www.elastic.co/guide/en/elasticsearch/reference/master/modules.html
[1]: https://www.elastic.co/guide/en/elasticsearch/reference/master/settings.html
2020-06-01 12:22:55 -04:00
James Rodewig 4dd05ccf9e
[DOCS] Relocate `node` module content (#56376) (#57435)
Moves `node` content from the [Modules][0] section to the
[Configuring Elasticsearch][1] section.

Supporting changes:
* Retitles page `Node settings`
* Increments several headings
* Removes several unneeded `[float]` attributes
* Replaces remaining `[float]` attributes with `[discrete]`
* Updates the `Other node settings` section

Relates to #53307

[0]: https://www.elastic.co/guide/en/elasticsearch/reference/master/modules.html
[1]: https://www.elastic.co/guide/en/elasticsearch/reference/master/settings.html
2020-06-01 12:04:15 -04:00
James Rodewig 131ce74840
[DOCS] Relocate thread pools content (#55814) (#57432)
Moves [thread pools content][0] from [Modules][1] to
[Configuring Elasticsearch][2].

Supporting changes:
* Changes page title to "Thread pools"
* Increments several headings
* Removes several unneeded `[float]` attributes
* Updates the anchors of several headings

Relates to #53307

[0]: https://www.elastic.co/guide/en/elasticsearch/reference/master/modules-threadpool.html
[1]: https://www.elastic.co/guide/en/elasticsearch/reference/master/modules.html
[2]: https://www.elastic.co/guide/en/elasticsearch/reference/master/settings.html

Co-authored-by: debadair <debadair@elastic.co>
2020-06-01 11:46:16 -04:00
James Rodewig 462a39bc2a
[7.x] [DOCS] Relocate network settings content (#55865) (#57429)
* [DOCS] Relocate network settings content (#55865)

Moves [network settings content][0] from [Modules][1] to the
[Configuring Elasticsearch][2] section.

Supporting changes:
* Changes page title and headings to sentence case
* Increments several headings
* Removes several unneeded `[float]` and `[horizontal]` attributes
* Removes several unneeded blank links in definition lists

[0]: https://www.elastic.co/guide/en/elasticsearch/reference/master/modules-network.html
[1]: https://www.elastic.co/guide/en/elasticsearch/reference/master/modules.html
[2]: https://www.elastic.co/guide/en/elasticsearch/reference/master/settings.html

* tweak for consistency
2020-06-01 11:27:27 -04:00
James Rodewig 1fcd73cd34
[DOCS] Relocate `transport` module content (#55472) (#57425)
Moves `transport` content from the [Modules][0] section to the
[Configuring Elasticsearch][1] section.

Supporting changes:
* Adds explicit anchors to several headings
* Changes several headings to sentence case
* Increments several headings
* Removes several unneeded `[float]` attributes
* Replaces `transport module` with `transport layer`

Relates to #53305

[0]: https://www.elastic.co/guide/en/elasticsearch/reference/master/modules.html
[1]: https://www.elastic.co/guide/en/elasticsearch/reference/master/settings.html
2020-06-01 10:56:28 -04:00
James Rodewig de5de4bf58
[DOCS] Relocate "Remote Clusters" page (#55172) (#57422)
Relocates the "Remote Clusters" documentation from the "Modules" section to the "Set up Elasticsearch" section.

Supporting changes:
* Reorders the "Bootstrap checks for X-Pack" section to immediately follow the "Bootstrap checks"chapter.
* Removes an outdated X-Pack `idef` from the "Remote Clusters" intro.
2020-06-01 10:28:34 -04:00
James Rodewig 8c0e7c8a6e
[DOCS] Relocate "Plugins" page (#54974) (#57418) 2020-06-01 10:07:50 -04:00
James Rodewig 43ef469570
[DOCS] Relocate `indices` module content (#54903) (#57413)
Moves `indices` content from the [Modules][0] section to the [Configuring
Elasticsearch][1] section.

Also removes the [Indices][2] landing page and adds a related redirect.

[0]: https://www.elastic.co/guide/en/elasticsearch/reference/master/modules.html
[1]: https://www.elastic.co/guide/en/elasticsearch/reference/master/settings.html
[2]: https://www.elastic.co/guide/en/elasticsearch/reference/master/modules-indices.html
2020-06-01 09:44:32 -04:00
Florian Kelbert 25f69547d9 [DOCS] Fix typo in rank feature query docs (#57401) 2020-06-01 08:42:45 -04:00
Nik Everett 07c76f2894
Update date_histogram docs (#56922) (#57387)
* Make it more clear that you can use `month` or `1M`.
* Explain rounding rules
* Consistently use "time zone" instead of "timezone". It looks like both
  are right but I see "time zone" much more. And the parameter in
  elasticsearch is `time_zone` so we may as well line up.

Closes #56760

Co-authored-by: James Rodewig <james.rodewig@elastic.co>
2020-05-29 17:40:40 -04:00
James Rodewig ebe433343f [DOCS] Correct link for ESMS redirect 2020-05-29 16:43:04 -04:00
Tomasz Elendt a7c36c8af5 Support multiple tokens on LHS in stemmer_override rules (#56113) (#56484)
This commit adds support for rules with multiple tokens on LHS, also
known as "contraction rules", into stemmer override token
filter. Contraction rules are handy into translating multiple
inflected words into the same root form. One side effect of this change is
that it brings stemmer override rules format closer to synonym rules
format so that it makes it easier to translate one into another.

This change also makes stemmer override rules parser more strict so
that it should catch more errors which were previously accepted.

Closes #56113
2020-05-29 22:34:31 +02:00
Benjamin Trent 35d5126cea
[7.x] [ML] adds new for_export flag to GET _ml/inference API (#57351) (#57368)
* [ML] adds new for_export flag to GET _ml/inference API (#57351)

Adds a new boolean flag, `for_export` to the `GET _ml/inference/<model_id>` API.

This flag is useful for moving models between clusters.
2020-05-29 14:01:08 -04:00
James Rodewig 7dbf5baf60 [DOCS] Remove Elastic Stack Monitoring Service docs (#57371) 2020-05-29 13:16:57 -04:00
Gordon Brown 1d5e476256
Add expand_wildcards to _cat/indices and _cat/aliases docs (#56964)
This commit adds the `expand_wildcards` parameter documentation to the
`_cat/indices` and `_cat/aliases` docs, as those APIs now support
`expand_wildcards`. Additionally, clarifies the `expand_wildcards` docs with
respect to hidden indices.
2020-05-29 11:01:53 -06:00
Benjamin Trent c8374dc9f3
[ML] add max_model_memory parameter to forecast request (#57254) (#57355)
This adds a max_model_memory setting to forecast requests. 
This setting can take a string value that is formatted according to byte sizes (i.e. "50mb", "150mb").

The default value is `20mb`.

There is a HARD limit at `500mb` which will throw an error if used.

If the limit is larger than 40% the anomaly job's configured model limit, the forecast limit is reduced to be strictly lower than that value. This reduction is logged and audited.

related native change: https://github.com/elastic/ml-cpp/pull/1238

closes: https://github.com/elastic/elasticsearch/issues/56420
2020-05-29 11:16:08 -04:00
Marios Trivyzas b2651323fd
SQL: Implement TIME_PARSE function for parsing strings into TIME values (#55223) (#57342)
Implement TIME_PARSE(<time_str>, <pattern_str>) function
which allows to parse a time string according to the specified
pattern into a time object. The patterns allowed are those of
java.time.format.DateTimeFormatter.

Closes #54963

Co-authored-by: Andrei Stefan <astefan@users.noreply.github.com>
Co-authored-by: Patrick Jiang(白泽) <patrickjiang0530@gmail.com>

(cherry picked from commit 1fe1188d449cad7d0782a202372edc52a4014135)
2020-05-29 15:48:37 +02:00
Adam Locke 11ae062877
[7.6] [DOCS] Explain flood stage watermark (#57321)
* [7.7] [DOCS] Explain flood stage watermark

* Adding id to fix cross-linking issue.
2020-05-28 19:29:32 -04:00
James Rodewig 47024ccb3c
[DOCS] Add verify snapshot repository API docs (#57253) (#57288) 2020-05-28 09:48:04 -04:00
James Rodewig 9277ce6f9e
[DOCS] Adds a doc value field example to Mapper Size docs (#57257) (#57280)
Changes:

* Updates snippet to include doc value field example.
* Fixes a broken link to inline script settings.
2020-05-28 09:26:49 -04:00
Martijn van Groningen 225ccd1cfa
Ensure template exists when creating data stream (#57275)
Backporting #56888 to 7.x branch.

Limit the creation of data streams only for namespaces that have a composable template with a data stream definition.

This way we ensure that mappings/settings have been specified and will be used at data stream creation and data stream rollover.

Also remove `timestamp_field` parameter from create data stream request and
let the create data stream api resolve the timestamp field
from the data stream definition snippet inside a composable template.

Relates to #53100
2020-05-28 15:08:25 +02:00
István Zoltán Szabó e1cab4feb4 [DOCS] Puts a link into the loss_function variable description (#56678) 2020-05-28 09:46:11 +02:00
James Rodewig a0ca0325fe
[DOCS] Reformat `min_hash` token filter docs (#57181) (#57246)
Changes:

* Rewrites description and adds a Lucene link
* Reformats the configurable parameters as a definition list
* Changes the `Theory` heading to `Using the min_hash token filter for
  similarity search`
* Adds some additional detail to the analyzer example
2020-05-27 15:08:23 -04:00
Lee Hinman c0f732b9f6
[7.x] Rename template V2 classes to ComposableTemplate (#57183) (#57232)
Backports the following commits to 7.x:

    Rename template V2 classes to ComposableTemplate (#57183)
2020-05-27 11:01:59 -06:00
Nik Everett 4d5be7c817
Save memory on numeric sig terms when not top (backport of #56789) (#57221)
This saves memory when running numeric significant terms which are not
at the top level by merging its collection into numeric terms and relying
on the optimization that we made in #55873.
2020-05-27 12:03:28 -04:00
Toby Sutor 2f6089c2b9 [DOCS] Add note about restoring snapshot versions (#55225)
As discussed at https://elastic.slack.com/archives/C0D1XEXEZ/p1586939752242300 is it not possible to restore snapshots taken on newer versions into clusters running lower versions. For example, a snapshot created in a 7.6.0 cluster cannot be restored on a 7.5.0 cluster. This needs to be documented.
2020-05-27 11:40:16 -04:00
James Rodewig 6dbe0022fa
[DOCS] Add redirects for 404 pages (#57161)
Adds several redirects for removed pages that return a
404 response.
2020-05-27 11:15:31 -04:00
James Rodewig f4a673c722
[DOCS] Add get snapshot repo API docs (#57150) (#57218) 2020-05-27 11:12:55 -04:00
James Rodewig 5ae650ecfd
[DOCS] Fix clarity of URL snapshot repo docs (#56920) (#57212)
Makes following changes to better clarify docs for read-only URL
snapshot repositories:

* Adds an example snippet for registering a URL repository
* Rewrites the protocols paragraph
* Adds a note to explicitly point out that only URLs using the `ftp`,
  `http`, `http`, and `jar` protocols do not need the `path.repo`
  setting.

Fixes #16280
2020-05-27 10:48:19 -04:00
James Rodewig ff63bda4be
[DOCS] Document index alias swaps are atomic (#55418) (#57209)
Co-authored-by: Tugberk Ugurlu <tugberk@outlook.com>
2020-05-27 10:23:51 -04:00
Samidh fdd378e3fb [Docs] Fix typo in start-watch-service.asciidoc (#57182) 2020-05-27 15:50:31 +02:00
James Rodewig 2c2673d759
[DOCS] Fix remote cluster client node docs (#57176) (#57205)
Changes:

* Documents the `r` role in the _cat/nodes API docs

* Fixes a typo in the node docs

Co-authored-by: Nick Canzoneri <nickcanz@github.com>
2020-05-27 08:58:29 -04:00
István Zoltán Szabó 27f258711a [DOCS] Fixes formatting of admonition paragraph in PUT inference API docs. (#57196) 2020-05-27 13:43:55 +02:00
James Rodewig 9f1e3bc82b
[DOCS] Document impact of restore API's `include_global_state` param (#57067) (#57155)
Previously, the restore API snippet included a `include_global_state` value of `true`.

Some users copy and paste the code example verbatim, updating only the index and
snapshot value names. Running the snippet could inadvertently wipe out a
cluster's current ILM policies, index templates, and ingest pipelines.

This change updates the snippet to use a `include_global_state` value of
`false`. It also adds a callout that better describes impacts of
using a `include_global_state` argument of `true`.

Co-authored-by: Mike Wong <mike.wong@elastic.co>
Co-authored-by: James Rodewig <james.rodewig@elastic.co>
Co-authored-by: David Turner <david.turner@elastic.co>
2020-05-26 11:39:39 -04:00
Russ Cam 8fd73d1e4f [DOCS] Add .NET client link to Bulk API docs (#57095)
This commit adds a link in the Bulk API documentation to
the .NET client's BulkAllObservable type and associated methods.
2020-05-26 09:12:57 -04:00
István Zoltán Szabó 47bf95cee3 [DOCS] Improves navigation between forecast APIs and adds short description. (#57035) 2020-05-25 09:11:00 +02:00
Marios Trivyzas b91bae30b1
SQL: [Tests] Move JDBC integration tests to new module (#56872) (#57072)
Move the JDBC functionality integration tests from `:sql:qa` to a separate
module `:sql:qa:jdbc`. This way the tests are isolated from the rest of the
integration tests and they only depend to the `:sql:jdbc` module, thus
removing the danger of accidentally pulling in some dependency that may
hide bugs.

Moreover this is a preparation for #56722, so that we can run those tests
between different JDBC and ES node versions and ensure forward
compatibility.

Move the rest of existing tests inside a new `:sql:qa:server` project, so that
the `:sql:qa` becomes the parent project for both and one can run all the integration
tests by using this parent project.

(cherry picked from commit c09f4a04484b8a43934fe58fbc41bd90b7dbcc76)
2020-05-22 17:49:36 +02:00
James Rodewig b3426dd558
[DOCS] Add delete snapshot repo API docs (#57043)
Changes:

* Adds API reference docs for the delete snapshot repo API.

* Corrects an error in the delete snapshot repo API spec. Comma-separated
repository names are not supported.

* Relocates the existing delete snapshot repo API example docs.
2020-05-21 14:47:07 -04:00
James Rodewig a2de43d468
[DOCS] Reformat `shingle` token filter (#57040)
Changes:

* Rewrites description and adds Lucene link
* Adds analyze example
* Rewrites parameter documentation
* Updates custom analyzer and filter examples
* Adds anchor to `index.max_shingle_diff` index-level setting
2020-05-21 13:56:54 -04:00
István Zoltán Szabó 9b7356d6af [DOCS] Removes the Jobs section from the ML anomaly detection APIs page. (#57031) 2020-05-21 17:32:07 +02:00
James Rodewig 9455bc84d8 [DOCS] Fix anchor in put snapshot repo API docs 2020-05-21 09:36:49 -04:00
James Rodewig 5cb34d9a6e
[DOCS] Reformat `hunspell` token filter (#56955)
Changes:

* Rewrites description and adds Lucene link
* Adds analyze example
* Rewrites parameter documentation
* Updates custom analyzer example
* Rewrites related setting documentation
2020-05-20 14:47:53 -04:00
Brandon Morelli ec41d36c62
docs: update links to beats security docs (#56875) (#56953) 2020-05-20 11:28:39 -07:00
James Rodewig b7c6f0d02f
[DOCS] Fix default for `http.compression` setting (#56899) (#57002)
Elasticsearch enables HTTP compression by default. However, to mitigate
potential security risks like the BREACH attack, compression is disabled by
default if HTTPS is enabled.

This updates the `http.compression` setting definition accordingly and adds
additional context.

Co-authored-by: Leaf-Lin <39002973+Leaf-Lin@users.noreply.github.com>
2020-05-20 12:12:05 -04:00
Albert Zaharovits fd54ed328c DOCS Correct audit emit_node_id default value as false (#56995)
Since version 7, the `xpack.security.audit.logfile.emit_node_id` setting defaults
to `false`, yet the docs say otherwise. This commit fixes that.
2020-05-20 16:35:30 +03:00
Théophile Helleboid - chtitux a2c6d61ed5 [DOCS] Fix typo in search API `explain` param def (#56991)
Co-authored-by: James Rodewig <james.rodewig@elastic.co>
2020-05-20 09:22:23 -04:00
Eddie Turizo 27673aa59c [DOCS] Fix ulimit value in system settings docs (#56973)
Correctly documents the ulimit value as 65535.

The limit was lowered as part of #37537.
2020-05-20 08:44:07 -04:00
Andrei Balici 19a336e8d3 Add `max_token_length` setting to the CharGroupTokenizer (#56860)
Adds `max_token_length` option to the CharGroupTokenizer.
Updates documentation as well to reflect the changes.

Closes #56676
2020-05-20 14:28:40 +02:00
James Rodewig cc12361a82 [DOCS] EQL: Fix whitespace in EQL snippet 2020-05-19 17:04:49 -04:00
James Rodewig f6d2688de2
[DOCS] Add JS client helper links to docs (#55216) (#56968)
Adds links for the Elasticsearch-js client to the bulk and scroll docs.

Co-authored-by: Tomas Della Vedova <delvedor@users.noreply.github.com>
2020-05-19 16:53:22 -04:00
Adam Locke cbd35e9a2b
[DOCS] Add links to `flattened` datatype (#56794) (#56963)
* Changes for #52239.

* Incorporating review feedback from Julie T. Also single-sourcing nexted options in the Mapping page and referencing them in the Nested page.

* Moving tip after the introduction and clarifying limits.

* Update docs/reference/mapping.asciidoc

Co-authored-by: James Rodewig <james.rodewig@elastic.co>

* Update docs/reference/mapping/types/nested.asciidoc

Co-authored-by: James Rodewig <james.rodewig@elastic.co>

Co-authored-by: James Rodewig <james.rodewig@elastic.co>

Co-authored-by: James Rodewig <james.rodewig@elastic.co>
2020-05-19 16:37:30 -04:00
James Rodewig 771ddbf083
[DOCS] EQL: Add sequence example to tutorial (#56965) (#56966)
Adds an example using the sequence syntax to the 'Run an EQL search'
tutorial.

Supplements other examples added with #56721
2020-05-19 16:14:57 -04:00
James Rodewig cc43d67eb1 [DOCS] Add leading slashes to EQL API examples 2020-05-19 15:38:37 -04:00
Adam Mohammed fd84fdf581 [DOCS] Fix typo in secure settings docs (#56778) 2020-05-19 12:04:25 -04:00
James Rodewig 0f233b1b0b
[DOCS] Remove outdated links for `similarity` mapping param args (#56925) 2020-05-19 11:04:53 -04:00
Lee Hinman e208925465
[7.x] Add template simulation API for simulating template composition (#56842) (#56924) 2020-05-19 08:12:21 -06:00
James Rodewig 342e713e2a
[DOCS] Fix fingerprint token filter's analyzer example (#56811) (#56943)
Co-authored-by: Abhilash Bolla <2282894+ivssh@users.noreply.github.com>
2020-05-19 09:30:00 -04:00
James Rodewig 22f54ba205 [DOCS] EQL: Fix API example headings 2020-05-18 16:29:29 -04:00
James Rodewig 74554f1ae8
[DOCS] Add put snapshot repo API docs (#56827) (#56900) 2020-05-18 08:55:22 -04:00
Benjamin Trent 297f864884
[ML] relax throttling on expired data cleanup (#56711) (#56895)
Throttling nightly cleanup as much as we do has been over cautious.

Night cleanup should be more lenient in its throttling. We still
keep the same batch size, but now the requests per second scale
with the number of data nodes. If we have more than 5 data nodes,
we don't throttle at all.

Additionally, the API now has `requests_per_second` and `timeout` set.
So users calling the API directly can set the throttling.

This commit also adds a new setting `xpack.ml.nightly_maintenance_requests_per_second`.
This will allow users to adjust throttling of the nightly maintenance.
2020-05-18 08:46:42 -04:00
James Rodewig e492c23944
[DOCS] Sort metric and pipeline agg docs (#56613) (#56846)
Co-authored-by: Gil Raphaelli <gil@elastic.co>
2020-05-15 17:15:53 -04:00
Gabriel Petrovay cb4d5f5042 Fixed calendar intervals documentation (#56666)
- the 1-letter intervals are not parseable (`m`, `h`, `d`, `w`,  `M`, `q`, `y`)
- fixed formatting broken by new lines
2020-05-15 16:55:57 -04:00
James Rodewig c50f86fbba
[DOCS] EQL: Document `case_sensitive` param (#56697) (#56818) 2020-05-15 11:47:19 -04:00
Dan Hermann 66871c5342
[7.x] Rename endpoint from plural "_data_streams" to singular "_data_stream" (#56825) 2020-05-15 10:27:53 -05:00
Dan Hermann 2a21d4d976
Docs for data stream REST APIs 2020-05-15 09:37:45 -05:00
James Rodewig 5e09762a27 [DOCS] EQL: Align comments in `between` fn examples 2020-05-15 09:20:45 -04:00
James Rodewig 24cd45345e [DOCS] EQL: Remove references to arrays/multi-value fields (#56772) 2020-05-15 09:09:07 -04:00
Benjamin Trent f71c305090
[7.x] [Transform] add support for terms agg in transforms (#56696) (#56809)
* [Transform] add support for terms agg in transforms (#56696)

This adds support for `terms` and `rare_terms` aggs in transforms. 

The default behavior is that the results are collapsed in the following manner:
`<AGG_NAME>.<BUCKET_NAME>.<SUBAGGS...>...`
Or if no sub aggs exist
`<AGG_NAME>.<BUCKET_NAME>.<_doc_count>`

The mapping is also defined as `flattened` by default. This is to avoid field explosion while still providing (limited) search and aggregation capabilities.
2020-05-15 08:08:43 -04:00
Tal Levy 5e90ff32f7
Add Normalize Pipeline Aggregation (#56399) (#56792)
This aggregation will perform normalizations of metrics
for a given series of data in the form of bucket values.

The aggregations supports the following normalizations

- rescale 0-1
- rescale 0-100
- percentage of sum
- mean normalization
- z-score normalization
- softmax normalization

To specify which normalization is to be used, it can be specified
in the normalize agg's `normalizer` field.

For example:

```
{
  "normalize": {
    "buckets_path": <>,
    "normalizer": "percent"
  }
}
```
2020-05-14 17:40:15 -07:00
Lee Hinman a73d7d9e2b
[7.x] Don't allow invalid template combinations (#56397) (#56795)
Backports the following commits to 7.x:

- Don't allow invalid template combinations (#56397)
2020-05-14 16:20:53 -06:00
James Rodewig 2a943a58a4
[DOCS] EQL: Document `number` function (#56770)
Co-authored-by: Ross Wolf <31489089+rw-access@users.noreply.github.com>
2020-05-14 15:44:04 -04:00
James Rodewig 2921747b23
[7.x] [DOCS] EQL: Document sequences (#56721) (#56774)
Co-authored-by: Ross Wolf <31489089+rw-access@users.noreply.github.com>
2020-05-14 11:51:40 -04:00
Lisa Cawley 6a8e10189f [DOCS] Add throttling based on configuration parameter (#56653) 2020-05-14 08:45:29 -07:00