Commit Graph

9094 Commits

Author SHA1 Message Date
James Rodewig 3a9bf33993
[DOCS] Remove metrics sidebar in `_source` docs (#60777) (#60786) 2020-08-05 16:19:47 -04:00
James Rodewig 029869eb35
[DOCS] Fix metadata field refs (#60764) (#60769) 2020-08-05 14:04:55 -04:00
James Rodewig 815f3d526e
[DOCS] Move named query content to bool query (#60748) (#60772) 2020-08-05 13:42:13 -04:00
Lisa Cawley d77ba58cfd
[DOCS] Add ml-cpp PRs to 7.9.0 release notes (#60689)
Co-authored-by: David Roberts <dave.roberts@elastic.co>
2020-08-05 10:12:11 -07:00
James Rodewig 3f9152c835
[DOCS] Fix query docs formatting (#60752) (#60760) 2020-08-05 12:47:42 -04:00
Francisco Fernández Castaño f64e572e69
[DOCS] Include reference to AWS VPC endpoints in s3 repository docs. (#60753)
Add VPC endpoint as the recommended way of connecting to s3 in private subnets

Backport of #60654

Co-authored-by: Bill Mitchell <vocatan@users.noreply.github.com>
Co-authored-by: David Turner <david.turner@elastic.co>
2020-08-05 18:39:47 +02:00
James Rodewig a1c27b0833
[DOCS] Refactor EQL docs (#60700) (#60745)
Changes:

* Moves sample data to reusable rest test
* Combines EQL index, requirements, and run a search pages
* Combines EQL syntax and limitations pages
* Adds related redirects
2020-08-05 11:25:18 -04:00
István Zoltán Szabó 35b9f2b46b [DOCS] Adds inference phase to get DFA job stats. (#60737) 2020-08-05 16:26:02 +02:00
Jake Landis f3752ba1d5
7.x suport new path for re-index java-api doc (#60319)
This commit uses the new location for the reindex java-api documentation.
Temporary files have been left behind to pacify the docs build.

related #60339
2020-08-05 09:05:07 -05:00
James Rodewig 8db3f0ca27
[DOCS] Refactor snippets for `Search your data` (#60701) (#60738)
Changes:
* Moves sample data to reusable REST test
* Add xref to pagination docs
* Removes duplicated results
* Updates the wildcard example
2020-08-05 09:52:35 -04:00
James Rodewig e214c70f8d [DOCS] Fix outdated twitter reference 2020-08-05 09:29:51 -04:00
James Rodewig e2553d5884
[DOCS] Add soft redirect for sliced scroll (#60699) (#60733) 2020-08-05 09:23:15 -04:00
Martijn van Groningen 160f27f77c
Fix mistake in notes around dynamic template validation. (#60726)
The double bracket notation is incorrect.
2020-08-05 14:42:15 +02:00
Przemysław Witek 0afa1bd972
Deprecate allow_no_jobs and allow_no_datafeeds in favor of allow_no_match (#60601) (#60727) 2020-08-05 13:39:40 +02:00
Pius 1ca58398c5 Highlight `cluster.initial_master_nodes` removal after cluster formation (#60631)
Explicitly ask users to remove `cluster.initial_master_nodes` once the cluster
has formed for the first time.
2020-08-05 08:58:22 +01:00
Yang Wang 1329c982c9
Ensure license is ready for xpack info doc tests (#60706) (#60708)
Fix another variant of missing license test failure similar to the cases fixed by #60498.
2020-08-05 16:36:43 +10:00
James Rodewig 5885f6ae66
[DOCS] Add missing lang values to snowball token filter (#60489) (#60692) 2020-08-04 17:46:03 -04:00
James Rodewig 704395e792
[DOCS] Update Debian APT repo command (#60679) (#60685)
The current `tee` command appends a definition to
`/etc/apt/sources.list.d/elastic-{version}.list`.

This can lead to duplicate lines and significantly slow apt-get
operations.

This updates the command to overwrite rather than append.
2020-08-04 16:00:32 -04:00
Lisa Cawley b1c10f457a
[DOCS] Adds scope to monitoring (#57852) (#60665) 2020-08-04 12:40:11 -07:00
James Rodewig a21ec410c7
[DOCS] Replace `twitter` dataset in search/agg docs (#60667) (#60675) 2020-08-04 14:16:38 -04:00
James Rodewig 0587199fa6
[DOCS] Update ILM docs to use composable index templates (#60323) (#60670) 2020-08-04 13:01:19 -04:00
Russ Cam 667309e6a6 [DOCS] Fix list dangling indices documentation (#60099)
This commit fixes the list dangling indices response.
The dangling_indices array is an array of objects
that represent aggregated dangling index information

(cherry picked from commit 24c72d4e71c95f2d7690090933e0657152f6af9b)
2020-08-04 10:32:00 +10:00
debadair 80584d266d
[DOCS] Update link to ILM tutorial (#60557) (#60624) 2020-08-03 13:04:24 -07:00
debadair e9ac195756
[DOCS] Add info about why we removed test fw docs (#60346) (#60558)
* [DOCS] Add info about why we removed test fw docs

* Apply suggestions from code review

Co-authored-by: James Rodewig <40268737+jrodewig@users.noreply.github.com>

Co-authored-by: James Rodewig <40268737+jrodewig@users.noreply.github.com>
2020-08-03 12:05:35 -07:00
James Rodewig 26d51089da
[DOCS] Replace `twitter` dataset in docs (#60604) (#60609) 2020-08-03 13:31:19 -04:00
James Rodewig c5f4f91ac4 [DOCS] Clarify reindex does not require existing dest 2020-08-03 12:46:40 -04:00
James Rodewig 5be515f126 [DOCS] Unhide EQL search in data streams docs 2020-08-03 11:59:12 -04:00
James Rodewig cfab3bccab
[DOCS] Replace `twitter` dataset in cat API docs (#60588) (#60597) 2020-08-03 10:22:36 -04:00
James Rodewig bffe82449a
[DOCS] Update Amazon Linux AMI link in EC2 docs (#60589) (#60593) 2020-08-03 09:32:31 -04:00
Yannick Welsch b0d601fa63 Adjust searchable snapshot license (#60578)
No longer needs Platinum license for testing on staging.
2020-08-03 13:19:53 +02:00
Yang Wang a76fc324d4
Fix get-license test failure by ensure cluster is ready (#60498) (#60569)
When a new cluster starts, the HTTP layer becomes ready to accept incoming 
requests while the basic license is still being populated in the background. 
When a get license request comes in before the license is ready, it can get 
404 error. This PR fixes it by either wrap the license check in assertBusy or 
ensure the license is ready before perform the check.

This is a backport for both #60498 and #60573
2020-08-03 19:40:03 +10:00
James Rodewig fcc53d9e0e
[DOCS] Note refresh requests are synchronous (#60540) (#60550) 2020-07-31 16:22:34 -04:00
James Rodewig 5a2c6f0d4f
[DOCS] http -> https, remove outdated plugin docs (#60380) (#60545)
Plugin discovery documentation contained information about installing
Elasticsearch 2.0 and installing an oracle JDK, both of which is no
longer valid.

While noticing that the instructions used cleartext HTTP to install
packages, this commit replaces HTTPs links instead of HTTP where possible.

In addition a few community links have been removed, as they do not seem
to exist anymore.

Co-authored-by: Alexander Reelsen <alexander@reelsen.net>
2020-07-31 16:16:31 -04:00
James Rodewig fb599dc343
[DOCS] Add rollups to `Tune for disk usage` (#60436) (#60542)
Co-authored-by: Leaf-Lin <39002973+Leaf-Lin@users.noreply.github.com>
2020-07-31 16:10:57 -04:00
James Rodewig 771e9f142a
[DOCS] Move search pagination content to one page (#60515) (#60525) 2020-07-31 12:40:40 -04:00
James Rodewig 9eba7f39b0
[DOCS] Replace `twitter` dataset in docs APIs (#60521) (#60529) 2020-07-31 12:40:03 -04:00
James Rodewig 4b12e69e8e
[DOCS] Replace `twitter` dataset in index API docs (#60473) (#60510) 2020-07-31 09:51:47 -04:00
James Rodewig 0022d316bb
[DOCS] Merge search topic and overview pages (#60459) (#60479) 2020-07-30 16:45:18 -04:00
James Rodewig 134b69d3aa
[DOCS] Fix `template` param in put index template API (#60474) (#60476) 2020-07-30 16:44:50 -04:00
James Rodewig 2d2b74dd32
[DOCS] Note remote reindex is not fwd compatible (#60425) (#60454) 2020-07-30 09:23:55 -04:00
James Rodewig b17ae33b3a
[DOCS] Move field collapse content to separate page (#60424) (#60451) 2020-07-30 09:19:05 -04:00
James Rodewig b3da548dab [DOCS] Fix awkward wording in kuromoji analyzer docs 2020-07-30 08:56:59 -04:00
Bogdan Pintea 8c22adc447
SQL: Add option to provide the delimiter for the CSV format (#59907) (#60420)
* SQL: Add option to provide the delimiter for the CSV format (#59907)

* Add option to provide the delimiter to the CSV fmt

This adds the option to provide the desired character as the separator
for the CSV format (the default remains comma).
A set of characters are excluded though - like CR, LF, `"` - to avoid
slipping onto the CSV-dialects slope. The tab is also forbidden, the
user needs to choose the "tsv" format explicitely.

Update the doc to make it clear that the textual CSV, TSV and TXT
formats pass the cursor back to the user through the Cursor HTTP header.

(cherry picked from commit 3a8b00cc7480f7ada57fcea3cbac957facac08fc)

* Java8 fixes

- replace Set#of();
- URLDecoder#decode() requires a string (vs a charset) as 2nd arg.
2020-07-29 21:40:11 +02:00
Tim Brooks 85fdf959ad
Add configured indexing memory limit to node stats (#60414)
This commit adds the configured memory limit to the node stats API.
2020-07-29 12:28:21 -06:00
James Rodewig 6054d33a63
[DOCS] Replace `twitter` dataset in API conventions + README (#60408) (#60410) 2020-07-29 14:14:01 -04:00
Tim Brooks e73c8eed33
Fix documentation about `indexing_pressure.memory.limit` (#60341)
The documentation about this setting is currently mislabelled. This
commit fixes the issue.
2020-07-29 10:57:29 -06:00
James Rodewig d08e7633f8
[DOCS] Add `number_of_routing_shards` index setting to index modules (#60311) (#60400)
Changes:

* Adds the `number_of_routing_shards` index setting to index modules docs.
* Updates the split API docs to mention that `number_of_routing_shards`
is a static setting.
2020-07-29 10:53:50 -04:00
James Rodewig ac6c806ec7
[DOCS] Fix typo in Watcher docs (#60326) (#60388)
Co-authored-by: Martin-Kemp <30285179+Martin-Kemp@users.noreply.github.com>
2020-07-29 10:15:09 -04:00
James Rodewig 1cfdb4fc08
[DOCS] Fix formatting in 7.0 breaking changes (#60372) (#60385)
Co-authored-by: Jacob Dreesen <jacob@hdreesen.de>
2020-07-29 09:16:29 -04:00
James Rodewig 9667aeadd2
[DOCS] Fix typo in take snapshot docs (#60204) (#60383)
Co-authored-by: VLADIMIR MIRONOV <mironov.v@torrowtech.com>
2020-07-29 09:16:00 -04:00
David Turner cf0cab614d Clarify remote clusters' use of transport layer (#60268)
Today there are a few places in the transport layer docs where we talk
about communication between nodes _within a cluster_. We also use the
transport layer for remote cluster connections, and these statements
also apply there, but this is not clear from today's docs. This commit
generalises these statements to make it clear that they apply to remote
cluster connections too.

It also adds a link from the docs on configuring TCP retries to the
(deeply-buried) docs on preserving long-lived connections.
2020-07-29 13:04:10 +01:00
James Rodewig 1a172afbb2
[DOCS] Add full-width char section to kuromoji analyzer docs (#60317) (#60324) 2020-07-28 14:17:23 -04:00
Julie Tibshirani c7bfb5de41
Add search `fields` parameter to support high-level field retrieval. (#60258)
This feature adds a new `fields` parameter to the search request, which
consults both the document `_source` and the mappings to fetch fields in a
consistent way. The PR merges the `field-retrieval` feature branch.

Addresses #49028 and #55363.
2020-07-28 10:58:20 -07:00
James Rodewig 8edae3cd15
[DOCS] Update pre-existing data stream refs (#60289) (#60293) 2020-07-28 13:51:43 -04:00
markharwood e0286e9bd3
Search - remove allow-expensive-query checks from wildcard field. (#60273) (#60308)
Removing allow-expensive-query checks because we think this field type is fast enough.

Closes #60139
2020-07-28 17:12:33 +01:00
David Turner 9450ea08b4 Log and track open/close of transport connections (#60297)
Transport connections between nodes remain in place until one or other
node shuts down or the connection is disrupted by a flaky network.
Today it is very difficult to demonstrate that transient failures and
cluster instability are caused by the network even though this is often
the case. In particular, transport connections open and close without
logging anything, even at `DEBUG` level, making it very hard to quantify
the scale of the problem or to correlate the networking problems with
external events.

This commit adds the missing `DEBUG`-level logging when transport
connections open and close, and also tracks the total number of
transport connections a node has opened as a measure of the stability of
the underlying network.
2020-07-28 17:08:04 +01:00
Adam Locke ee18538fd7
[DOCS] Adds table with icons for version compatibility (#60159) (#60302)
* Adds table with icons for simplicity.

* Updating table for clarity.

* Changing table formatting and incorporating more feedback.

* Changing table alignment.
2020-07-28 11:08:58 -04:00
James Rodewig 4eee6d274d
[DOCS] Fix broken link to Lucene docs (#59365) (#60290)
Co-authored-by: Alexander Reelsen <alexander@reelsen.net>
2020-07-28 09:09:48 -04:00
Yannick Welsch a55c869aab Properly document keepalive and other tcp options (#60216)
Keepalive options are not well-documented (only in transport section, although also available at http and network level).

Co-authored-by: David Turner <david.turner@elastic.co>
Co-authored-by: James Rodewig <40268737+jrodewig@users.noreply.github.com>
2020-07-28 11:10:04 +02:00
Yannick Welsch ffe114b890 Set specific keepalive options by default on supported platforms (#59278)
keepalives tell any intermediate devices that the connection remains alive, which helps with overzealous firewalls that are
killing idle connections. keepalives are enabled by default in Elasticsearch, but use system defaults for their
configuration, which often times do not have reasonable defaults (e.g. 7200s for TCP_KEEP_IDLE) in the context of
distributed systems such as Elasticsearch.

This PR sets the socket-level keep_alive options for network.tcp.{keep_idle,keep_interval} to 5 minutes on configurations
that support it (>= Java 11 & (MacOS || Linux)) and where the system defaults are set to something higher than 5
minutes. This helps keep the connections alive while not interfering with system defaults or user-specified settings
unless they are deemed to be set too high by providing better out-of-the-box defaults.
2020-07-28 11:10:04 +02:00
James Rodewig aba785cb6e
[DOCS] Update my-index examples (#60132) (#60248)
Changes the following example index names to `my-index-000001` for consistency:

* `my-index`
* `my_index`
* `myindex`
2020-07-27 15:58:26 -04:00
James Rodewig 3bb58eb5c1
[DOCS] Fix `fuzzy_rewrite` ref in match query docs (#60237) (#60251) 2020-07-27 15:36:09 -04:00
James Rodewig 1178f5c6db
[DOCS] Fix ingest processor docs for autogen doc IDs (#60147) (#60242)
If you autogen doc IDs, you cannot use the `{{_id}}` value in an ingest
processor.

This adds a related admonition to the ingest processor docs.
2020-07-27 13:55:21 -04:00
James Rodewig 95d7ce76ec
[DOCS] Fix `rewrite` => `fuzzy_rewrite` in multi match query docs (#60175) (#60233)
Co-authored-by: homersimpsons <guillaume.alabre@gmail.com>
2020-07-27 12:33:14 -04:00
James Rodewig 7a23c6b6ec
[DOCS] Fix formatting in simple query string query docs (#60226)
Co-authored-by: Ulas Keles <ulaskeles@users.noreply.github.com>
2020-07-27 12:20:02 -04:00
James Rodewig dbd7e7793a
[DOCS] Fix default gap policy for moving fn, moving avg aggs (#60223) 2020-07-27 12:08:33 -04:00
lcawl a27d630bdf [DOCS] Removes coming tag 2020-07-27 07:55:18 -07:00
James Rodewig 747f8bfe79
[DOCS] Add Kibana screenshots to data stream docs (#60118) (#60217) 2020-07-27 10:39:32 -04:00
James Rodewig 608a5b9e71
[DOCS] Clarify compatibility for upgrade via reindex (#60045) (#60209)
Co-authored-by: Inbar Shimshon <inbar.shimshon@elastic.co>
2020-07-27 09:38:39 -04:00
James Rodewig 08e11814c3
[DOCS] Fix clarity of 7.6 derived key breaking change (#60154) 2020-07-27 08:36:48 -04:00
David Turner 53fa52d618 Fix whitespace bug in #59222 2020-07-27 12:26:33 +01:00
David Turner d8fdb82efb Suggest reducing tcp_retries2 (#59222)
Adds documentation suggesting reducing `tcp_retries2` on Linux to detect
network partitions more quickly.

Relates #34405
2020-07-27 11:40:12 +01:00
debadair 284c61ad19
[DOCS] Refactored index-templates topic. (#59737) (#60165)
* [DOCS] Refactored index-templates topic.

* [DOCS] Add separate files.

* [DOCS] Add delete component template.

* Apply suggestions from code review

Co-authored-by: James Rodewig <james.rodewig@elastic.co>

* [DOCS] Incorporated review comments
2020-07-23 19:48:19 -07:00
Lisa Cawley 2665bfffce
[DOCS] Fix security links in machine learning APIs (#60098) (#60152) 2020-07-23 16:43:10 -07:00
Lisa Cawley cc6edc39a1
[DOCS] Refresh transform screenshots with histograms (#59264) (#60145) 2020-07-23 11:14:50 -07:00
James Rodewig 2e01f652c1
[DOCS] Move search sort docs to separate page (#60123) (#60142)
Moves the search sort docs from the deprecated 'Request Body Search'
page to a new subpage of 'Run a search'.

No substantive changes were made to the content.
2020-07-23 13:44:47 -04:00
Albert Zaharovits 2eaf5e1c25
[DOCS] Mapping updates are deprecated for ingestion privileges (#60024)
This PR contains the deprecation notice that `create`, `create_doc`, `index` and
`write` ingest privileges do not permit mapping updates in version 8. It also
updates the docs description of said privileges. 

This should've been part of #58784
2020-07-23 19:49:23 +03:00
James Rodewig 988e8c8fc6
[DOCS] Swap `[float]` for `[discrete]` (#60134)
Changes instances of `[float]` in our docs for `[discrete]`.

Asciidoctor prefers the `[discrete]` tag for floating headings:
https://asciidoctor.org/docs/asciidoc-asciidoctor-diffs/#blocks
2020-07-23 12:42:33 -04:00
Adrien Grand 716a3d5a21 Mention how CCR can help optimize indexing throughput. (#54870) 2020-07-23 18:40:40 +02:00
Martijn Laarman 890d35f74d [DOCS] note breaking change from 7.8.1 in migration guide (#59642)
Co-authored-by: Adam Locke <adam.locke@elastic.co>
(cherry picked from commit b0c34020ed6a10fda8e6efa9af343bd283954ec5)
2020-07-23 13:18:46 +02:00
Julie Tibshirani aa57bbd422
Consolidate validation for 'docvalue_fields'. (#60065)
This improves modularity and also fixes some issues when `docvalues_fields` is
used within `inner_hits` or the `top_hits` agg:
* We previously didn't resolve wildcards in field names.
* We also forgot to enforce the limit `index.max_docvalue_fields_search`.
2020-07-22 17:26:58 -07:00
James Rodewig 67b07ec386
[DOCS] Remove SQL access settings page (#60078) (#60089)
This page previously documented `xpack.sql.enabled`.

However, in 7.8 and above, `xpack.sql.enabled` is always enabled and
the setting has no effect. There is no reason to maintain this page.
2020-07-22 16:59:21 -04:00
James Rodewig f8976505cb
[DOCS] Correct the default value of `ignore_throttled` param (#60036) (#60086)
Co-authored-by: bellengao <gbl_long@163.com>
2020-07-22 16:53:18 -04:00
James Rodewig 0c9791798d
[7.x] [DOCS] Reformat snippets to use two-space indents (#60080) 2020-07-22 15:57:49 -04:00
Lisa Cawley 9ba017f699
[DOCS] Changes level offset of transform pages (#60066) (#60075) 2020-07-22 11:22:57 -07:00
Tim Brooks ba01540d7e
Implement human readable indexing pressure stats (#60058)
The indexing pressure stats do not currently have human readable
variants. This commit add human readable variants and updates the
documentation.
2020-07-22 12:07:59 -06:00
James Rodewig ed10d7407c
[DOCS] Fix shrink index API prereqs (#59985) (#60067) 2020-07-22 14:06:40 -04:00
Tim Brooks ceb54ed655
Add indexing pressure documentation (#59456)
This commit adds documentation about the new indexing pressure memory
limit setting and exposure of this metrics in node stats.
2020-07-22 10:09:18 -06:00
Adam Locke 0a73225cd8
[DOCS] Adding new page for restore snapshot API (#59937) (#60055)
* Adding new page for restore snapshot API.

* Improving test cases, lots of edits, and streamlining content.

* Incorporating review suggestions and feedback.

* Specify `index alias` vs `alias`

* Change parameter order

* Provide clarity around regular expression

* Add link to SLM parameters

* Split sentences in example

* Adding link to master node page.
2020-07-22 12:08:55 -04:00
Lisa Cawley 46d33b1586
[DOCS] 7.9.0 release notes (#60053) 2020-07-22 08:40:59 -07:00
Emily Li 5f27a95346 Fix grammar mistake in SQL data type docs. (#60028)
Remove an extra 'when'.
2020-07-21 16:15:06 -07:00
James Rodewig 293cb8d48c
[DOCS] Fix typo in thread pools docs (#59944) (#60019)
Fix typo where available processors should be allocated processors.

Co-authored-by: Leaf-Lin <39002973+Leaf-Lin@users.noreply.github.com>
2020-07-21 17:04:36 -04:00
James Rodewig 401e12dc2b
[DOCS] Fix data stream docs (#59818) (#60010) 2020-07-21 17:04:13 -04:00
James Rodewig 04c68ba740
[DOCS] Update search docs to use `my-index` dataset (#60005) (#60012) 2020-07-21 16:14:44 -04:00
James Rodewig b302b09b85
[DOCS] Reformat snippets to use two-space indents (#59973) (#59994) 2020-07-21 15:49:58 -04:00
David Roberts 606b7ea139 [DOCS] Adds extra ml-cpp PRs to release notes (#59967) 2020-07-21 11:47:36 -07:00
Tim Brooks ed315442ac
Update thread pool docs about WRITE queue size (#59643)
This commit updates the thread pool documentation to reflect the change
in the WRITE thread pool default queue size.
2020-07-21 12:38:03 -06:00
James Rodewig 32d7fa1541
[DOCS] Introduce basic ECS logs test (#59713) (#59997)
Adds a new `my-index-00001` REST test for docs snippets.

This test can serve as a lightweight replacement for
our existing `twitter` REST tests.

The new dataset is:

* Based on Apache logs, which is better aligned with Elastic use cases
* Compliant with ECS
* Similar to the existing `twitter` data set, containing the same field data types
* Lightweight, which should keep existing test runtimes roughly the same

Also updates the search API reference docs to use the new test.
2020-07-21 13:25:53 -04:00
James Rodewig fb40ccf8a4
[DOCS] Mark data stream stats API as stable (#59978) (#59987)
Removes experimental admon from data stream stats API.
Relates to #59860.
2020-07-21 11:22:36 -04:00
malpani 0555fef799 Support ignore_keywords flag for word delimiter graph token filter (#59563)
This commit allows customizing the word delimiter token filters to skip processing 
tokens tagged as keyword through the `ignore_keywords` flag Lucene's 
WordDelimiterGraphFilter already exposes.

Fix for #59491
2020-07-21 16:11:55 +01:00
Howard 466e947b0e
[DOCS] Fix missing punctuation in agg docs (#59823) 2020-07-21 10:19:29 -04:00
Przemysław Witek 283a1f605c
Rename binary_soft_classification evaluation to outlier_detection (#59951) (#59970) 2020-07-21 15:15:04 +02:00
Lisa Cawley fb212269ce
[DOCS] Changes level offset of anomaly detection pages (#59911) (#59940) 2020-07-20 17:04:59 -07:00
Julie Tibshirani 8dc5880c3f Add 'point' to the top-level field type docs. (#59731)
Before it was missing from the list. This PR also renames the 'geo data types'
section to 'spatial data types' and consolidates the geo and cartesian types
into that section.
2020-07-20 16:30:12 -07:00
Lisa Cawley 9633d503d8
[DOCS] Changes level offset for anomaly detection APIs (#59920) (#59928) 2020-07-20 13:10:54 -07:00
Lisa Cawley 8f8d24b3c1
[DOCS] Changes level offset in data frame analytics APIs (#59919) (#59923) 2020-07-20 13:06:29 -07:00
James Rodewig ff8a042580
[DOCS] Reformat agg snippets to use two-space indents (#59912) (#59922) 2020-07-20 15:59:00 -04:00
James Rodewig bb80c56e04
[DOCS] Reformat Plugin snippets to use two-space indents (#59895) (#59916) 2020-07-20 15:06:12 -04:00
James Rodewig 24fec52447
[DOCS] Add performance warning for scripts (#59890) (#59913) 2020-07-20 15:05:33 -04:00
Armin Braun e16e565c5e
Fix Snapshot Status API Docs Test (#59902) (#59908)
The clock resolution for this API is our default 200ms. It is unlikely but
possible that a shard snapshot starts and ends on separate clock ticks and that breaks the test.
Just allowing any value here seems fine to me (seems we can't match for integer specifically).
2020-07-20 18:43:40 +02:00
Igor Motov 96a5284484
Add hard_bounds documentation (#59809) (#59883)
Fixes #59774
2020-07-20 10:51:23 -04:00
Nik Everett fe10141108
Document supported scenarios for CCS (#58120) (#59886)
Documents the supported scenarios for CCS.

Co-authored-by: Adam Locke <adam.locke@elastic.co>
2020-07-20 10:41:53 -04:00
David Turner b75207a09f Remove sporadic min/max usage estimates from stats (#59755)
Today `GET _nodes/stats/fs` includes `{least,most}_usage_estimate`
fields for some nodes. These fields have rather strange semantics. They
are only reported on the elected master and on nodes that have been the
elected master since they were last restarted; when a node stops being
the elected master these stats remain in place but we stop updating them
so they may become arbitrarily stale.

This means that these statistics are pretty meaningless and impossible
to use correctly. Even if they were kept up to date they're never
reported for data-only nodes anyway, despite the fact that data nodes
are the ones where we care most about disk usage. The information needed
to compute the path with the least/most available space is already
provided in the rest the stats output, so we can treat the inclusion of
these stats as a bug and fix it by simply removing them in this commit.
Since these stats were always optional and mostly omitted (for opaque
reasons) this is not considered a breaking change.
2020-07-20 15:22:04 +01:00
James Rodewig e7c7ed6493
[DOCS] Fix `requests_per_second` reindex param (#59871) (#59876)
Corrects the `requests_per_second` query parameter used in the reindex,
delete by query, and update by query API docs.

The parameter defaults to `-1` (no throttle). `0` is not an allowed value.
2020-07-20 10:08:51 -04:00
James Rodewig 76b2dd23e2
[DOCS] Document data stream stats API (#59435) (#59874) 2020-07-20 09:50:26 -04:00
James Rodewig 32c8df68ba
[DOCS] Fix erroneous data stream ref (#59805) (#59868)
Removes an erroneous data stream reference added in #58513.

While technically possible, we don't encourage using date math to name
data streams.
2020-07-20 09:30:30 -04:00
James Rodewig 82a8d9aa0c
[DOCS] Fix keyword marker docs (#59834) (#59863)
Co-authored-by: Rui Almeida <ruial@outlook.com>
2020-07-20 09:27:42 -04:00
James Rodewig 828aa6f640
[DOCS] EQL: Remove collapsible sections from EQL search docs (#59819) (#59861) 2020-07-20 09:26:32 -04:00
James Rodewig a160daa5d9
[DOCS] Remove collapsible examples (#59820) (#59857)
Snippets are now visible without additional clicks.
2020-07-20 09:14:36 -04:00
Nik Everett 514b2f3414
Clean up a few of vwh's rough edges (#59341) (#59807)
This cleans up a few rough edged in the `variable_width_histogram`,
mostly found by @wwang500:
1. Setting its tuning parameters in an unexpected order could cause the
   request to fail.
2. We checked that the maximum number of buckets was both less than
   50000 and MAX_BUCKETS. This drops the 50000.
3. Fixes a divide by 0 that can occur of the `shard_size` is 1.
4. Fixes a divide by 0 that can occur if the `shard_size * 3` overflows
   a signed int.
5. Requires `shard_size * 3 / 4` to be at least `buckets`. If it is less
   than `buckets` we will very consistently return fewer buckets than
   requested. For the most part we expect folks to leave it at the
   default. If they change it, we expect it to be much bigger than
   `buckets`.
6. Allocate a smaller `mergeMap` in when initially bucketing requests
   that don't use the entire `shard_size * 3 / 4`. Its just a waste.
7. Default `shard_size` to `10 * buckets` rather than `100`. It *looks*
   like that was our intention the whole time. And it feels like it'd
   keep the algorithm humming along more smoothly.
8. Default the `initial_buffer` to `min(10 * shard_size, 50000)` like
   we've documented it rather than `5000`. Like the point above, this
   feels like the right thing to do to keep the algorithm happy.

Co-authored-by: Elastic Machine <elasticmachine@users.noreply.github.com>

Co-authored-by: Elastic Machine <elasticmachine@users.noreply.github.com>
2020-07-17 15:16:09 -04:00
Adam Locke 29ff05cbac
[7.x] [DOCS] Update snapshot/restore docs to align with API changes (#59730) (#59803)
* [DOCS] Updating snapshot/restore pages to align with API changes (#59730)

* Updating snapshot/restore pages to align with API changes.

* Fixing texts in delete snapshot page.

* Removing duplicate code sample and making editorial changes.

* Change "deleted" to "delete"

* Incorporating review feedback and making minor editorial changes.

* Remove titleabbrev

* Add paragraph break

* Remove titleabbrev from restore page

* Remove titleabbrev from create page

* Change "Create" to lowercase

* Change API names to lowercase

* Remove extraneous delimiters

* Change "Delete" to lowercase

* Single-sourcing warning and clarifying warning text.

* Fixing tests and removing erroneous example.
2020-07-17 14:33:18 -04:00
Dan Hermann 48df9b1a0e
Update regex file for es user agent node processor (#59697) (#59794) 2020-07-17 11:04:01 -05:00
James Rodewig 40f319a1f6
[DOCS] Reformat Painless snippets to use two-space indents (#59776) (#59791) 2020-07-17 11:31:37 -04:00
Adam Locke 6ccf3548e7
Fix Snapshot Status API Docs Test (#59775) (#59787)
Introduce a fix to tests by snapshotting a single index+shard in the snapshot that
we get the status for and verifying consistency instead of equality
for total file counts.

Co-authored-by: Armin Braun <me@obrown.io>
2020-07-17 11:11:23 -04:00
James Rodewig a672a2a2d4
[DOCS] Move highlighting docs to separate page (#59768) (#59781)
Moves the highlighting docs from the deprecated 'Request Body Search'
chapter to the new subpage of the 'Run a search chapter' section.

No substantive changes were made to the content.
2020-07-17 10:57:00 -04:00
Benjamin Trent b7f30fc929
[7.x] Adding new `require_alias` option to indexing requests (#58917) (#59769)
* Adding new `require_alias` option to indexing requests (#58917)

This commit adds the `require_alias` flag to requests that create new documents.

This flag, when `true` prevents the request from automatically creating an index. Instead, the destination of the request MUST be an alias.

When the flag is not set, or `false`, the behavior defaults to the `action.auto_create_index` settings.

This is useful when an alias is required instead of a concrete index.

closes https://github.com/elastic/elasticsearch/issues/55267
2020-07-17 10:24:58 -04:00
Stuart Tettemer 9aa476196f
Scripting: Update ctx docs with sha augment #59719 (#59736)
Backport: 352a90c
2020-07-16 18:33:08 -05:00
Stuart Tettemer c491212dc1
Scripting: fix generateContextDoc path and url #59676 (#59722)
* Add doc runtime class path
* Use getAllHttpSocketURI.get(0) instead of getAllHttpSocketURI to get a single
test cluster URL rather than a list

Backport: 3057e0f
2020-07-16 15:03:36 -05:00
James Rodewig fa2167af0a
[7.x] [DOCS] Update upgrade docs and release highlights for 7.9 (#59674) 2020-07-16 15:58:40 -04:00
James Rodewig da85a40e7e
[DOCS] Reformat `predicate_token_filter` tokenfilter (#57705) (#59714) 2020-07-16 13:35:09 -04:00
lcawl f2b530dbdb [DOCS] Re-adds coming macro in release notes 2020-07-16 09:12:39 -07:00
István Zoltán Szabó 35512a9284 [DOCS] Adds security privilege info to inference bucket aggregation (#59604) 2020-07-16 18:03:19 +02:00
Marios Trivyzas c7efbc1b83
SQL: Implement DATE_PARSE function for parsing strings into DATE values (#57391) (#59699)
Implement DATE_PARSE(<date_str>, <pattern_str>) function
which allows to parse a date string according to the specified
pattern into a date object. The patterns allowed are those of
java.time.format.DateTimeFormatter.

Closes #54962

Co-authored-by: Marios Trivyzas <matriv@users.noreply.github.com>
Co-authored-by: Patrick Jiang(白泽) <dreamlike.sky@foxmail.com>

(cherry picked from commit 647a413d9b21bd3938f1716bb19f8407e1334125)
2020-07-16 17:24:30 +02:00
Adam Locke 305b46c7cd
[DOCS] Adding get snapshot status API docs (#59355) (#59670)
* Adding get snapshot status API docs.

* Adding more fields and a link to the new page.

* Adding missing spaces in TESTRESPONSES

* Adding more parameters and making some edits.

* Marking snapshot as optional

* Marking repository as optional

* Add data type for stats

* Add data type for shard_stats

* Incorporating review feedback.

* Lots of review feedback incorporated.

* Fixing tests to unbreak CI builds.

* Changing indices to index.
2020-07-16 11:21:17 -04:00
Benjamin Trent a28547c4b4
[7.x] [ML] add new `custom` field to trained model processors (#59542) (#59700)
* [ML] add new `custom` field to trained model processors (#59542)

This commit adds the new configurable field `custom`.

`custom` indicates if the preprocessor was submitted by a user or automatically created by the analytics job.

Eventually, this field will be used in calculating feature importance. When `custom` is true, the feature importance for
the processed fields is calculated. When `false` the current behavior is the same (we calculate the importance for the originating field/feature).

This also adds new required methods to the preprocessor interface. If users are to supply their own preprocessors
in the analytics job configuration, we need to know the input and output field names.
2020-07-16 10:57:38 -04:00
István Zoltán Szabó 76fbe0a6d9 [DOCS] Sorts agg and grouping names alphabetically in PUT Transforms API docs. (#59688) 2020-07-16 12:45:29 +02:00
Przemysław Witek df4fea79cb
Add a "verbose" option to the data frame analytics stats endpoint (#59589) (#59621) 2020-07-16 09:51:31 +02:00
lcawl 4ad8bef33b [DOCS] Removes docs PR from release notes 2020-07-15 16:07:43 -07:00
James Rodewig 43481441e9
[DOCS] EQL: Update EQL search response format (#59554) (#59668) 2020-07-15 17:23:48 -04:00
James Rodewig e30af2fc35
[DOCS] Fix syntax and wording in EQL docs (#59623) (#59650) 2020-07-15 14:45:56 -04:00
Adam Locke 776e9507fb
[DOCS] Update similarity.asciidoc (#59400) (#59644)
Community contribution to fix linking issues in the Similarity module docs.

Co-authored-by: Xin Yan <SHU_Yanx@hotmail.com>
2020-07-15 14:12:00 -04:00
James Rodewig ef9b14b07e
[DOCS] Add `write_index_only` param to ds mapping tutorials (#59618) (#59639) 2020-07-15 13:02:01 -04:00
Rory Hunter b8d73a1e7e
Default gateway.auto_import_dangling_indices to false (#59302)
Backport of #58898.

Part of #48366. Now that there is a dedicated API for dangling indices, the auto-import
behaviour can default to off. Also add a note to the breaking changes for 7.9.0.
2020-07-15 17:10:42 +01:00
James Rodewig 8cac702171 [DOCS] Note that EQL timestamp field can also be date_nanos 2020-07-15 09:55:55 -04:00
James Rodewig 4e58f967de
[DOCS] Update ds overview for optional `@timestamp` mapping (#59558) (#59614) 2020-07-15 09:46:55 -04:00
Martijn Laarman a699c89133 [DOCS] Add release notes for 7.8.1 (#59594)
(cherry picked from commit f43a233948f13e487d4d0f4be668687c404a71f4)
2020-07-15 11:42:03 +02:00
Armin Braun ecf97e9415
Remove Outdated Documentation On Snapshots (#59358) (#59585)
* We now have concurrent repository operations so the one at a time limit does not apply any longer
* Initialization was never slow solely due to loading information about all existing snaphots (though this contributed)
but also because two cluster state updates and a few writes to the repository had to happen before initialization could return
   * Repo data necessary for a snapshot create operation is now cached on heap so loading it is effectively instant
   * Snapshot initialization is just a single CS update now
   * Initialization does no writes to the repository whatsoever
* Fixed missing `repository`
2020-07-15 07:49:18 +02:00
James Rodewig e5baacbe2e
[DOCS] Simplify index template snippets for data streams (#59533) (#59553)
Removes the `@timestamp` field mapping from several data stream index
template snippets.

With #59317, the `@timestamp` field defaults to a `date` field data type
for data streams.
2020-07-14 17:28:43 -04:00
James Rodewig be4483034c
[DOCS] Add example of ds index template with date_nanos mapping (#59535) (#59570) 2020-07-14 17:28:31 -04:00
Costin Leau 679619c798 EQL: Improve retrieval of results (#59552)
Instead of retrieving an entire SearchHit, get just a reference and
postpone the document retrieval when assembling the final results.
Remove sort information from results to make them consistent.
Move TumblingWindow under the sequence package.

Co-authored-by: James Rodewig <james.rodewig@elastic.co>
(cherry picked from commit bccfbcd81f2f1d3552e95e4a9ee2618fb3059bd9)
2020-07-14 23:53:57 +03:00
Armin Braun d18b434e62
Remove Artificially Low Chunk Size Limits from GCS + Azure Blob Stores (#59279) (#59564)
Removing these limits as they cause unnecessarily many object in the blob stores.
We do not have to worry about BwC of this change since we do not support any 3rd party
implementations of Azure or GCS.
Also, since there is no valid reason to set a different than the default maximum chunk size at this
point, removing the documentation (which was incorrect in the case of Azure to begin with) for the setting
from the docs.

Closes #56018
2020-07-14 22:31:07 +02:00
Julie Tibshirani 3ccc767003 Expand docs for component template merging. (#59466)
This change clarifies the order in which components are merged. It also adds
information on mapping merging, now that this has been implemented.
2020-07-14 11:08:34 -07:00
James Baiera 55c4dec360
LLRC RequestOptions add RequestConfig (#57972) (#59344)
Different kinds of requests may need different request options from the client 
default. Users can optionally set RequestConfig on a single request's 
RequestOptions to override the default. Without this, socketTimeout can only 
set at RestClient initialization.

Co-authored-by: weizijun <weizijun1989@gmail.com>
2020-07-14 13:34:53 -04:00
James Rodewig f4c46075b4
[DOCS] Add data streams to index template API docs (#59462) (#59549) 2020-07-14 12:51:47 -04:00
Tim Brooks a46e5e0f04
Increase default write queue size (#59464)
This commit increases the default write queue size to 10000. This is to
allow a greater number of pending indexing requests. This work is safe
as we have added additional memory limits. Relates to #59263.
2020-07-14 10:35:25 -06:00
Andrei Dan 7dcdaeae49
Default to @timestamp in composable template datastream definition (#59317) (#59516)
This makes the data_stream timestamp field specification optional when
defining a composable template.
When there isn't one specified it will default to `@timestamp`.

(cherry picked from commit 5609353c5d164e15a636c22019c9c17fa98aac30)
Signed-off-by: Andrei Dan <andrei.dan@elastic.co>
2020-07-14 12:36:54 +01:00
Andrei Dan 4180333bbc
[7.x] Composable templates: add a default mapping for @timestamp (#59244) (#59510)
This adds a low precendece mapping for the `@timestamp` field with
type `date`.
This will aid with the bootstrapping of data streams as a timestamp
mapping can be omitted when nanos precision is not needed.

(cherry picked from commit 4e72f43d62edfe52a934367ce9809b5efbcdb531)
Signed-off-by: Andrei Dan <andrei.dan@elastic.co>
2020-07-14 11:29:33 +01:00
debadair 7d20d32a8c
Update node.asciidoc (#59201) (#59479)
TIP block was missing due to the lack of line break prior to the "TIP"

Co-authored-by: Leaf-Lin <39002973+Leaf-Lin@users.noreply.github.com>
2020-07-13 16:51:14 -07:00
James Rodewig db89764539
[DOCS] Add data streams to rollup APIs (#59423) (#59465) 2020-07-13 16:57:40 -04:00
James Rodewig a1cf955dbd
[DOCS] Clarify that passwords are not preserved for `kibana_system` user (#59449) (#59460) 2020-07-13 16:34:11 -04:00
Lee Hinman bf1a60130d
[7.x] Add telemetery for data streams (#59433) (#59454)
This commit adds data stream info to the `/_xpack` and `/_xpack/usage` APIs. Currently the usage is
pretty minimal, returning only the number of data streams and the number of indices currently
abstracted by a data stream:

```
  ...
  "data_streams" : {
    "available" : true,
    "enabled" : true,
    "data_streams" : 3,
    "indices_count" : 17
  }
  ...
```
2020-07-13 14:30:11 -06:00
Adam Locke aa260636e5
Indicating that the size parameter defaults to 10. (#59438) (#59461) 2020-07-13 16:27:20 -04:00
James Rodewig d293e1ae36
[DOCS] Add data streams to reload search analyzers API (#59422) (#59437) 2020-07-13 12:50:47 -04:00
James Rodewig 0a7664e190
[DOCS] Add data streams to validate query API (#59420) (#59436) 2020-07-13 12:50:34 -04:00
homersimpsons f95658d1f8 [DOCS] MatchQuery: `transpositions` to `fuzzy_transpositions` (#59371) 2020-07-13 12:37:30 -04:00
Christos Soulios 3868bcc7b8
[7.x] Histogram integration on Histogram field type (#59431)
Backports #58930 to 7.x
Implements histogram aggregation over histogram fields as requested in #53285.
2020-07-13 19:36:33 +03:00
Igor Motov 1acb4aeba9
EQL: Prepare for release (#59331) (#59426)
Enables eql setting in release builds.

Relates #51613
2020-07-13 11:54:32 -04:00
Dan Hermann c228532ebd
Update docs for delete data stream API to show that multiple names are supported 2020-07-13 09:11:25 -05:00
James Rodewig 27a87c9d0c
[DOCS] Update snapshot/restore and SLM docs for data streams (#58513) (#59403)
Updates the existing snapshot/restore and SLM docs to make them
aware of data streams.
2020-07-13 09:26:51 -04:00
James Rodewig 2629a95e14
[DOCS] EQL: Document `until` keyword support (#59320) (#59408) 2020-07-13 09:05:47 -04:00
James Rodewig 85101fa487
[DOCS] Add data streams to searchable snapshot API docs (#59325) (#59409) 2020-07-13 09:05:27 -04:00
James Rodewig a357ec59f2
[DOCS] Add data streams to index APIs (#59329) (#59410) 2020-07-13 09:05:03 -04:00
James Rodewig 35a78b88ab
[DOCS] Add data streams to ILM explain API (#59343) (#59411) 2020-07-13 09:04:42 -04:00
James Rodewig 896d0ffd9b
[DOCS] EQL: Prepare docs for release (#59259) (#59407)
Changes:

* Swaps the `dev` admonitions for `experimental` admonitions
* Removes `ifdef` statements preventing the docs from appearing in
  released branches
2020-07-13 09:04:15 -04:00
James Rodewig 9d5c091f7a
[DOCS] Add data streams to EQL search docs (#58611) (#59404) 2020-07-13 09:03:55 -04:00
James Rodewig 39bcc4a1a7
[DOCS] Add ingest pipeline ex to data stream docs (#58343) (#59402) 2020-07-13 09:03:36 -04:00
Kartika Prasad 8ab0c1b4a0 Update indexing-speed.asciidoc (#59347)
typo fix
2020-07-13 12:19:43 +01:00
Dylan Mann 42035a5df3 Update ICU Analyzer Documentation (#59305) 2020-07-13 12:15:13 +01:00
István Zoltán Szabó cdf6a054c6 [DOCS] Fixes getting time features example in Painless in Transforms (#59379) 2020-07-13 10:57:59 +02:00
David Roberts 2f9d4a1c7a [DOCS] Adds extra ml-cpp PRs to release notes (#59354)
Following the rebuild of 7.8.1 two extra ml-cpp PRs will
now be released in 7.8.1.
2020-07-13 09:36:21 +01:00
David Kyle a6a27b76dc
Fix broken links to aggregation javadoc (#59083) (#59319)
Fixes links from the Java High Level Rest Client to the aggregations java docs
2020-07-11 13:28:03 +01:00
James Rodewig 1402f787f8
[DOCS] Add data streams to field caps API docs (#59326) (#59340) 2020-07-09 16:54:33 -04:00
James Rodewig 41345d4dd3
[DOCS] Add data streams to clear cache API docs (#59324) (#59339) 2020-07-09 16:54:04 -04:00
James Rodewig 77e227bf9b
[DOCS] Document custom routing support for data streams (#59323) (#59338) 2020-07-09 16:52:30 -04:00
James Rodewig ef74a68bcc
[DOCS] Document index aliases do not support data streams (#59321) (#59337) 2020-07-09 16:51:58 -04:00
Lisa Cawley 54483394ae
[DOCS] Clarify subscription requirements (#58958) (#59307) 2020-07-09 12:24:45 -07:00
James Rodewig fca722cee1
[DOCS] Add x-pack tag to data stream docs (#59241) (#59299) 2020-07-09 13:12:38 -04:00
Dimitris Athanasiou b2243337d8
[7.x][ML] Data frame analytics max_num_threads setting (#59254) (#59308)
This adds a setting to data frame analytics jobs called
`max_number_threads`. The setting expects a positive integer.
When used the user specifies the max number of threads that may
be used by the analysis. Note that the actual number of threads
used is limited by the number of processors on the node where
the job is assigned. Also, the process may use a couple more threads
for operational functionality that is not the analysis itself.

This setting may also be updated for a stopped job.

More threads may reduce the time it takes to complete the job at the cost
of using more CPU.

Backport of #59254 and #57274
2020-07-09 19:15:46 +03:00
Rory Hunter 5debd09808 Dangling indices documentation (#58751)
Part of #48366. Add documentation for the dangling indices
API added in #58176.

Co-authored-by: David Turner <david.turner@elastic.co>
Co-authored-by: Adam Locke <adam.locke@elastic.co>
2020-07-09 14:02:23 +01:00
David Kyle c5443f78ce
Add Inference Pipeline aggregation to HLRC (#59086) (#59250)
Adds InferencePipelineAggregationBuilder to the HLRC duplicating 
the server side classes
2020-07-09 13:38:45 +01:00
Andrei Stefan c0e0bca84c
Remove search_after and implicit_join_key_field (#59232) (#59280)
(cherry picked from commit 6ede6c59eff321b9fedad30e19508b9e4f788b54)
2020-07-09 12:34:01 +03:00
Bogdan Pintea acfff7b896
Add sample versions of standard deviation and variance funcs (#59093) (#59274)
* Add sample versions of standard deviation and variance functions (#59093)

* Add STDDEV_SAMP, VAR_SAMP

This commit adds the sampling variations of the standard deviation and
variance agg functions.

(cherry picked from commit 8b29817b49e386215f29cb5b3356d0183fd5d9de)

* Fix: workaround for lack of Map#of() in Java8

Replace Map#of() with a HashMap static init.
2020-07-09 10:17:13 +02:00
Adam Locke 96a06685cf
[7.x] [DOCS] Adding get snapshot api docs (#59238)
* [DOCS] Adding get snapshot API docs (#59098)

* Adding page for get snapshot API.

* Adding values for state and cleaning up some other formatting.

* Adding missing forward slash to GET request.

* Updating values for start_time and end_time in TESTRESPONSE.

* Swap "return" for "retrieve"

* Swap "return" for "retrieve" 2

* Change .snapshot to .response

* Adding response parameters and incorporating edits from review.

* Update response example to include repository info

* Change dash to underscore

* Add data type for snapshot in response

* Incorporating review comments and adding missing response definitions.

* Minor rewording in description.

* Removing multi-snapshot support for 7.x.

* Changing end_time value from build error.

* Removing .response from snippet testing.
2020-07-08 16:40:35 -04:00
James Rodewig d2c5a4c5e9
[7.x] [DOCS] Update get data stream API response (#59197) (#59221)
Updates docs and snippets for changes made to the get data stream API with
PR #59128.
2020-07-08 14:04:14 -04:00
James Rodewig 838f717e5f
[DOCS] Add data streams to security docs (#59084) (#59237) 2020-07-08 12:53:56 -04:00
James Rodewig 93a5eb0688
[DOCS] EQL: Document `size` limit for pipes (#59085) (#59236)
Changes:
* Documents the `size` default as `10`.
* Updates `size` param def to note its relation to pipes.
* Updates the `head` and `tail` pipe docs to modify sequences.
* Documents the `fetch_size` parameter.

Relates to #59014 and #59063
2020-07-08 12:22:57 -04:00
Martijn van Groningen 17bd559253
Fix the timestamp field of a data stream to @timestamp (#59210)
Backport of #59076 to 7.x branch.

The commit makes the following changes:
* The timestamp field of a data stream definition in a composable
  index template can only be set to '@timestamp'.
* Removed custom data stream timestamp field validation and reuse the validation from `TimestampFieldMapper` and
  instead only check that the _timestamp field mapping has been defined on a backing index of a data stream.
* Moved code that injects _timestamp meta field mapping from `MetadataCreateIndexService#applyCreateIndexRequestWithV2Template58956(...)` method
  to `MetadataIndexTemplateService#collectMappings(...)` method.
* Fixed a bug (#58956) that cases timestamp field validation to be performed
  for each template and instead of the final mappings that is created.
* only apply _timestamp meta field if index is created as part of a data stream or data stream rollover,
this fixes a docs test, where a regular index creation matches (logs-*) with a template with a data stream definition.

Relates to #58642
Relates to #53100
Closes #58956
Closes #58583
2020-07-08 17:30:46 +02:00
James Rodewig b27de36b5d
[DOCS] EQL: Document `maxspan` keyword (#58931) (#59223) 2020-07-08 11:04:28 -04:00
James Rodewig 37be56ab97
[DOCS] EQL: Document unsupported var comparison (#58941) (#59224)
ES EQL queries do not support the comparison of a variable, such as
a field value, to another variable.

This adds a related para and example to the EQL syntax docs.
2020-07-08 11:04:05 -04:00
David Kyle b87cef6fe7
Include the ml inference aggregation doc (#59219) (#59226)
Add to the list of pipeline aggregations
2020-07-08 14:35:08 +01:00
Yannick Welsch 0b9eb210b8
Add basic searchable snapshots usage information (#58828) (#59160)
Adds super basic usage information for searchable snapshots, to be extended later.

Backport of #58828
2020-07-08 13:09:29 +02:00
Nhat Nguyen ef5c397c0f
Sending operations concurrently in peer recovery (#58018)
Today, we send operations in phase2 of peer recoveries batch by batch
sequentially. Normally that's okay as we should have a fairly small of
operations in phase 2 due to the file-based threshold. However, if
phase1 takes a lot of time and we are actively indexing, then phase2 can
have a lot of operations to replay.

With this change, we will send multiple batches concurrently (defaults
to 1) to reduce the recovery time.

Backport of #58018
2020-07-07 22:03:31 -04:00
Lisa Cawley 2e71db71b6
[DOCS] Clarifies transform node settings (#59023) (#59192) 2020-07-07 13:54:54 -07:00
James Rodewig 6ed356ffc3
[DOCS] Replace `datatype` with `data type` (#58972) (#59184) 2020-07-07 14:59:35 -04:00
James Rodewig 045b893dd1
[DOCS] Add data streams to shard stores API docs (#59070) (#59181) 2020-07-07 14:57:47 -04:00
James Rodewig 225506f0e4
[DOCS] Add data streams to rank eval API docs (#59069) (#59179) 2020-07-07 14:57:35 -04:00
James Rodewig 1431a5436b
[DOCS] Add data streams to force merge API docs (#58951) (#59178) 2020-07-07 14:57:19 -04:00
Lisa Cawley 233857ef6e [DOCS] Adds ml-cpp PRs to release notes (#59188) 2020-07-07 11:56:40 -07:00
James Rodewig 189d69d826
[DOCS] Clarify atomic change for alias swaps (#59154) (#59164)
Small edit highlighting the fact that atomic cluster state change does not guarantee lack of errors for in-flight requests.

Co-authored-by: James Rodewig <james.rodewig@elastic.co>

Co-authored-by: Grzegorz Banasiak <grzegorz.banasiak@elastic.co>
2020-07-07 13:03:12 -04:00
Nik Everett eb169ae226
Fix lookup support in adjacency matrix (backport of #59099) (#59108)
This request:
```
POST /_search
{
  "aggs": {
    "a": {
      "adjacency_matrix": {
        "filters": {
          "1": {
            "terms": { "t": { "index": "lookup", "id": "1", "path": "t" } }
          }
        }
      }
    }
  }
}
```

Would fail with a 500 error and a message like:
```
{
  "error": {
    "root_cause": [
      {
        "type": "illegal_state_exception",
        "reason":"async actions are left after rewrite"
      }
    ]
  }
}
```

This fixes that by moving the query rewrite phase from a synchronous
call on the data nodes into the standard aggregation rewrite phase which
can properly handle the asynchronous actions.
2020-07-07 10:28:20 -04:00
David Turner 8f4f844e6e Add docs for filesystem health checks (#59134)
Documents the feature and settings introduced in #52680.

Co-authored-by: James Rodewig <james.rodewig@elastic.co>
2020-07-07 14:14:58 +01:00
James Rodewig 664b546771 [DOCS] Fix anchor syntax 2020-07-07 09:02:33 -04:00
James Rodewig a8220ad51e [DOCS] Fix anchor syntax 2020-07-07 08:57:20 -04:00
Jake Landis 604c6dd528
7.x - Create plugin for yamlTest task (#56841) (#59090)
This commit creates a new Gradle plugin to provide a separate task name
and source set for running YAML based REST tests. The only project
converted to use the new plugin in this PR is distribution/archives/integ-test-zip.
For which the testing has been moved to :rest-api-spec since it makes the most
sense and it avoids a small but awkward change to the distribution plugin.

The remaining cases in modules, plugins, and x-pack will be handled in followups.

This plugin is distinctly different from the plugin introduced in #55896 since
the YAML REST tests are intended to be black box tests over HTTP. As such they
should not (by default) have access to the classpath for that which they are testing.

The YAML based REST tests will be moved to separate source sets (yamlRestTest).
The which source is the target for the test resources is dependent on if this
new plugin is applied. If it is not applied, it will default to the test source
set.

Further, this introduces a breaking change for plugin developers that
use the YAML testing framework. They will now need to either use the new source set
and matching task, or configure the rest resources to use the old "test" source set that
matches the old integTest task. (The former should be preferred).

As part of this change (which is also breaking for plugin developers) the
rest resources plugin has been removed from the build plugin and now requires
either explicit application or application via the new YAML REST test plugin.

Plugin developers should be able to fix the breaking changes to the YAML tests
by adding apply plugin: 'elasticsearch.yaml-rest-test' and moving the YAML tests
under a yamlRestTest folder (instead of test)
2020-07-06 14:16:26 -05:00
James Rodewig d66084dcaf
[DOCS] Update data stream mapping and setting docs (#58874) (#59067) 2020-07-06 11:58:43 -04:00
Adam Locke e3469bb6e2
Removing ESS icon for xpack.security.audit.enabled. (#59078) (#59079) 2020-07-06 11:20:53 -04:00
James Rodewig 31c71914b7
[DOCS] Clean up `Use a data stream` test snippets (#58968) (#58978) 2020-07-06 08:39:04 -04:00
Przemysław Witek f35ad0d4e1
Report peak model memory in ModelSizeStats (#59017) (#59055) 2020-07-06 12:55:12 +02:00
David Kyle c651135562
[ML] Make Inference processor field_map and inference_config optional (#59010)
Relaxes the requirement that the inference ingest processor must has a
field_map and inference_config defined even if they are empty.
2020-07-06 11:35:30 +01:00
Martijn van Groningen f0dd9b4ace
Add data stream timestamp validation via metadata field mapper (#59002)
Backport of #58582 to 7.x branch.

This commit adds a new metadata field mapper that validates,
that a document has exactly a single timestamp value in the data stream timestamp field and
that the timestamp field mapping only has `type`, `meta` or `format` attributes configured.
Other attributes can affect the guarantee that an index with this meta field mapper has a
useable timestamp field.

The MetadataCreateIndexService inserts a data stream timestamp field mapper whenever
a new backing index of a data stream is created.

Relates to #53100
2020-07-06 11:32:33 +02:00
David Turner 2fdd8f3a2c Restores do not cause red health (#59015)
Since 2.0.0 (56a264cf6d) we have documented that restoring a snapshot
typically results in `red` cluster health. However since 5.0.0 (#19516)
this hasn't been true, we report `yellow` health for unassigned
primaries that will be recovered from a snapshot in the future. This
commit adjusts these docs to match today's behaviour.
2020-07-04 11:16:28 +01:00
debadair c8e3128fe4
[DOCS] Combo version of ILM docs. (#57909) (#59029)
* [DOCS] Combo version of ILM docs.

* [DOCS] Moved tutorial from Kibana.

* Adds documentation for index lifecycle policies (#28705)

* [DOCS] Adds documentation for index lifecycle policies

* [DOCS] Updated image for policy options to show all menu items

* Update create-policy.asciidoc

* [DOCS] Incorporated review comments on hot and warm phase

* [DOCS] Additional changes to warm phase

* [DOCS] Removed the word open in the warm phase

* Adds X-Pack icon for ILM (#34178)

* Add ILM tutorial (#59502)

* Add tutorial for ILM with filebeat

* Change screenshots and add additional steps

* Update screenshots, add numbered steps, and other minor edits

* Incorporate feedback: update links, formatting, and minor edits

* Move tip inline with list

* Apply suggestions from code review

Co-Authored-By: James Rodewig <james.rodewig@elastic.co>

* Move TIP inline . . . again

* Put TIP inline

Co-authored-by: James Rodewig <james.rodewig@elastic.co>

* Updates for navigation redesign (#68709)

* [DOCS] Updates for navigation redesign

* Getting started

* Set up text

* Discover

* Dashboard, Graph, ML, Maps, APM, SIEM, Dev tools

* Dev Tools, Stack Monitoring, Management

* Management

* Final changes

* [DOCS] Updates for navigation redesign

* [DOCS] Updates CCR monitoring screenshots

* updates SIEM screenshot and Cases overview text

* Added Brandon's APM image

* [DOCS] Refines CCR shard screenshot

* Removed merge conflict image file

Co-authored-by: lcawl <lcawley@elastic.co>
Co-authored-by: Ben Skelker <ben.skelker@elastic.co>

* [DOCS] Put API examples in collapsible sections like ML does

* Fix include

* Added tutorial images

* Fixed images

* Add short title for FB tutorial

* Add missing files

* Incorporate review feedback

* review feedback

* Incorporated review feedback

Co-authored-by: gchaps <33642766+gchaps@users.noreply.github.com>
Co-authored-by: Lisa Cawley <lcawley@elastic.co>
Co-authored-by: Melori Arellano <melori@elastic.co>
Co-authored-by: James Rodewig <james.rodewig@elastic.co>
Co-authored-by: Kaarina Tungseth <kaarina.tungseth@elastic.co>
Co-authored-by: Ben Skelker <ben.skelker@elastic.co>

Co-authored-by: gchaps <33642766+gchaps@users.noreply.github.com>
Co-authored-by: Lisa Cawley <lcawley@elastic.co>
Co-authored-by: Melori Arellano <melori@elastic.co>
Co-authored-by: James Rodewig <james.rodewig@elastic.co>
Co-authored-by: Kaarina Tungseth <kaarina.tungseth@elastic.co>
Co-authored-by: Ben Skelker <ben.skelker@elastic.co>
2020-07-03 13:01:08 -07:00
Benjamin Trent b9d9964d10
[ML] add exponent output aggregator to inference (#58933) (#59016)
* [ML] add exponent output aggregator to inference

* fixing docs

Co-authored-by: Elastic Machine <elasticmachine@users.noreply.github.com>
2020-07-03 14:51:00 -04:00
Lisa Cawley 935a49a8d6 [DOCS] Deprecates node.ml (#59024) 2020-07-03 11:10:05 -07:00
Lisa Cawley f9b365db6c [DOCS] Edits ML circuit breaker settings (#59026) 2020-07-03 11:07:46 -07:00
Bogdan Pintea 3d96d91efb
[7.x] SQL: fix handling of escaped chars in JDBC connection string (#58429) (#58977)
SQL: fix handling of escaped chars in JDBC connection string (#58429)

This commit fixes an issue emerging when the connection string URI
contains escaped characters.

The original URI is pre-parsed in order to re-assemble a new URI having
the optional elements filled in with defaults. The new URI has been
using however the unescaped query and fragment parts. So if these
contained any escaped `&` or `=` (such as in the password option value),
the unescaping would reveal them and make them later interfere with the
options parsing.

The commit changes that, so that the new URI be built from the unescaped
"raw" parts of the original URI.

(cherry picked from commit 94eb5a05e79c6e203de548d05b13e00295bd4489)
2020-07-03 17:03:00 +02:00
David Kyle f6a0c2c59d
[7.x] Pipeline Inference Aggregation (#58965)
Adds a pipeline aggregation that loads a model and performs inference on the
input aggregation results.
2020-07-03 09:29:04 +01:00
debadair b769e9143d
[DOCS] Add simulate ref (#58579) (#58987)
* [DOCS] Add simulate ref pages

* Add links & experimental tags

* Fixed simulate index response

* Apply suggestions from code review

Co-authored-by: James Rodewig <james.rodewig@elastic.co>

*Incorporate review feedback.
2020-07-02 19:05:32 -07:00
DeDe Morton 2c43421208 [DOCS] Change Beats links to refactored getting started docs (#58790) 2020-07-02 17:11:25 -07:00
Adam Locke 20d04081ec
[7.x] [DOCS] Add supported ESS settings to ES docs (#57953) (#58981)
* Adding ESS icons to supported ES settings.

* Adding new file for supported ESS settings.

* Adding supported ESS settings for HTTP and disk-based shard allocation.

* Adding more supported settings for ESS.

* Adding descriptions for each Cloud section, plus additional settings.

* Adding new warehouse file for Cloud, plus additional settings.

* Adding node settings for Cloud.

* Adding audit settings for Cloud.

* Resolving merge conflict.

* Adding SAML settings (part 1).

* Adding SAML realm encryption and signing settings.

* Adding SAML SSL settings.

* Adding Kerberos realm settings.

* Adding OpenID Connect Realm settings.

* Adding OpenID Connect SSL settings.

* Resolving leftover Git merge markers.

* Removing Cloud settings page and link to it.

* Add link to mapping source

* Update docs/reference/docs/reindex.asciidoc

* Incorporate edit of HTTP settings

* Remove "cloud" from tag and ID

* Remove "cloud" from tag and update description

* Remove "cloud" from tag and ID

* Change "whitelists" to "specifies"

* Remove "cloud" from end tag

* Removing cloud from IDs and tags.

* Changing link reference to fix build issue.

* Adding index management page for missing settings.

* Removing warehouse file for Cloud and moving settings elsewhere.

* Clarifying true/false usage of http.detailed_errors.enabled.

* Changing underscore to dash in link to fix ci build.
2020-07-02 19:40:45 -04:00
James Rodewig 84444e1a54
[DOCS] Add data streams to flush API docs (#58950) (#58975) 2020-07-02 17:26:49 -04:00
James Rodewig e64fe15c1f
[DOCS] Add data streams to cluster APIs docs (#58945) (#58974)
Makes existing docs for the cluster health and cluster state APIs aware
of data streams.
2020-07-02 17:20:55 -04:00
Przemysław Witek 751e84e4c8
Rename regression evaluation metrics to make the names consistent with loss functions (#58887) (#58927) 2020-07-02 17:35:55 +02:00
James Rodewig 9569375ae7
[DOCS] Add data streams to remove lifecycle policy API (#58777) (#58921) 2020-07-02 09:58:54 -04:00
James Rodewig 04ff24ca8e
[DOCS] Add data streams to deprecation info API docs (#58685) (#58920) 2020-07-02 09:52:33 -04:00
James Rodewig 6436792aac
[DOCS] Fix headings for simple analyzer docs (#58910) (#58918) 2020-07-02 09:52:05 -04:00
James Rodewig ff04212257
[DOCS] Fix snippet tests for resolve API docs (#58908) (#58911) 2020-07-02 09:27:14 -04:00
James Rodewig 57d679ff64
[DOCS] Add data streams to put mapping and update settings API docs (#58849) (#58913) 2020-07-02 09:26:57 -04:00
James Rodewig fc5b2b2be4 [DOCS] Fix `scroll` param typo 2020-07-02 08:48:07 -04:00
Yang Wang a5a8b4ae1d
Add cache for application privileges (#55836) (#58798)
Add caching support for application privileges to reduce number of round-trips to security index when building application privilege descriptors.

Privilege retrieving in NativePrivilegeStore is changed to always fetching all privilege documents for a given application. The caching is applied to all places including "get privilege", "has privileges" APIs and CompositeRolesStore (for authentication).
2020-07-02 11:50:03 +10:00
debadair 4471cd7527 [DOCS] Fix cannot must typo. (#58884) 2020-07-01 17:46:09 -07:00
Adam Locke 9c77862a23
[7.x] [DOCS] Adding delete snapshot API docs. (#58865) (#58871)
* [DOCS] Adding delete snapshot API docs.

* Adding TESTSETUP snippets and fixing original TEST.

* Removing extraneous TESTSETUP.

* Revising <snapshot> description.

* Removing TEST.

* Streamline delete API description.

* Improve TESTSETUP for snippets.
2020-07-01 15:03:02 -04:00
James Rodewig a966513eae
[DOCS] Remove problematic terms (#58832) (#58851) 2020-07-01 13:47:14 -04:00
Nik Everett 326cce624b Document using stored scripts for ingest (#58783)
This documents using stored scripts for complex conditionals in indest.
2020-07-01 13:36:00 -04:00
James Rodewig 860c94ca60
[DOCS] Add redirects for 404 pages (#58846) (#58853) 2020-07-01 12:02:59 -04:00
Lee Hinman d3d03fc1c6
[7.x] Add default composable templates for new indexing strategy (#57629) (#58757)
Backports the following commits to 7.x:

    Add default composable templates for new indexing strategy (#57629)
2020-07-01 09:32:32 -06:00
James Rodewig 7a8da9daa3
[DOCS] Document open requests for data streams (#58615) (#58621)
Adds an open API example to the data streams docs. Also updates the
existing open API docs to make them aware of data streams.
2020-07-01 11:22:45 -04:00
Dan Hermann 27e95d2162
[DOCS] Resolve index API (#58206) (#58843) 2020-07-01 09:41:15 -05:00
Przemysław Witek 909649dd15
[7.x] Implement pseudo Huber loss (PseudoHuber) evaluation metric for regression analysis (#58734) (#58825) 2020-07-01 14:52:06 +02:00
David Turner 822b7421ce Forbid read-only-allow-delete block in blocks API (#58727)
The read-only-allow-delete block is not really under the user's control
since Elasticsearch adds/removes it automatically. This commit removes
support for it from the new API for adding blocks to indices that was
introduced in #58094.
2020-07-01 13:18:26 +01:00