7544 Commits

Author SHA1 Message Date
David Turner
9450ea08b4 Log and track open/close of transport connections (#60297)
Transport connections between nodes remain in place until one or other
node shuts down or the connection is disrupted by a flaky network.
Today it is very difficult to demonstrate that transient failures and
cluster instability are caused by the network even though this is often
the case. In particular, transport connections open and close without
logging anything, even at `DEBUG` level, making it very hard to quantify
the scale of the problem or to correlate the networking problems with
external events.

This commit adds the missing `DEBUG`-level logging when transport
connections open and close, and also tracks the total number of
transport connections a node has opened as a measure of the stability of
the underlying network.
2020-07-28 17:08:04 +01:00
Adam Locke
ee18538fd7
[DOCS] Adds table with icons for version compatibility (#60159) (#60302)
* Adds table with icons for simplicity.

* Updating table for clarity.

* Changing table formatting and incorporating more feedback.

* Changing table alignment.
2020-07-28 11:08:58 -04:00
James Rodewig
4eee6d274d
[DOCS] Fix broken link to Lucene docs (#59365) (#60290)
Co-authored-by: Alexander Reelsen <alexander@reelsen.net>
2020-07-28 09:09:48 -04:00
Yannick Welsch
a55c869aab Properly document keepalive and other tcp options (#60216)
Keepalive options are not well-documented (only in transport section, although also available at http and network level).

Co-authored-by: David Turner <david.turner@elastic.co>
Co-authored-by: James Rodewig <40268737+jrodewig@users.noreply.github.com>
2020-07-28 11:10:04 +02:00
Yannick Welsch
ffe114b890 Set specific keepalive options by default on supported platforms (#59278)
keepalives tell any intermediate devices that the connection remains alive, which helps with overzealous firewalls that are
killing idle connections. keepalives are enabled by default in Elasticsearch, but use system defaults for their
configuration, which often times do not have reasonable defaults (e.g. 7200s for TCP_KEEP_IDLE) in the context of
distributed systems such as Elasticsearch.

This PR sets the socket-level keep_alive options for network.tcp.{keep_idle,keep_interval} to 5 minutes on configurations
that support it (>= Java 11 & (MacOS || Linux)) and where the system defaults are set to something higher than 5
minutes. This helps keep the connections alive while not interfering with system defaults or user-specified settings
unless they are deemed to be set too high by providing better out-of-the-box defaults.
2020-07-28 11:10:04 +02:00
James Rodewig
aba785cb6e
[DOCS] Update my-index examples (#60132) (#60248)
Changes the following example index names to `my-index-000001` for consistency:

* `my-index`
* `my_index`
* `myindex`
2020-07-27 15:58:26 -04:00
James Rodewig
3bb58eb5c1
[DOCS] Fix fuzzy_rewrite ref in match query docs (#60237) (#60251) 2020-07-27 15:36:09 -04:00
James Rodewig
1178f5c6db
[DOCS] Fix ingest processor docs for autogen doc IDs (#60147) (#60242)
If you autogen doc IDs, you cannot use the `{{_id}}` value in an ingest
processor.

This adds a related admonition to the ingest processor docs.
2020-07-27 13:55:21 -04:00
James Rodewig
95d7ce76ec
[DOCS] Fix rewrite => fuzzy_rewrite in multi match query docs (#60175) (#60233)
Co-authored-by: homersimpsons <guillaume.alabre@gmail.com>
2020-07-27 12:33:14 -04:00
James Rodewig
7a23c6b6ec
[DOCS] Fix formatting in simple query string query docs (#60226)
Co-authored-by: Ulas Keles <ulaskeles@users.noreply.github.com>
2020-07-27 12:20:02 -04:00
James Rodewig
dbd7e7793a
[DOCS] Fix default gap policy for moving fn, moving avg aggs (#60223) 2020-07-27 12:08:33 -04:00
lcawl
a27d630bdf [DOCS] Removes coming tag 2020-07-27 07:55:18 -07:00
James Rodewig
747f8bfe79
[DOCS] Add Kibana screenshots to data stream docs (#60118) (#60217) 2020-07-27 10:39:32 -04:00
James Rodewig
608a5b9e71
[DOCS] Clarify compatibility for upgrade via reindex (#60045) (#60209)
Co-authored-by: Inbar Shimshon <inbar.shimshon@elastic.co>
2020-07-27 09:38:39 -04:00
James Rodewig
08e11814c3
[DOCS] Fix clarity of 7.6 derived key breaking change (#60154) 2020-07-27 08:36:48 -04:00
David Turner
53fa52d618 Fix whitespace bug in #59222 2020-07-27 12:26:33 +01:00
David Turner
d8fdb82efb Suggest reducing tcp_retries2 (#59222)
Adds documentation suggesting reducing `tcp_retries2` on Linux to detect
network partitions more quickly.

Relates #34405
2020-07-27 11:40:12 +01:00
debadair
284c61ad19
[DOCS] Refactored index-templates topic. (#59737) (#60165)
* [DOCS] Refactored index-templates topic.

* [DOCS] Add separate files.

* [DOCS] Add delete component template.

* Apply suggestions from code review

Co-authored-by: James Rodewig <james.rodewig@elastic.co>

* [DOCS] Incorporated review comments
2020-07-23 19:48:19 -07:00
Lisa Cawley
2665bfffce
[DOCS] Fix security links in machine learning APIs (#60098) (#60152) 2020-07-23 16:43:10 -07:00
Lisa Cawley
cc6edc39a1
[DOCS] Refresh transform screenshots with histograms (#59264) (#60145) 2020-07-23 11:14:50 -07:00
James Rodewig
2e01f652c1
[DOCS] Move search sort docs to separate page (#60123) (#60142)
Moves the search sort docs from the deprecated 'Request Body Search'
page to a new subpage of 'Run a search'.

No substantive changes were made to the content.
2020-07-23 13:44:47 -04:00
Albert Zaharovits
2eaf5e1c25
[DOCS] Mapping updates are deprecated for ingestion privileges (#60024)
This PR contains the deprecation notice that `create`, `create_doc`, `index` and
`write` ingest privileges do not permit mapping updates in version 8. It also
updates the docs description of said privileges. 

This should've been part of #58784
2020-07-23 19:49:23 +03:00
James Rodewig
988e8c8fc6
[DOCS] Swap [float] for [discrete] (#60134)
Changes instances of `[float]` in our docs for `[discrete]`.

Asciidoctor prefers the `[discrete]` tag for floating headings:
https://asciidoctor.org/docs/asciidoc-asciidoctor-diffs/#blocks
2020-07-23 12:42:33 -04:00
Adrien Grand
716a3d5a21 Mention how CCR can help optimize indexing throughput. (#54870) 2020-07-23 18:40:40 +02:00
Martijn Laarman
890d35f74d [DOCS] note breaking change from 7.8.1 in migration guide (#59642)
Co-authored-by: Adam Locke <adam.locke@elastic.co>
(cherry picked from commit b0c34020ed6a10fda8e6efa9af343bd283954ec5)
2020-07-23 13:18:46 +02:00
Julie Tibshirani
aa57bbd422
Consolidate validation for 'docvalue_fields'. (#60065)
This improves modularity and also fixes some issues when `docvalues_fields` is
used within `inner_hits` or the `top_hits` agg:
* We previously didn't resolve wildcards in field names.
* We also forgot to enforce the limit `index.max_docvalue_fields_search`.
2020-07-22 17:26:58 -07:00
James Rodewig
67b07ec386
[DOCS] Remove SQL access settings page (#60078) (#60089)
This page previously documented `xpack.sql.enabled`.

However, in 7.8 and above, `xpack.sql.enabled` is always enabled and
the setting has no effect. There is no reason to maintain this page.
2020-07-22 16:59:21 -04:00
James Rodewig
f8976505cb
[DOCS] Correct the default value of ignore_throttled param (#60036) (#60086)
Co-authored-by: bellengao <gbl_long@163.com>
2020-07-22 16:53:18 -04:00
James Rodewig
0c9791798d
[7.x] [DOCS] Reformat snippets to use two-space indents (#60080) 2020-07-22 15:57:49 -04:00
Lisa Cawley
9ba017f699
[DOCS] Changes level offset of transform pages (#60066) (#60075) 2020-07-22 11:22:57 -07:00
Tim Brooks
ba01540d7e
Implement human readable indexing pressure stats (#60058)
The indexing pressure stats do not currently have human readable
variants. This commit add human readable variants and updates the
documentation.
2020-07-22 12:07:59 -06:00
James Rodewig
ed10d7407c
[DOCS] Fix shrink index API prereqs (#59985) (#60067) 2020-07-22 14:06:40 -04:00
Tim Brooks
ceb54ed655
Add indexing pressure documentation (#59456)
This commit adds documentation about the new indexing pressure memory
limit setting and exposure of this metrics in node stats.
2020-07-22 10:09:18 -06:00
Adam Locke
0a73225cd8
[DOCS] Adding new page for restore snapshot API (#59937) (#60055)
* Adding new page for restore snapshot API.

* Improving test cases, lots of edits, and streamlining content.

* Incorporating review suggestions and feedback.

* Specify `index alias` vs `alias`

* Change parameter order

* Provide clarity around regular expression

* Add link to SLM parameters

* Split sentences in example

* Adding link to master node page.
2020-07-22 12:08:55 -04:00
Lisa Cawley
46d33b1586
[DOCS] 7.9.0 release notes (#60053) 2020-07-22 08:40:59 -07:00
Emily Li
5f27a95346 Fix grammar mistake in SQL data type docs. (#60028)
Remove an extra 'when'.
2020-07-21 16:15:06 -07:00
James Rodewig
293cb8d48c
[DOCS] Fix typo in thread pools docs (#59944) (#60019)
Fix typo where available processors should be allocated processors.

Co-authored-by: Leaf-Lin <39002973+Leaf-Lin@users.noreply.github.com>
2020-07-21 17:04:36 -04:00
James Rodewig
401e12dc2b
[DOCS] Fix data stream docs (#59818) (#60010) 2020-07-21 17:04:13 -04:00
James Rodewig
04c68ba740
[DOCS] Update search docs to use my-index dataset (#60005) (#60012) 2020-07-21 16:14:44 -04:00
James Rodewig
b302b09b85
[DOCS] Reformat snippets to use two-space indents (#59973) (#59994) 2020-07-21 15:49:58 -04:00
David Roberts
606b7ea139 [DOCS] Adds extra ml-cpp PRs to release notes (#59967) 2020-07-21 11:47:36 -07:00
Tim Brooks
ed315442ac
Update thread pool docs about WRITE queue size (#59643)
This commit updates the thread pool documentation to reflect the change
in the WRITE thread pool default queue size.
2020-07-21 12:38:03 -06:00
James Rodewig
32d7fa1541
[DOCS] Introduce basic ECS logs test (#59713) (#59997)
Adds a new `my-index-00001` REST test for docs snippets.

This test can serve as a lightweight replacement for
our existing `twitter` REST tests.

The new dataset is:

* Based on Apache logs, which is better aligned with Elastic use cases
* Compliant with ECS
* Similar to the existing `twitter` data set, containing the same field data types
* Lightweight, which should keep existing test runtimes roughly the same

Also updates the search API reference docs to use the new test.
2020-07-21 13:25:53 -04:00
James Rodewig
fb40ccf8a4
[DOCS] Mark data stream stats API as stable (#59978) (#59987)
Removes experimental admon from data stream stats API.
Relates to #59860.
2020-07-21 11:22:36 -04:00
malpani
0555fef799 Support ignore_keywords flag for word delimiter graph token filter (#59563)
This commit allows customizing the word delimiter token filters to skip processing 
tokens tagged as keyword through the `ignore_keywords` flag Lucene's 
WordDelimiterGraphFilter already exposes.

Fix for #59491
2020-07-21 16:11:55 +01:00
Howard
466e947b0e
[DOCS] Fix missing punctuation in agg docs (#59823) 2020-07-21 10:19:29 -04:00
Przemysław Witek
283a1f605c
Rename binary_soft_classification evaluation to outlier_detection (#59951) (#59970) 2020-07-21 15:15:04 +02:00
Lisa Cawley
fb212269ce
[DOCS] Changes level offset of anomaly detection pages (#59911) (#59940) 2020-07-20 17:04:59 -07:00
Julie Tibshirani
8dc5880c3f Add 'point' to the top-level field type docs. (#59731)
Before it was missing from the list. This PR also renames the 'geo data types'
section to 'spatial data types' and consolidates the geo and cartesian types
into that section.
2020-07-20 16:30:12 -07:00
Lisa Cawley
9633d503d8
[DOCS] Changes level offset for anomaly detection APIs (#59920) (#59928) 2020-07-20 13:10:54 -07:00