Add release 7.10 highlights (#63468)

Co-authored-by: James Rodewig <40268737+jrodewig@users.noreply.github.com>
Co-authored-by: Jay Modi <jaymode@users.noreply.github.com>
Co-authored-by: Adrien Grand <jpountz@gmail.com>
This commit is contained in:
Andrei Dan 2020-10-13 13:25:08 +01:00 committed by GitHub
parent 26f58c8bdb
commit b3aa575f0d
No known key found for this signature in database
GPG Key ID: 4AEE18F83AFDEB23
1 changed files with 155 additions and 15 deletions

View File

@ -45,7 +45,7 @@ with this change.
// tag::notable-highlights[] // tag::notable-highlights[]
[discrete] [discrete]
[[data-tier-formalization]] [[data-tier-formalization]]
=== Data Tier Formalization === Data tiers
7.10 introduces the concept of formalized data tiers within Elasticsearch. <<data-tiers,Data tiers>> 7.10 introduces the concept of formalized data tiers within Elasticsearch. <<data-tiers,Data tiers>>
are a simple, integrated approach that gives users control over optimizing for cost, are a simple, integrated approach that gives users control over optimizing for cost,
performance, and breadth/depth of data. Prior to this formalization, many users configured their own performance, and breadth/depth of data. Prior to this formalization, many users configured their own
@ -62,19 +62,159 @@ Newly created indices abstracted by a <<data-streams,data stream>> will be alloc
the `data_hot` tier automatically, while standalone indices will be allocated to the `data_hot` tier automatically, while standalone indices will be allocated to
the `data_content` tier automatically. Nodes with the pre-existing `data` role are the `data_content` tier automatically. Nodes with the pre-existing `data` role are
considered to be part of all tiers. considered to be part of all tiers.
// end::notable-highlights[]
// Use the notable-highlights tag to mark entries that [discrete]
// should be featured in the Stack Installation and Upgrade Guide: [[points-in-time-for-search]]
// tag::notable-highlights[] === Points in time (PITs) for search
// [discrete]
// === Heading
//
// Description.
// end::notable-highlights[]
// Omit the notable highlights tag for entries that only need to appear in the ES ref: In 7.10, we're introducing points in time (PITs), a lightweight way to preserve
// [discrete] index state over searches. PITs improve end-user experience by making UIs more
// === Heading reactive.
//
// Description. By default, a search request waits for complete results before returning a
response. For example, a search that retrieves top hits and aggregations returns
a response only after both top hits and aggregations are computed. However,
aggregations are usually slower and more expensive to compute than top hits.
Instead of sending a combined request, you can send two separate requests: one
for top hits and another one for aggregations. With separate search requests, a
UI can display top hits as soon as they're available and display aggregation
data after the slower aggregation request completes. You can use a PIT to ensure
both search requests run on the same data and index state.
To use a PIT in a search, you must first explicitly create the PIT using the new
{ref}/point-in-time-api.html[open PIT API]. PITs get automatically garbage-collected
after `keep_alive` if no follow-up request extends their duration.
[source,console]
----
POST /my-index-000001/_pit?keep_alive=1m
----
// TEST[setup:my_index]
The API returns a PIT ID you can use in search requests. You can also
configure by how long to extend your PIT's lifespan using the search request's
`keep_alive` parameter.
[source,console]
----
POST /_search
{
"size": 100,
"query": {
"match" : {
"title" : "elasticsearch"
}
},
"pit": {
"id": "46ToAwMDaWR4BXV1aWQxAgZub2RlXzEAAAAAAAAAAAEBYQNpZHkFdXVpZDIrBm5vZGVfMwAAAAAAAAAAKgFjA2lkeQV1dWlkMioGbm9kZV8yAAAAAAAAAAAMAWICBXV1aWQyAAAFdXVpZDEAAQltYXRjaF9hbGw_gAAAAA==",
"keep_alive": "1m"
}
}
----
// TEST[catch:missing]
PITs automatically close when their `keep_alive` period ends. You can
also manually close PITs you no longer need using the
{ref}/point-in-time-api.html[close PIT API]. Closing a PIT releases the
resources needed to maintain the PIT's index state.
[source,console]
----
DELETE /_pit
{
"id" : "46ToAwMDaWR4BXV1aWQxAgZub2RlXzEAAAAAAAAAAAEBYQNpZHkFdXVpZDIrBm5vZGVfMwAAAAAAAAAAKgFjA2lkeQV1dWlkMioGbm9kZV8yAAAAAAAAAAAMAWIBBXV1aWQyAAA="
}
----
// TEST[catch:missing]
For more information about using PITs in search, see
{ref}/paginate-search-results.html#search-after[Paginate search results with
`search_after`] or the {ref}/point-in-time-api.html[PIT API documentation].
[discrete]
[[support-for-request-level-circuit-breakers]]
=== Request-level circuit breakers on coordinating nodes
You can now use a coordinating node to account for memory used to perform
partial and final reduce of aggregations in the request circuit breaker. The
search coordinator adds the memory that it used to save and reduce the results
of shard aggregations in the request circuit breaker. Before any partial or
final reduce, the memory needed to reduce the aggregations is estimated and a
CircuitBreakingException is thrown if exceeds the maximum memory allowed in this
breaker.
This size is estimated as roughly 1.5 times the size of the serialized
aggregations that need to be reduced. This estimation can be completely off for
some aggregations but it is corrected with the real size after the reduce
completes. If the reduce is successful, we update the circuit breaker to remove
the size of the source aggregations and replace the estimation with the
serialized size of the newly reduced result.
[discrete]
[[eql-case-sensitivity-operator]]
=== EQL: Case-sensitivity and the `:` operator
In 7.10, we made most EQL operators and functions case-sensitive by default.
We've also added `:`, a new case-insensitive equal operator. Designed for
security use cases, you can use the `:` operator to search for strings in
Windows event logs and other event data containing a mix of letter cases.
[source,console]
----
GET /my-index-000001/_eql/search
{
"query": """
process where process.executable : "c:\\\\windows\\\\system32\\\\cmd.exe"
"""
}
----
// TEST[setup:sec_logs]
For more information, see the {ref}/eql-syntax.html[EQL
syntax documentation].
[discrete]
[[deprecate-rest-api-access-to-system-indices]]
=== REST API access to system indices is deprecated
We are deprecating REST API access to system indices. Most REST API
requests that attempt to access system indices will return the following
deprecation warning:
[source,text]
----
this request accesses system indices: [.system_index_name], but in a future
major version, direct access to system indices will be prevented by default
----
The following REST API endpoints access system indices as part of their
implementation and will not return the deprecation warning:
* `GET _cluster/health`
* `GET {index}/_recovery`
* `GET _cluster/allocation/explain`
* `GET _cluster/state`
* `POST _cluster/reroute`
* `GET {index}/_stats`
* `GET {index}/_segments`
* `GET {index}/_shard_stores`
* `GET _cat/[indices,aliases,health,recovery,shards,segments]`
We are also adding a new metadata flag to track indices. {es} will automatically
add this flag to any existing system indices during upgrade.
[discrete]
[[add-system-read-thread-pool]]
=== New thread pools for system indices
We've added two new thread pools for system indices: `system_read` and
`system_write`. These thread pools ensure system indices critical to the Elastic
Stack, such as those used by security or Kibana, remain responsive when
a cluster is under heavy query or indexing load.
`system_read` is a `fixed` thread pool used to manage resources for
read operations targeting system indices. Similarly, `system_write` is a
`fixed` thread pool used to manage resources for write operations targeting
system indices. Both have a maximum number of threads equal to `5`
or half of the available processors, whichever is smaller.
// end::notable-highlights[]