[DOCS] Removed unused pages (#58209)
This commit is contained in:
debadair
GitHub
parent
3249ee9a86
commit
cfef2b2bec
|
|
@ -1,371 +0,0 @@
|
||||||
[[release-highlights-7.0.0]]
|
|
||||||
== 7.0.0 release highlights
|
|
||||||
++++
|
|
||||||
<titleabbrev>7.0.0</titleabbrev>
|
|
||||||
++++
|
|
||||||
|
|
||||||
//NOTE: The notable-highlights tagged regions are re-used in the
|
|
||||||
//Installation and Upgrade Guide
|
|
||||||
|
|
||||||
//tag::notable-highlights[]
|
|
||||||
[float]
|
|
||||||
==== Adaptive replica selection enabled by default
|
|
||||||
|
|
||||||
In Elasticsearch 6.x and prior, a series of search requests to the same shard
|
|
||||||
would be forwarded to the primary and each replica in round-robin fashion. This
|
|
||||||
could prove problematic if one node starts a long garbage collection --- search
|
|
||||||
requests could still be forwarded to the slow node regardless and would have an
|
|
||||||
impact on search latency.
|
|
||||||
|
|
||||||
In 6.1, we added an experimental
|
|
||||||
{ref}/search.html#search-adaptive-replica[adaptive replica selection] feature.
|
|
||||||
Each node tracks and compares how long search requests to
|
|
||||||
other nodes take, and uses this information to adjust how frequently to send
|
|
||||||
requests to shards on particular nodes. In our benchmarks, this results in an
|
|
||||||
overall improvement in search throughput and reduced 99th percentile latencies.
|
|
||||||
|
|
||||||
This option was disabled by default throughout 6.x, but we’ve heard feedback
|
|
||||||
from our users that have found the setting to be very beneficial, so we’ve
|
|
||||||
turned it on by default starting in Elasticsearch 7.0.0.
|
|
||||||
//end::notable-highlights[]
|
|
||||||
|
|
||||||
//tag::notable-highlights[]
|
|
||||||
[float]
|
|
||||||
=== Skip shard refreshes if a shard is "search idle"
|
|
||||||
|
|
||||||
Elasticsearch 6.x and prior {ref}/indices-refresh.html[refreshed] indices
|
|
||||||
automatically in the background, by default every second. This provides the
|
|
||||||
“near real-time” search capabilities Elasticsearch is known for: results are
|
|
||||||
available for search requests within one second after they'd been added, by
|
|
||||||
default. However, this behavior has a significant impact on indexing performance
|
|
||||||
if the refreshes are not needed, (e.g., if Elasticsearch isn’t servicing any
|
|
||||||
active searches).
|
|
||||||
|
|
||||||
Elasticsearch 7.0 is much smarter about this behavior by introducing the
|
|
||||||
notion of a shard being "search idle". A shard now transitions to being search
|
|
||||||
idle after it hasn't had any searches for
|
|
||||||
{ref}/index-modules.html#dynamic-index-settings[thirty seconds], by default.
|
|
||||||
Once a shard is search idle, all scheduled refreshes will
|
|
||||||
be skipped until a search comes through, which will trigger the next scheduled
|
|
||||||
refresh. We know that this is going to significantly increase the indexing
|
|
||||||
throughput for many users. The new behavior is only applied if there is no
|
|
||||||
explicit {ref}/index-modules.html#dynamic-index-settings[refresh interval set],
|
|
||||||
so do set the refresh
|
|
||||||
interval explicitly for any indices on which you prefer the old behavior.
|
|
||||||
//end::notable-highlights[]
|
|
||||||
|
|
||||||
//tag::notable-highlights[]
|
|
||||||
[float]
|
|
||||||
=== Default to one shard
|
|
||||||
|
|
||||||
One of the biggest sources of troubles we’ve seen over the years from our users
|
|
||||||
has been over-sharding and defaults play a big role in that. In Elasticsearch
|
|
||||||
6.x and prior, we defaulted to five shards by default per index. If you had one
|
|
||||||
daily index for ten different applications and each had the default of five
|
|
||||||
shards, you were creating fifty shards per day and it wasn't long before you had
|
|
||||||
thousands of shards even if you were only indexing a few gigabytes of data per
|
|
||||||
day. Index Lifecycle Management was a first step to help with this: providing
|
|
||||||
native rollover functions to create indexes by size instead of (just) by day and
|
|
||||||
built-in shrink functionality to shrink the number of shards per
|
|
||||||
index. Defaulting indices to one shard is the next step in helping to reduce
|
|
||||||
over-sharding. Of course, if you have another preferred primary shard count, you
|
|
||||||
can set it via the index settings.
|
|
||||||
//end::notable-highlights[]
|
|
||||||
|
|
||||||
//tag::notable-highlights[]
|
|
||||||
[float]
|
|
||||||
=== Lucene 8
|
|
||||||
|
|
||||||
As with every major release, we look to support the latest major version of
|
|
||||||
Lucene, along with all the goodness that comes with it. That includes all the
|
|
||||||
developments that we contributed to the new Lucene version. Elasticsearch 7.0
|
|
||||||
bundles Lucene 8, which is the latest version of Lucene. Lucene version 8 serves
|
|
||||||
as the foundation for many functional improvements in the rest of Elasticsearch,
|
|
||||||
including improved search performance for top-k queries and better ways to
|
|
||||||
combine relevance signals for your searches while still maintaining speed.
|
|
||||||
//end::notable-highlights[]
|
|
||||||
|
|
||||||
//tag::notable-highlights[]
|
|
||||||
[float]
|
|
||||||
=== Introduce the ability to minimize round-trips in {ccs}
|
|
||||||
|
|
||||||
In Elasticsearch 5.3, we released a feature called
|
|
||||||
{ref}/modules-cross-cluster-search.html[{ccs}] for users to query across multiple
|
|
||||||
clusters. We’ve since improved on the {ccs} framework, adding features to
|
|
||||||
ultimately use it to deprecate and replace tribe nodes as a way to federate
|
|
||||||
queries. In Elasticsearch 7.0, we’re adding a new execution mode for {ccs}: one
|
|
||||||
which has fewer round-trips when they aren't necessary. This mode
|
|
||||||
(`ccs_minimize_roundtrips`) can result in faster searches when the {ccs} query
|
|
||||||
spans high-latencies (e.g., across a WAN).
|
|
||||||
//end::notable-highlights[]
|
|
||||||
|
|
||||||
//tag::notable-highlights[]
|
|
||||||
[float]
|
|
||||||
=== New cluster coordination implementation
|
|
||||||
|
|
||||||
Since the beginning, we focused on making Elasticsearch easy to scale and
|
|
||||||
resilient to catastrophic failures. To support these requirements, we created a
|
|
||||||
pluggable cluster coordination system, with the default implementation known as
|
|
||||||
Zen Discovery. Zen Discovery was meant to be effortless, and give our users
|
|
||||||
peace of mind (as the name implies). The meteoric rise in Elasticsearch usage
|
|
||||||
has taught us a great deal. For instance, Zen's `minimum_master_nodes` setting
|
|
||||||
was often misconfigured, which put clusters at a greater risk of split brains
|
|
||||||
and losing data. Maintaining this setting across large and dynamically resizing
|
|
||||||
clusters was also difficult.
|
|
||||||
|
|
||||||
In Elasticsearch 7.0, we have completely rethought and rebuilt the cluster
|
|
||||||
coordination layer. The new implementation gives safe sub-second master election
|
|
||||||
times, where Zen may have taken several seconds to elect a new master, valuable
|
|
||||||
time for a mission-critical deployment. With the `minimum_master_nodes` setting
|
|
||||||
removed, growing and shrinking clusters becomes safer and easier, and leaves
|
|
||||||
much less room to misconfigure the system. Most importantly, the new cluster
|
|
||||||
coordination layer gives us strong building blocks for the future of
|
|
||||||
Elasticsearch, ensuring we can build functionality for even more advanced
|
|
||||||
use-cases to come.
|
|
||||||
//end::notable-highlights[]
|
|
||||||
|
|
||||||
//tag::notable-highlights[]
|
|
||||||
[float]
|
|
||||||
=== Better support for small heaps (the real-memory circuit breaker)
|
|
||||||
|
|
||||||
Elasticsearch 7.0 adds an all-new {ref}/circuit-breaker.html[circuit breaker]
|
|
||||||
that keeps track of the total memory used by the JVM and will reject requests if
|
|
||||||
they would cause the reserved plus actual heap usage to exceed 95%. We'll also
|
|
||||||
be changing the default maximum buckets to return as part of an aggregation
|
|
||||||
(`search.max_buckets`) to 10,000, which is unbounded by default in 6.x and
|
|
||||||
prior. These two show great signs at seriously improving the out-of-memory
|
|
||||||
protection of Elasticsearch in 7.x, helping you keep your cluster alive even in
|
|
||||||
the face of adversarial or novice users running large queries and aggregations.
|
|
||||||
//end::notable-highlights[]
|
|
||||||
|
|
||||||
//tag::notable-highlights[]
|
|
||||||
[float]
|
|
||||||
=== {ccr-cap} is production-ready
|
|
||||||
|
|
||||||
We introduced {ccr-cap} as a beta feature in Elasticsearch
|
|
||||||
6.5. {ccr-cap} was the most heavily requested features for Elasticsearch. We're
|
|
||||||
excited to announce {ccr-cap} is now generally available and ready for production use
|
|
||||||
in Elasticsearch 6.7 and 7.0! {ccr-cap} has a variety of use cases, including
|
|
||||||
cross-datacenter and cross-region replication, replicating data to get closer to
|
|
||||||
the application server and user, and maintaining a centralized reporting cluster
|
|
||||||
replicated from a large number of smaller clusters.
|
|
||||||
|
|
||||||
In addition to maturing to a GA feature, there were a number of important
|
|
||||||
technical advancements in CCR for 6.7 and 7.0. Previous versions of {ccr-cap} required
|
|
||||||
replication to start on new indices only: existing indices could not be
|
|
||||||
replicated. {ccr-cap} can now start replicating existing indices that have soft
|
|
||||||
deletes enabled in 6.7 and 7.0, and new indices default to having soft deletes
|
|
||||||
enabled. We also introduced new technology to prevent a follower index from
|
|
||||||
falling fatally far behind its leader index. We’ve added a management UI in
|
|
||||||
Kibana for configuring remote clusters, indices to replicate, and index naming
|
|
||||||
patterns for automatic replication (e.g. for replicating `metricbeat-*`
|
|
||||||
indices). We've also added a monitoring UI for insight into {ccr} progress and
|
|
||||||
alerting on errors. Check out the Getting started with {ccr}
|
|
||||||
guide, or visit the reference documentation to learn more.
|
|
||||||
//end::notable-highlights[]
|
|
||||||
|
|
||||||
//tag::notable-highlights[]
|
|
||||||
[float]
|
|
||||||
=== {ilm-cap} is production-ready
|
|
||||||
|
|
||||||
Index Lifecycle Management (ILM) was
|
|
||||||
https://www.elastic.co/blog/elastic-stack-6-6-0-released[released] as a beta
|
|
||||||
feature in Elasticsearch 6.6. We’ve officially moved ILM out of beta and into
|
|
||||||
GA, ready for production usage! ILM makes it easy to manage the lifecycle of
|
|
||||||
data in Elasticsearch, including how data progresses between
|
|
||||||
https://www.elastic.co/guide/en/elasticsearch/reference/7.0/ilm-policy-definition.html[hot, warm, cold, and deletion phases].
|
|
||||||
Specific rules regarding how data moves through these phases can be created via
|
|
||||||
APIs in Elasticsearch, or a beautiful management UI in Kibana.
|
|
||||||
|
|
||||||
In Elasticsearch 6.7 and 7.0, ILM can now manage frozen indices. Frozen indices
|
|
||||||
are valuable for long term data storage in Elasticsearch, and require a smaller
|
|
||||||
amount of memory (heap) in relation to the amount of data managed by a node. In
|
|
||||||
6.7 and 7.0,
|
|
||||||
https://www.elastic.co/guide/en/elasticsearch/reference/7.0/_actions.html[frozen indices]
|
|
||||||
can now be frozen as part of the cold phase in ILM. In addition, ILM now works
|
|
||||||
directly with Cross-Cluster Replication (CCR), which also GA’d in the
|
|
||||||
Elasticsearch 6.7 and 7.0 releases. The potential actions available in each ILM
|
|
||||||
phase can be found in the
|
|
||||||
https://www.elastic.co/guide/en/elasticsearch/reference/7.0/_actions.html[documentation].
|
|
||||||
ILM is free to use and part of the default distribution of Elasticsearch.
|
|
||||||
//end::notable-highlights[]
|
|
||||||
|
|
||||||
//tag::notable-highlights[]
|
|
||||||
[float]
|
|
||||||
=== SQL is production-ready
|
|
||||||
|
|
||||||
The SQL interface to Elasticsearch is now GA.
|
|
||||||
https://www.elastic.co/blog/elasticsearch-6-3-0-released[Introduced in 6.3] as
|
|
||||||
an alpha release, the SQL interface allows developers and data scientists
|
|
||||||
familiar with SQL to use the speed, scalability, and full-text power of
|
|
||||||
Elasticsearch that others know and love. It also allows BI tools using SQL to
|
|
||||||
easily access data in Elasticsearch. In addition to approving SQL access as a GA
|
|
||||||
feature in Elasticsearch, we’ve designated our
|
|
||||||
https://www.elastic.co/downloads/jdbc-client[JDBC] and
|
|
||||||
https://www.elastic.co/downloads/odbc-client[ODBC] drivers as GA. There are four
|
|
||||||
methods to access Elasticsearch SQL: through the
|
|
||||||
https://www.elastic.co/guide/en/elasticsearch/reference/7.0/sql-rest.html[Elasticsearch
|
|
||||||
REST endpoints], the
|
|
||||||
https://www.elastic.co/guide/en/elasticsearch/reference/7.0/sql-cli.html[Elasticsearch
|
|
||||||
SQL command line interface], the
|
|
||||||
https://www.elastic.co/guide/en/elasticsearch/reference/7.0/sql-jdbc.html[JDBC
|
|
||||||
driver], and the
|
|
||||||
https://www.elastic.co/guide/en/elasticsearch/reference/7.0/sql-odbc.html[ODBC
|
|
||||||
driver].
|
|
||||||
//end::notable-highlights[]
|
|
||||||
|
|
||||||
//tag::notable-highlights[]
|
|
||||||
[float]
|
|
||||||
=== High-level REST client is feature-complete
|
|
||||||
|
|
||||||
If you’ve been following our
|
|
||||||
https://www.elastic.co/blog/the-elasticsearch-java-high-level-rest-client-is-out[blog]
|
|
||||||
or our https://github.com/elastic/elasticsearch/issues/27205[GitHub repository],
|
|
||||||
you may be aware of a task we’ve been working on for quite a while now: creating
|
|
||||||
a next-generation Java client for accessing an Elasticsearch cluster. We
|
|
||||||
started off by working on the most commonly-used features like search and
|
|
||||||
aggregations, and have been working our way through administrative and
|
|
||||||
monitoring APIs. Many of you that use Java are already using this new client,
|
|
||||||
but for those that are still using the TransportClient, now is a great time to
|
|
||||||
upgrade to our High Level REST Client, or HLRC.
|
|
||||||
|
|
||||||
As of 7.0.0, the HLRC now has all the API checkboxes checked to call it
|
|
||||||
“complete” so those of you still using the TransportClient should be able to
|
|
||||||
migrate. We’ll of course continue to develop our REST APIs and will add them to
|
|
||||||
this client as we go. For a list of all of the APIs that are available, have a
|
|
||||||
look at our
|
|
||||||
https://www.elastic.co/guide/en/elasticsearch/client/java-rest/7.0/java-rest-high.html[HLRC
|
|
||||||
documentation]. To get started, have a look at the
|
|
||||||
https://www.elastic.co/guide/en/elasticsearch/client/java-rest/7.0/java-rest-high-getting-started.html[getting
|
|
||||||
started with the HLRC] section of our docs and if you need help migrating from
|
|
||||||
the TransportClient, have a look at our
|
|
||||||
https://www.elastic.co/guide/en/elasticsearch/client/java-rest/7.0/java-rest-high-level-migration.html[migration
|
|
||||||
guide].
|
|
||||||
//end::notable-highlights[]
|
|
||||||
|
|
||||||
//tag::notable-highlights[]
|
|
||||||
[float]
|
|
||||||
=== Support nanosecond timestamps
|
|
||||||
|
|
||||||
Up until 7.0 Elasticsearch could only store timestamps with millisecond
|
|
||||||
precision. If you wanted to process events that occur at a higher rate -- for
|
|
||||||
example if you want to store and analyze tracing or network packet data in
|
|
||||||
Elasticsearch -- you may want higher precision. Historically, we have used the
|
|
||||||
https://www.joda.org/joda-time/[Joda time library] to handle dates and times,
|
|
||||||
and Joda lacked support for such high precision timestamps.
|
|
||||||
|
|
||||||
With JDK 8, an official Java time API has been introduced which can also handle
|
|
||||||
nanosecond precision timestamps and over the past year, we’ve been working to
|
|
||||||
migrate our Joda time usage to the native Java time while trying to maintain
|
|
||||||
backwards compatibility. As of 7.0.0, you can now make use of these nanosecond
|
|
||||||
timestamps via a dedicated
|
|
||||||
https://www.elastic.co/guide/en/elasticsearch/reference/7.0/date_nanos.html[date_nanos
|
|
||||||
field mapper]. Note that aggregations are still on a millisecond resolution
|
|
||||||
with this field to avoid having an explosion of buckets.
|
|
||||||
//end::notable-highlights[]
|
|
||||||
|
|
||||||
//tag::notable-highlights[]
|
|
||||||
[float]
|
|
||||||
=== Faster retrieval of top hits
|
|
||||||
|
|
||||||
When it comes to search, query performance is a key feature. We have achieved a
|
|
||||||
significant improvement to search performance in Elasticsearch 7.0 for
|
|
||||||
situations in which the exact hit count is not needed and it is sufficient to
|
|
||||||
set a lower boundary to the number of results. For example, if your users
|
|
||||||
typically just look at the first page of results on your site and don’t care
|
|
||||||
about exactly how many documents matched, you may be able to show them “more
|
|
||||||
than 10,000 hits” and then provide them with paginated results. It’s quite
|
|
||||||
common to have users enter frequently-occurring terms like “the” and “a” in
|
|
||||||
their queries, which has historically forced Elasticsearch to score a lot of
|
|
||||||
documents even when those frequent terms couldn’t possibly add much to the
|
|
||||||
score.
|
|
||||||
|
|
||||||
In these conditions Elasticsearch can now skip calculating scores for records
|
|
||||||
that are identified at an early stage as records that will not be ranked at the
|
|
||||||
top of the result set. This can significantly improve the query speed. The
|
|
||||||
actual number of top results that are scored is
|
|
||||||
https://www.elastic.co/guide/en/elasticsearch/reference/7.0/search-request-track-total-hits.html[configurable],
|
|
||||||
but the default is 10,000. The behavior of queries that have a result set that
|
|
||||||
is smaller than this threshold will not change - i.e. the results count is
|
|
||||||
accurate but there is no performance improvement for queries that match a small
|
|
||||||
number of documents. Because the improvement is based on skipping low ranking
|
|
||||||
records, it does not apply to aggregations. You can read more about this
|
|
||||||
powerful algorithmic development in our blog post
|
|
||||||
https://www.elastic.co/blog/faster-retrieval-of-top-hits-in-elasticsearch-with-block-max-wand[Magic
|
|
||||||
WAND: Faster Retrieval of Top Hits in Elasticsearch].
|
|
||||||
//end::notable-highlights[]
|
|
||||||
|
|
||||||
//tag::notable-highlights[]
|
|
||||||
[float]
|
|
||||||
=== Support for TLS 1.3
|
|
||||||
|
|
||||||
Elasticsearch has supported encrypted communications for a long time, however,
|
|
||||||
we recently started https://www.elastic.co/support/matrix#matrix_jvm[supporting
|
|
||||||
JDK 11], which gives us new capabilities. JDK 11 now has TLSv1.3 support so
|
|
||||||
starting with 7.0, we’re now supporting TLSv1.3 within Elasticsearch for those
|
|
||||||
of you running JDK 11. In order to help new users from inadvertently running
|
|
||||||
with low security, we’ve also dropped TLSv1.0 from our defaults. For those
|
|
||||||
running older versions of Java, we have default options of TLSv1.2 and
|
|
||||||
TLSv1.1. Have a look at our
|
|
||||||
https://www.elastic.co/guide/en/elasticsearch/reference/7.0/ssl-tls.html[TLS
|
|
||||||
setup instructions] if you need help getting started.
|
|
||||||
//end::notable-highlights[]
|
|
||||||
|
|
||||||
//tag::notable-highlights[]
|
|
||||||
[float]
|
|
||||||
=== Bundle JDK in Elasticsearch distribution
|
|
||||||
|
|
||||||
One of the more prominent "getting started hurdles" we’ve seen users run into
|
|
||||||
has been not knowing that Elasticsearch is a Java application and that they need
|
|
||||||
to install one of the supported JDKs first. With 7.0, we’re now bundling a
|
|
||||||
distribution of OpenJDK to help users get started with Elasticsearch even
|
|
||||||
faster. We understand that some users have preferred JDK distributions, so we
|
|
||||||
also support bringing your own JDK. If you want to bring your own JDK, you can
|
|
||||||
still do so by
|
|
||||||
https://www.elastic.co/guide/en/elasticsearch/reference/7.0/setup.html#jvm-version[setting
|
|
||||||
JAVA_HOME] before starting Elasticsearch.
|
|
||||||
//end::notable-highlights[]
|
|
||||||
|
|
||||||
//tag::notable-highlights[]
|
|
||||||
[float]
|
|
||||||
=== Rank features
|
|
||||||
|
|
||||||
Elasticsearch 7.0 has several new field types to get the most out of your data.
|
|
||||||
Two to help with core search use cases are
|
|
||||||
https://www.elastic.co/guide/en/elasticsearch/reference/7.0/rank-feature.html[`rank_feature`]
|
|
||||||
and
|
|
||||||
https://www.elastic.co/guide/en/elasticsearch/reference/7.0/rank-features.html[`rank_features`].
|
|
||||||
These can be used to boost documents based on numeric or categorical values
|
|
||||||
while still maintaining the performance of the new fast top hits query
|
|
||||||
capabilities. For more information on these fields and how to use them, read our
|
|
||||||
https://www.elastic.co/blog/easier-relevance-tuning-elasticsearch-7-0[blog
|
|
||||||
post].
|
|
||||||
//end::notable-highlights[]
|
|
||||||
|
|
||||||
//tag::notable-highlights[]
|
|
||||||
[float]
|
|
||||||
=== JSON logging
|
|
||||||
|
|
||||||
JSON logging is now enabled in Elasticsearch in addition to plaintext
|
|
||||||
logs. Starting in 7.0, you will find new files with `.json` extensions in your
|
|
||||||
log directory. This means you can now use filtering tools like
|
|
||||||
https://stedolan.github.io/jq/[`jq`] to pretty print and process your logs in a
|
|
||||||
much more structured manner. You can also expect finding additional information
|
|
||||||
like `node.id`, `cluster.uuid`, `type` (and more) in each log line. The `type`
|
|
||||||
field per each JSON log line will let you to distinguish log streams when
|
|
||||||
running on docker.
|
|
||||||
//end::notable-highlights[]
|
|
||||||
|
|
||||||
//tag::notable-highlights[]
|
|
||||||
[float]
|
|
||||||
=== Script score query (aka function score 2.0)
|
|
||||||
|
|
||||||
With 7.0, we are introducing the
|
|
||||||
https://www.elastic.co/guide/en/elasticsearch/reference/7.0/query-dsl-script-score-query.html[next
|
|
||||||
generation of our function score capability]. This new script_score query
|
|
||||||
provides a new, simpler, and more flexible way to generate a ranking score per
|
|
||||||
record. The script_score query is constructed of a set of functions, including
|
|
||||||
arithmetic and distance functions, which the user can mix and match to construct
|
|
||||||
arbitrary function score calculations. The modular structure is simpler to use
|
|
||||||
and will open this important functionality to additional users.
|
|
||||||
//end::notable-highlights[]
|
|
||||||
|
|
@ -1,40 +0,0 @@
|
||||||
[[release-highlights-7.1.0]]
|
|
||||||
== 7.1.0 release highlights
|
|
||||||
++++
|
|
||||||
<titleabbrev>7.1.0</titleabbrev>
|
|
||||||
++++
|
|
||||||
|
|
||||||
See also <<release-notes-7.1.0,{es} 7.1.0 release notes>>.
|
|
||||||
|
|
||||||
//tag::notable-highlights[]
|
|
||||||
[float]
|
|
||||||
==== TLS is now licensed under the Elastic Basic license
|
|
||||||
|
|
||||||
Transport Layer Security (TLS), commonly referred to as SSL, is now
|
|
||||||
licensed under the free-of-charge Elastic Basic license. Previously, this security feature
|
|
||||||
required a paid Gold-tier subscription. With the default distribution,
|
|
||||||
you can now encrypt all Elasticsearch communication, within a cluster and across remotes
|
|
||||||
clusters. Download https://www.elastic.co/downloads/elasticsearch[Elasticsearch],
|
|
||||||
https://www.elastic.co/guide/en/elasticsearch/reference/7.1/configuring-tls.html[configure TLS],
|
|
||||||
and run your cluster in production, knowing all Elasticsearch communication is safely encrypted.
|
|
||||||
For details, see https://www.elastic.co/subscriptions
|
|
||||||
//end::notable-highlights[]
|
|
||||||
|
|
||||||
//tag::notable-highlights[]
|
|
||||||
[float]
|
|
||||||
==== RBAC is now licensed under the Elastic Basic license
|
|
||||||
|
|
||||||
RBAC (Role Based Access Control) is now licenced under the free-of-charge Elastic Basic licence.
|
|
||||||
Previously, this security feature required a paid Gold-tier subscription.
|
|
||||||
With the default distribution you can take advantage of RBAC by configuring users, groups, roles
|
|
||||||
and permissions for any user from the
|
|
||||||
https://www.elastic.co/guide/en/elasticsearch/reference/7.1/configuring-file-realm.html[file realm]
|
|
||||||
or the https://www.elastic.co/guide/en/elasticsearch/reference/7.1/configuring-native-realm.html[native realm]
|
|
||||||
. Download https://www.elastic.co/downloads/elasticsearch[Elasticsearch],
|
|
||||||
https://www.elastic.co/guide/en/elasticsearch/reference/7.1/authorization.html[configure RBAC],
|
|
||||||
and run your cluster in production, knowing your private data stays private.
|
|
||||||
Note that our advanced security features, such as single sign-on and Active Directory/LDAP
|
|
||||||
authentication to field-level and document-level security, remain paid features.
|
|
||||||
For details, see https://www.elastic.co/subscriptions
|
|
||||||
|
|
||||||
//end::notable-highlights[]
|
|
||||||
|
|
@ -1,101 +0,0 @@
|
||||||
[[release-highlights-7.2.0]]
|
|
||||||
== 7.2.0 release highlights
|
|
||||||
++++
|
|
||||||
<titleabbrev>7.2.0</titleabbrev>
|
|
||||||
++++
|
|
||||||
|
|
||||||
//NOTE: The notable-highlights tagged regions are re-used in the
|
|
||||||
//Installation and Upgrade Guide
|
|
||||||
|
|
||||||
// tag::notable-highlights[]
|
|
||||||
[discrete]
|
|
||||||
==== {dataframes-cap}
|
|
||||||
|
|
||||||
beta[] You can now transform your data with
|
|
||||||
{ref}/transforms.html[data frames]. There is a new {kib} wizard that
|
|
||||||
guides you through the process of creating a {dataframe-transform} to pivot and
|
|
||||||
summarize your data and store it in a new index. Alternatively, you can use
|
|
||||||
{ref}/data-frame-apis.html[{dataframe} APIs] to preview, create, and manage
|
|
||||||
the transforms.
|
|
||||||
|
|
||||||
// end::notable-highlights[]
|
|
||||||
|
|
||||||
// tag::notable-highlights[]
|
|
||||||
[float]
|
|
||||||
==== Closed indices are now replicated
|
|
||||||
|
|
||||||
Elasticsearch 7.2.0 brings better support for closed indices by allowing
|
|
||||||
shards of closed indices to be replicated.
|
|
||||||
As soon as an index is closed, Elasticsearch takes care of safely tearing down
|
|
||||||
the "opened" shards before reinitializing them as "closed" shards, which require
|
|
||||||
fewer resources. Closed shards can later be promoted to primary shards or
|
|
||||||
automatically recovered during {ref}/recovery.html[peer recovery]
|
|
||||||
The data is also automatically replicated by the
|
|
||||||
cluster to ensure that enough shard copies are safely kept around at all
|
|
||||||
times (configurable with `index.number_of_replicas`).
|
|
||||||
|
|
||||||
In addition to that, it is now possible to snapshot closed indices using
|
|
||||||
the {ref}/modules-snapshots.html[Snapshot/Restore API]. To include a closed index
|
|
||||||
when creating a snapshot on Elasticsearch 7.2+, the `expand_wildcards`
|
|
||||||
parameter must be explicitly set to either `all` or `closed` .
|
|
||||||
|
|
||||||
Note that only indices closed in Elasticsearch 7.2+ are automatically
|
|
||||||
replicated. Indices closed on previous versions of Elasticsearch will
|
|
||||||
remain non replicated unless they are opened and closed again in 7.2+.
|
|
||||||
|
|
||||||
// end::notable-highlights[]
|
|
||||||
|
|
||||||
// tag::notable-highlights[]
|
|
||||||
[float]
|
|
||||||
==== Geo features in SQL
|
|
||||||
beta[] This release introduces the first set of geo features to SQL.
|
|
||||||
The implementation is based on the OpenGIS® Implementation Standard for Geographic
|
|
||||||
information - "Simple feature access". This is the current de-facto standard for GIS
|
|
||||||
system implementation. This release includes a small subset of SQL option AKA ISO 19125-2.
|
|
||||||
|
|
||||||
For this initial beta release, we added support for returning
|
|
||||||
geo_shapes and geo_points as results, added support for a few geo functions
|
|
||||||
(ST_AsText, ST_Distance, ST_GeometryType, ST_GeometryFromText, ST_X, ST_Y, and ST_Z)
|
|
||||||
, and added a limited support for using geo_points in distance queries. For example:
|
|
||||||
SELECT * FROM my_index WHERE ST_Distance(point, ST_WKTToSQL('point (10 20)')) < 20.
|
|
||||||
|
|
||||||
// end::notable-highlights[]
|
|
||||||
|
|
||||||
// tag::notable-highlights[]
|
|
||||||
[float]
|
|
||||||
==== OpenId Connect authentication realm
|
|
||||||
|
|
||||||
This release introduces OpenId Connect as an authentication realm.
|
|
||||||
Elasticsearch (with the assistance of Kibana or another web component) can now serve as an
|
|
||||||
OpenID Connect Relying Party (RP). {es} supports the Authorization Code Grant and Implicit
|
|
||||||
flows as described in http://ela.st/oidc-spec. It also supports consuming and verifying signed ID Tokens
|
|
||||||
, RP initiated single sign on (SSO), 3rd party initiated SSO, and RP initiated single logout.
|
|
||||||
|
|
||||||
|
|
||||||
// end::notable-highlights[]
|
|
||||||
|
|
||||||
// tag::notable-highlights[]
|
|
||||||
[float]
|
|
||||||
==== Search as you type field mapping type
|
|
||||||
|
|
||||||
The `search_as_you_type` field type is a text-like field optimized to
|
|
||||||
provide out-of-the-box support for queries that serve an as-you-type completion
|
|
||||||
use case. It creates a series of subfields that are analyzed to index terms
|
|
||||||
that can be efficiently matched by a query that partially matches the entire
|
|
||||||
indexed text value. Both prefix completion (i.e matching terms starting at the
|
|
||||||
beginning of the input) and infix completion (i.e. matching terms at any
|
|
||||||
position within the input) are supported.
|
|
||||||
|
|
||||||
// end::notable-highlights[]
|
|
||||||
|
|
||||||
|
|
||||||
// tag::notable-highlights[]
|
|
||||||
[float]
|
|
||||||
==== Distance Feature Query
|
|
||||||
|
|
||||||
The `distance_feature` query is a specialized query that only works on `date`, `date_nanos`, or `geo_point`
|
|
||||||
fields. The query boosts documents scores based on proximity to some given origin.
|
|
||||||
For example, you can use this query to give higher scores to documents with dates
|
|
||||||
closer to a certain date or locations closer to a certain location.
|
|
||||||
|
|
||||||
// end::notable-highlights[]
|
|
||||||
|
|
@ -1,186 +0,0 @@
|
||||||
[[release-highlights-7.3.0]]
|
|
||||||
== 7.3.0 release highlights
|
|
||||||
++++
|
|
||||||
<titleabbrev>7.3.0</titleabbrev>
|
|
||||||
++++
|
|
||||||
|
|
||||||
//NOTE: The notable-highlights tagged regions are re-used in the
|
|
||||||
//Installation and Upgrade Guide
|
|
||||||
|
|
||||||
// tag::notable-highlights[]
|
|
||||||
[float]
|
|
||||||
==== Voting-only master nodes
|
|
||||||
|
|
||||||
A new {ref}/modules-node.html#voting-only-node[`node.voting-only`] role has been
|
|
||||||
introduced that allows nodes to participate in elections even though they are
|
|
||||||
not eligible to become the master.
|
|
||||||
The benefit is that these nodes still help with high availability while
|
|
||||||
requiring less CPU and heap than master nodes.
|
|
||||||
|
|
||||||
NOTE: The `node.voting-only` role is only available with the default
|
|
||||||
distribution of {es}.
|
|
||||||
|
|
||||||
// end::notable-highlights[]
|
|
||||||
|
|
||||||
// tag::notable-highlights[]
|
|
||||||
[float]
|
|
||||||
==== Reloading of search-time synonyms
|
|
||||||
|
|
||||||
A new {ref}/indices-reload-analyzers.html[Analyzer reload API] allows to reload
|
|
||||||
the definition of search-time analyzers and their associated resources. A common
|
|
||||||
use-case for this API is the reloading of search-time synonyms. In earlier
|
|
||||||
versions of Elasticsearch, users could force synonyms to be reloaded by closing
|
|
||||||
the index and then opening it again. With this new API, synonyms can be updated
|
|
||||||
without closing the index.
|
|
||||||
|
|
||||||
NOTE: The Analyzer reload API is only available with the default distribution
|
|
||||||
of {es}.
|
|
||||||
|
|
||||||
// end::notable-highlights[]
|
|
||||||
|
|
||||||
// tag::notable-highlights[]
|
|
||||||
[float]
|
|
||||||
==== New `flattened` field type
|
|
||||||
|
|
||||||
A new {ref}/flattened.html[`flattened`] field type has been added, which can index
|
|
||||||
arbitrary `json` objects into a single field. This helps avoid hitting issues
|
|
||||||
due to many fields in mappings, at the cost of more limited search
|
|
||||||
functionality.
|
|
||||||
|
|
||||||
NOTE: The `flattened` field type is only available with the
|
|
||||||
default distribution of {es}.
|
|
||||||
|
|
||||||
// end::notable-highlights[]
|
|
||||||
|
|
||||||
// tag::notable-highlights[]
|
|
||||||
[float]
|
|
||||||
==== Functions on vector fields
|
|
||||||
|
|
||||||
Painless now support computing the
|
|
||||||
{ref}/query-dsl-script-score-query.html#vector-functions[cosine similarity] and
|
|
||||||
the {ref}/query-dsl-script-score-query.html#vector-functions[dot product] of a
|
|
||||||
query vector and either values of a
|
|
||||||
{ref}/sparse-vector.html[`sparse_vector`] or
|
|
||||||
{ref}/dense-vector.html[`dense_vector`] field.
|
|
||||||
|
|
||||||
NOTE: These functions are only available with the default distribution of {es}.
|
|
||||||
|
|
||||||
// end::notable-highlights[]
|
|
||||||
|
|
||||||
// tag::notable-highlights[]
|
|
||||||
[float]
|
|
||||||
==== Prefix and wildcard support for intervals
|
|
||||||
|
|
||||||
{ref}/query-dsl-intervals-query.html[Intervals] now support querying by
|
|
||||||
{ref}/query-dsl-intervals-query.html#intervals-prefix[prefix] or
|
|
||||||
{ref}/query-dsl-intervals-query.html#intervals-wildcard[wildcard].
|
|
||||||
|
|
||||||
// end::notable-highlights[]
|
|
||||||
|
|
||||||
// tag::notable-highlights[]
|
|
||||||
[float]
|
|
||||||
==== Rare terms aggregation
|
|
||||||
|
|
||||||
A new
|
|
||||||
{ref}/search-aggregations-bucket-rare-terms-aggregation.html[Rare Terms aggregation]
|
|
||||||
allows to find the least frequent values in a field. It is intended to replace
|
|
||||||
the `"order" : { "_count" : "asc" }` option of the
|
|
||||||
{ref}/search-aggregations-bucket-terms-aggregation.html[terms aggregations].
|
|
||||||
|
|
||||||
// end::notable-highlights[]
|
|
||||||
|
|
||||||
// tag::notable-highlights[]
|
|
||||||
[float]
|
|
||||||
==== Aliases are replicated via {ccr}
|
|
||||||
|
|
||||||
Read aliases are now replicated via {ref}/ccr-put-follow.html[{ccr}]. Note that
|
|
||||||
write aliases are still not replicated since they only make sense for indices that
|
|
||||||
are being written to while follower indices do not receive direct writes.
|
|
||||||
|
|
||||||
// end::notable-highlights[]
|
|
||||||
|
|
||||||
// tag::notable-highlights[]
|
|
||||||
[float]
|
|
||||||
==== SQL supports frozen indices
|
|
||||||
|
|
||||||
{es-sql} now supports querying {ref}/frozen-indices.html[frozen indices] via the
|
|
||||||
new {ref}/sql-index-frozen.html[`FROZEN`] keyword.
|
|
||||||
|
|
||||||
// end::notable-highlights[]
|
|
||||||
|
|
||||||
// tag::notable-highlights[]
|
|
||||||
[float]
|
|
||||||
==== Fixed memory leak when using templates in document-level security
|
|
||||||
|
|
||||||
{ref}/document-level-security.html[Document-level security] was using an
|
|
||||||
unbounded cache for the set of visible documents. This could lead to a memory
|
|
||||||
leak when using a templated query as a role query. The cache has been fixed to
|
|
||||||
evict based on memory usage and has a limit of 50MB.
|
|
||||||
|
|
||||||
// end::notable-highlights[]
|
|
||||||
|
|
||||||
// tag::notable-highlights[]
|
|
||||||
[float]
|
|
||||||
==== More memory-efficient aggregations on `keyword` fields
|
|
||||||
|
|
||||||
{ref}/search-aggregations-bucket-terms-aggregation.html[Terms aggregations]
|
|
||||||
generally need to build
|
|
||||||
{ref}/search-aggregations-bucket-terms-aggregation.html#search-aggregations-bucket-terms-aggregation-execution-hint[global ordinals]
|
|
||||||
in order to run. Unfortunately this operation became more memory-intensive in
|
|
||||||
6.0 due to the move to doc-value iterators in order to improve handling of
|
|
||||||
sparse fields. Memory pressure of global ordinals now goes back to a more
|
|
||||||
similar level as what you could have on pre-6.0 releases.
|
|
||||||
|
|
||||||
// end::notable-highlights[]
|
|
||||||
|
|
||||||
// tag::notable-highlights[]
|
|
||||||
[discrete]
|
|
||||||
[[release-highlights-7.3.0-transforms]]
|
|
||||||
==== {dataframes-cap}: transform and pivot your streaming data
|
|
||||||
|
|
||||||
beta[] {ref}/transforms.html[{dataframe-transforms-cap}] are a core new
|
|
||||||
feature in {es} that enable you to transform an existing index to a secondary,
|
|
||||||
summarized index. {dataframe-transforms-cap} enable you to pivot your data and
|
|
||||||
create entity-centric indices that can summarize the behavior of an entity. This
|
|
||||||
organizes the data into an analysis-friendly format.
|
|
||||||
|
|
||||||
{dataframe-transforms-cap} were originally available in 7.2. With 7.3 they can
|
|
||||||
now run either as a single batch transform or continuously incorporating new
|
|
||||||
data as it is ingested.
|
|
||||||
|
|
||||||
{dataframes-cap} enable new possibilities for {ml} analysis (such as
|
|
||||||
_outlier detection_), but they can also be useful for other types of
|
|
||||||
visualizations and custom types of analysis.
|
|
||||||
|
|
||||||
// end::notable-highlights[]
|
|
||||||
|
|
||||||
// tag::notable-highlights[]
|
|
||||||
[discrete]
|
|
||||||
[[release-highlights-7.3.0-outlier-detection]]
|
|
||||||
==== Discover your most unusual data using {oldetection}
|
|
||||||
|
|
||||||
The goal of {ml-docs}/dfa-outlier-detection.html[{oldetection}] is to find
|
|
||||||
the most unusual data points in an index. We analyse the numerical fields of
|
|
||||||
each data point (document in an index) and annotate them with how unusual they
|
|
||||||
are.
|
|
||||||
|
|
||||||
We use unsupervised {oldetection} which means there is no need to provide a
|
|
||||||
training data set to teach {oldetection} to recognize outliers. In practice,
|
|
||||||
this is achieved by using an ensemble of distance based and density based
|
|
||||||
techniques to identify those data points which are the most different from the
|
|
||||||
bulk of the data in the index. We assign to each analysed data point an
|
|
||||||
{olscore}, which captures how different the entity is from other entities in the
|
|
||||||
index.
|
|
||||||
|
|
||||||
In addition to new {oldetection} functionality, we are introducing the
|
|
||||||
{ref}/evaluate-dfanalytics.html[evaluate {dfanalytics} API], which enables you
|
|
||||||
to compute a range of performance metrics such
|
|
||||||
as confusion matrices, precision, recall, the
|
|
||||||
https://en.wikipedia.org/wiki/Receiver_operating_characteristic[receiver-operating characteristics (ROC) curve]
|
|
||||||
and the area under the ROC curve. If you are running {oldetection} on a source
|
|
||||||
index that has already been labeled to indicate which points are truly outliers
|
|
||||||
and which are normal, you can use the
|
|
||||||
evaluate {dfanalytics} API to assess the performance of the
|
|
||||||
{oldetection} analytics on your dataset.
|
|
||||||
|
|
||||||
// end::notable-highlights[]
|
|
||||||
|
|
@ -1,156 +0,0 @@
|
||||||
[[release-highlights-7.4.0]]
|
|
||||||
== 7.4.0 release highlights
|
|
||||||
++++
|
|
||||||
<titleabbrev>7.4.0</titleabbrev>
|
|
||||||
++++
|
|
||||||
|
|
||||||
//NOTE: The notable-highlights tagged regions are re-used in the
|
|
||||||
//Installation and Upgrade Guide
|
|
||||||
|
|
||||||
// tag::notable-highlights[]
|
|
||||||
[float]
|
|
||||||
==== Results pinning
|
|
||||||
|
|
||||||
You can use the new {ref}/query-dsl-pinned-query.html[pinned query]
|
|
||||||
to define the first records
|
|
||||||
(and the order in which they are returned)
|
|
||||||
in a result set directly within {es}.
|
|
||||||
|
|
||||||
// end::notable-highlights[]
|
|
||||||
|
|
||||||
// tag::notable-highlights[]
|
|
||||||
[float]
|
|
||||||
==== New `shape` field type
|
|
||||||
|
|
||||||
A new {ref}/shape.html[`shape`] field type has been added,
|
|
||||||
which allows you to position and query shapes
|
|
||||||
in a geometry of your choosing.
|
|
||||||
|
|
||||||
// end::notable-highlights[]
|
|
||||||
|
|
||||||
// tag::notable-highlights[]
|
|
||||||
[float]
|
|
||||||
==== Circle ingest processor
|
|
||||||
|
|
||||||
A new {ref}/ingest-circle-processor.html[circle ingest processor] has been added,
|
|
||||||
which translates circles into regular polygons (bounded by the circles).
|
|
||||||
This makes ingesting, indexing, searching, and aggregating circles both easy and efficient.
|
|
||||||
|
|
||||||
// end::notable-highlights[]
|
|
||||||
|
|
||||||
// tag::notable-highlights[]
|
|
||||||
[float]
|
|
||||||
==== Aggregations on range fields
|
|
||||||
|
|
||||||
The {ref}/search-aggregations-bucket-histogram-aggregation.html[histogram]
|
|
||||||
and {ref}/search-aggregations-bucket-datehistogram-aggregation.html[date histogram]
|
|
||||||
aggregations now support the {ref}/range.html[`range`] field type.
|
|
||||||
|
|
||||||
Range aggregations are useful
|
|
||||||
when counting ranges that overlap with specific buckets
|
|
||||||
(e.g. the number of phone calls that took place during a specific minute).
|
|
||||||
|
|
||||||
// end::notable-highlights[]
|
|
||||||
|
|
||||||
// tag::notable-highlights[]
|
|
||||||
[float]
|
|
||||||
==== Cumulative cardinality aggregation
|
|
||||||
|
|
||||||
A new {ref}/search-aggregations-pipeline-cumulative-cardinality-aggregation.html[cumulative cardinality aggregation]
|
|
||||||
has been added
|
|
||||||
as part of our ongoing effort to provide advanced aggregations.
|
|
||||||
|
|
||||||
You can use this new pipeline aggregation
|
|
||||||
to calculate a net-new total of document occurrences
|
|
||||||
within a given time range.
|
|
||||||
|
|
||||||
// end::notable-highlights[]
|
|
||||||
|
|
||||||
// tag::notable-highlights[]
|
|
||||||
[float]
|
|
||||||
==== Snapshot lifecycle management
|
|
||||||
|
|
||||||
We’re introducing {ref}/getting-started-snapshot-lifecycle-management.html[snapshot lifecycle management (SLM)],
|
|
||||||
which allows an administrator to define policies,
|
|
||||||
via API or {kibana-ref}/index-lifecycle-policies.html[{kib} UI],
|
|
||||||
that manage when and how often snapshots are taken.
|
|
||||||
You can use SLM
|
|
||||||
to ensure that appropriate, recent backups are ready
|
|
||||||
if disaster strikes
|
|
||||||
or you need to restore {es} data.
|
|
||||||
|
|
||||||
// end::notable-highlights[]
|
|
||||||
|
|
||||||
// tag::notable-highlights[]
|
|
||||||
[float]
|
|
||||||
==== API key management
|
|
||||||
|
|
||||||
New {ref}/security-privileges.html[cluster privileges] to manage API keys have been added,
|
|
||||||
allowing cluster administrators to manage everything,
|
|
||||||
and regular users to manage their own keys.
|
|
||||||
Users can create API keys
|
|
||||||
and use them to provide long-term credentials
|
|
||||||
while interacting with {es}.
|
|
||||||
|
|
||||||
// end::notable-highlights[]
|
|
||||||
|
|
||||||
// tag::notable-highlights[]
|
|
||||||
[float]
|
|
||||||
==== TLS settings for email notifications
|
|
||||||
|
|
||||||
Notifications may contain sensitive information that must be protected over the wire. This requires that communication with the mail server is encrypted and authenticated properly.
|
|
||||||
{es} now supports custom {ref}/notification-settings.html#ssl-notification-smtp-settings[TLS settings] for email notifications,
|
|
||||||
allowing secure connections to servers with custom security configuration.
|
|
||||||
|
|
||||||
// end::notable-highlights[]
|
|
||||||
|
|
||||||
// tag::notable-highlights[]
|
|
||||||
[float]
|
|
||||||
==== Automatic query cancellation
|
|
||||||
|
|
||||||
{es} now automatically terminates queries
|
|
||||||
sent through the `_search` endpoint
|
|
||||||
when the initiating connection is closed.
|
|
||||||
|
|
||||||
// end::notable-highlights[]
|
|
||||||
|
|
||||||
// tag::notable-highlights[]
|
|
||||||
[float]
|
|
||||||
==== Support for AdoptOpenJDK
|
|
||||||
|
|
||||||
AdoptOpenJDK 13 is now supported and shipped with {es} as the pre-bundled JDK.
|
|
||||||
|
|
||||||
If you want to use your own JDK,
|
|
||||||
you can still do so by setting `JAVA_HOME` before starting Elasticsearch.
|
|
||||||
|
|
||||||
The availability of a notarized AdoptOpenJDK package
|
|
||||||
(per the new requirements for software running on macOS Catalina)
|
|
||||||
facilitates notarization of {es} for continued support on macOS.
|
|
||||||
|
|
||||||
// end::notable-highlights[]
|
|
||||||
|
|
||||||
// tag::notable-highlights[]
|
|
||||||
[float]
|
|
||||||
==== Regression analysis - Experimental
|
|
||||||
|
|
||||||
{ml-docs}/dfa-regression.html[Regression analysis] is an experimental machine learning process
|
|
||||||
for estimating the relationships among a number of feature variables and a dependent variable,
|
|
||||||
then making further predictions based on the described relationship.
|
|
||||||
|
|
||||||
// end::notable-highlights[]
|
|
||||||
|
|
||||||
// tag::notable-highlights[]
|
|
||||||
[float]
|
|
||||||
==== New vector distance functions for document script scoring - Experimental
|
|
||||||
|
|
||||||
Two experimential similarity measurements—
|
|
||||||
Manhattan distance (L1 norm)
|
|
||||||
and Euclidean distance (L2 norm)—
|
|
||||||
have been added.
|
|
||||||
Like the dot product and cosine similarity,
|
|
||||||
the Euclidean and Manhattan distances are provided as {ref}/query-dsl-script-score-query.html#vector-functions[predefined Painless functions]
|
|
||||||
so that they may be incorporated with other query elements
|
|
||||||
as part of a {ref}/query-dsl-script-score-query.html[script_score] query.
|
|
||||||
|
|
||||||
// end::notable-highlights[]
|
|
||||||
|
|
||||||
|
|
@ -1,64 +0,0 @@
|
||||||
[[release-highlights-7.5.0]]
|
|
||||||
== 7.5.0 release highlights
|
|
||||||
++++
|
|
||||||
<titleabbrev>7.5.0</titleabbrev>
|
|
||||||
++++
|
|
||||||
|
|
||||||
//NOTE: The notable-highlights tagged regions are re-used in the
|
|
||||||
//Installation and Upgrade Guide
|
|
||||||
|
|
||||||
// tag::notable-highlights[]
|
|
||||||
[float]
|
|
||||||
==== Enrich processor
|
|
||||||
|
|
||||||
A new {ref}/enrich-processor.html[enrich ingest processor] has been added,
|
|
||||||
which can enrich documents with data from another index.
|
|
||||||
|
|
||||||
// end::notable-highlights[]
|
|
||||||
|
|
||||||
// tag::notable-highlights[]
|
|
||||||
[float]
|
|
||||||
==== Shape support in SQL
|
|
||||||
|
|
||||||
{ref}/xpack-sql.html[SQL] functionality that worked for geo_shape will
|
|
||||||
now work for the {ref}/shape.html[`shape`] field type introduced in 7.4.
|
|
||||||
|
|
||||||
|
|
||||||
// end::notable-highlights[]
|
|
||||||
|
|
||||||
|
|
||||||
// tag::notable-highlights[]
|
|
||||||
[float]
|
|
||||||
==== Snapshot lifecycle management retention
|
|
||||||
|
|
||||||
There is a new {ref}/slm-retention.html[snapshot lifecycle management retention],
|
|
||||||
which allows you to delete older snapshots automatically through a custom
|
|
||||||
policy’s schedule.
|
|
||||||
|
|
||||||
// end::notable-highlights[]
|
|
||||||
|
|
||||||
|
|
||||||
// tag::notable-highlights[]
|
|
||||||
[float]
|
|
||||||
==== Pause {ccr}
|
|
||||||
|
|
||||||
New {ref}/ccr-post-pause-follow.html[pause] and
|
|
||||||
{ref}/ccr-post-pause-follow.html[resume] API endpoints for
|
|
||||||
{ref}/xpack-ccr.html[{ccr}] have been added, which enable you to temporarily
|
|
||||||
pause auto-follow patterns.
|
|
||||||
|
|
||||||
// end::notable-highlights[]
|
|
||||||
|
|
||||||
// tag::notable-highlights[]
|
|
||||||
[float]
|
|
||||||
==== {ml-cap} {classanalysis}
|
|
||||||
|
|
||||||
{ml-docs}/dfa-classification.html[{classanalysis-cap}] is a supervised {ml}
|
|
||||||
process for predicting a class or category of a given data point in a dataset.
|
|
||||||
For example, it can determine whether an email is spam or not.
|
|
||||||
{classification-cap} is for predicting discrete, categorical values, unlike
|
|
||||||
{reganalysis}, which predicts continuous, numerical values. 7.5.0 introduces
|
|
||||||
binary classification, which can label data points into two possible categories.
|
|
||||||
|
|
||||||
// end::notable-highlights[]
|
|
||||||
|
|
||||||
|
|
@ -1,72 +0,0 @@
|
||||||
[[release-highlights-7.6.0]]
|
|
||||||
== 7.6.0 release highlights
|
|
||||||
++++
|
|
||||||
<titleabbrev>7.6.0</titleabbrev>
|
|
||||||
++++
|
|
||||||
|
|
||||||
//NOTE: The notable-highlights tagged regions are re-used in the
|
|
||||||
//Installation and Upgrade Guide
|
|
||||||
|
|
||||||
// tag::notable-highlights[]
|
|
||||||
[float]
|
|
||||||
==== New histogram field type
|
|
||||||
|
|
||||||
A new {ref}/histogram.html[histogram field type] has been added. The new `histogram` field accepts
|
|
||||||
pre-aggregated histograms which can later be used directly in the
|
|
||||||
{ref}/search-aggregations-metrics-percentile-aggregation.html[percentiles] and
|
|
||||||
{ref}/search-aggregations-metrics-percentile-rank-aggregation.html[percentile_ranks] aggregations.
|
|
||||||
This allows users to pre-aggregate histogram data locally and only send the final
|
|
||||||
data structure, saving storage and network bandwidth while retaining the ability to
|
|
||||||
aggregate it like any other data.
|
|
||||||
|
|
||||||
// end::notable-highlights[]
|
|
||||||
|
|
||||||
// tag::notable-highlights[]
|
|
||||||
[float]
|
|
||||||
==== Optimized sorting on long field types
|
|
||||||
|
|
||||||
Sorting on a {ref}/number.html[`long`] field now internally rewrites into a Lucene `DistanceFeatureQuery`.
|
|
||||||
This lets {es} skip non-competitive hits, which often improves query speed.
|
|
||||||
In benchmarking tests, this sped up sorts on `long` fields by 10x.
|
|
||||||
|
|
||||||
// end::notable-highlights[]
|
|
||||||
|
|
||||||
// tag::notable-highlights[]
|
|
||||||
[float]
|
|
||||||
==== Simplifying and operationalizing machine learning
|
|
||||||
|
|
||||||
With the release of 7.6 the {stack} delivers an end-to-end {ml} pipeline
|
|
||||||
providing the path from raw data to building, testing, and deploying {ml} models
|
|
||||||
in production. Up to this point {ml} in the {stack} had primarily focused on
|
|
||||||
unsupervised techniques by using sophisticated pattern recognition that builds
|
|
||||||
time series models used for {anomaly-detect}. With the new {dfanalytics}, you
|
|
||||||
can now use labelled data to train and test your own models, store those models
|
|
||||||
as {es} indices, and use {ml-docs}/ml-inference.html[inference] to add predicted
|
|
||||||
values to the indices based on your trained models.
|
|
||||||
|
|
||||||
One packaged model that we are releasing in 7.6 is
|
|
||||||
{ml-docs}/ml-lang-ident.html[{lang-ident}]. If you have documents or sources
|
|
||||||
that come in a variety of languages, {lang-ident} can be used to determine the
|
|
||||||
language of text so you can improve the overall search relevance.
|
|
||||||
{lang-ident-cap} is a trained model that can provide a prediction of the
|
|
||||||
language of any text field.
|
|
||||||
// end::notable-highlights[]
|
|
||||||
|
|
||||||
// tag::notable-highlights[]
|
|
||||||
[float]
|
|
||||||
==== {ccs-cap} in {transforms}
|
|
||||||
|
|
||||||
{ref}/transforms.html[{transforms-cap}] can now use {ccs} (CCS) for the source
|
|
||||||
index. Now you can have separate clusters (for example, project clusters) build
|
|
||||||
entity-centric or feature indices against a primary cluster.
|
|
||||||
|
|
||||||
// end::notable-highlights[]
|
|
||||||
|
|
||||||
[float]
|
|
||||||
=== Learn more
|
|
||||||
|
|
||||||
Get more details on these features in the
|
|
||||||
https://www.elastic.co/blog/elasticsearch-7-6-0-released[{es} 7.6 release blog].
|
|
||||||
For a complete list of enhancements and other changes, check out the
|
|
||||||
<<release-notes-7.6.0,{es} 7.6 release notes>>.
|
|
||||||
|
|
||||||
|
|
@ -1,191 +0,0 @@
|
||||||
[[release-highlights-7.7.0]]
|
|
||||||
== 7.7.0 release highlights
|
|
||||||
++++
|
|
||||||
<titleabbrev>7.7.0</titleabbrev>
|
|
||||||
++++
|
|
||||||
|
|
||||||
//NOTE: The notable-highlights tagged regions are re-used in the
|
|
||||||
//Installation and Upgrade Guide
|
|
||||||
|
|
||||||
// tag::notable-highlights[]
|
|
||||||
[float]
|
|
||||||
=== Fixed index corruption on shrunk indices
|
|
||||||
|
|
||||||
Applying deletes or updates on an index after it had been shrunk would likely
|
|
||||||
corrupt the index. We advise users of Elasticsearch 6.x who opt in for soft
|
|
||||||
deletes on some of their indices and all users of Elasticsearch 7.x to upgrade
|
|
||||||
to 7.7 as soon as possible to no longer be subject to this corruption bug. In
|
|
||||||
case upgrading in the near future is not an option, we recommend to completely
|
|
||||||
stop using `_shrink` on read-write indices and to do a force-merge right after
|
|
||||||
shrinking on read-only indices, which significantly reduces the likeliness of
|
|
||||||
being affected by this bug in case deletes or updates get applied by mistake.
|
|
||||||
This bug is fixed as of {es} 7.7.0. Low-level details can be found on the
|
|
||||||
https://issues.apache.org/jira/browse/LUCENE-9300[corresponding issue].
|
|
||||||
|
|
||||||
// end::notable-highlights[]
|
|
||||||
|
|
||||||
// tag::notable-highlights[]
|
|
||||||
[float]
|
|
||||||
=== Significant reduction of heap usage of segments
|
|
||||||
|
|
||||||
This release of Elasticsearch significantly reduces the amount of heap memory
|
|
||||||
that is needed to keep Lucene segments open. In addition to helping with cluster
|
|
||||||
stability, this helps reduce costs by storing much more data per node before
|
|
||||||
hitting memory limits.
|
|
||||||
|
|
||||||
// end::notable-highlights[]
|
|
||||||
|
|
||||||
// tag::notable-highlights[]
|
|
||||||
[discrete]
|
|
||||||
=== {transforms-cap} – now in GA!
|
|
||||||
|
|
||||||
In 7.7, we move {transforms} from beta to general availability.
|
|
||||||
|
|
||||||
{ref}/transforms.html[{transforms-cap}] enable you to pivot existing {es}
|
|
||||||
indices using group-by and aggregations into a destination feature index, which
|
|
||||||
provides opportunities for new insights and analytics. For example, you can use
|
|
||||||
{transforms} to pivot your data into entity-centric indices that summarize the
|
|
||||||
behavior of users or sessions or other entities in your data.
|
|
||||||
|
|
||||||
{transforms-cap} now include support for cross-cluster search. Allowing you to
|
|
||||||
create your destination feature index on a separate cluster from the source
|
|
||||||
indices.
|
|
||||||
|
|
||||||
Aggregation support has been expanded within {transforms} to include support for
|
|
||||||
{ref}/search-aggregations-metrics-percentile-aggregation.html[multi-value (percentiles)]
|
|
||||||
and
|
|
||||||
{ref}/search-aggregations-bucket-filter-aggregation.html[filter aggregations].
|
|
||||||
We also optimized the performance of the
|
|
||||||
{ref}/search-aggregations-bucket-datehistogram-aggregation.html[date histogram aggregations].
|
|
||||||
|
|
||||||
// end::notable-highlights[]
|
|
||||||
|
|
||||||
// tag::notable-highlights[]
|
|
||||||
[discrete]
|
|
||||||
=== Introducing multiclass {classification}
|
|
||||||
|
|
||||||
{ml-docs}/dfa-classification.html[{classification-cap}] using multiple classes
|
|
||||||
is now available in {dfanalytics}. {classification-cap} is a supervised {ml}
|
|
||||||
technique which has been already available as a binary process in the previous
|
|
||||||
release. Multiclass {classification} works well with up to 30 distinct
|
|
||||||
categories.
|
|
||||||
|
|
||||||
// end::notable-highlights[]
|
|
||||||
|
|
||||||
// tag::notable-highlights[]
|
|
||||||
[discrete]
|
|
||||||
=== {feat-imp-cap} at {infer} time
|
|
||||||
|
|
||||||
{feat-imp-cap} now can be calculated at {infer} time. This value provides
|
|
||||||
further insight into the results of a {classification} or {regression} job and
|
|
||||||
therefore helps interpret these results.
|
|
||||||
|
|
||||||
// end::notable-highlights[]
|
|
||||||
|
|
||||||
// tag::notable-highlights[]
|
|
||||||
[float]
|
|
||||||
=== Finer memory control for bucket aggregations
|
|
||||||
|
|
||||||
While building buckets, aggregations will now periodically check the
|
|
||||||
real-memory circuit breaker before continuing to allocate more buckets. This
|
|
||||||
allows better responsivity to memory pressure and avoids `OutOfMemory`
|
|
||||||
situations due to allocating more buckets than the node can handle.
|
|
||||||
|
|
||||||
// end::notable-highlights[]
|
|
||||||
|
|
||||||
// tag::notable-highlights[]
|
|
||||||
[float]
|
|
||||||
=== A new way of searching: asynchronously
|
|
||||||
|
|
||||||
You can now submit {ref}/async-search-intro.html[long-running searches] using
|
|
||||||
the new {ref}/async-search.html[`_async_search` API]. The new API accepts the
|
|
||||||
same parameters and request body as the {ref}/search-search.html[Search API].
|
|
||||||
However, instead of blocking and returning the final response only when it's
|
|
||||||
entirely finished, you can retrieve results from an async search as they become
|
|
||||||
available.
|
|
||||||
|
|
||||||
The request takes a parameter, `wait_for_completion`, which controls how long
|
|
||||||
the server will wait until it sends back a response. The first response
|
|
||||||
contains among others a search unique ID, a response version, an indication if
|
|
||||||
this response is partial or not, plus the usual metadata (shards involved,
|
|
||||||
number of hits etc) and potentially results. If the response is not complete
|
|
||||||
and final, the client can continue polling for results, issuing a new request
|
|
||||||
using the provided search ID. If new results are available, the returned
|
|
||||||
version is incremented and the new batch of results are returned. This can
|
|
||||||
continue until all the results are fetched.
|
|
||||||
|
|
||||||
Unless deleted earlier by the user, the asynchronous searches are kept alive
|
|
||||||
for a given interval. This defaults to 5 days and can be controlled by another
|
|
||||||
request parameter, `keep_alive`.
|
|
||||||
// end::notable-highlights[]
|
|
||||||
|
|
||||||
// tag::notable-highlights[]
|
|
||||||
[float]
|
|
||||||
=== Password protection for the keystore
|
|
||||||
|
|
||||||
{es} uses a custom on-disk {ref}/secure-settings.html[keystore] for secure settings such as
|
|
||||||
passwords and SSL certificates. Up until now, this prevented users with
|
|
||||||
{ref}/elasticsearch-keystore.html[command-line access] from viewing secure files by listing commands, but nothing
|
|
||||||
prevented such users from changing values in the keystore, or removing values
|
|
||||||
from it. Furthermore, the values were only obfuscated by a hash; no
|
|
||||||
user-specific secret protected the secure settings.
|
|
||||||
|
|
||||||
This new feature changes all of that by adding password-protection to the
|
|
||||||
keystore. This is not be a breaking change: if a keystore has no password,
|
|
||||||
there won’t be any new prompts. A user must choose to password-protect their
|
|
||||||
keystore in order to benefit from the new behavior.
|
|
||||||
|
|
||||||
// end::notable-highlights[]
|
|
||||||
|
|
||||||
// tag::notable-highlights[]
|
|
||||||
[float]
|
|
||||||
=== A new aggregation: `top_metrics`
|
|
||||||
|
|
||||||
The new {ref}//search-aggregations-metrics-top-metrics.html[`top_metrics` aggregation] "selects" a metric from a document according
|
|
||||||
to a criteria on a given, different field. That criteria is currently the
|
|
||||||
largest or smallest "sort" value. It is fairly similar to `top_hits` in spirit,
|
|
||||||
but because it is more limited, `top_metrics` uses less memory and
|
|
||||||
is often faster.
|
|
||||||
|
|
||||||
// end::notable-highlights[]
|
|
||||||
|
|
||||||
// tag::notable-highlights[]
|
|
||||||
[float]
|
|
||||||
=== Query speed-up for sorted queries on time-based indices
|
|
||||||
|
|
||||||
We've optimized sorted, top-documents-only queries run on time-based indices.
|
|
||||||
The optimization stems from the fact that the ranges of (document) timestamps
|
|
||||||
in the shards don't overlap. It is implemented by rewriting the shard search
|
|
||||||
requests based on the partial results already available from other shards, if
|
|
||||||
it can be determined that the query will not yield any result from the current
|
|
||||||
shard; i.e. we know in advance that the bottom entry of the (sorted) result set
|
|
||||||
after a partial merge is better than the values contained in this current
|
|
||||||
shard.
|
|
||||||
|
|
||||||
// end::notable-highlights[]
|
|
||||||
|
|
||||||
// tag::notable-highlights[]
|
|
||||||
[float]
|
|
||||||
=== A new aggregation: `boxplot`
|
|
||||||
|
|
||||||
The https://en.wikipedia.org/wiki/Interquartile_range[interquartile range (IQR)] is a common robust measure of statistical dispersion.
|
|
||||||
Compared to the standard deviation, the IQR is less sensitive to outliers in
|
|
||||||
the data, with a breakdown point of 0.25. Along with the median, it is often
|
|
||||||
used in creating a box plot, a simple yet common way to summarize data and
|
|
||||||
identify potential outliers.
|
|
||||||
|
|
||||||
The new {ref}/search-aggregations-metrics-boxplot-aggregation.html[`boxplot`
|
|
||||||
aggregation] calculates the min, max, and medium as well as the first and third
|
|
||||||
quartiles of a given data set.
|
|
||||||
|
|
||||||
// end::notable-highlights[]
|
|
||||||
|
|
||||||
// tag::notable-highlights[]
|
|
||||||
[float]
|
|
||||||
=== AArch64 support
|
|
||||||
|
|
||||||
{es} now provides AArch64 packaging, including bundling an AArch64 JDK
|
|
||||||
distribution. There are some restrictions in place, namely no {ml} support and
|
|
||||||
depending on underlying page sizes, class data sharing is disabled.
|
|
||||||
|
|
||||||
// end::notable-highlights[]
|
|
||||||
|
|
@ -1,167 +0,0 @@
|
||||||
[[release-highlights-7.8.0]]
|
|
||||||
== 7.8.0 release highlights
|
|
||||||
++++
|
|
||||||
<titleabbrev>7.8.0</titleabbrev>
|
|
||||||
++++
|
|
||||||
|
|
||||||
//NOTE: The notable-highlights tagged regions are re-used in the
|
|
||||||
//Installation and Upgrade Guide
|
|
||||||
|
|
||||||
// tag::notable-highlights[]
|
|
||||||
[float]
|
|
||||||
=== Geo improvements
|
|
||||||
|
|
||||||
We have made several improvements to geo support in {es} 7.8.
|
|
||||||
|
|
||||||
- You can now run an aggregation that finds the bounding box (top left point and
|
|
||||||
bottom right point) that contains all shapes matching a query. A shape is
|
|
||||||
anything that is defined by multiple points. See
|
|
||||||
{ref}/search-aggregations-metrics-geobounds-aggregation.html[Geo Bounds Aggregations].
|
|
||||||
- {ref}/search-aggregations-bucket-geohashgrid-aggregation.html[GeoHash grid aggregations]
|
|
||||||
and {ref}/search-aggregations-bucket-geotilegrid-aggregation.html[map tile grid aggregations]
|
|
||||||
allow you to group geo_points into buckets.
|
|
||||||
- {ref}/search-aggregations-metrics-geocentroid-aggregation.html[Geo centroid aggregations]
|
|
||||||
allow you to compute the weighted https://en.wikipedia.org/wiki/Centroid[centroid]
|
|
||||||
from all coordinate values for a geo_point field.
|
|
||||||
|
|
||||||
// end::notable-highlights[]
|
|
||||||
|
|
||||||
// tag::notable-highlights[]
|
|
||||||
[float]
|
|
||||||
=== Add support for t-test aggregations
|
|
||||||
|
|
||||||
{es} now supports a `t_test` metrics
|
|
||||||
aggregation, which performs a statistical hypothesis test in which the test
|
|
||||||
statistic follows a
|
|
||||||
https://en.wikipedia.org/wiki/Student%27s_t-distribution[Student’s
|
|
||||||
t-distribution] under the null hypothesis on numeric values extracted from
|
|
||||||
the aggregated documents or generated by provided scripts. In practice,
|
|
||||||
this will tell you if the difference between two population means are
|
|
||||||
statistically significant and did not occur by chance alone. See
|
|
||||||
{ref}/search-aggregations-metrics-ttest-aggregation.html[T-Test Aggregation].
|
|
||||||
|
|
||||||
// end::notable-highlights[]
|
|
||||||
|
|
||||||
// tag::notable-highlights[]
|
|
||||||
[float]
|
|
||||||
=== Expose aggregation usage in feature usage API
|
|
||||||
|
|
||||||
It is now possible to fetch a count of aggregations that have been executed
|
|
||||||
via the {ref}/cluster-nodes-usage.html[node features API]. This is broken down per
|
|
||||||
combination of aggregation and data type, per shard on each node, from the
|
|
||||||
last restart until the time when the counts are fetched. When trying to
|
|
||||||
analyze how {es} is being used in practice, it is useful to know
|
|
||||||
the usage distribution across aggregations and field types. For example,
|
|
||||||
you might be able to conclude that a certain part of an index is not used a
|
|
||||||
lot and could perhaps can be eliminated.
|
|
||||||
|
|
||||||
|
|
||||||
// end::notable-highlights[]
|
|
||||||
|
|
||||||
|
|
||||||
// tag::notable-highlights[]
|
|
||||||
[float]
|
|
||||||
=== Support `value_count` and `avg` aggregations over histogram fields
|
|
||||||
|
|
||||||
{es} now implements `value_count` and `avg` aggregations over histogram
|
|
||||||
fields.
|
|
||||||
|
|
||||||
When the `value_count` aggregation is computed on {ref}/histogram.html[histogram
|
|
||||||
fields], the result of the aggregation is the sum of all numbers in the
|
|
||||||
`counts` array of the histogram.
|
|
||||||
|
|
||||||
When the average is computed on histogram fields, the result of the
|
|
||||||
aggregation is the weighted average of all elements in the `values` array
|
|
||||||
taking into consideration the number in the same position in the `counts`
|
|
||||||
array.
|
|
||||||
|
|
||||||
// end::notable-highlights[]
|
|
||||||
|
|
||||||
// tag::notable-highlights[]
|
|
||||||
[float]
|
|
||||||
=== Reduce aggregation memory consumption
|
|
||||||
|
|
||||||
{es} now attempts to save memory on the coordinating node by delaying
|
|
||||||
deserialization of the shard results for an aggregation until the last
|
|
||||||
second. This is helpful as it makes the shard-aggregations results "short
|
|
||||||
lived" garbage. It also should shrink the memory usage of aggregations when
|
|
||||||
they are waiting to be merged.
|
|
||||||
|
|
||||||
Additionally, when the search is in batched reduce mode, {es} will force
|
|
||||||
the results to be serialized between batch reduces in an attempt to keep
|
|
||||||
the memory usage as low as possible between reductions.
|
|
||||||
|
|
||||||
// end::notable-highlights[]
|
|
||||||
|
|
||||||
// tag::notable-highlights[]
|
|
||||||
[float]
|
|
||||||
=== Scalar functions now supported in SQL aggregations
|
|
||||||
|
|
||||||
When querying {es} using SQL, it is now possible to use scalar functions
|
|
||||||
inside aggregations. This allows for more complex expressions, including
|
|
||||||
within `GROUP BY` or `HAVING` clauses. For example:
|
|
||||||
|
|
||||||
[source, sql]
|
|
||||||
----
|
|
||||||
SELECT
|
|
||||||
MAX(CASE WHEN a IS NULL then -1 ELSE abs(a * 10) + 1 END) AS max,
|
|
||||||
b
|
|
||||||
FROM test
|
|
||||||
GROUP BY b
|
|
||||||
HAVING
|
|
||||||
MAX(CASE WHEN a IS NULL then -1 ELSE abs(a * 10) + 1 END) > 5
|
|
||||||
----
|
|
||||||
|
|
||||||
// end::notable-highlights[]
|
|
||||||
// tag::notable-highlights[]
|
|
||||||
[float]
|
|
||||||
[[release-highlights-7.8.0-throttling]]
|
|
||||||
=== Increase the performance and scalability of {transforms} with throttling
|
|
||||||
|
|
||||||
{transforms-cap} achieved GA status in 7.7 and now in 7.8 they are even better
|
|
||||||
with the introduction of
|
|
||||||
{ref}/transform-overview.html#transform-performance[throttling]. You can spread
|
|
||||||
out the impact of the {transforms} on your cluster by defining the rate at which
|
|
||||||
they perform search and index requests. Set the `docs_per_second` limit when you
|
|
||||||
create or update your {transform}.
|
|
||||||
|
|
||||||
// end::notable-highlights[]
|
|
||||||
// tag::notable-highlights[]
|
|
||||||
[float]
|
|
||||||
[[release-highlights-7.8.0-mml]]
|
|
||||||
=== Better estimates for {ml} model memory usage
|
|
||||||
|
|
||||||
For 7.8, we introduce dynamic estimation of the model memory limit for jobs in
|
|
||||||
{ml-docs}/ootb-ml-jobs.html[ML solution modules]. The estimate is generated
|
|
||||||
during the job creation. It uses a calculation based on the specific detectors
|
|
||||||
of the job and the cardinality of the partitioning and influencer fields. It
|
|
||||||
means the job setup has better default values depending on the size of the data
|
|
||||||
being analyzed.
|
|
||||||
|
|
||||||
// end::notable-highlights[]
|
|
||||||
// tag::notable-highlights[]
|
|
||||||
[float]
|
|
||||||
[[release-highlights-7.8.0-loss-functions]]
|
|
||||||
=== Additional loss functions for {regression}
|
|
||||||
|
|
||||||
{ml-docs}/dfa-regression.html#dfa-regression-lossfunction[Loss functions]
|
|
||||||
measure how well a {ml} model fits a specific data set. In 7.8, we added two new
|
|
||||||
loss functions for {regression} analysis. In addition to the existing mean
|
|
||||||
squared error function, there are now mean squared logarithmic error and
|
|
||||||
Pseudo-Huber loss functions. These additions enable you to choose the
|
|
||||||
loss function that fits best with your data set.
|
|
||||||
|
|
||||||
// end::notable-highlights[]
|
|
||||||
|
|
||||||
// tag::notable-highlights[]
|
|
||||||
[float]
|
|
||||||
[[release-highlights-7.8.0-data-visualizer]]
|
|
||||||
=== Extended upload limit and explanations for Data Visualizer
|
|
||||||
|
|
||||||
You can now upload files up to 1 GB in Data Visualizer. The file structure
|
|
||||||
finder functionality of the Data Visualizer provides more detailed explanations
|
|
||||||
after both successful and unsuccessful analysis which makes it easier to
|
|
||||||
diagnose issues with file upload.
|
|
||||||
|
|
||||||
// end::notable-highlights[]
|
|
||||||
|
|
||||||
Loading…
Reference in New Issue