Docs: forward port release docs of 7.7.0 (#56706)

Forward port the release docs of 7.7.0: breaking changes, release notes,
release highlights.

docs/reference/migration/migrate_7_7.asciidoc

@@ -9,15 +9,10 @@ your application to Elasticsearch 7.7.
 
 See also <<release-highlights>> and <<es-release-notes>>.
 
-coming[7.7.0]
-
 //NOTE: The notable-breaking-changes tagged regions are re-used in the
 //Installation and Upgrade Guide
 
 //tag::notable-breaking-changes[]
-
-//end::notable-breaking-changes[]
-
 [discrete]
 [[breaking_77_logging_changes]]
 === Logging changes
@@ -30,7 +25,22 @@ loggers in the `org.elasticsearch.action.*` hierarchy emitted log messages at
 `DEBUG` level by default. This sometimes resulted in a good deal of unnecessary
 log noise. From 7.7 onwards the default log level for logger in this hierarchy
 is now `INFO`, in line with most other loggers. If needed, you can recover the
-pre-7.7 default behaviour by adjusting your <<logging>>.
+pre-7.7 default behaviour by adjusting your {ref}/logging.html[logging].
+
+[discrete]
+[[breaking_77_mapping_changes]]
+=== Mapping changes
+
+[discrete]
+[[stricter-mapping-validation]]
+==== Validation for dynamic templates
+
+So far misconfiguration of dynamic templates have been discovered when indexing
+a document with an unmapped field only. In {es} 8.0 and later versions, dynamic mappings
+have stricter validation, done at mapping update time. Invalid updates, such as using
+incorrect analyzer settings or unknown field types, fail. For
+indices created in {es} 7.7 and later version, the update succeeds but emits a warning.
+
 
 [discrete]
 [[breaking_77_settings_changes]]
@@ -86,7 +96,14 @@ happen when using date math or dates that don't specify up to the last
 millisecond. While queries on `date` field round up to the latest millisecond
 for `gt` and `lte` boundaries, the same queries on `date_range` fields didn't
 do this. The behavior is now the same for both field types like documented in
-<<range-query-date-math-rounding>>.
+{ref}/query-dsl-range-query.html#range-query-date-math-rounding[Date math and rounding].
+
+[discrete]
+==== Pipeline aggregation validation errors
+The pipeline aggregation validation has been moved to the coordinating node.
+Those errors that used to return HTTP 500s/Internal Server Errors now return
+400/Bad Request and we now return a list of validation errors rather than the
+first one we encounter.
 
 [discrete]
 [[breaking_77_highlighters_changes]]
@@ -98,3 +115,4 @@ If a keyword value was ignored during indexing because of its length
 (`ignore_above` parameter was applied), {es} doesn't attempt to
 highlight it anymore, which means no highlights are produced for
 ignored values.
+//end::notable-breaking-changes[]

docs/reference/release-notes/7.7.asciidoc

@@ -131,6 +131,7 @@ Search::
 === Enhancements
 
 Aggregations::
+* Fixed rewrite of time zone without DST {pull}54398[#54398]
 * Try to save memory on aggregations {pull}53793[#53793]
 * Speed up partial reduce of terms aggregations {pull}53216[#53216] (issue: {issue}51857[#51857])
 * Simplify SiblingPipelineAggregator {pull}53144[#53144]
@@ -182,6 +183,8 @@ Features/Features::
 * Enable deprecation checks for removed settings {pull}53317[#53317]
 
 Features/ILM+SLM::
+* Use Priority.IMMEDIATE for stop ILM cluster update {pull}54909[#54909]
+* Add cluster update timeout on step retry {pull}54878[#54878]
 * Hide ILM & SLM history aliases {pull}53564[#53564]
 * Avoid race condition in ILMHistorySotre {pull}53039[#53039] (issues: {issue}50353[#50353], {issue}52853[#52853])
 * Make FreezeStep retryable {pull}52540[#52540]
@@ -213,9 +216,6 @@ Features/Monitoring::
 * Secure password for monitoring HTTP exporter {pull}50919[#50919] (issue: {issue}50197[#50197])
 * Validate SSL settings at parse time {pull}49196[#49196] (issue: {issue}47711[#47711])
 
-Features/Stats::
-* Add analysis components and mapping types to the usage API. {pull}51031[#51031]
-
 Features/Watcher::
 * Make watch history indices hidden {pull}52962[#52962] (issue: {issue}50251[#50251])
 * Upgrade to the latest OWASP HTML sanitizer {pull}50765[#50765] (issue: {issue}50395[#50395])
@@ -235,6 +235,7 @@ Infra/Logging::
 * Capture stdout and stderr to log4j log {pull}50259[#50259] (issue: {issue}50156[#50156])
 
 Infra/Packaging::
+* Use AdoptOpenJDK API to Download JDKs {pull}55127[#55127] (issue: {issue}55125[#55125])
 * Introduce aarch64 Docker image {pull}53936[#53936] (issue: {issue}53914[#53914])
 * Introduce jvm.options.d for customizing JVM options {pull}51882[#51882] (issue: {issue}51626[#51626])
 
@@ -414,6 +415,7 @@ Audit::
 * Logfile audit settings validation {pull}52537[#52537] (issues: {issue}47038[#47038], {issue}47711[#47711], {issue}52357[#52357])
 
 Authentication::
+* Fix responses for the token APIs {pull}54532[#54532] (issue: {issue}53323[#53323])
 * Fix potential bug in concurrent token refresh support {pull}53668[#53668]
 * Respect runas realm for ApiKey security operations {pull}52178[#52178] (issue: {issue}51975[#51975])
 * Preserve ApiKey credentials for async verification {pull}51244[#51244]
@@ -425,12 +427,14 @@ Authorization::
 * Allow _rollup_search with read privilege {pull}52043[#52043] (issue: {issue}50245[#50245])
 
 CCR::
+* Clear recent errors when auto-follow successfully {pull}54997[#54997]
 * Put CCR tasks on (data && remote cluster clients) {pull}54146[#54146] (issue: {issue}53924[#53924])
 * Handle no such remote cluster exception in ccr {pull}53415[#53415] (issue: {issue}53225[#53225])
 * Fix shard follow task cleaner under security {pull}52347[#52347] (issues: {issue}44702[#44702], {issue}51971[#51971])
 
 CRUD::
 * Force execution of finish shard bulk request {pull}51957[#51957] (issue: {issue}51904[#51904])
+* Block too many concurrent mapping updates {pull}51038[#51038] (issue: {issue}50670[#50670])
 * Return 429 status code when there's a read_only cluster block {pull}50166[#50166] (issue: {issue}49393[#49393])
 
 Cluster Coordination::
@@ -444,21 +448,28 @@ Distributed::
 * Exclude nested documents in LuceneChangesSnapshot {pull}51279[#51279]
 
 Engine::
+* Update translog policy before the next safe commit {pull}54839[#54839] (issue: {issue}52223[#52223])
 * Fix doc_stats and segment_stats of ReadOnlyEngine {pull}53345[#53345] (issues: {issue}51303[#51303], {issue}51331[#51331])
 * Do not wrap soft-deletes reader for segment stats {pull}51331[#51331] (issues: {issue}51192[#51192], {issue}51303[#51303])
 * Account soft-deletes in FrozenEngine {pull}51192[#51192] (issue: {issue}50775[#50775])
 * Fixed an index corruption bug that would occur when applying deletes or updates on an index after it has been shrunk. More details can be found on the https://issues.apache.org/jira/browse/LUCENE-9300[corresponding issue].
 
 Features/CAT APIs::
+* Cat tasks output should respect time display settings {pull}54536[#54536]
 * Fix NPE in RestPluginsAction {pull}52620[#52620] (issue: {issue}45321[#45321])
 
 Features/ILM+SLM::
+* Ensure error handler is called during SLM retention callback failure {pull}55252[#55252] (issue: {issue}55217[#55217])
+* Ignore ILM indices in the TerminalPolicyStep {pull}55184[#55184] (issue: {issue}51631[#51631])
 * Disallow negative TimeValues {pull}53913[#53913] (issue: {issue}54041[#54041])
 * Fix null config in SnapshotLifecyclePolicy.toRequest {pull}53328[#53328] (issues: {issue}44465[#44465], {issue}53171[#53171])
 * Freeze step retry when not acknowledged {pull}53287[#53287]
 * Make the set-single-node-allocation retryable {pull}52077[#52077] (issue: {issue}43401[#43401])
 * Fix the init step to actually be retryable {pull}52076[#52076]
 
+Features/Indices APIs::
+* Read the index.number_of_replicas from template so that wait_for_active_shards is interpreted correctly {pull}54231[#54231]
+
 Features/Ingest::
 * Fix ingest pipeline _simulate api with empty docs never returns a response {pull}52937[#52937] (issue: {issue}52833[#52833])
 * Handle errors when evaluating if conditions in processors {pull}52543[#52543] (issue: {issue}52339[#52339])
@@ -492,6 +503,8 @@ Highlighting::
 * Fix highlighter support in PinnedQuery and added test {pull}53716[#53716] (issue: {issue}53699[#53699])
 
 Infra/Core::
+* Make feature usage version aware {pull}55246[#55246] (issues: {issue}44589[#44589], {issue}55248[#55248])
+* Avoid StackOverflowError if write circular reference exception {pull}54147[#54147] (issue: {issue}53589[#53589])
 * Fix Joda compatibility in stream protocol {pull}53823[#53823] (issue: {issue}53586[#53586])
 * Avoid self-suppression on grouped action listener {pull}53262[#53262] (issue: {issue}53174[#53174])
 * Ignore virtual ethernet devices that disappear {pull}51581[#51581] (issue: {issue}49914[#49914])
@@ -537,6 +550,7 @@ Mapping::
 * Throw better exception on wrong `dynamic_templates` syntax {pull}51783[#51783] (issue: {issue}51486[#51486])
 
 Network::
+* Add support for more named curves {pull}55179[#55179] (issue: {issue}55031[#55031])
 * Allow proxy mode server name to be updated {pull}54107[#54107]
 * Invoke response handler on failure to send {pull}53631[#53631]
 * Do not log no-op reconnections at DEBUG {pull}53469[#53469]
@@ -553,6 +567,7 @@ Reindex::
 * Allow comma separated source indices {pull}52044[#52044] (issue: {issue}51949[#51949])
 
 SQL::
+* Fix ODBC metadata for DATE & TIME data types {pull}55316[#55316] (issue: {issue}41086[#41086])
 * Fix NPE for parameterized LIKE/RLIKE {pull}53573[#53573] (issue: {issue}53557[#53557])
 * Add support for index aliases for SYS COLUMNS command {pull}53525[#53525] (issue: {issue}31609[#31609])
 * Fix issue with LIKE/RLIKE as painless script {pull}53495[#53495] (issue: {issue}53486[#53486])
@@ -567,6 +582,7 @@ SQL::
 * Selecting a literal from grouped by query generates error {pull}41964[#41964] (issues: {issue}41413[#41413], {issue}41951[#41951])
 
 Search::
+* Improve robustness of Query Result serializations {pull}54692[#54692] (issue: {issue}54665[#54665])
 * Fix Term Vectors with artificial docs and keyword fields {pull}53504[#53504] (issue: {issue}53494[#53494])
 * Fix concurrent requests race over scroll context limit {pull}53449[#53449]
 * Fix pre-sorting of shards in the can_match phase {pull}53397[#53397]
@@ -576,6 +592,8 @@ Search::
 * Correct boost in `script_score` query and error on negative scores {pull}52478[#52478] (issue: {issue}48465[#48465])
 
 Snapshot/Restore::
+* Exclude Snapshot Shard Status Update Requests from Circuit Breaker {pull}55376[#55376] (issue: {issue}54714[#54714])
+* Fix Snapshot Completion Listener Lost on Master Failover {pull}54286[#54286]
 * Fix Non-Verbose Snapshot List Missing Empty Snapshots {pull}52433[#52433]
 * Fix Inconsistent Shard Failure Count in Failed Snapshots {pull}51416[#51416] (issue: {issue}47550[#47550])
 * Fix Overly Aggressive Request DeDuplication {pull}51270[#51270] (issue: {issue}51253[#51253])

docs/reference/release-notes/highlights-7.7.0.asciidoc

@@ -24,6 +24,17 @@ https://issues.apache.org/jira/browse/LUCENE-9300[corresponding issue].
 
 // end::notable-highlights[]
 
+// tag::notable-highlights[]
+[float]
+=== Significant reduction of heap usage of segments
+
+This release of Elasticsearch significantly reduces the amount of heap memory
+that is needed to keep Lucene segments open. In addition to helping with cluster
+stability, this helps reduce costs by storing much more data per node before
+hitting memory limits.
+
+// end::notable-highlights[]
+
 // tag::notable-highlights[]
 [discrete]
 === {transforms-cap} – now in GA!
@@ -69,4 +80,113 @@ categories.
 further insight into the results of a {classification} or {regression} job and 
 therefore helps interpret these results.
 
-// end::notable-highlights[]
+// end::notable-highlights[]
+
+// tag::notable-highlights[]
+[float]
+=== Finer memory control for bucket aggregations
+
+We introduced a new `search.check_buckets_step_size` setting to
+better control how the coordinating node allocates memory when aggregating
+buckets. The allocation of buckets is now be done in steps, each step
+allocating a number of buckets equal to this setting. To avoid an `OutOfMemory`
+error, a parent circuit breaker check is performed on allocation.
+
+// end::notable-highlights[]
+
+// tag::notable-highlights[]
+[float]
+=== A new way of searching: asynchronously
+
+You can now submit {ref}/async-search-intro.html[long-running searches] using
+the new {ref}/async-search.html[`_async_search` API]. The new API accepts the
+same parameters and request body as the {ref}/search-search.html[Search API].
+However, instead of blocking and returning the final response only when it's 
+entirely finished, you can retrieve results from an async search as they become 
+available.
+
+The request takes a parameter, `wait_for_completion`, which controls how long
+the server will wait until it sends back a response. The first response
+contains among others a search unique ID, a response version, an indication if
+this response is partial or not, plus the usual metadata (shards involved,
+number of hits etc) and potentially results. If the response is not complete
+and final, the client can continue polling for results, issuing a new request
+using the provided search ID. If new results are available, the returned
+version is incremented and the new batch of results are returned. This can 
+continue until all the results are fetched.
+
+Unless deleted earlier by the user, the asynchronous searches are kept alive
+for a given interval. This defaults to 5 days and can be controlled by another
+request parameter, `keep_alive`.
+// end::notable-highlights[]
+
+// tag::notable-highlights[]
+[float]
+=== Password protection for the keystore
+
+{es} uses a custom on-disk {ref}/secure-settings.html[keystore] for secure settings such as
+passwords and SSL certificates. Up until now, this prevented users with
+{ref}/elasticsearch-keystore.html[command-line access] from viewing secure files by listing commands, but nothing
+prevented such users from changing values in the keystore, or removing values
+from it. Furthermore, the values were only obfuscated by a hash; no
+user-specific secret protected the secure settings.
+
+This new feature changes all of that by adding password-protection to the
+keystore. This is not be a breaking change: if a keystore has no password,
+there won’t be any new prompts. A user must choose to password-protect their
+keystore in order to benefit from the new behavior.
+
+// end::notable-highlights[]
+
+// tag::notable-highlights[]
+[float]
+=== A new aggregation: `top_metrics`
+
+The new {ref}//search-aggregations-metrics-top-metrics.html[`top_metrics` aggregation] "selects" a metric from a document according
+to a criteria on a given, different field. That criteria is currently the
+largest or smallest "sort" value. It is fairly similar to `top_hits` in spirit,
+but because it is more limited, `top_metrics` uses less memory and
+is often faster.
+
+// end::notable-highlights[]
+
+// tag::notable-highlights[]
+[float]
+=== Query speed-up for sorted queries on time-based indices
+
+We've optimized sorted, top-documents-only queries run on time-based indices.
+The optimization stems from the fact that the ranges of (document) timestamps
+in the shards don't overlap. It is implemented by rewriting the shard search
+requests based on the partial results already available from other shards, if
+it can be determined that the query will not yield any result from the current
+shard; i.e. we know in advance that the bottom entry of the (sorted) result set
+after a partial merge is better than the values contained in this current
+shard.
+
+// end::notable-highlights[]
+
+// tag::notable-highlights[]
+[float]
+=== A new aggregation: `boxplot`
+
+The https://en.wikipedia.org/wiki/Interquartile_range[interquartile range (IQR)] is a common robust measure of statistical dispersion.
+Compared to the standard deviation, the IQR is less sensitive to outliers in
+the data, with a breakdown point of 0.25. Along with the median, it is often
+used in creating a box plot, a simple yet common way to summarize data and
+identify potential outliers.
+
+The new {ref}/search-aggregations-metrics-boxplot-aggregation.html[`boxplot`
+aggregation] calculates the min, max, and medium as well as the first and third
+quartiles of a given data set.
+
+// end::notable-highlights[]
+
+// tag::notable-highlights[]
+[float]
+=== AArch64 support
+
+{es} now provides AArch64 packaging, including bundling an AArch64 JDK
+distribution. There are some restrictions in place, namely no {ml} support and
+depending on underlying page sizes, class data sharing is disabled.
+
+// end::notable-highlights[]