Add missing "long form" 7.0 breaking changes (#41049)

These are breaking changes that were present in the release notes but the PR didn't include any "narrative long form" description for the docs.

docs/reference/migration/migrate_7_0.asciidoc

@@ -13,7 +13,9 @@ See also <<release-highlights>> and <<es-release-notes>>.
 * <<breaking_70_cluster_changes>>
 * <<breaking_70_discovery_changes>>
 * <<breaking_70_indices_changes>>
+* <<breaking_70_ingest_changes>>
 * <<breaking_70_mappings_changes>>
+* <<breaking_70_ml_changes>>
 * <<breaking_70_search_changes>>
 * <<breaking_70_suggesters_changes>>
 * <<breaking_70_packaging_changes>>
@@ -50,8 +52,10 @@ include::migrate_7_0/aggregations.asciidoc[]
 include::migrate_7_0/analysis.asciidoc[]
 include::migrate_7_0/cluster.asciidoc[]
 include::migrate_7_0/discovery.asciidoc[]
+include::migrate_7_0/ingest.asciidoc[]
 include::migrate_7_0/indices.asciidoc[]
 include::migrate_7_0/mappings.asciidoc[]
+include::migrate_7_0/ml.asciidoc[]
 include::migrate_7_0/search.asciidoc[]
 include::migrate_7_0/suggesters.asciidoc[]
 include::migrate_7_0/packaging.asciidoc[]

docs/reference/migration/migrate_7_0/aggregations.asciidoc

@@ -47,3 +47,11 @@ explicitly defining how their data is processed.
 The `percentiles` and `percentile_ranks` aggregations used to return `NaN` in
 the response if they were applied to an empty set of values. Because `NaN` is
 not officially supported by JSON, it has been replaced with `null`.
+
+[float]
+==== `stats` and `extended_stats` now return 0 instead of `null` for zero docs
+
+When the `stats` and `extended_stats` aggregations collected zero docs (`doc_count: 0`),
+their value would be `null`.  This was in contrast with the `sum` aggregation which
+would return `0`.  The `stats` and `extended_stats` aggs are now consistent with
+`sum` and also return zero.

docs/reference/migration/migrate_7_0/api.asciidoc

@@ -80,6 +80,28 @@ and `size` will be populated for fixed thread pools.
 and Update request. The Update API returns `400 - Bad request` if request contains
 unknown parameters (instead of ignored in the previous version).
 
+[float]
+==== PUT Document with Version error message changed when document is missing
+
+If you attempt to `PUT` a document with versioning (e.g. `PUT /test/_doc/1?version=4`)
+but the document does not exist, a cryptic message is returned:
+
+[source,text]
+----------
+version conflict, current version [-1] is different than the one provided [4]
+----------
+
+Now if the document is missing a more helpful message is returned:
+
+[source,text]
+----------
+document does not exist (expected version [4])
+----------
+
+Although exceptions messages are liable to change and not generally subject to
+backwards compatibility, the nature of this message might mean clients are relying
+on parsing the version numbers and so the format change might impact some users.
+
 [float]
 [[remove-suggest-metric]]
 ==== Remove support for `suggest` metric/index metric in indices stats and nodes stats APIs
@@ -168,3 +190,39 @@ privilege).
 
 The `_cache/clear` API no longer supports the `GET` HTTP verb. It must be called
 with `POST`.
+
+[float]
+==== Cluster state size metrics removed from Cluster State API Response
+
+The `compressed_size` / `compressed_size_in_bytes` fields were removed from
+the Cluster State API response.  The calculation of the size was expensive and had
+dubious value, so the field was removed from the response.
+
+[float]
+==== Migration Assistance API has been removed
+
+The Migration Assistance API has been functionally replaced by the
+Deprecation Info API, and the Migration Upgrade API is not used for the
+transition from ES 6.x to 7.x, and does not need to be kept around to
+repair indices that were not properly upgraded before upgrading the
+cluster, as was the case in 6.
+
+[float]
+==== Changes to thread pool naming in Node and Cat APIs
+The `thread_pool` information returned from the Nodes and Cat APIs has been
+standardized to use the same terminology as the thread pool configurations.
+This means the response will align with the configuration instead of being
+the same across all the thread pools, regardless of type.
+
+[float]
+==== Return 200 when cluster has valid read-only blocks
+If the cluster was configured with `no_master_block: write` and lost its master,
+it would return a `503` status code from a main request (`GET /`) even though
+there are viable read-only nodes available.  The cluster now returns 200 status
+in this situation.
+
+[float]
+==== Clearing indices cache is now POST-only
+Clearing the cache indices could previously be done via GET and POST. As GET should
+only support read only non state-changing operations, this is no longer allowed.
+Only POST can be used to clear the cache.

docs/reference/migration/migrate_7_0/ingest.asciidoc

@@ -0,0 +1,27 @@
+[float]
+[[breaking_70_ingest_changes]]
+=== API changes
+
+//NOTE: The notable-breaking-changes tagged regions are re-used in the
+//Installation and Upgrade Guide
+
+//tag::notable-breaking-changes[]
+
+[float]
+==== Ingest configuration exception information is now transmitted in metadata field
+
+Previously, some ingest configuration exception information about ingest processors
+was sent to the client in the HTTP headers, which is inconsistent with how
+exceptions are conveyed in other parts of Elasticsearch.
+
+Configuration exception information is now conveyed as a field in the response
+body.
+//end::notable-breaking-changes[]
+[float]
+==== Ingest plugin special handling has been removed
+There was some special handling for installing and removing the `ingest-geoip` and
+`ingest-user-agent` plugins after they were converted to modules. This special handling
+was done to minimize breaking users in a minor release, and would exit with a status code
+zero to avoid breaking automation.
+
+This special handling has now been removed.

docs/reference/migration/migrate_7_0/low_level_restclient.asciidoc

@@ -29,3 +29,8 @@ backwards compatibility.
 
 We deprecated `setHosts` in 6.4.0 in favor of `setNodes` because it supports
 host metadata used by the `NodeSelector`.
+
+[float]
+==== Minimum compiler version change
+The minimum compiler version on the low-level REST client has been bumped
+to JDK 8.

docs/reference/migration/migrate_7_0/mappings.asciidoc

@@ -21,12 +21,14 @@ This field used to index a composite key formed of the `_type` and the `_id`.
 Now that indices cannot have multiple types, this has been removed in favour
 of `_id`.
 
+//tag::notable-breaking-changes[]
 [float]
 ==== The `_default_` mapping is no longer allowed
 
 The `_default_` mapping has been deprecated in 6.0 and is now no longer allowed
 in 7.0. Trying to configure a `_default_` mapping on 7.x indices will result in
 an error.
+//end::notable-breaking-changes[]
 
 [float]
 ==== `index_options` for numeric fields has been removed
@@ -86,3 +88,8 @@ will be removed in a future version.
 The maximum allowed number of completion contexts in a mapping will be limited
 to 10 in the next major version. Completion fields that define more than 10
 contexts in a mapping will log a deprecation warning in this version.
+
+[float]
+==== `include_type_name` now defaults to `false`
+The default for `include_type_name` is now `false` for all APIs that accept
+the parameter.

docs/reference/migration/migrate_7_0/ml.asciidoc

@@ -0,0 +1,15 @@
+[float]
+[[breaking_70_ml_changes]]
+=== ML changes
+
+//NOTE: The notable-breaking-changes tagged regions are re-used in the
+//Installation and Upgrade Guide
+
+//tag::notable-breaking-changes[]
+
+// end::notable-breaking-changes[]
+
+[float]
+==== Types in Datafeed config are no longer valid
+Types have been removed from the datafeed config and are no longer
+valid parameters.

docs/reference/migration/migrate_7_0/packaging.asciidoc

@@ -29,3 +29,9 @@ for windows. These files have been removed. Use the `zip` package instead.
 
 Ubuntu 14.04 will reach end-of-life on April 30, 2019. As such, we are no longer
 supporting Ubuntu 14.04.
+
+[float]
+==== CLI secret prompting is no longer supported
+The ability to use `${prompt.secret}` and `${prompt.text}` to collect secrets
+from the CLI at server start is no longer supported.  Secure settings have replaced
+the need for these prompts.

docs/reference/migration/migrate_7_0/search.asciidoc

@@ -212,6 +212,7 @@ on whether queries need to access score or not. As a result `bool` queries with
 `minimum_should_match` to 1. This behavior has been deprecated in the previous
 major version.
 
+//tag::notable-breaking-changes[]
 [float]
 ==== `hits.total` is now an object in the search response
 
@@ -241,6 +242,7 @@ You can also retrieve `hits.total` as a number in the rest response by adding
 `rest_total_hits_as_int=true` in the request parameter of the search request.
 This parameter has been added to ease the transition to the new format and
 will be removed in the next major version (8.0).
+//end::notable-breaking-changes[]
 
 [float]
 ==== `hits.total` is omitted in the response if `track_total_hits` is disabled (false)
@@ -250,6 +252,7 @@ will set `hits.total` to null and the object will not be displayed in the rest
 layer. You can add `rest_total_hits_as_int=true` in the search request parameters
 to get the old format back (`"total": -1`).
 
+//tag::notable-breaking-changes[]
 [float]
 ==== `track_total_hits` defaults to 10,000
 
@@ -280,3 +283,22 @@ documents. If the total number of hits that match the query is greater than this
 
 You can force the count to always be accurate by setting `"track_total_hits`
 to true explicitly in the search request.
+//end::notable-breaking-changes[]
+
+[float]
+==== Limitations on Similarities
+Lucene 8 introduced more constraints on similarities, in particular:
+
+- scores must not be negative,
+- scores must not decrease when term freq increases,
+- scores must not increase when norm (interpreted as an unsigned long) increases.
+
+[float]
+==== Weights in Function Score must be positive
+Negative `weight` parameters in the `function_score` are no longer allowed.
+
+[float]
+==== Query string and Simple query string limit expansion of fields to 1024
+The number of automatically expanded fields for the "all fields"
+mode (`"default_field": "*"`) for the query_string and simple_query_string
+queries is now 1024 fields.

docs/reference/migration/migrate_7_0/settings.asciidoc

@@ -211,3 +211,22 @@ Elastic Stack to handle the indexing part.
 ==== Ingest User Agent processor defaults uses `ecs` output format
 https://github.com/elastic/ecs[ECS] format is now the default.
 The `ecs` setting for the user agent ingest processor now defaults to true.
+
+[float]
+[[remove-action-master-force_local]]
+==== Remove `action.master.force_local`
+
+The `action.master.force_local` setting was an undocumented setting, used
+internally by the tribe node to force reads to local cluster state (instead of
+forwarding to a master, which tribe nodes did not have).  Since the tribe
+node was removed, this setting was removed too.
+
+[float]
+==== Enforce cluster-wide shard limit
+The cluster-wide shard limit is now enforced and not optional. The limit can
+still be adjusted as desired using the cluster settings API.
+
+[float]
+==== HTTP Max content length setting is no longer parsed leniently
+Previously, `http.max_content_length` would reset to `100mb` if the setting was
+`Integer.MAX_VALUE`.  This leniency has been removed.

docs/reference/migration/migrate_7_0/suggesters.asciidoc

@@ -14,3 +14,8 @@
 
 Plugins must now explicitly indicate the type of suggestion that they produce.
 
+[float]
+==== Phrase suggester now multiples alpha
+Previously, the laplace smoothing used by the phrase suggester added `alpha`,
+when it should instead multiply.  This behavior has been changed and will
+affect suggester scores.