From 0a85d1fe03ead83e39208d1e27ab1a7ba6a550c0 Mon Sep 17 00:00:00 2001 From: Zachary Tong Date: Wed, 10 Apr 2019 09:41:04 -0400 Subject: [PATCH] Add missing "long form" 7.0 breaking changes (#41049) These are breaking changes that were present in the release notes but the PR didn't include any "narrative long form" description for the docs. --- docs/reference/migration/migrate_7_0.asciidoc | 4 ++ .../migrate_7_0/aggregations.asciidoc | 8 +++ .../migration/migrate_7_0/api.asciidoc | 62 ++++++++++++++++++- .../migration/migrate_7_0/ingest.asciidoc | 27 ++++++++ .../migrate_7_0/low_level_restclient.asciidoc | 5 ++ .../migration/migrate_7_0/mappings.asciidoc | 9 ++- .../migration/migrate_7_0/ml.asciidoc | 15 +++++ .../migration/migrate_7_0/packaging.asciidoc | 6 ++ .../migration/migrate_7_0/search.asciidoc | 22 +++++++ .../migration/migrate_7_0/settings.asciidoc | 19 ++++++ .../migration/migrate_7_0/suggesters.asciidoc | 5 ++ 11 files changed, 179 insertions(+), 3 deletions(-) create mode 100644 docs/reference/migration/migrate_7_0/ingest.asciidoc create mode 100644 docs/reference/migration/migrate_7_0/ml.asciidoc diff --git a/docs/reference/migration/migrate_7_0.asciidoc b/docs/reference/migration/migrate_7_0.asciidoc index 3d723074e9d..f0bdd1de439 100644 --- a/docs/reference/migration/migrate_7_0.asciidoc +++ b/docs/reference/migration/migrate_7_0.asciidoc @@ -13,7 +13,9 @@ See also <> and <>. * <> * <> * <> +* <> * <> +* <> * <> * <> * <> @@ -50,8 +52,10 @@ include::migrate_7_0/aggregations.asciidoc[] include::migrate_7_0/analysis.asciidoc[] include::migrate_7_0/cluster.asciidoc[] include::migrate_7_0/discovery.asciidoc[] +include::migrate_7_0/ingest.asciidoc[] include::migrate_7_0/indices.asciidoc[] include::migrate_7_0/mappings.asciidoc[] +include::migrate_7_0/ml.asciidoc[] include::migrate_7_0/search.asciidoc[] include::migrate_7_0/suggesters.asciidoc[] include::migrate_7_0/packaging.asciidoc[] diff --git a/docs/reference/migration/migrate_7_0/aggregations.asciidoc b/docs/reference/migration/migrate_7_0/aggregations.asciidoc index 6fb974a1095..85dd2af04f5 100644 --- a/docs/reference/migration/migrate_7_0/aggregations.asciidoc +++ b/docs/reference/migration/migrate_7_0/aggregations.asciidoc @@ -47,3 +47,11 @@ explicitly defining how their data is processed. The `percentiles` and `percentile_ranks` aggregations used to return `NaN` in the response if they were applied to an empty set of values. Because `NaN` is not officially supported by JSON, it has been replaced with `null`. + +[float] +==== `stats` and `extended_stats` now return 0 instead of `null` for zero docs + +When the `stats` and `extended_stats` aggregations collected zero docs (`doc_count: 0`), +their value would be `null`. This was in contrast with the `sum` aggregation which +would return `0`. The `stats` and `extended_stats` aggs are now consistent with +`sum` and also return zero. diff --git a/docs/reference/migration/migrate_7_0/api.asciidoc b/docs/reference/migration/migrate_7_0/api.asciidoc index cdcf6a93e36..8fae000457f 100644 --- a/docs/reference/migration/migrate_7_0/api.asciidoc +++ b/docs/reference/migration/migrate_7_0/api.asciidoc @@ -76,10 +76,32 @@ pools. Note that `core` and `max` will be populated for scaling thread pools, and `size` will be populated for fixed thread pools. [float] -==== The parameter `fields` deprecated in 6.x has been removed from Bulk request -and Update request. The Update API returns `400 - Bad request` if request contains +==== The parameter `fields` deprecated in 6.x has been removed from Bulk request +and Update request. The Update API returns `400 - Bad request` if request contains unknown parameters (instead of ignored in the previous version). +[float] +==== PUT Document with Version error message changed when document is missing + +If you attempt to `PUT` a document with versioning (e.g. `PUT /test/_doc/1?version=4`) +but the document does not exist, a cryptic message is returned: + +[source,text] +---------- +version conflict, current version [-1] is different than the one provided [4] +---------- + +Now if the document is missing a more helpful message is returned: + +[source,text] +---------- +document does not exist (expected version [4]) +---------- + +Although exceptions messages are liable to change and not generally subject to +backwards compatibility, the nature of this message might mean clients are relying +on parsing the version numbers and so the format change might impact some users. + [float] [[remove-suggest-metric]] ==== Remove support for `suggest` metric/index metric in indices stats and nodes stats APIs @@ -168,3 +190,39 @@ privilege). The `_cache/clear` API no longer supports the `GET` HTTP verb. It must be called with `POST`. + +[float] +==== Cluster state size metrics removed from Cluster State API Response + +The `compressed_size` / `compressed_size_in_bytes` fields were removed from +the Cluster State API response. The calculation of the size was expensive and had +dubious value, so the field was removed from the response. + +[float] +==== Migration Assistance API has been removed + +The Migration Assistance API has been functionally replaced by the +Deprecation Info API, and the Migration Upgrade API is not used for the +transition from ES 6.x to 7.x, and does not need to be kept around to +repair indices that were not properly upgraded before upgrading the +cluster, as was the case in 6. + +[float] +==== Changes to thread pool naming in Node and Cat APIs +The `thread_pool` information returned from the Nodes and Cat APIs has been +standardized to use the same terminology as the thread pool configurations. +This means the response will align with the configuration instead of being +the same across all the thread pools, regardless of type. + +[float] +==== Return 200 when cluster has valid read-only blocks +If the cluster was configured with `no_master_block: write` and lost its master, +it would return a `503` status code from a main request (`GET /`) even though +there are viable read-only nodes available. The cluster now returns 200 status +in this situation. + +[float] +==== Clearing indices cache is now POST-only +Clearing the cache indices could previously be done via GET and POST. As GET should +only support read only non state-changing operations, this is no longer allowed. +Only POST can be used to clear the cache. diff --git a/docs/reference/migration/migrate_7_0/ingest.asciidoc b/docs/reference/migration/migrate_7_0/ingest.asciidoc new file mode 100644 index 00000000000..a8c9b8084d6 --- /dev/null +++ b/docs/reference/migration/migrate_7_0/ingest.asciidoc @@ -0,0 +1,27 @@ +[float] +[[breaking_70_ingest_changes]] +=== API changes + +//NOTE: The notable-breaking-changes tagged regions are re-used in the +//Installation and Upgrade Guide + +//tag::notable-breaking-changes[] + +[float] +==== Ingest configuration exception information is now transmitted in metadata field + +Previously, some ingest configuration exception information about ingest processors +was sent to the client in the HTTP headers, which is inconsistent with how +exceptions are conveyed in other parts of Elasticsearch. + +Configuration exception information is now conveyed as a field in the response +body. +//end::notable-breaking-changes[] +[float] +==== Ingest plugin special handling has been removed +There was some special handling for installing and removing the `ingest-geoip` and +`ingest-user-agent` plugins after they were converted to modules. This special handling +was done to minimize breaking users in a minor release, and would exit with a status code +zero to avoid breaking automation. + +This special handling has now been removed. diff --git a/docs/reference/migration/migrate_7_0/low_level_restclient.asciidoc b/docs/reference/migration/migrate_7_0/low_level_restclient.asciidoc index e3132b59680..0c7539873cc 100644 --- a/docs/reference/migration/migrate_7_0/low_level_restclient.asciidoc +++ b/docs/reference/migration/migrate_7_0/low_level_restclient.asciidoc @@ -29,3 +29,8 @@ backwards compatibility. We deprecated `setHosts` in 6.4.0 in favor of `setNodes` because it supports host metadata used by the `NodeSelector`. + +[float] +==== Minimum compiler version change +The minimum compiler version on the low-level REST client has been bumped +to JDK 8. diff --git a/docs/reference/migration/migrate_7_0/mappings.asciidoc b/docs/reference/migration/migrate_7_0/mappings.asciidoc index 422c649e33b..cbbe20bbf84 100644 --- a/docs/reference/migration/migrate_7_0/mappings.asciidoc +++ b/docs/reference/migration/migrate_7_0/mappings.asciidoc @@ -21,12 +21,14 @@ This field used to index a composite key formed of the `_type` and the `_id`. Now that indices cannot have multiple types, this has been removed in favour of `_id`. +//tag::notable-breaking-changes[] [float] ==== The `_default_` mapping is no longer allowed The `_default_` mapping has been deprecated in 6.0 and is now no longer allowed in 7.0. Trying to configure a `_default_` mapping on 7.x indices will result in an error. +//end::notable-breaking-changes[] [float] ==== `index_options` for numeric fields has been removed @@ -85,4 +87,9 @@ will be removed in a future version. The maximum allowed number of completion contexts in a mapping will be limited to 10 in the next major version. Completion fields that define more than 10 -contexts in a mapping will log a deprecation warning in this version. \ No newline at end of file +contexts in a mapping will log a deprecation warning in this version. + +[float] +==== `include_type_name` now defaults to `false` +The default for `include_type_name` is now `false` for all APIs that accept +the parameter. diff --git a/docs/reference/migration/migrate_7_0/ml.asciidoc b/docs/reference/migration/migrate_7_0/ml.asciidoc new file mode 100644 index 00000000000..89b77232821 --- /dev/null +++ b/docs/reference/migration/migrate_7_0/ml.asciidoc @@ -0,0 +1,15 @@ +[float] +[[breaking_70_ml_changes]] +=== ML changes + +//NOTE: The notable-breaking-changes tagged regions are re-used in the +//Installation and Upgrade Guide + +//tag::notable-breaking-changes[] + +// end::notable-breaking-changes[] + +[float] +==== Types in Datafeed config are no longer valid +Types have been removed from the datafeed config and are no longer +valid parameters. diff --git a/docs/reference/migration/migrate_7_0/packaging.asciidoc b/docs/reference/migration/migrate_7_0/packaging.asciidoc index db5df312356..f2b4c2a8a2f 100644 --- a/docs/reference/migration/migrate_7_0/packaging.asciidoc +++ b/docs/reference/migration/migrate_7_0/packaging.asciidoc @@ -29,3 +29,9 @@ for windows. These files have been removed. Use the `zip` package instead. Ubuntu 14.04 will reach end-of-life on April 30, 2019. As such, we are no longer supporting Ubuntu 14.04. + +[float] +==== CLI secret prompting is no longer supported +The ability to use `${prompt.secret}` and `${prompt.text}` to collect secrets +from the CLI at server start is no longer supported. Secure settings have replaced +the need for these prompts. diff --git a/docs/reference/migration/migrate_7_0/search.asciidoc b/docs/reference/migration/migrate_7_0/search.asciidoc index f0591408964..8b23201f42c 100644 --- a/docs/reference/migration/migrate_7_0/search.asciidoc +++ b/docs/reference/migration/migrate_7_0/search.asciidoc @@ -212,6 +212,7 @@ on whether queries need to access score or not. As a result `bool` queries with `minimum_should_match` to 1. This behavior has been deprecated in the previous major version. +//tag::notable-breaking-changes[] [float] ==== `hits.total` is now an object in the search response @@ -241,6 +242,7 @@ You can also retrieve `hits.total` as a number in the rest response by adding `rest_total_hits_as_int=true` in the request parameter of the search request. This parameter has been added to ease the transition to the new format and will be removed in the next major version (8.0). +//end::notable-breaking-changes[] [float] ==== `hits.total` is omitted in the response if `track_total_hits` is disabled (false) @@ -250,6 +252,7 @@ will set `hits.total` to null and the object will not be displayed in the rest layer. You can add `rest_total_hits_as_int=true` in the search request parameters to get the old format back (`"total": -1`). +//tag::notable-breaking-changes[] [float] ==== `track_total_hits` defaults to 10,000 @@ -280,3 +283,22 @@ documents. If the total number of hits that match the query is greater than this You can force the count to always be accurate by setting `"track_total_hits` to true explicitly in the search request. +//end::notable-breaking-changes[] + +[float] +==== Limitations on Similarities +Lucene 8 introduced more constraints on similarities, in particular: + +- scores must not be negative, +- scores must not decrease when term freq increases, +- scores must not increase when norm (interpreted as an unsigned long) increases. + +[float] +==== Weights in Function Score must be positive +Negative `weight` parameters in the `function_score` are no longer allowed. + +[float] +==== Query string and Simple query string limit expansion of fields to 1024 +The number of automatically expanded fields for the "all fields" +mode (`"default_field": "*"`) for the query_string and simple_query_string +queries is now 1024 fields. diff --git a/docs/reference/migration/migrate_7_0/settings.asciidoc b/docs/reference/migration/migrate_7_0/settings.asciidoc index f2488092c85..499e6ce3a0f 100644 --- a/docs/reference/migration/migrate_7_0/settings.asciidoc +++ b/docs/reference/migration/migrate_7_0/settings.asciidoc @@ -211,3 +211,22 @@ Elastic Stack to handle the indexing part. ==== Ingest User Agent processor defaults uses `ecs` output format https://github.com/elastic/ecs[ECS] format is now the default. The `ecs` setting for the user agent ingest processor now defaults to true. + +[float] +[[remove-action-master-force_local]] +==== Remove `action.master.force_local` + +The `action.master.force_local` setting was an undocumented setting, used +internally by the tribe node to force reads to local cluster state (instead of +forwarding to a master, which tribe nodes did not have). Since the tribe +node was removed, this setting was removed too. + +[float] +==== Enforce cluster-wide shard limit +The cluster-wide shard limit is now enforced and not optional. The limit can +still be adjusted as desired using the cluster settings API. + +[float] +==== HTTP Max content length setting is no longer parsed leniently +Previously, `http.max_content_length` would reset to `100mb` if the setting was +`Integer.MAX_VALUE`. This leniency has been removed. diff --git a/docs/reference/migration/migrate_7_0/suggesters.asciidoc b/docs/reference/migration/migrate_7_0/suggesters.asciidoc index 213958382aa..5e00b0d249d 100644 --- a/docs/reference/migration/migrate_7_0/suggesters.asciidoc +++ b/docs/reference/migration/migrate_7_0/suggesters.asciidoc @@ -14,3 +14,8 @@ Plugins must now explicitly indicate the type of suggestion that they produce. +[float] +==== Phrase suggester now multiples alpha +Previously, the laplace smoothing used by the phrase suggester added `alpha`, +when it should instead multiply. This behavior has been changed and will +affect suggester scores.