2016-02-29 09:35:48 -05:00
|
|
|
[[breaking-changes-5.0]]
|
|
|
|
|
== Breaking changes in 5.0
|
2015-09-03 09:00:52 -04:00
|
|
|
|
|
|
|
|
This section discusses the changes that you need to be aware of when migrating
|
2016-02-29 09:35:48 -05:00
|
|
|
your application to Elasticsearch 5.0.
|
|
|
|
|
|
2016-08-10 04:16:25 -04:00
|
|
|
[IMPORTANT]
|
|
|
|
|
.Known networking bug in 5.0.0-alpha5
|
|
|
|
|
======================================================
|
|
|
|
|
|
2016-08-10 10:42:58 -04:00
|
|
|
There is a bug in the new Netty4 implementation in this release which affects any REST request with
|
|
|
|
|
a body that is sent in two requests, the first with an `Expect: 100-continue` header. This bug will
|
|
|
|
|
manifest with an exception similar to the following:
|
2016-08-10 04:16:25 -04:00
|
|
|
|
|
|
|
|
[source,txt]
|
|
|
|
|
----
|
|
|
|
|
[WARN ][http.netty4] [wtOV9Vb] caught exception while handling client http traffic, closing connection [id: 0x1320b717, L:/0:0:0:0:0:0:0:1:9200 - R:/0:0:0:0:0:0:0:1:54732]
|
|
|
|
|
java.lang.UnsupportedOperationException: unsupported message type: DefaultFullHttpResponse (expected: ByteBuf, FileRegion)
|
|
|
|
|
----
|
|
|
|
|
|
|
|
|
|
This is due to incorrect handling of the `Expect` HTTP header, and it can be
|
|
|
|
|
worked around in one of three ways:
|
|
|
|
|
|
|
|
|
|
* Use a client which does not add `Expect` headers (including the official clients).
|
|
|
|
|
|
|
|
|
|
* Pass a blank `Except` header, e.g.
|
|
|
|
|
+
|
|
|
|
|
[source,sh]
|
|
|
|
|
----
|
|
|
|
|
curl -H 'Expect:' ...
|
|
|
|
|
----
|
|
|
|
|
|
|
|
|
|
* Use Netty3 for the HTTP layer by passing the following setting at startup:
|
|
|
|
|
+
|
|
|
|
|
[source,sh]
|
|
|
|
|
----
|
|
|
|
|
./bin/elasticsearch -Ehttp.type=netty3
|
|
|
|
|
----
|
|
|
|
|
|
|
|
|
|
======================================================
|
|
|
|
|
|
2016-06-23 07:10:50 -04:00
|
|
|
[float]
|
|
|
|
|
[[migration-plugin]]
|
|
|
|
|
=== Migration Plugin
|
|
|
|
|
|
|
|
|
|
The https://github.com/elastic/elasticsearch-migration/blob/2.x/README.asciidoc[`elasticsearch-migration` plugin]
|
|
|
|
|
(compatible with Elasticsearch 2.3.0 and above) will help you to find issues
|
2016-08-10 04:16:25 -04:00
|
|
|
that need to be addressed when upgrading to Elasticsearch 5.0.
|
2016-06-23 07:10:50 -04:00
|
|
|
|
2016-03-14 23:13:06 -04:00
|
|
|
[float]
|
|
|
|
|
=== Indices created before 5.0
|
|
|
|
|
|
2016-06-21 12:02:55 -04:00
|
|
|
Elasticsearch 5.0 can read indices created in version 2.0 or above. An
|
|
|
|
|
Elasticsearch 5.0 node will not start in the presence of indices created in a
|
|
|
|
|
version of Elasticsearch before 2.0.
|
2016-03-14 23:13:06 -04:00
|
|
|
|
2016-03-13 16:17:48 -04:00
|
|
|
[IMPORTANT]
|
|
|
|
|
.Reindex indices from Elasticseach 1.x or before
|
|
|
|
|
=========================================
|
|
|
|
|
|
|
|
|
|
Indices created in Elasticsearch 1.x or before will need to be reindexed with
|
2016-06-23 07:10:50 -04:00
|
|
|
Elasticsearch 2.x in order to be readable by Elasticsearch 5.x. It is not
|
|
|
|
|
sufficient to use the <<indices-upgrade,`upgrade`>> API. The easiest
|
|
|
|
|
way to reindex old indices is to upgrade to Elasticsearch 2.3 or later and to use the
|
|
|
|
|
`reindex` API, or the reindex UI provided by the <<migration-plugin,Migration Plugin>>.
|
2016-03-13 16:17:48 -04:00
|
|
|
|
|
|
|
|
=========================================
|
|
|
|
|
|
2016-06-21 12:02:55 -04:00
|
|
|
The first time Elasticsearch 5.0 starts, it will automatically rename index
|
|
|
|
|
folders to use the index UUID instead of the index name. If you are using
|
|
|
|
|
<<indices-shadow-replicas,shadow replicas>> with shared data folders, first
|
|
|
|
|
start a single node with access to all data folders, and let it rename all
|
|
|
|
|
index folders before starting other nodes in the cluster.
|
|
|
|
|
|
2016-03-13 16:17:48 -04:00
|
|
|
[float]
|
|
|
|
|
=== Also see:
|
|
|
|
|
|
2016-02-29 09:35:48 -05:00
|
|
|
* <<breaking_50_search_changes>>
|
2016-03-13 16:17:48 -04:00
|
|
|
* <<breaking_50_mapping_changes>>
|
|
|
|
|
* <<breaking_50_percolator>>
|
2016-04-26 03:40:30 -04:00
|
|
|
* <<breaking_50_suggester>>
|
2016-03-13 16:17:48 -04:00
|
|
|
* <<breaking_50_index_apis>>
|
2016-05-12 16:07:46 -04:00
|
|
|
* <<breaking_50_document_api_changes>>
|
2016-03-13 16:17:48 -04:00
|
|
|
* <<breaking_50_settings_changes>>
|
|
|
|
|
* <<breaking_50_allocation>>
|
2016-05-03 02:53:15 -04:00
|
|
|
* <<breaking_50_http_changes>>
|
2016-02-29 09:35:48 -05:00
|
|
|
* <<breaking_50_rest_api_changes>>
|
|
|
|
|
* <<breaking_50_cat_api>>
|
|
|
|
|
* <<breaking_50_java_api_changes>>
|
|
|
|
|
* <<breaking_50_packaging>>
|
2016-03-13 16:17:48 -04:00
|
|
|
* <<breaking_50_plugins>>
|
2016-04-08 06:18:02 -04:00
|
|
|
* <<breaking_50_fs>>
|
Use the new points API to index numeric fields. #17746
This makes all numeric fields including `date`, `ip` and `token_count` use
points instead of the inverted index as a lookup structure. This is expected
to perform worse for exact queries, but faster for range queries. It also
requires less storage.
Notes about how the change works:
- Numeric mappers have been split into a legacy version that is essentially
the current mapper, and a new version that uses points, eg.
LegacyDateFieldMapper and DateFieldMapper.
- Since new and old fields have the same names, the decision about which one
to use is made based on the index creation version.
- If you try to force using a legacy field on a new index or a field that uses
points on an old index, you will get an exception.
- IP addresses now support IPv6 via Lucene's InetAddressPoint and store them
in SORTED_SET doc values using the same encoding (fixed length of 16 bytes
and sortable).
- The internal MappedFieldType that is stored by the new mappers does not have
any of the points-related properties set. Instead, it keeps setting the index
options when parsing the `index` property of mappings and does
`if (fieldType.indexOptions() != IndexOptions.NONE) { // add point field }`
when parsing documents.
Known issues that won't fix:
- You can't use numeric fields in significant terms aggregations anymore since
this requires document frequencies, which points do not record.
- Term queries on numeric fields will now return constant scores instead of
giving better scores to the rare values.
Known issues that we could work around (in follow-up PRs, this one is too large
already):
- Range queries on `ip` addresses only work if both the lower and upper bounds
are inclusive (exclusive bounds are not exposed in Lucene). We could either
decide to implement it, or drop range support entirely and tell users to
query subnets using the CIDR notation instead.
- Since IP addresses now use a different representation for doc values,
aggregations will fail when running a terms aggregation on an ip field on a
list of indices that contains both pre-5.0 and 5.0 indices.
- The ip range aggregation does not work on the new ip field. We need to either
implement range aggs for SORTED_SET doc values or drop support for ip ranges
and tell users to use filters instead. #17700
Closes #16751
Closes #17007
Closes #11513
2016-04-01 05:07:35 -04:00
|
|
|
* <<breaking_50_aggregations_changes>>
|
2016-02-28 07:12:05 -05:00
|
|
|
* <<breaking_50_scripting>>
|
Use the new points API to index numeric fields. #17746
This makes all numeric fields including `date`, `ip` and `token_count` use
points instead of the inverted index as a lookup structure. This is expected
to perform worse for exact queries, but faster for range queries. It also
requires less storage.
Notes about how the change works:
- Numeric mappers have been split into a legacy version that is essentially
the current mapper, and a new version that uses points, eg.
LegacyDateFieldMapper and DateFieldMapper.
- Since new and old fields have the same names, the decision about which one
to use is made based on the index creation version.
- If you try to force using a legacy field on a new index or a field that uses
points on an old index, you will get an exception.
- IP addresses now support IPv6 via Lucene's InetAddressPoint and store them
in SORTED_SET doc values using the same encoding (fixed length of 16 bytes
and sortable).
- The internal MappedFieldType that is stored by the new mappers does not have
any of the points-related properties set. Instead, it keeps setting the index
options when parsing the `index` property of mappings and does
`if (fieldType.indexOptions() != IndexOptions.NONE) { // add point field }`
when parsing documents.
Known issues that won't fix:
- You can't use numeric fields in significant terms aggregations anymore since
this requires document frequencies, which points do not record.
- Term queries on numeric fields will now return constant scores instead of
giving better scores to the rare values.
Known issues that we could work around (in follow-up PRs, this one is too large
already):
- Range queries on `ip` addresses only work if both the lower and upper bounds
are inclusive (exclusive bounds are not exposed in Lucene). We could either
decide to implement it, or drop range support entirely and tell users to
query subnets using the CIDR notation instead.
- Since IP addresses now use a different representation for doc values,
aggregations will fail when running a terms aggregation on an ip field on a
list of indices that contains both pre-5.0 and 5.0 indices.
- The ip range aggregation does not work on the new ip field. We need to either
implement range aggs for SORTED_SET doc values or drop support for ip ranges
and tell users to use filters instead. #17700
Closes #16751
Closes #17007
Closes #11513
2016-04-01 05:07:35 -04:00
|
|
|
|
2016-03-03 13:44:56 -05:00
|
|
|
|
2016-03-13 16:17:48 -04:00
|
|
|
include::migrate_5_0/search.asciidoc[]
|
2016-01-22 06:50:28 -05:00
|
|
|
|
2016-03-13 16:17:48 -04:00
|
|
|
include::migrate_5_0/mapping.asciidoc[]
|
2016-01-22 06:50:28 -05:00
|
|
|
|
2016-03-13 16:17:48 -04:00
|
|
|
include::migrate_5_0/percolator.asciidoc[]
|
2016-01-28 04:31:43 -05:00
|
|
|
|
2016-04-26 03:40:30 -04:00
|
|
|
include::migrate_5_0/suggest.asciidoc[]
|
|
|
|
|
|
2016-03-13 16:17:48 -04:00
|
|
|
include::migrate_5_0/index-apis.asciidoc[]
|
2016-02-29 09:25:39 -05:00
|
|
|
|
2016-05-17 16:47:20 -04:00
|
|
|
include::migrate_5_0/docs.asciidoc[]
|
|
|
|
|
|
2016-03-13 16:17:48 -04:00
|
|
|
include::migrate_5_0/settings.asciidoc[]
|
2016-02-29 09:25:39 -05:00
|
|
|
|
2016-03-13 16:17:48 -04:00
|
|
|
include::migrate_5_0/allocation.asciidoc[]
|
2016-01-28 04:31:43 -05:00
|
|
|
|
2016-05-03 02:53:15 -04:00
|
|
|
include::migrate_5_0/http.asciidoc[]
|
|
|
|
|
|
2016-03-13 16:17:48 -04:00
|
|
|
include::migrate_5_0/rest.asciidoc[]
|
2016-02-04 10:23:58 -05:00
|
|
|
|
2016-03-13 16:17:48 -04:00
|
|
|
include::migrate_5_0/cat.asciidoc[]
|
2016-01-22 07:00:50 -05:00
|
|
|
|
2016-03-13 16:17:48 -04:00
|
|
|
include::migrate_5_0/java.asciidoc[]
|
2016-01-22 07:00:50 -05:00
|
|
|
|
2016-03-13 16:17:48 -04:00
|
|
|
include::migrate_5_0/packaging.asciidoc[]
|
2016-03-08 15:20:15 -05:00
|
|
|
|
2016-03-13 16:17:48 -04:00
|
|
|
include::migrate_5_0/plugins.asciidoc[]
|
2016-03-08 15:20:15 -05:00
|
|
|
|
2016-04-08 06:18:02 -04:00
|
|
|
include::migrate_5_0/fs.asciidoc[]
|
Use the new points API to index numeric fields. #17746
This makes all numeric fields including `date`, `ip` and `token_count` use
points instead of the inverted index as a lookup structure. This is expected
to perform worse for exact queries, but faster for range queries. It also
requires less storage.
Notes about how the change works:
- Numeric mappers have been split into a legacy version that is essentially
the current mapper, and a new version that uses points, eg.
LegacyDateFieldMapper and DateFieldMapper.
- Since new and old fields have the same names, the decision about which one
to use is made based on the index creation version.
- If you try to force using a legacy field on a new index or a field that uses
points on an old index, you will get an exception.
- IP addresses now support IPv6 via Lucene's InetAddressPoint and store them
in SORTED_SET doc values using the same encoding (fixed length of 16 bytes
and sortable).
- The internal MappedFieldType that is stored by the new mappers does not have
any of the points-related properties set. Instead, it keeps setting the index
options when parsing the `index` property of mappings and does
`if (fieldType.indexOptions() != IndexOptions.NONE) { // add point field }`
when parsing documents.
Known issues that won't fix:
- You can't use numeric fields in significant terms aggregations anymore since
this requires document frequencies, which points do not record.
- Term queries on numeric fields will now return constant scores instead of
giving better scores to the rare values.
Known issues that we could work around (in follow-up PRs, this one is too large
already):
- Range queries on `ip` addresses only work if both the lower and upper bounds
are inclusive (exclusive bounds are not exposed in Lucene). We could either
decide to implement it, or drop range support entirely and tell users to
query subnets using the CIDR notation instead.
- Since IP addresses now use a different representation for doc values,
aggregations will fail when running a terms aggregation on an ip field on a
list of indices that contains both pre-5.0 and 5.0 indices.
- The ip range aggregation does not work on the new ip field. We need to either
implement range aggs for SORTED_SET doc values or drop support for ip ranges
and tell users to use filters instead. #17700
Closes #16751
Closes #17007
Closes #11513
2016-04-01 05:07:35 -04:00
|
|
|
|
|
|
|
|
include::migrate_5_0/aggregations.asciidoc[]
|
2016-02-28 07:12:05 -05:00
|
|
|
|
|
|
|
|
include::migrate_5_0/scripting.asciidoc[]
|
