[Doc] migration guide joda (#51986)

The joda to java.time migration requires users to upgrade their mappings. We allow them to still use 6.x created indices with joda patterns in 7 but ask them to upgrade their patterns in 7.x.
This migration guide is to help them understand how they could be affected and what needs to be changed in their mappings.
closes #51614
closes #51236

docs/reference/migration/migrate_7_0/java_time.asciidoc

@@ -89,6 +89,7 @@ When you are running Elasticsearch 7 with Java 8, you are not able to parse
 the timezone `GMT0` properly anymore. The reason for this is a bug in the
 JDK, which has not been fixed for JDK8. You can read more in the
 https://bugs.openjdk.java.net/browse/JDK-8138664[official issue]
+This bug is fixed in JDK9 and later versions.
 
 [float]
 ==== Scripting with dates should use java time based methods
@@ -125,3 +126,10 @@ and should be replaced.
 
 With the switch to java time, support for negative timestamps has been removed.
 For dates before 1970, use a date format containing a year.
+
+
+[float]
+==== Migration guide
+For a detailed migration guide, see <<migrate-to-java-time>>.
+
+include::migrate_to_java_time.asciidoc[]

docs/reference/migration/migrate_7_0/migrate_to_java_time.asciidoc

@@ -0,0 +1,410 @@
+[[migrate-to-java-time]]
+=== Java time migration guide
+
+With 7.0, {es} switched from joda time to java time for date-related parsing,
+formatting, and calculations. This guide is designed to help you determine
+if your cluster is impacted and, if so, prepare for the upgrade.
+
+[discrete]
+[[java-time-convert-date-formats]]
+=== Convert date formats
+
+To upgrade to 7.0, you'll need to convert any joda-time date formats
+to their java-time equivalents.
+
+To help track this effort, you can prefix java-time date formats with an `8`
+ in {es} 6.8 and later versions.
+
+For example, you can change the date format `YYYY-MM-dd` to `8yyyy-MM-dd` to
+indicate the date format uses java time.
+
+{es} treats date formats starting with the `8` prefix differently depending on
+the version:
+
+*6.8*: Date formats with an `8` prefix are handled as java-time formats. Date
+formats without an `8` prefix are treated as joda-time formats. We recommend
+converting these joda-time formats to java-time _before_ upgrading to 7.x.
+
+*7.x and later*: For indices created in 6.x, date formats without an `8` prefix
+are treated as joda-time formats. For indices created in 7.x and later versions,
+all date formats are treated as java-time formats, regardless of whether it
+starts with an `8` prefix.
+
+[[java-time-migration-impacted-features]]
+==== Impacted features
+The switch to java time only impacts custom <<date,`date`>> and
+<<date_nanos,`date_nanos`>> formats.
+
+These formats are commonly used in:
+
+* <<mapping,Index mappings>>
+* <<indices-templates,Index templates>>
+* <<pipeline,Ingest pipelines>>
+
+If you don't use custom date formats, you can skip the rest of this guide.
+Most custom date formats are compatible. However, several require
+an update.
+
+To see if your date format is impacted, use the <<migration-api-deprecation,deprecation info API>>
+or the {kibana-ref}/upgrade-assistant.html[Kibana upgrade assistant].
+
+[[java-time-migration-incompatible-date-formats]]
+==== Incompatible date formats
+Custom date formats containing the following joda-time literals should be
+changed before upgrading.
+
+`Y` (Year of era)::
++
+--
+Replace with `y`.
+
+*Example:*
+`YYYY-MM-dd` should become `yyyy-MM-dd`.
+
+In java time, `Y` is used for
+https://docs.oracle.com/javase/8/docs/api/java/time/temporal/WeekFields.html[week-based year].
+Using `Y` in place of `y` could result in off-by-one errors in year calculation.
+
+For pattern `YYYY-ww` and date `2019-01-01T00:00:00.000Z`  will give `2019-01`
+For pattern `YYYY-ww` and date `2018-12-31T00:00:00.000Z`  will give `2019-01` (counter-intuitive) because there is >4 days of that week in 2019
+--
+
+`y` (Year)::
++
+--
+Replace with `u`.
+
+*Example:*
+`yyyy-MM-dd` should become `uuuu-MM-dd`.
+
+In java time, `y` is used for year of era. `u` can contain non-positive
+values while `y` cannot. `y` can also be associated with an era field.
+--
+
+
+`C` (Century of era)::
++
+--
+Century of era is not supported in java time.
+There is no replacement. Instead, we recommend you preprocess your input.
+--
+
+`x` (Week year)::
++
+--
+Replace with `Y`.
+
+In java time, `x` means https://docs.oracle.com/javase/8/docs/api/java/time/format/DateTimeFormatter.html[zone-offset]. 
+
+[WARNING]
+====
+Failure to properly convert `x` (Week year) to `Y` could result in data loss.
+====
+--
+
+`Z` (Zone offset/id)::
++
+--
+Replace with multiple `X`'s.
+
+`Z` has a similar meaning in java time. However, java time expects different
+numbers of literals to parse different forms.
+
+Consider migrating to `X`, which gives you more control over how time is parsed.
+For example, the joda-time format `YYYY-MM-dd'T'hh:mm:ssZZ` accepts the following dates:
+
+```
+2010-01-01T01:02:03Z
+2010-01-01T01:02:03+01
+2010-01-01T01:02:03+01:02
+2010-01-01T01:02:03+01:02:03
+```
+
+In java time, you cannot parse all these dates using a single format
+Instead, you must specify 3 separate formats:
+
+```
+2010-01-01T01:02:03Z
+2010-01-01T01:02:03+01
+both parsed with yyyy-MM-dd'T'hh:mm:ssX
+
+2010-01-01T01:02:03+01:02
+yyyy-MM-dd'T'hh:mm:ssXXX
+
+2010-01-01T01:02:03+01:02:03
+yyyy-MM-dd'T'hh:mm:ssXXXXX
+```
+
+
+The formats must then be delimited using `||`:
+[source,txt]
+--------------------------------------------------
+yyyy-MM-dd'T'hh:mm:ssX||yyyy-MM-dd'T'hh:mm:ssXXX||yyyy-MM-dd'T'hh:mm:ssXXXXX
+--------------------------------------------------
+
+The same applies if you expect your pattern to occur without a colon (`:`):
+For example, the `YYYY-MM-dd'T'hh:mm:ssZ` format accepts the following date forms:
+```
+2010-01-01T01:02:03Z
+2010-01-01T01:02:03+01
+2010-01-01T01:02:03+0102
+2010-01-01T01:02:03+010203
+```
+To accept all these forms in java time, you must use the `||` delimiter:
+[source,txt]
+--------------------------------------------------
+yyyy-MM-dd'T'hh:mm:ssX||yyyy-MM-dd'T'hh:mm:ssXX||yyyy-MM-dd'T'hh:mm:ssXXXX
+--------------------------------------------------
+--
+
+`d` (Day)::
++
+--
+In java time, `d` is still interpreted as "day" but is less flexible.
+
+For example, the joda-time date format `YYYY-MM-dd` accepts `2010-01-01` or
+`2010-01-1`.
+
+In java time, you must use the `||` delimiter to provide specify each format:
+
+[source,txt]
+--------------------------------------------------
+yyyy-MM-dd||yyyy-MM-d
+--------------------------------------------------
+
+In java time, `d` also does not accept more than 2 digits. To accept days with more
+than two digits, you must include a text literal in your java-time date format.
+For example, to parse `2010-01-00001`, you must use the following java-time date format: 
+
+[source,txt]
+--------------------------------------------------
+yyyy-MM-'000'dd
+--------------------------------------------------
+--
+
+`e` (Name of day)::
++
+--
+In java time, `e` is still interpreted as "name of day" but does not parse
+short- or full-text forms.
+
+For example, the joda-time date format `EEE YYYY-MM` accepts both
+`Wed 2020-01` and `Wednesday 2020-01`.
+
+To accept both of these dates in java time, you must specify each format using
+the `||` delimiter: 
+
+[source,txt]
+--------------------------------------------------
+cccc yyyy-MM||ccc yyyy-MM
+--------------------------------------------------
+
+The joda-time literal `E` is interpreted as "day of week."
+The java-time literal `c` is interpreted as "localized day of week."
+`E` does not accept full-text day formats, such as `Wednesday`.
+--
+
+`EEEE` and similar text forms::
++
+--
+Support for full-text forms depends on the locale data provided with your Java
+Development Kit (JDK) and other implementation details. We recommend you
+test formats containing these patterns carefully before upgrading.
+--
+
+`z` (Time zone text)::
++
+--
+In java time, `z` outputs 'Z' for Zulu when given a UTC timezone.
+--
+
+[[java-time-migration-test]]
+===== Test with your data
+
+We strongly recommend you test any date format changes using real data before
+deploying in production.
+
+For help with date debugging, consider using
+https://esddd.herokuapp.com/[https://esddd.herokuapp.com/.]
+
+[[java-time-migrate-update-mappings]]
+==== Update index mappings
+To update joda-time date formats in index mappings, you must create a new index
+with an updated mapping and reindex your data to it.
+You can however update your pipelines or templates.
+
+The following `my_index_1` index contains a mapping for the `datetime` field, a
+`date` field with a custom joda-time date format.
+////
+[source,console]
+--------------------------------------------------
+PUT my_index_1
+{
+    "mappings" : {
+      "properties" : {
+         "datetime": {
+           "type": "date",
+           "format": "yyyy/MM/dd HH:mm:ss||yyyy/MM/dd||epoch_millis"
+         }
+      }
+    }
+}
+--------------------------------------------------
+////
+
+[source,console]
+--------------------------------------------------
+GET my_index_1/_mapping
+--------------------------------------------------
+// TEST[continued]
+
+[source,console-result]
+--------------------------------------------------
+{
+  "my_index_1" : {
+    "mappings" : {
+      "properties" : {
+         "datetime": {
+           "type": "date",
+           "format": "yyyy/MM/dd HH:mm:ss||yyyy/MM/dd||epoch_millis"
+         }
+      }
+    }
+  }
+}
+--------------------------------------------------
+
+
+To change the date format for the `datetime` field, create a separate index
+containing an updated mapping and date format.
+
+For example, the following `my_index_2` index changes the `datetime` field's 
+date format to `8uuuu/MM/dd HH:mm:ss||uuuu/MM/dd||epoch_millis`. The `8` prefix 
+indicates this date format uses java time.
+
+[source,console]
+--------------------------------------------------
+PUT my_index_2
+{
+  "mappings": {
+    "properties": {
+      "datetime": {
+        "type": "date",
+        "format": "8uuuu/MM/dd HH:mm:ss||uuuu/MM/dd||epoch_millis"
+      }
+    }
+  }
+}
+--------------------------------------------------
+// TEST[continued]
+
+Next, reindex data from the old index to the new index.
+
+The following <<docs-reindex,reindex>> API request reindexes data from
+`my_index_1` to `my_index_2`.
+
+[source,console]
+--------------------------------------------------
+POST _reindex
+{
+  "source": {
+    "index": "my_index_1"
+  },
+  "dest": {
+    "index": "my_index_2"
+  }
+}
+--------------------------------------------------
+// TEST[continued]
+
+If you use index aliases, update them to point to the new index.
+
+[source,console]
+--------------------------------------------------
+POST /_aliases
+{
+    "actions" : [
+        { "remove" : { "index" : "my_index_1", "alias" : "my_index" } },
+        { "add" : { "index" : "my_index_2", "alias" : "my_index" } }
+    ]
+}
+--------------------------------------------------
+// TEST[continued]
+
+[[java-time-migration-update-ingest-pipelines]]
+===== Update ingest pipelines
+If your ingest pipelines contain joda-time date formats, you can update them
+using the <<put-pipeline-api,put ingest pipeline>> API.
+
+[source,console]
+--------------------------------------------------
+PUT _ingest/pipeline/my_pipeline
+{
+  "description": "Pipeline for routing data to specific index",
+  "processors": [
+    {
+      "date": {
+        "field": "createdTime",
+        "formats": [
+         "8uuuu-w"
+        ]
+      },
+      "date_index_name": {
+        "field": "@timestamp",
+        "date_rounding": "d",
+        "index_name_prefix": "x-",
+        "index_name_format": "8uuuu-w"
+      }
+    }
+  ]
+}
+--------------------------------------------------
+
+
+[[java-time-migration-update-index-templates]]
+===== Update index templates
+
+If your index templates contain joda-time date formats, you can update them
+using the <<indices-templates,put index template>> API.
+
+[source,console]
+--------------------------------------------------
+PUT _template/template_1
+{
+  "index_patterns": [
+    "te*",
+    "bar*"
+  ],
+  "settings": {
+    "number_of_shards": 1
+  },
+  "mappings": {
+    "_source": {
+      "enabled": false
+    },
+    "properties": {
+      "host_name": {
+        "type": "keyword"
+      },
+      "created_at": {
+        "type": "date",
+        "format": "8EEE MMM dd HH:mm:ss Z yyyy"
+      }
+    }
+  }
+}
+--------------------------------------------------
+
+////
+[source,console]
+--------------------------------------------------
+DELETE /_template/template_1
+--------------------------------------------------
+// TEST[continued]
+////
+
+[[java-time-migration-update-external-tools-templates]]
+===== Update external tools and templates
+Ensure you also update any date formats in templates or tools outside of {es}.
+This can include tools such as {beats-ref}/getting-started.html[{beats}] or
+{logstash-ref}/index.html[Logstash].

docs/reference/release-notes/7.0.asciidoc

@@ -6,6 +6,15 @@ releases of 7.0.0.
 
 Also see <<breaking-changes-7.0,Breaking changes in 7.0>>.
 
+[float]
+=== Known issues
+
+* Indices created in 6.x with <<date,`date`>> and <<date_nanos,`date_nanos`>> fields using formats
+that are incompatible with java.time patterns will cause parsing errors, incorrect date calculations or wrong search results.
+https://github.com/elastic/elasticsearch/pull/52555
+This is fixed in {es} 7.7 and later versions.
+
+
 [[breaking-7.0.0]]
 [float]
 === Breaking changes

docs/reference/release-notes/7.1.asciidoc

@@ -3,6 +3,14 @@
 
 Also see <<breaking-changes-7.1,Breaking changes in 7.1>>.
 
+[float]
+=== Known issues
+
+* Indices created in 6.x with <<date,`date`>> and <<date_nanos,`date_nanos`>> fields using formats
+that are incompatible with java.time patterns will cause parsing errors, incorrect date calculations or wrong search results.
+https://github.com/elastic/elasticsearch/pull/52555
+This is fixed in {es} 7.7 and later versions.
+
 [[bug-7.1.1]]
 [float]
 === Bug fixes

docs/reference/release-notes/7.2.asciidoc

@@ -85,6 +85,14 @@ Security::
 
 Also see <<breaking-changes-7.2,Breaking changes in 7.2>>.
 
+[float]
+=== Known issues
+
+* Indices created in 6.x with <<date,`date`>> and <<date_nanos,`date_nanos`>> fields using formats
+that are incompatible with java.time patterns will cause parsing errors, incorrect date calculations or wrong search results.
+https://github.com/elastic/elasticsearch/pull/52555
+This is fixed in {es} 7.7 and later versions.
+
 [[breaking-7.2.0]]
 [float]
 === Breaking changes

docs/reference/release-notes/7.3.asciidoc

@@ -125,6 +125,7 @@ Security::
 * Use system context for looking up connected nodes {pull}43991[#43991] (issue: {issue}43974[#43974])
 
 
+
 [[upgrade-7.3.1]]
 [float]
 === Upgrades
@@ -138,6 +139,16 @@ Infra/Packaging::
 
 Also see <<breaking-changes-7.3,Breaking changes in 7.3>>.
 
+
+[float]
+=== Known issues
+
+* Indices created in 6.x with <<date,`date`>> and <<date_nanos,`date_nanos`>> fields using formats
+that are incompatible with java.time patterns will cause parsing errors, incorrect date calculations or wrong search results.
+https://github.com/elastic/elasticsearch/pull/52555
+This is fixed in {es} 7.7 and later versions.
+
+
 [[breaking-7.3.0]]
 [float]
 === Breaking changes

docs/reference/release-notes/7.4.asciidoc

@@ -116,6 +116,11 @@ Any attempt to log a slow search can throw an AIOOBE due to a bug that
 performs concurrent modifications on a shared byte array.
 (issue: {issue}/48358[#48358])
 
+* Indices created in 6.x with <<date,`date`>> and <<date_nanos,`date_nanos`>> fields using formats
+that are incompatible with java.time patterns will cause parsing errors, incorrect date calculations or wrong search results.
+https://github.com/elastic/elasticsearch/pull/52555
+This is fixed in {es} 7.7 and later versions.
+
 [[breaking-7.4.0]]
 [float]
 === Breaking changes

docs/reference/release-notes/7.5.asciidoc

@@ -98,6 +98,11 @@ Also see <<breaking-changes-7.5,Breaking changes in 7.5>>.
 If a {transform} is running during upgrade, the {transform} audit index might disappear.
 (issue: {issue}/49730[#49730])
 
+* Indices created in 6.x with <<date,`date`>> and <<date_nanos,`date_nanos`>> fields using formats
+that are incompatible with java.time patterns will cause parsing errors, incorrect date calculations or wrong search results.
+https://github.com/elastic/elasticsearch/pull/52555
+This is fixed in {es} 7.7 and later versions.
+
 [[breaking-7.5.0]]
 [float]
 === Breaking changes