From 6f03a6c54656a08112168e8a3b29bac1f9cd9037 Mon Sep 17 00:00:00 2001 From: Lisa Cawley Date: Mon, 25 Mar 2019 10:23:10 -0700 Subject: [PATCH] [DOCS] Simplify ML upgrade step (#40006) --- docs/reference/upgrade/close-ml.asciidoc | 44 ++++++++++--------- .../upgrade/cluster_restart.asciidoc | 2 +- .../upgrade/reindex_upgrade.asciidoc | 20 +++++++-- .../upgrade/rolling_upgrade.asciidoc | 2 +- 4 files changed, 41 insertions(+), 27 deletions(-) diff --git a/docs/reference/upgrade/close-ml.asciidoc b/docs/reference/upgrade/close-ml.asciidoc index e30210b16a1..179d4f9adff 100644 --- a/docs/reference/upgrade/close-ml.asciidoc +++ b/docs/reference/upgrade/close-ml.asciidoc @@ -11,33 +11,35 @@ POST _ml/set_upgrade_mode?enabled=false // TEARDOWN //////////// -If your {ml} indices were created earlier than the previous major version, they -must be reindexed. In those circumstances, there must be no machine learning -jobs running during the upgrade. +If your {ml} indices were created before {prev-major-version}, you must +<>. -In all other circumstances, there is no requirement to close your {ml} jobs. -There are, however, advantages to doing so. If you choose to leave your jobs -running during the upgrade, they are affected when you stop the {ml} nodes. The -jobs move to another {ml} node and restore the model states. This scenario has -the least disruption to the active {ml} jobs but incurs the highest load on the -cluster. +If your {ml} indices were created in {prev-major-version}, you can: -To close all {ml} jobs before you upgrade, see -{stack-ov}/stopping-ml.html[Stopping {ml}]. This method persists the model -state at the moment of closure, which means that when you open your jobs after -the upgrade, they use the exact same model. This scenario takes the most time, -however, especially if you have many jobs or jobs with large model states. - -To temporarily halt the tasks associated with your {ml} jobs and {dfeeds} and -prevent new jobs from opening, use the <>: +* Leave your {ml} jobs running during the upgrade. When you shut down a +{ml} node, its jobs automatically move to another node and restore the model +states. This option enables your jobs to continue running during the upgrade but +it puts increased load on the cluster. +* Temporarily halt the tasks associated with your {ml} jobs and {dfeeds} and +prevent new jobs from opening by using the +<>: ++ +-- [source,js] -------------------------------------------------- POST _ml/set_upgrade_mode?enabled=true -------------------------------------------------- // CONSOLE -This method does not persist the absolute latest model state, rather it uses the -last model state that was automatically saved. By halting the tasks, you avoid -incurring the cost of managing active jobs during the upgrade and it's quicker -than stopping {dfeeds} and closing jobs. +When you disable upgrade mode, the jobs resume using the last model +state that was automatically saved. This option avoids the overhead of managing +active jobs during the upgrade and is faster than explicitly stopping {dfeeds} +and closing jobs. +-- + +* {stack-ov}/stopping-ml.html[Stop all {dfeeds} and close all jobs]. This option +saves the model state at the time of closure. When you reopen the jobs after the +upgrade, they use the exact same model. However, saving the latest model state +takes longer than using upgrade mode, especially if you have a lot of jobs or +jobs with large model states. diff --git a/docs/reference/upgrade/cluster_restart.asciidoc b/docs/reference/upgrade/cluster_restart.asciidoc index 1402bbd9bb7..20786857993 100644 --- a/docs/reference/upgrade/cluster_restart.asciidoc +++ b/docs/reference/upgrade/cluster_restart.asciidoc @@ -26,7 +26,7 @@ recovery. include::synced-flush.asciidoc[] -- -. *Stop any machine learning jobs that are running.* +. *Temporarily stop the tasks associated with active {ml} jobs and {dfeeds}.* (Optional) + -- include::close-ml.asciidoc[] diff --git a/docs/reference/upgrade/reindex_upgrade.asciidoc b/docs/reference/upgrade/reindex_upgrade.asciidoc index 1dc20ec6e70..7fc6c320263 100644 --- a/docs/reference/upgrade/reindex_upgrade.asciidoc +++ b/docs/reference/upgrade/reindex_upgrade.asciidoc @@ -59,9 +59,10 @@ ifdef::include-xpack[] [TIP] ==== If you use {ml-features} and your {ml} indices were created before -{prev-major-version}, you must -{stack-ov}/stopping-ml.html[stop all {dfeeds} and close all {ml} jobs] before -you reindex the indices. +{prev-major-version}, you must temporarily halt the tasks associated with your +{ml} jobs and {dfeeds} and prevent new jobs from opening during the reindex. Use +the <> or +{stack-ov}/stopping-ml.html[stop all {dfeeds} and close all {ml} jobs]. If you use {es} {security-features}, before you reindex `.security*` internal indices it is a good idea to create a temporary superuser account in the `file` @@ -112,6 +113,17 @@ indices from the previous major version to be upgraded to the current major version. Skipping a major version means that you must resolve any backward compatibility issues yourself. +ifdef::include-xpack[] +If you use {ml-features} and you're migrating indices from a 6.5 or earlier +cluster, the job and {dfeed} configuration information are not stored in an +index. You must recreate your {ml} jobs in the new cluster. If you are migrating +from a 6.6 or later cluster, it is a good idea to temporarily halt the tasks +associated with your {ml} jobs and {dfeeds} to prevent inconsistencies between +different {ml} indices that are reindexed at slightly different times. Use the +<> or +{stack-ov}/stopping-ml.html[stop all {dfeeds} and close all {ml} jobs]. +endif::include-xpack[] + ============================================= To migrate your indices: @@ -184,4 +196,4 @@ monitor progress of the reindex job with the <>: `30s` and `1`). .. Once reindexing is complete and the status of the new index is `green`, - you can delete the old index. + you can delete the old index. \ No newline at end of file diff --git a/docs/reference/upgrade/rolling_upgrade.asciidoc b/docs/reference/upgrade/rolling_upgrade.asciidoc index 9871f35a2cc..e6ed7bb4846 100644 --- a/docs/reference/upgrade/rolling_upgrade.asciidoc +++ b/docs/reference/upgrade/rolling_upgrade.asciidoc @@ -35,7 +35,7 @@ include::synced-flush.asciidoc[] -- -. *Stop any machine learning jobs that are running.* +. *Temporarily stop the tasks associated with active {ml} jobs and {dfeeds}.* (Optional) + -- include::close-ml.asciidoc[]