OpenSearch/docs/en/ml/troubleshooting.asciidoc

[[ml-troubleshooting]]
== {xpackml} Troubleshooting
++++
<titleabbrev>{xpackml}</titleabbrev>
++++

Use the information in this section to troubleshoot common problems and find
answers for frequently asked questions.

* <<ml-rollingupgrade>>
* <<ml-mappingclash>>

To get help, see <<xpack-help>>.

[[ml-rollingupgrade]]
=== Machine learning features unavailable after rolling upgrade

This problem occurs after you upgrade all of the nodes in your cluster to
{version} by using rolling upgrades. When you try to use {xpackml} features for
the first time, all attempts fail, though `GET _xpack` and `GET _xpack/usage`
indicate that {xpack} is enabled.

*Symptoms:*

* Errors when you click *Machine Learning* in {kib}.
For example: `Jobs list could not be created` and `An internal server error occurred`.
* Null pointer and remote transport exceptions when you run {ml} APIs such as
`GET _xpack/ml/anomaly_detectors` and `GET _xpack/ml/datafeeds`.
* Errors in the log files on the master nodes.
For example: `unable to install ml metadata upon startup`

*Resolution:*

After you upgrade all master-eligible nodes to {es} {version} and {xpack}
{version}, restart the current master node, which triggers the {xpackml}
features to re-initialize.

For more information, see {ref}/rolling-upgrades.html[Rolling upgrades].

[[ml-mappingclash]]
=== Job creation failure due to mapping clash

This problem occurs when you try to create a job.

*Symptoms:*

* Illegal argument exception occurs when you click *Create Job* in {kib} or run
the create job API. For example:
`Save failed: [status_exception] This job would cause a mapping clash
with existing field [field_name] - avoid the clash by assigning a dedicated
results index` or `Save failed: [illegal_argument_exception] Can't merge a non
object mapping [field_name] with an object mapping [field_name]`.

*Resolution:*

This issue typically occurs when two or more jobs store their results in the
same index and the results contain fields with the same name but different
data types or different `fields` settings.

By default, {ml} results are stored in the `.ml-anomalies-shared` index in {es}.
To resolve this issue, click *Advanced > Use dedicated index* when you create
the job in {kib}. If you are using the create job API, specify an index name in
the `results_index_name` property.

[[ml-jobnames]]
=== {kib} cannot display jobs with invalid characters in their name

This problem occurs when you create a job by using the
{ref}/ml-put-job.html[Create Jobs API] then try to view that job in {kib}. In
particular, the problem occurs when you use a period(.) in the job identifier.

*Symptoms:*

* When you try to open a job (named, for example, `job.test` in the
**Anomaly Explorer** or the **Single Metric Viewer**, the job name is split and
the text after the period is assumed to be the job name. If a job does not exist
with that abbreviated name, an error occurs. For example:
`Warning Requested job test does not exist`. If a job exists with that
abbreviated name, it is displayed.

*Resolution:*

Create jobs in {kib} or ensure that you create jobs with valid identifiers when
you use the {ml} APIs. For more information about valid identifiers, see
{ref}/ml-put-job.html[Create Jobs API] or
{ref}/ml-job-resource.html[Job Resources].

[[ml-upgradedf]]

=== Upgraded nodes fail to start due to {dfeed} issues

This problem occurs when you have a {dfeed} that contains search or query
domain specific language (DSL) that was discontinued. For example, if you
created a {dfeed} query in 5.x using search syntax that was deprecated in 5.x
and removed in 6.0, you must fix the {dfeed} before you upgrade to 6.0.

*Symptoms:*

* If {ref}/logging.html#deprecation-logging[deprecation logging] is enabled
before the upgrade, deprecation messages are generated when the {dfeeds} attempt
to retrieve data.
* After the upgrade, nodes fail to start and the error indicates that they
failed to read the local state.

*Resolution:*

Before you upgrade, identify the problematic search or query DSL. In 5.6.5 and
later, the Upgrade Assistant detects these scenarios. If you cannot fix the DSL
before the upgrade, you must delete the {dfeed} then re-create it with valid DSL
after the upgrade.

If you do not fix or delete the {dfeed} before the upgrade, in order to successfully
start the failing nodes you must downgrade the nodes then fix the problem per
above.

See also {stack-ref}/upgrading-elastic-stack.html[Upgrading the Elastic Stack].