Remove `node.attr.xpack.ml.allocation_enabled` node attribute and instead use `node.attr.max_running_jobs` attribute to indicate a node is a ml node.
(in addition to indicating how many jobs can hosted on a node)
Original commit: elastic/x-pack-elasticsearch@7979bc55b4
Fields were missing from the job's copy constructor.
This was the cause of intermittent test failures.
Original commit: elastic/x-pack-elasticsearch@3cd705ab3c
...correctly update the job's model_snapshot_id whenever
a new model snapshot is persisted. Load the snapshot that
corresponds to the job's model_snapshot_id when a job is
opened.
This PR makes a series of changes that completes the change
of having job's active model snapshot persisted on the job.
Original commit: elastic/x-pack-elasticsearch@c2e23fa1ee
A job allocation is either a job task in OPENING state or a job task that has been assigned to an executor node, but the executor node hasn't had the opportunity to set the job task status to OPENING.
In order to keep track of restarted tasks, `allocationIdOnLastStatusUpdate` field was added to `PersistentTaskInProgress` class.
This will allow persistent task implementors to detect whether the executor node has changed or has been unset since the last status update has occured.
Original commit: elastic/x-pack-elasticsearch@b7b85a8274
This commit removes the usages and definition of deprecated methods that do not accept the XContentType for the source.
Additionally, usages of *Entity classes from the http client library have been changed to always specify the content
type.
Original commit: elastic/x-pack-elasticsearch@29d336a008
All validation should be done in the builder and the Job object should just be a dumb, immutable blob.
Original commit: elastic/x-pack-elasticsearch@a78e6edbe9
* Update running process from cluster state update
* Put a big lock on the UpdateProcessAction
* Protect concurrent update of the process with a semaphore
* Review comments
Original commit: elastic/x-pack-elasticsearch@d08b53247a
This commit moves persistent tasks from ClusterState.Custom to MetaData.Custom and adds ability for the task to remain in the metadata after completion.
Original commit: elastic/x-pack-elasticsearch@751b40de2a
The messages stored in the .ml-notifications index include a
message field that is mapped to text. It makes sense to keep
that mapping as it can improve search experience in messages.
However, the ML kibana app needs to aggregate over messages,
thus we also need to provide a keyword mapping.
Original commit: elastic/x-pack-elasticsearch@5a5df53d85
* Update input field mappings if job results index exists
* Add field names to the index mapping in shared indices
* Throw error if creating job will violate the index.mapping.total_fields.limit
* Review comments
Original commit: elastic/x-pack-elasticsearch@d7712d9263
This commit ensures that the authentication sets the correct version on the stream when it is serialized over the wire
so that there is not a version mismatch between the authentication and the connection it came from.
Original commit: elastic/x-pack-elasticsearch@267d7068f4
Any failure to communicate with an LDAP server should be treated as if authentication failed rather than a fatal "stop the world" error.
Because the AD realm validates the user's password by attempting to bind to the directory with that username/password combination, connection failures are to be expected and must be handled gracefully because the AD realm may not be the last realm in the order-chain, and any exception would prevent later realms from being tested.
Closes: elastic/x-pack-elasticsearch#564
Original commit: elastic/x-pack-elasticsearch@5282290b00
This change adds checks to the following machine learning actions to check that machine learning is permitted by the license before executing the request and throw and exception if the license forbids it:
* Create job
* Create data feed
* Open job
* Start data feed
This is not a final list of the restrictions we want to have in place if the license forbids ML access but it serves as a starting point and can be easily updated following further discussions.
The change also moves the `transportClientMode` check to `createComponents` so we don’t try to start the server parts of machine learning (job manager, connection to named pipes etc.) if we are running in a transport client.
Original commit: elastic/x-pack-elasticsearch@6c19ebd3bc
Also added a test to multi node qa module that tests the datafeeder, which should have caught this.
Original commit: elastic/x-pack-elasticsearch@89e4875f6c
Now that TLS is required for node to node communication, we no longer need to sign the
authentication information to prevent tampering.
Original commit: elastic/x-pack-elasticsearch@1f86cf2395
Was accidentally injecting the script object, not the string version of the code. Also
added a check so we only inject for Painless scripts (and not groovy, etc).
Minor style tweaks too.
Original commit: elastic/x-pack-elasticsearch@58c7275bd8
Essentially an update to https://github.com/elastic/prelert-legacy/pull/736 . Still does not
default to using shared indices, but adds the capability for two jobs to share the same one
without conflict
Still does not default to using shared indices, just adds the capability for two jobs to share the same one without conflict.
Original commit: elastic/x-pack-elasticsearch@60d93a06ea
By injecting immediately before use, we don't have to store it in the config and return
huge blobs to the user when they request the config
Original commit: elastic/x-pack-elasticsearch@2e53c930ea
Source filtering is currently written as if the fact that an object is accepted
means that all sub fields and objects are implicitly accepted, which is not
true.
Original commit: elastic/x-pack-elasticsearch@afc4440e7a
This node attribute setting defaults to true when ml is enabled and jobs are only opened on nodes where the allocation enabled node attribute is set to true.
Original commit: elastic/x-pack-elasticsearch@fd3f1b3058
This change adds licensing to the maching learning feature, and only allows access to machine learning if a trial or platinum license is installed.
Further, this change also renames `MlPlugin` to `MachineLearning` in line with the other feature plugin names and move the enabled setting to `XPackSettings`
Original commit: elastic/x-pack-elasticsearch@48ea9d781b
This commit removes the undocumented, test used only, setting xpack.monitoring.collection.collectors. It should not be used by users at all, and has been initially created for collector tests.
From elastic/elasticsearch#4957
Original commit: elastic/x-pack-elasticsearch@0fa39a94b5
The Elastic Secrets vault is served via HTTPS with a Let's Encrypt
certificate. The root certificate that cross-signed the Let's Encrypt
certificates were not trusted by the JDK until 8u101. This commit adds a
version check at the start of the build to make it clear the cause of
the underlying issue, and what the fix is.
Relates elastic/x-pack-elasticsearch#541
Original commit: elastic/x-pack-elasticsearch@6bf8076cb6
Martijn van Groningen
Dimitris Athanasiou
Alexander Reelsen
David Roberts
Jay Modi
David Kyle
Zachary Tong
Igor Motov
Zach
jaymode
Tim Vernum
Colin Goodheart-Smithe
Zachary Tong
Adrien Grand
Jason Tedor
Tanguy Leroux
Ryan Ernst