mirror of
https://github.com/honeymoose/OpenSearch.git
synced 2025-03-09 14:34:43 +00:00
15e53280dc
The goal of this change is to allow datafeeds to start when the job is in the opening state. This makes the API more async and it allows clients like the ML UI to open a job and start its datafeed without having to manage the complexity of dealing with timeouts due to the job taking time to open due to restoring a large state. In order to achieve this, this commit does a number of things: - accepts a start datafeed request when the job is opening - adds logic to the DatafeedManager to wait before running the datafeed task until the job is opened - refactord the datafeed node selection logic into its own class - splitd selection issues in critical and non-critical with regard to creating the datafeed task - refactord the unit tests to make simpler to write & understand - adds unit tests for added and modified functionality - changes the response when the datafeed cannot be started to be a conflict exception relates elastic/x-pack-elasticsearch#1535 Original commit: elastic/x-pack-elasticsearch@c83196155d
= Elasticsearch X-Pack A set of Elastic's commercial plugins for Elasticsearch: - License - Security - Watcher - Monitoring - Machine Learning - Graph = Setup You must checkout `x-pack-elasticsearch` and `elasticsearch` with a specific directory structure. The `elasticsearch` checkout will be used when building `x-pack-elasticsearch`. The structure is: - /path/to/elastic/elasticsearch - /path/to/elastic/elasticsearch-extra/x-pack-elasticsearch == Vault Secret The build requires a Vault Secret ID. You can use a GitHub token by following these steps: 1. Go to https://github.com/settings/tokens 2. Click *Generate new token* 3. Set permissions to `read:org` 4. Copy the token into `~/.elastic/github.token` 5. Set the token's file permissions to `600` ``` $ mkdir ~/.elastic $ vi ~/.elastic/github.token # Add your_token exactly as it is into the file and save it $ chmod 600 ~/.elastic/github.token ``` If you do not create the token, then you will see something along the lines of this as the failure when trying to build X-Pack: ``` * What went wrong: Missing ~/.elastic/github.token file or VAULT_SECRET_ID environment variable, needed to authenticate with vault for secrets ``` === Offline Mode When running the build in offline mode (`--offline`), it will not required to have the vault secret setup. == Native Code **This is mandatory as tests depend on it** Machine Learning requires platform specific binaries, build from https://github.com/elastic/machine-learning-cpp via CI servers. The native artifacts are stored in S3. To retrieve them infra's team Vault service is utilized, which requires a github token. Please setup a github token as documented: https://github.com/elastic/infra/blob/master/docs/vault.md#github-auth The github token has to be put into ~/.elastic/github.token, while the file rights must be set to 0600. = Build - Run unit tests: + [source, txt] ----- gradle clean test ----- - Run all tests: + [source, txt] ----- gradle clean check ----- - Run integration tests: + [source, txt] ----- gradle clean integTest ----- - Package X-Pack (without running tests) + [source, txt] ----- gradle clean assemble ----- - Install X-Pack (without running tests) + [source, txt] ----- gradle clean install -----