No more relative links
This commit is contained in:
aetter
parent
da466be751
commit
9ea68d488a
|
|
@ -56,4 +56,4 @@ If you discover a potential security issue in this project we ask that you notif
|
|||
|
||||
## Licensing
|
||||
|
||||
See the [LICENSE](LICENSE) file for our project's licensing. We will ask you to confirm the licensing of your contribution.
|
||||
See the [LICENSE]({{site.url}}{{site.baseurl}}/LICENSE) file for our project's licensing. We will ask you to confirm the licensing of your contribution.
|
||||
|
|
|
|||
|
|
@ -263,7 +263,7 @@ This project has adopted an [Open Source Code of Conduct](https://opensearch.org
|
|||
|
||||
## Security
|
||||
|
||||
See [CONTRIBUTING](CONTRIBUTING.md#security-issue-notifications) for more information.
|
||||
See [CONTRIBUTING]({{site.url}}{{site.baseurl}}/CONTRIBUTING.md#security-issue-notifications) for more information.
|
||||
|
||||
|
||||
## License
|
||||
|
|
|
|||
|
|
@ -3,13 +3,16 @@ layout: default
|
|||
title: OpenSearch CLI
|
||||
nav_order: 52
|
||||
has_children: false
|
||||
redirect_from:
|
||||
- /docs/odfe-cli/
|
||||
- /docs/cli/
|
||||
---
|
||||
|
||||
# OpenSearch CLI
|
||||
|
||||
The OpenSearch CLI command line interface (opensearch-cli) lets you manage your OpenSearch cluster from the command line and automate tasks.
|
||||
|
||||
Currently, opensearch-cli supports the [Anomaly Detection](../ad/) and [k-NN](../knn/) plugins, along with arbitrary REST API paths. Among other things, you can use opensearch-cli to create and delete detectors, start and stop them, and check k-NN statistics.
|
||||
Currently, opensearch-cli supports the [Anomaly Detection]({{site.url}}{{site.baseurl}}/monitoring-plugins/ad/) and [k-NN]({{site.url}}{{site.baseurl}}/search-plugins/knn/) plugins, along with arbitrary REST API paths. Among other things, you can use opensearch-cli to create and delete detectors, start and stop them, and check k-NN statistics.
|
||||
|
||||
Profiles let you easily access different clusters or sign requests with different credentials. opensearch-cli supports unauthenticated requests, HTTP basic signing, and IAM signing for Amazon Web Services.
|
||||
|
||||
|
|
@ -20,6 +20,6 @@ To create a Gantt chart, perform the following steps:
|
|||
1. Choose **Panel settings** to adjust axis labels, time format, and colors.
|
||||
1. Choose **Update**.
|
||||
|
||||
|
||||
|
||||
|
||||
This Gantt chart displays the ID of each log on the y-axis. Each bar is a unique event that spans some amount of time. Hover over a bar to see the duration of that event.
|
||||
|
|
|
|||
|
|
@ -4,6 +4,9 @@ title: About Dashboards
|
|||
nav_order: 1
|
||||
has_children: false
|
||||
has_toc: false
|
||||
redirect_from:
|
||||
- /docs/opensearch-dashboards/
|
||||
- /opensearch-dashboards/
|
||||
---
|
||||
|
||||
# OpenSearch Dashboards
|
||||
|
|
|
|||
|
|
@ -11,13 +11,13 @@ You *can* start OpenSearch Dashboards using `docker run` after [creating a Docke
|
|||
|
||||
1. Run `docker pull opensearchproject/opensearch-dashboards:{{site.opensearch_version}}`.
|
||||
|
||||
1. Create a [`docker-compose.yml`](https://docs.docker.com/compose/compose-file/) file appropriate for your environment. A sample file that includes OpenSearch Dashboards is available on the OpenSearch [Docker installation page](../opensearch/docker/#sample-docker-compose-file).
|
||||
1. Create a [`docker-compose.yml`](https://docs.docker.com/compose/compose-file/) file appropriate for your environment. A sample file that includes OpenSearch Dashboards is available on the OpenSearch [Docker installation page]({{site.url}}{{site.baseurl}}/opensearch/install/docker/#sample-docker-compose-file).
|
||||
|
||||
Just like `opensearch.yml`, you can pass a custom `opensearch_dashboards.yml` to the container in the Docker Compose file.
|
||||
{: .tip }
|
||||
|
||||
1. Run `docker-compose up`.
|
||||
|
||||
Wait for the containers to start. Then see the [OpenSearch Dashboards documentation](../../../opensearch-dashboards/).
|
||||
Wait for the containers to start. Then see the [OpenSearch Dashboards documentation]({{site.url}}{{site.baseurl}}/).
|
||||
|
||||
1. When finished, run `docker-compose down`.
|
||||
|
|
|
|||
|
|
@ -65,8 +65,8 @@ traceAnalyticsDashboards 1.0.0.0-beta1
|
|||
## Prerequisites
|
||||
|
||||
- A compatible OpenSearch cluster
|
||||
- The corresponding OpenSearch plugins [installed on that cluster](../../install/plugins)
|
||||
- The corresponding version of [OpenSearch Dashboards](../) (e.g. OpenSearch Dashboards 1.0.0 works with OpenSearch 1.0.0)
|
||||
- The corresponding OpenSearch plugins [installed on that cluster]({{site.url}}{{site.baseurl}}/opensearch/install/plugins/)
|
||||
- The corresponding version of [OpenSearch Dashboards]({{site.url}}{{site.baseurl}}/) (e.g. OpenSearch Dashboards 1.0.0 works with OpenSearch 1.0.0)
|
||||
|
||||
|
||||
## Install
|
||||
|
|
|
|||
|
|
@ -27,4 +27,4 @@ nav_order: 30
|
|||
./bin/opensearch-dashboards
|
||||
```
|
||||
|
||||
1. See the [OpenSearch Dashboards documentation](../../opensearch-dashboards/).
|
||||
1. See the [OpenSearch Dashboards documentation]({{site.url}}{{site.baseurl}}/opensearch-dashboards/).
|
||||
|
|
|
|||
|
|
@ -8,7 +8,7 @@ has_children: false
|
|||
|
||||
# OpenSearch Dashboards notebooks (experimental)
|
||||
|
||||
Notebooks have a known issue with [tenants](../../security/access-control/multi-tenancy/). If you open a notebook and can't see its visualizations, you might be under the wrong tenant, or you might not have access to the tenant at all.
|
||||
Notebooks have a known issue with [tenants]({{site.url}}{{site.baseurl}}/security-plugin/access-control/multi-tenancy/). If you open a notebook and can't see its visualizations, you might be under the wrong tenant, or you might not have access to the tenant at all.
|
||||
{: .warning }
|
||||
|
||||
An OpenSearch Dashboards notebook is an interface that lets you easily combine live visualizations and narrative text in a single notebook interface.
|
||||
|
|
@ -45,7 +45,7 @@ Paragraphs combine text and visualizations for describing data.
|
|||
1. To add text, choose **Add markdown paragraph**.
|
||||
1. Add rich text with markdown syntax.
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
#### Add a visualization paragraph
|
||||
|
|
|
|||
|
|
@ -7,7 +7,7 @@ nav_order: 20
|
|||
|
||||
# Reporting
|
||||
|
||||
You can use OpenSearch Dashboards to create PNG, PDF, and CSV reports. To create reports, you must have the correct permissions. For a summary of the predefined roles and the permissions they grant, see the [security plugin](../../security/access-control/users-roles/#predefined-roles).
|
||||
You can use OpenSearch Dashboards to create PNG, PDF, and CSV reports. To create reports, you must have the correct permissions. For a summary of the predefined roles and the permissions they grant, see the [security plugin]({{site.url}}{{site.baseurl}}/security-plugin/access-control/users-roles/#predefined-roles).
|
||||
|
||||
|
||||
## Create reports from Discovery, Visualize, or Dashboard
|
||||
|
|
@ -35,7 +35,7 @@ Definitions let you generate reports on a periodic schedule.
|
|||
1. (Optional) Add a header or footer to the report. Headers and footers are only available for dashboard or visualization reports.
|
||||
1. Under **Report trigger**, choose either **On-demand** or **Schedule**.
|
||||
|
||||
For scheduled reports, select either **Recurring** or **Cron based**. You can receive reports daily or at some other time interval. Cron expressions give you even more flexiblity. See [Cron expression reference](../../alerting/cron/) for more information.
|
||||
For scheduled reports, select either **Recurring** or **Cron based**. You can receive reports daily or at some other time interval. Cron expressions give you even more flexiblity. See [Cron expression reference]({{site.url}}{{site.baseurl}}/monitoring-plugins/alerting/cron/) for more information.
|
||||
|
||||
1. Choose **Create**.
|
||||
|
||||
|
|
@ -45,7 +45,7 @@ Definitions let you generate reports on a periodic schedule.
|
|||
|
||||
While creating a report for dashboards or visualizations, you might see a the following error:
|
||||
|
||||
|
||||
|
||||
|
||||
This problem can occur for two reasons:
|
||||
|
||||
|
|
|
|||
|
|
@ -51,7 +51,7 @@ The order in which you select attributes is critical. A city followed by a demog
|
|||
Specify a schedule to roll up your indices as it’s being ingested. The index rollup job is enabled by default.
|
||||
|
||||
1. Specify if the data is continuous or not.
|
||||
3. For roll up execution frequency, select **Define by fixed interval** and specify the **Rollup interval** and the time unit or **Define by cron expression** and add in a cron expression to select the interval. To learn how to define a cron expression, see [Alerting](../alerting/cron/).
|
||||
3. For roll up execution frequency, select **Define by fixed interval** and specify the **Rollup interval** and the time unit or **Define by cron expression** and add in a cron expression to select the interval. To learn how to define a cron expression, see [Alerting]({{site.url}}{{site.baseurl}}/monitoring-plugins/alerting/cron/).
|
||||
4. Specify the number of pages per execution process. A larger number means faster execution and more cost for memory.
|
||||
5. (Optional) Add a delay to the roll up executions. This is the amount of time the job waits for data ingestion to accommodate any processing time. For example, if you set this value to 10 minutes, an index rollup that executes at 2 PM to roll up 1 PM to 2 PM of data starts at 2:10 PM.
|
||||
6. Choose **Next**.
|
||||
|
|
|
|||
|
|
@ -29,7 +29,7 @@ If you don't have any data in your cluster, you can use the sample flight data w
|
|||
### Step 1: Choose indices
|
||||
|
||||
1. In the **Job name and description** section, specify a name and an optional description for your job.
|
||||
2. In the **Indices** section, select the source and target index. You can either select an existing target index or create a new one by entering a name for your new index. If you want to transform just a subset of your source index, choose **Add Data Filter**, and use the OpenSearch query DSL to specify a subset of your source index. For more information about the OpenSearch query DSL, see [query DSL](../../opensearch/query-dsl/).
|
||||
2. In the **Indices** section, select the source and target index. You can either select an existing target index or create a new one by entering a name for your new index. If you want to transform just a subset of your source index, choose **Add Data Filter**, and use the OpenSearch query DSL to specify a subset of your source index. For more information about the OpenSearch query DSL, see [query DSL]({{site.url}}{{site.baseurl}}/opensearch/query-dsl/).
|
||||
3. Choose **Next**.
|
||||
|
||||
### Step 2: Select fields to transform
|
||||
|
|
@ -42,7 +42,7 @@ On the other hand, aggregations let you perform simple calculations. For example
|
|||
|
||||
1. In the data table, select the fields you want to transform and expand the drop-down menu within the column header to choose the grouping or aggregation you want to use.
|
||||
|
||||
Currently, transform jobs support histogram, date_histogram, and terms groupings. For more information about groupings, see [Bucket Aggregations](../../opensearch/bucket-agg/). In terms of aggregations, you can select from `sum`, `avg`, `max`, `min`, `value_count`, `percentiles`, and `scripted_metric`. For more information about aggregations, see [Metric Aggregations](../../opensearch/metric-agg/).
|
||||
Currently, transform jobs support histogram, date_histogram, and terms groupings. For more information about groupings, see [Bucket Aggregations]({{site.url}}{{site.baseurl}}/opensearch/bucket-agg/). In terms of aggregations, you can select from `sum`, `avg`, `max`, `min`, `value_count`, `percentiles`, and `scripted_metric`. For more information about aggregations, see [Metric Aggregations]({{site.url}}{{site.baseurl}}/opensearch/metric-agg/).
|
||||
|
||||
2. Repeat step 1 for any other fields that you want to transform.
|
||||
3. After selecting the fields that you want to transform and verifying the transformation, choose **Next**.
|
||||
|
|
|
|||
|
|
@ -133,11 +133,11 @@ description | String | Describes the transform job. | No
|
|||
metadata_id | String | Any metadata to be associated with the transform job. | No
|
||||
source_index | String | The source index whose data to transform. | Yes
|
||||
target_index | String | The target index the newly transformed data is added into. You can create a new index or update an existing one. | Yes
|
||||
data_selection_query | JSON | The query DSL to use to filter a subset of the source index for the transform job. See [query DSL](../../../opensearch/query-dsl) for more information. | Yes
|
||||
data_selection_query | JSON | The query DSL to use to filter a subset of the source index for the transform job. See [query DSL]({{site.url}}{{site.baseurl}}/opensearch/query-dsl) for more information. | Yes
|
||||
page_size | Integer | The number of fields to transform at a time. Higher number means higher performance but requires more memory and can cause higher latency. (Default: 1) | Yes
|
||||
groups | Array | Specifies the grouping(s) to use in the transform job. Supported groups are `terms`, `histogram`, and `date_histogram`. For more information, see [Bucket Aggregations](../../../opensearch/bucket-agg). | Yes if not using aggregations
|
||||
groups | Array | Specifies the grouping(s) to use in the transform job. Supported groups are `terms`, `histogram`, and `date_histogram`. For more information, see [Bucket Aggregations]({{site.url}}{{site.baseurl}}/opensearch/bucket-agg). | Yes if not using aggregations
|
||||
source_field | String | The field(s) to transform | Yes
|
||||
aggregations | JSON | The aggregations to use in the transform job. Supported aggregations are: `sum`, `max`, `min`, `value_count`, `avg`, `scripted_metric`, and `percentiles`. For more information, see [Metric Aggregations](../../../opensearch/metric-agg). | Yes if not using groups
|
||||
aggregations | JSON | The aggregations to use in the transform job. Supported aggregations are: `sum`, `max`, `min`, `value_count`, `avg`, `scripted_metric`, and `percentiles`. For more information, see [Metric Aggregations]({{site.url}}{{site.baseurl}}/opensearch/metric-agg). | Yes if not using groups
|
||||
|
||||
## Update a transform job
|
||||
|
||||
|
|
|
|||
|
|
@ -19,7 +19,7 @@ For example, you can define a policy that moves your index into a `read_only` st
|
|||
|
||||
You might want to perform an index rollover after a certain amount of time or run a `force_merge` operation on an index during off-peak hours to improve search performance during peak hours.
|
||||
|
||||
To use the ISM plugin, your user role needs to be mapped to the `all_access` role that gives you full access to the cluster. To learn more, see [Users and roles](../security/access-control/users-roles/).
|
||||
To use the ISM plugin, your user role needs to be mapped to the `all_access` role that gives you full access to the cluster. To learn more, see [Users and roles]({{site.url}}{{site.baseurl}}/security-plugin/access-control/users-roles/).
|
||||
{: .note }
|
||||
|
||||
## Get started with ISM
|
||||
|
|
@ -28,7 +28,7 @@ To get started, choose **Index Management** in OpenSearch Dashboards.
|
|||
|
||||
### Step 1: Set up policies
|
||||
|
||||
A policy is a set of rules that describes how an index should be managed. For information about creating a policy, see [Policies](policies/).
|
||||
A policy is a set of rules that describes how an index should be managed. For information about creating a policy, see [Policies]({{site.url}}{{site.baseurl}}/im-plugin/ism/policies/).
|
||||
|
||||
1. Choose the **Index Policies** tab.
|
||||
2. Choose **Create policy**.
|
||||
|
|
@ -54,7 +54,7 @@ PUT _plugins/_ism/policies/policy_id
|
|||
}
|
||||
```
|
||||
|
||||
For an example ISM template policy, see [Sample policy with ISM template](policies/#sample-policy-with-ism-template).
|
||||
For an example ISM template policy, see [Sample policy with ISM template]({{site.url}}{{site.baseurl}}/im-plugin/ism/policies#sample-policy-with-ism-template).
|
||||
|
||||
Older versions of the plugin include the `policy_id` in an index template, so when an index is created that matches the index template pattern, the index will have the policy attached to it:
|
||||
|
||||
|
|
@ -83,20 +83,20 @@ The `opendistro.index_state_management.policy_id` setting is deprecated. You can
|
|||
4. From the **Policy ID** menu, choose the policy that you created.
|
||||
You can see a preview of your policy.
|
||||
5. If your policy includes a rollover operation, specify a rollover alias.
|
||||
Make sure that the alias that you enter already exists. For more information about the rollover operation, see [rollover](policies/#rollover).
|
||||
Make sure that the alias that you enter already exists. For more information about the rollover operation, see [rollover]({{site.url}}{{site.baseurl}}/im-plugin/ism/policies#rollover).
|
||||
6. Choose **Apply**.
|
||||
|
||||
After you attach a policy to an index, ISM creates a job that runs every 5 minutes by default to perform policy actions, check conditions, and transition the index into different states. To change the default time interval for this job, see [Settings](settings/).
|
||||
After you attach a policy to an index, ISM creates a job that runs every 5 minutes by default to perform policy actions, check conditions, and transition the index into different states. To change the default time interval for this job, see [Settings]({{site.url}}{{site.baseurl}}/im-plugin/ism/settings/).
|
||||
|
||||
If you want to use an OpenSearch operation to create an index with a policy already attached to it, see [create index](api/#create-index).
|
||||
If you want to use an OpenSearch operation to create an index with a policy already attached to it, see [create index]({{site.url}}{{site.baseurl}}/im-plugin/ism/api#create-index).
|
||||
|
||||
### Step 3: Manage indices
|
||||
|
||||
1. Choose **Managed Indices**.
|
||||
2. To change your policy, see [Change Policy](managedindices/#change-policy).
|
||||
2. To change your policy, see [Change Policy]({{site.url}}{{site.baseurl}}/im-plugin/ism/managedindices#change-policy).
|
||||
3. To attach a rollover alias to your index, select your policy and choose **Add rollover alias**.
|
||||
Make sure that the alias that you enter already exists. For more information about the rollover operation, see [rollover](policies/#rollover).
|
||||
Make sure that the alias that you enter already exists. For more information about the rollover operation, see [rollover]({{site.url}}{{site.baseurl}}/im-plugin/ism/policies#rollover).
|
||||
4. To remove a policy, choose your policy, and then choose **Remove policy**.
|
||||
5. To retry a policy, choose your policy, and then choose **Retry policy**.
|
||||
|
||||
For information about managing your policies, see [Managed Indices](managedindices/).
|
||||
For information about managing your policies, see [Managed Indices]({{site.url}}{{site.baseurl}}/im-plugin/ism/managedindices/).
|
||||
|
|
|
|||
|
|
@ -88,7 +88,7 @@ The following example action has a timeout period of one hour. The policy retrie
|
|||
}
|
||||
```
|
||||
|
||||
For a list of available unit types, see [Supported units](../../../opensearch/units/).
|
||||
For a list of available unit types, see [Supported units]({{site.url}}{{site.baseurl}}/opensearch/units/).
|
||||
|
||||
## ISM supported operations
|
||||
|
||||
|
|
@ -159,7 +159,7 @@ Parameter | Description | Type | Required
|
|||
}
|
||||
```
|
||||
|
||||
For information about setting replicas, see [Primary and replica shards](../../../opensearch/#primary-and-replica-shards).
|
||||
For information about setting replicas, see [Primary and replica shards]({{site.url}}{{site.baseurl}}/opensearch/#primary-and-replica-shards).
|
||||
|
||||
### close
|
||||
|
||||
|
|
@ -308,7 +308,7 @@ Parameter | Description | Type
|
|||
|
||||
### snapshot
|
||||
|
||||
Backup your cluster’s indices and state. For more information about snapshots, see [Take and restore snapshots](../../../opensearch/snapshot-restore/).
|
||||
Backup your cluster’s indices and state. For more information about snapshots, see [Take and restore snapshots]({{site.url}}{{site.baseurl}}/opensearch/snapshot-restore/).
|
||||
|
||||
The `snapshot` operation has the following parameters:
|
||||
|
||||
|
|
@ -435,7 +435,7 @@ Note that this condition does not execute at exactly 5:00 PM; the job still exec
|
|||
|
||||
A window of an hour, which this example uses, is generally sufficient, but you might increase it to 2--3 hours to avoid missing the window and having to wait a week for the transition to occur. Alternately, you could use a broader expression such as `* * * * SAT,SUN` to have the transition occur at any time during the weekend.
|
||||
|
||||
For information on writing cron expressions, see [Cron expression reference](../../../alerting/cron/).
|
||||
For information on writing cron expressions, see [Cron expression reference]({{site.url}}{{site.baseurl}}/monitoring-plugins/alerting/cron/).
|
||||
|
||||
---
|
||||
|
||||
|
|
@ -662,4 +662,4 @@ After 30 days, the policy moves this index into a `delete` state. The service se
|
|||
|
||||
This diagram shows the `states`, `transitions`, and `actions` of the above policy as a finite-state machine. For more information about finite-state machines, see [Wikipedia](https://en.wikipedia.org/wiki/Finite-state_machine).
|
||||
|
||||
|
||||
|
||||
|
|
|
|||
|
|
@ -10,7 +10,7 @@ nav_order: 4
|
|||
|
||||
We don't recommend changing these settings; the defaults should work well for most use cases.
|
||||
|
||||
Index State Management (ISM) stores its configuration in the `.opendistro-ism-config` index. Don't modify this index without using the [ISM API operations](../api/).
|
||||
Index State Management (ISM) stores its configuration in the `.opendistro-ism-config` index. Don't modify this index without using the [ISM API operations]({{site.url}}{{site.baseurl}}/im-plugin/ism/api/).
|
||||
|
||||
All settings are available using the OpenSearch `_cluster/settings` operation. None require a restart, and all can be marked `persistent` or `transient`.
|
||||
|
||||
|
|
@ -31,7 +31,7 @@ Setting | Default | Description
|
|||
|
||||
## Audit history indices
|
||||
|
||||
If you don't want to disable ISM audit history or shorten the retention period, you can create an [index template](../../../opensearch/index-templates/) to reduce the shard count of the history indices:
|
||||
If you don't want to disable ISM audit history or shorten the retention period, you can create an [index template]({{site.url}}{{site.baseurl}}/opensearch/index-templates/) to reduce the shard count of the history indices:
|
||||
|
||||
```json
|
||||
PUT _index_template/ism_history_indices
|
||||
|
|
|
|||
|
|
@ -13,7 +13,7 @@ It can be challenging to discover anomalies using conventional methods such as c
|
|||
|
||||
Anomaly detection automatically detects anomalies in your OpenSearch data in near real-time using the Random Cut Forest (RCF) algorithm. RCF is an unsupervised machine learning algorithm that models a sketch of your incoming data stream to compute an `anomaly grade` and `confidence score` value for each incoming data point. These values are used to differentiate an anomaly from normal variations. For more information about how RCF works, see [Random Cut Forests](https://pdfs.semanticscholar.org/8bba/52e9797f2e2cc9a823dbd12514d02f29c8b9.pdf?_ga=2.56302955.1913766445.1574109076-1059151610.1574109076).
|
||||
|
||||
You can pair the anomaly detection plugin with the [alerting plugin](../alerting/) to notify you as soon as an anomaly is detected.
|
||||
You can pair the anomaly detection plugin with the [alerting plugin]({{site.url}}{{site.baseurl}}/monitoring-plugins/alerting/) to notify you as soon as an anomaly is detected.
|
||||
|
||||
To use the anomaly detection plugin, your computer needs to have more than one CPU core.
|
||||
{: .note }
|
||||
|
|
@ -100,11 +100,11 @@ Examine the sample preview and use it to fine-tune your feature configurations (
|
|||
Choose the **Anomaly results** tab. You need to wait for some time to see the anomaly results. If the detector interval is 10 minutes, the detector might take more than an hour to start, as it's waiting for sufficient data to generate anomalies.
|
||||
|
||||
A shorter interval means the model passes the shingle process more quickly and starts to generate the anomaly results sooner.
|
||||
Use the [profile detector](./api#profile-detector) operation to make sure you have sufficient data points.
|
||||
Use the [profile detector]({{site.url}}{{site.baseurl}}/monitoring-plugins/ad/api#profile-detector) operation to make sure you have sufficient data points.
|
||||
|
||||
If you see the detector pending in "initialization" for longer than a day, aggregate your existing data using the detector interval to check for any missing data points. If you find a lot of missing data points from the aggregated data, consider increasing the detector interval.
|
||||
|
||||
|
||||
|
||||
|
||||
Analize anomalies with the following visualizations:
|
||||
|
||||
|
|
@ -113,7 +113,7 @@ Analize anomalies with the following visualizations:
|
|||
- **Feature breakdown** - plots the features based on the aggregation method. You can vary the date-time range of the detector.
|
||||
- **Anomaly occurrence** - shows the `Start time`, `End time`, `Data confidence`, and `Anomaly grade` for each detected anomaly.
|
||||
|
||||
`Anomaly grade` is a number between 0 and 1 that indicates how anomalous a data point is. An anomaly grade of 0 represents “not an anomaly,” and a non-zero value represents the relative severity of the anomaly.
|
||||
`Anomaly grade` is a number between 0 and 1 that indicates how anomalous a data point is. An anomaly grade of 0 represents “not an anomaly,” and a non-zero value represents the relative severity of the anomaly.
|
||||
|
||||
`Data confidence` is an estimate of the probability that the reported anomaly grade matches the expected anomaly grade. Confidence increases as the model observes more data and learns the data behavior and trends. Note that confidence is distinct from model accuracy.
|
||||
|
||||
|
|
@ -124,7 +124,7 @@ Choose a filled rectangle to see a more detailed view of the anomaly.
|
|||
|
||||
### Step 4: Set up alerts
|
||||
|
||||
Choose **Set up alerts** and configure a monitor to notify you when anomalies are detected. For steps to create a monitor and set up notifications based on your anomaly detector, see [Monitors](../alerting/monitors/).
|
||||
Choose **Set up alerts** and configure a monitor to notify you when anomalies are detected. For steps to create a monitor and set up notifications based on your anomaly detector, see [Monitors]({{site.url}}{{site.baseurl}}/monitoring-plugins/alerting/monitors/).
|
||||
|
||||
If you stop or delete a detector, make sure to delete any monitors associated with it.
|
||||
|
||||
|
|
|
|||
|
|
@ -10,24 +10,24 @@ has_children: false
|
|||
|
||||
You can use the security plugin with anomaly detection in OpenSearch to limit non-admin users to specific actions. For example, you might want some users to only be able to create, update, or delete detectors, while others to only view detectors.
|
||||
|
||||
All anomaly detection indices are protected as system indices. Only a super admin user or an admin user with a TLS certificate can access system indices. For more information, see [System indices](../../security/configuration/system-indices/).
|
||||
All anomaly detection indices are protected as system indices. Only a super admin user or an admin user with a TLS certificate can access system indices. For more information, see [System indices]({{site.url}}{{site.baseurl}}/security-plugin/configuration/system-indices/).
|
||||
|
||||
|
||||
Security for anomaly detection works the same as [security for alerting](../../alerting/security/).
|
||||
Security for anomaly detection works the same as [security for alerting]({{site.url}}{{site.baseurl}}/monitoring-plugins/alerting/security/).
|
||||
|
||||
## Basic permissions
|
||||
|
||||
As an admin user, you can use the security plugin to assign specific permissions to users based on which APIs they need access to. For a list of supported APIs, see [Anomaly detection API](../api/).
|
||||
As an admin user, you can use the security plugin to assign specific permissions to users based on which APIs they need access to. For a list of supported APIs, see [Anomaly detection API]({{site.url}}{{site.baseurl}}/monitoring-plugins/ad/api/).
|
||||
|
||||
The security plugin has two built-in roles that cover most anomaly detection use cases: `anomaly_full_access` and `anomaly_read_access`. For descriptions of each, see [Predefined roles](../../security/access-control/users-roles/#predefined-roles).
|
||||
The security plugin has two built-in roles that cover most anomaly detection use cases: `anomaly_full_access` and `anomaly_read_access`. For descriptions of each, see [Predefined roles]({{site.url}}{{site.baseurl}}/security-plugin/access-control/users-roles/#predefined-roles).
|
||||
|
||||
If these roles don't meet your needs, mix and match individual anomaly detection [permissions](../../security/access-control/permissions/) to suit your use case. Each action corresponds to an operation in the REST API. For example, the `cluster:admin/opensearch/ad/detector/delete` permission lets you delete detectors.
|
||||
If these roles don't meet your needs, mix and match individual anomaly detection [permissions]({{site.url}}{{site.baseurl}}/security-plugin/access-control/permissions/) to suit your use case. Each action corresponds to an operation in the REST API. For example, the `cluster:admin/opensearch/ad/detector/delete` permission lets you delete detectors.
|
||||
|
||||
## (Advanced) Limit access by backend role
|
||||
|
||||
Use backend roles to configure fine-grained access to individual detectors based on roles. For example, users of different departments in an organization can view detectors owned by their own department.
|
||||
|
||||
First, make sure your users have the appropriate [backend roles](../../security/access-control/). Backend roles usually come from an [LDAP server](../../security/configuration/ldap/) or [SAML provider](../../security/configuration/saml/), but if you use the internal user database, you can use the REST API to [add them manually](../../security/access-control/api/#create-user).
|
||||
First, make sure your users have the appropriate [backend roles]({{site.url}}{{site.baseurl}}/security-plugin/access-control/). Backend roles usually come from an [LDAP server]({{site.url}}{{site.baseurl}}/security-plugin/configuration/ldap/) or [SAML provider]({{site.url}}{{site.baseurl}}/security-plugin/configuration/saml/), but if you use the internal user database, you can use the REST API to [add them manually]({{site.url}}{{site.baseurl}}/security-plugin/access-control/api/#create-user).
|
||||
|
||||
Next, enable the following setting:
|
||||
|
||||
|
|
|
|||
|
|
@ -174,7 +174,7 @@ If you use a custom webhook for your destination and need to embed JSON in the m
|
|||
}
|
||||
```
|
||||
|
||||
If you want to specify a timezone, you can do so by including a [cron expression](../cron/) with a timezone name in the `schedule` section of your request.
|
||||
If you want to specify a timezone, you can do so by including a [cron expression]({{site.url}}{{site.baseurl}}/monitoring-plugins/alerting/cron/) with a timezone name in the `schedule` section of your request.
|
||||
|
||||
The following example creates a monitor that runs at 12:10 PM Pacific Time on the 1st day of every month.
|
||||
|
||||
|
|
|
|||
|
|
@ -4,6 +4,9 @@ title: Cron
|
|||
nav_order: 20
|
||||
parent: Alerting
|
||||
has_children: false
|
||||
redirect_from:
|
||||
- /alerting/cron/
|
||||
- /docs/alerting/cron/
|
||||
---
|
||||
|
||||
# Cron expression reference
|
||||
|
|
@ -61,4 +64,4 @@ Every three hours on the first day of every other month:
|
|||
|
||||
## API
|
||||
|
||||
For an example of how to use a custom cron expression in an API call, see the [create monitor API operation](../api/#request-1).
|
||||
For an example of how to use a custom cron expression in an API call, see the [create monitor API operation]({{site.url}}{{site.baseurl}}/monitoring-plugins/alerting/api/#request-1).
|
||||
|
|
|
|||
|
|
@ -13,4 +13,4 @@ The alerting feature notifies you when data from one or more OpenSearch indices
|
|||
|
||||
To get started, choose **Alerting** in OpenSearch Dashboards.
|
||||
|
||||
|
||||
|
||||
|
|
|
|||
|
|
@ -102,7 +102,7 @@ POST _nodes/reload_secure_settings
|
|||
1. Choose **Alerting**, **Monitors**, **Create monitor**.
|
||||
1. Specify a name for the monitor.
|
||||
|
||||
The anomaly detection option is for pairing with the anomaly detection plugin. See [Anomaly Detection](../../ad/).
|
||||
The anomaly detection option is for pairing with the anomaly detection plugin. See [Anomaly Detection]({{site.url}}{{site.baseurl}}/monitoring-plugins/ad/).
|
||||
For anomaly detector, choose an appropriate schedule for the monitor based on the detector interval. Otherwise, the alerting monitor might miss reading the results.
|
||||
|
||||
For example, assume you set the monitor interval and the detector interval as 5 minutes, and you start the detector at 12:00. If an anomaly is detected at 12:05, it might be available at 12:06 because of the delay between writing the anomaly and it being available for queries. The monitor reads the anomaly results between 12:00 and 12:05, so it does not get the anomaly results available at 12:06.
|
||||
|
|
@ -114,13 +114,13 @@ Whenever you update a detector’s interval, make sure to update the associated
|
|||
|
||||
1. Choose one or more indices. You can also use `*` as a wildcard to specify an index pattern.
|
||||
|
||||
If you use the security plugin, you can only choose indices that you have permission to access. For details, see [Alerting security](../security/).
|
||||
If you use the security plugin, you can only choose indices that you have permission to access. For details, see [Alerting security]({{site.url}}{{site.baseurl}}/security-plugin/).
|
||||
|
||||
1. Define the monitor in one of three ways: visually, using a query, or using an anomaly detector.
|
||||
|
||||
- Visual definition works well for monitors that you can define as "some value is above or below some threshold for some amount of time."
|
||||
|
||||
- Query definition gives you flexibility in terms of what you query for (using [the OpenSearch query DSL](../../opensearch/full-text)) and how you evaluate the results of that query (Painless scripting).
|
||||
- Query definition gives you flexibility in terms of what you query for (using [the OpenSearch query DSL]({{site.url}}{{site.baseurl}}/opensearch/query-dsl/full-text)) and how you evaluate the results of that query (Painless scripting).
|
||||
|
||||
This example averages the `cpu_usage` field:
|
||||
|
||||
|
|
@ -172,12 +172,12 @@ Whenever you update a detector’s interval, make sure to update the associated
|
|||
|
||||
1. To define a monitor visually, choose **Define using visual graph**. Then choose an aggregation (for example, `count()` or `average()`), a set of documents, and a timeframe. Visual definition works well for most monitors.
|
||||
|
||||
To use a query, choose **Define using extraction query**, add your query (using [the OpenSearch query DSL](../../opensearch/full-text/)), and test it using the **Run** button.
|
||||
To use a query, choose **Define using extraction query**, add your query (using [the OpenSearch query DSL]({{site.url}}{{site.baseurl}}/opensearch/query-dsl/full-text/)), and test it using the **Run** button.
|
||||
|
||||
The monitor makes this query to OpenSearch as often as the schedule dictates; check the **Query Performance** section and make sure you're comfortable with the performance implications.
|
||||
|
||||
To use an anomaly detector, choose **Define using Anomaly detector** and select your **Detector**.
|
||||
1. Choose a frequency and timezone for your monitor. Note that you can only pick a timezone if you choose Daily, Weekly, Monthly, or [custom cron expression](../cron/) for frequency.
|
||||
1. Choose a frequency and timezone for your monitor. Note that you can only pick a timezone if you choose Daily, Weekly, Monthly, or [custom cron expression]({{site.url}}{{site.baseurl}}/monitoring-plugins/alerting/cron/) for frequency.
|
||||
1. Choose **Create**.
|
||||
|
||||
|
||||
|
|
@ -265,7 +265,7 @@ Below are some variables you can include in your message using Mustache template
|
|||
Variable | Data Type | Description
|
||||
:--- | :--- | :---
|
||||
`ctx.monitor` | JSON | Includes `ctx.monitor.name`, `ctx.monitor.type`, `ctx.monitor.enabled`, `ctx.monitor.enabled_time`, `ctx.monitor.schedule`, `ctx.monitor.inputs`, `triggers` and `ctx.monitor.last_update_time`.
|
||||
`ctx.monitor.user` | JSON | Includes information about the user who created the monitor. Includes `ctx.monitor.user.backend_roles` and `ctx.monitor.user.roles`, which are arrays that contain the backend roles and roles assigned to the user. See [alerting security](../security/) for more information.
|
||||
`ctx.monitor.user` | JSON | Includes information about the user who created the monitor. Includes `ctx.monitor.user.backend_roles` and `ctx.monitor.user.roles`, which are arrays that contain the backend roles and roles assigned to the user. See [alerting security]({{site.url}}{{site.baseurl}}/monitoring-plugins/alerting/security/) for more information.
|
||||
`ctx.monitor.enabled` | Boolean | Whether the monitor is enabled.
|
||||
`ctx.monitor.enabled_time` | Milliseconds | Unix epoch time of when the monitor was last enabled.
|
||||
`ctx.monitor.schedule` | JSON | Contains a schedule of how often or when the monitor should run.
|
||||
|
|
|
|||
|
|
@ -13,9 +13,9 @@ If you use the security plugin alongside alerting, you might want to limit certa
|
|||
|
||||
## Basic permissions
|
||||
|
||||
The security plugin has three built-in roles that cover most alerting use cases: `alerting_read_access`, `alerting_ack_alerts`, and `alerting_full_access`. For descriptions of each, see [Predefined roles](../../security/access-control/users-roles/#predefined-roles).
|
||||
The security plugin has three built-in roles that cover most alerting use cases: `alerting_read_access`, `alerting_ack_alerts`, and `alerting_full_access`. For descriptions of each, see [Predefined roles]({{site.url}}{{site.baseurl}}/security-plugin/access-control/users-roles/#predefined-roles).
|
||||
|
||||
If these roles don't meet your needs, mix and match individual alerting [permissions](../../security/access-control/permissions/) to suit your use case. Each action corresponds to an operation in the REST API. For example, the `cluster:admin/opensearch/alerting/destination/delete` permission lets you delete destinations.
|
||||
If these roles don't meet your needs, mix and match individual alerting [permissions]({{site.url}}{{site.baseurl}}/security-plugin/access-control/permissions/) to suit your use case. Each action corresponds to an operation in the REST API. For example, the `cluster:admin/opensearch/alerting/destination/delete` permission lets you delete destinations.
|
||||
|
||||
|
||||
## How monitors access data
|
||||
|
|
@ -29,14 +29,14 @@ Later, the user `psantos` wants to edit the monitor to run every two hours, but
|
|||
- Update the monitor so that it only checks `store1-returns`.
|
||||
- Ask an administrator for read access to the other two indices.
|
||||
|
||||
After making the change, the monitor now runs with the same permissions as `psantos`, including any [document-level security](../../security/access-control/document-level-security/) queries, [excluded fields](../../security/access-control/field-level-security/), and [masked fields](../../security/access-control/field-masking/). If you use an extraction query to define your monitor, use the **Run** button to ensure that the response includes the fields you need.
|
||||
After making the change, the monitor now runs with the same permissions as `psantos`, including any [document-level security]({{site.url}}{{site.baseurl}}/security-plugin/access-control/document-level-security/) queries, [excluded fields]({{site.url}}{{site.baseurl}}/security-plugin/access-control/field-level-security/), and [masked fields]({{site.url}}{{site.baseurl}}/security-plugin/access-control/field-masking/). If you use an extraction query to define your monitor, use the **Run** button to ensure that the response includes the fields you need.
|
||||
|
||||
|
||||
## (Advanced) Limit access by backend role
|
||||
|
||||
Out of the box, the alerting plugin has no concept of ownership. For example, if you have the `cluster:admin/opensearch/alerting/monitor/write` permission, you can edit *all* monitors, regardless of whether you created them. If a small number of trusted users manage your monitors and destinations, this lack of ownership generally isn't a problem. A larger organization might need to segment access by backend role.
|
||||
|
||||
First, make sure that your users have the appropriate [backend roles](../../security/access-control/). Backend roles usually come from an [LDAP server](../../security/configuration/ldap/) or [SAML provider](../../security/configuration/saml/). However, if you use the internal user database, you can use the REST API to [add them manually](../../security/access-control/api/#create-user).
|
||||
First, make sure that your users have the appropriate [backend roles]({{site.url}}{{site.baseurl}}/security-plugin/access-control/). Backend roles usually come from an [LDAP server]({{site.url}}{{site.baseurl}}/security-plugin/configuration/ldap/) or [SAML provider]({{site.url}}{{site.baseurl}}/security-plugin/configuration/saml/). However, if you use the internal user database, you can use the REST API to [add them manually]({{site.url}}{{site.baseurl}}/security-plugin/access-control/api/#create-user).
|
||||
|
||||
Next, enable the following setting:
|
||||
|
||||
|
|
@ -58,7 +58,7 @@ If `jdoe` creates a monitor, `jroe` can see and modify it, but `psantos` can't.
|
|||
|
||||
<!-- ## (Advanced) Limit access by individual
|
||||
|
||||
If you only want users to be able to see and modify their own monitors and destinations, duplicate the `alerting_full_access` role and add the following [DLS query](../../security/access-control/document-level-security/) to it:
|
||||
If you only want users to be able to see and modify their own monitors and destinations, duplicate the `alerting_full_access` role and add the following [DLS query]({{site.url}}{{site.baseurl}}/security-plugin/access-control/document-level-security/) to it:
|
||||
|
||||
```json
|
||||
{
|
||||
|
|
|
|||
|
|
@ -10,13 +10,13 @@ nav_order: 5
|
|||
|
||||
## Alerting indices
|
||||
|
||||
The alerting feature creates several indices and one alias. The security plugin demo script configures them as [system indices](../../security/configuration/system-indices/) for an extra layer of protection. Don't delete these indices or modify their contents without using the alerting APIs.
|
||||
The alerting feature creates several indices and one alias. The security plugin demo script configures them as [system indices]({{site.url}}{{site.baseurl}}/security-plugin/configuration/system-indices/) for an extra layer of protection. Don't delete these indices or modify their contents without using the alerting APIs.
|
||||
|
||||
Index | Purpose
|
||||
:--- | :---
|
||||
`.opendistro-alerting-alerts` | Stores ongoing alerts.
|
||||
`.opendistro-alerting-alert-history-<date>` | Stores a history of completed alerts.
|
||||
`.opendistro-alerting-config` | Stores monitors, triggers, and destinations. [Take a snapshot](../../opensearch/snapshot-restore) of this index to back up your alerting configuration.
|
||||
`.opendistro-alerting-config` | Stores monitors, triggers, and destinations. [Take a snapshot]({{site.url}}{{site.baseurl}}/opensearch/snapshot-restore) of this index to back up your alerting configuration.
|
||||
`.opendistro-alerting-alert-history-write` (alias) | Provides a consistent URI for the `.opendistro-alerting-alert-history-<date>` index.
|
||||
|
||||
All alerting indices are hidden by default. For a summary, make the following request:
|
||||
|
|
@ -51,7 +51,7 @@ Setting | Default | Description
|
|||
`plugins.alerting.alert_history_enabled` | true | Whether to create `.opendistro-alerting-alert-history-<date>` indices.
|
||||
`plugins.alerting.alert_history_retention_period` | 60d | The amount of time to keep history indices before automatically deleting them.
|
||||
`plugins.alerting.destination.allow_list` | ["chime", "slack", "custom_webhook", "email", "test_action"] | The list of allowed destinations. If you don't want to allow users to a certain type of destination, you can remove it from this list, but we recommend leaving this setting as-is.
|
||||
`plugins.alerting.filter_by_backend_roles` | "false" | Restricts access to monitors by backend role. See [Alerting security](../security/).
|
||||
`plugins.alerting.filter_by_backend_roles` | "false" | Restricts access to monitors by backend role. See [Alerting security]({{site.url}}{{site.baseurl}}/monitoring-plugins/alerting/security/).
|
||||
`plugins.scheduled_jobs.sweeper.period` | 5m | The alerting feature uses its "job sweeper" component to periodically check for new or updated jobs. This setting is the rate at which the sweeper checks to see if any jobs (monitors) have changed and need to be rescheduled.
|
||||
`plugins.scheduled_jobs.sweeper.page_size` | 100 | The page size for the sweeper. You shouldn't need to change this value.
|
||||
`plugins.scheduled_jobs.sweeper.backoff_millis` | 50ms | The amount of time the sweeper waits between retries---increases exponentially after each failed retry.
|
||||
|
|
|
|||
|
|
@ -19,7 +19,7 @@ Note the use of port 9600. Provide parameters for metrics, aggregations, dimensi
|
|||
?metrics=<metrics>&agg=<aggregations>&dim=<dimensions>&nodes=all"
|
||||
```
|
||||
|
||||
For a full list of metrics, see [Metrics reference](../reference/). Performance Analyzer updates its data every five seconds. If you create a custom client, we recommend using that same interval for calls to the API.
|
||||
For a full list of metrics, see [Metrics reference]({{site.url}}{{site.baseurl}}/monitoring-plugins/pa/reference/). Performance Analyzer updates its data every five seconds. If you create a custom client, we recommend using that same interval for calls to the API.
|
||||
|
||||
|
||||
#### Sample request
|
||||
|
|
|
|||
|
|
@ -32,7 +32,7 @@ The best way to get started with building custom dashboards is to duplicate and
|
|||
|
||||
PerfTop positions elements within a grid. For example, consider this 12 * 12 grid.
|
||||
|
||||
|
||||
|
||||
|
||||
The upper-left of the grid represents row 0, column 0, so the starting positions for the three boxes are:
|
||||
|
||||
|
|
@ -95,7 +95,7 @@ At this point, however, all the JSON does is define the size and position of thr
|
|||
|
||||
## Add queries
|
||||
|
||||
Queries use the same elements as the [REST API](../api/), just in JSON form:
|
||||
Queries use the same elements as the [REST API]({{site.url}}{{site.baseurl}}/monitoring-plugins/pa/api/), just in JSON form:
|
||||
|
||||
```json
|
||||
{
|
||||
|
|
@ -108,7 +108,7 @@ Queries use the same elements as the [REST API](../api/), just in JSON form:
|
|||
}
|
||||
```
|
||||
|
||||
For details on available metrics, see [Metrics reference](../reference/).
|
||||
For details on available metrics, see [Metrics reference]({{site.url}}{{site.baseurl}}/monitoring-plugins/pa/reference/).
|
||||
|
||||
|
||||
## Add options
|
||||
|
|
|
|||
|
|
@ -17,7 +17,7 @@ You can also install it using [npm](https://www.npmjs.com/):
|
|||
npm install -g @aws/opensearch-perftop
|
||||
```
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
## Get started with PerfTop
|
||||
|
|
@ -46,7 +46,7 @@ Otherwise, just specify the OpenSearch endpoint:
|
|||
./opensearch-perf-top-macos --dashboard dashboards/<dashboard>.json --endpoint my-cluster.my-domain.com
|
||||
```
|
||||
|
||||
PerfTop has four pre-built dashboards in the `dashboards` directory, but you can also [create your own](dashboards/).
|
||||
PerfTop has four pre-built dashboards in the `dashboards` directory, but you can also [create your own]({{site.url}}{{site.baseurl}}/dashboards/).
|
||||
|
||||
You can also load the pre-built dashboards (ClusterOverview, ClusterNetworkMemoryAnalysis, ClusterThreadAnalysis, or NodeAnalysis) without the JSON files, such as `--dashboard ClusterThreadAnalysis`.
|
||||
|
||||
|
|
|
|||
|
|
@ -7,7 +7,7 @@ nav_order: 25
|
|||
|
||||
# Data Prepper configuration reference
|
||||
|
||||
This page lists all supported Data Prepper sources, buffers, preppers, and sinks, along with their associated options. For example configuration files, see [Data Prepper](../data-prepper/).
|
||||
This page lists all supported Data Prepper sources, buffers, preppers, and sinks, along with their associated options. For example configuration files, see [Data Prepper]({{site.url}}{{site.baseurl}}/monitoring-plugins/trace/data-prepper/).
|
||||
|
||||
|
||||
## Data Prepper server options
|
||||
|
|
@ -149,7 +149,7 @@ aws_region | No | String, AWS region for the cluster (e.g. `"us-east-1"`) if you
|
|||
trace_analytics_raw | No | Boolean, default false. Whether to export as trace data to the `otel-v1-apm-span-*` index pattern (alias `otel-v1-apm-span`) for use with the Trace Analytics OpenSearch Dashboards plugin.
|
||||
trace_analytics_service_map | No | Boolean, default false. Whether to export as trace data to the `otel-v1-apm-service-map` index for use with the service map component of the Trace Analytics OpenSearch Dashboards plugin.
|
||||
index | No | String, name of the index to export to. Only required if you don't use the `trace_analytics_raw` or `trace_analytics_service_map` presets.
|
||||
template_file | No | String, the path to a JSON [index template](../../opensearch/index-templates/) file (e.g. `/your/local/template-file.json` if you do not use the `trace_analytics_raw` or `trace_analytics_service_map`. See [otel-v1-apm-span-index-template.json](https://github.com/opensearch-project/data-prepper/blob/main/data-prepper-plugins/opensearch/src/main/resources/otel-v1-apm-span-index-template.json) for an example.
|
||||
template_file | No | String, the path to a JSON [index template]({{site.url}}{{site.baseurl}}/opensearch/index-templates/) file (e.g. `/your/local/template-file.json` if you do not use the `trace_analytics_raw` or `trace_analytics_service_map`. See [otel-v1-apm-span-index-template.json](https://github.com/opensearch-project/data-prepper/blob/main/data-prepper-plugins/opensearch/src/main/resources/otel-v1-apm-span-index-template.json) for an example.
|
||||
document_id_field | No | String, the field from the source data to use for the OpenSearch document ID (e.g. `"my-field"`) if you don't use the `trace_analytics_raw` or `trace_analytics_service_map` presets.
|
||||
dlq_file | No | String, the path to your preferred dead letter queue file (e.g. `/your/local/dlq-file`). Data Prepper writes to this file when it fails to index a document on the OpenSearch cluster.
|
||||
bulk_size | No | Integer (long), default 5. The maximum size (in MiB) of bulk requests to the OpenSearch cluster. Values below 0 indicate an unlimited size. If a single document exceeds the maximum bulk request size, Data Prepper sends it individually.
|
||||
|
|
|
|||
|
|
@ -105,7 +105,7 @@ service-map-pipeline:
|
|||
trace_analytics_service_map: true
|
||||
```
|
||||
|
||||
To learn more, see the [Data Prepper configuration reference](../data-prepper-reference/).
|
||||
To learn more, see the [Data Prepper configuration reference]({{site.url}}{{site.baseurl}}/monitoring-plugins/trace/data-prepper-reference/).
|
||||
|
||||
## Configure the Data Prepper server
|
||||
Data Prepper itself provides administrative HTTP endpoints such as `/list` to list pipelines and `/metrics/prometheus` to provide Prometheus-compatible metrics data. The port which serves these endpoints, as well as TLS configuration, is specified by a separate YAML file. Example:
|
||||
|
|
|
|||
|
|
@ -12,7 +12,7 @@ OpenSearch Trace Analytics consists of two components---Data Prepper and the Tra
|
|||
|
||||
## Basic flow of data
|
||||
|
||||
|
||||
|
||||
|
||||
1. Trace Analytics relies on you adding instrumentation to your application and generating trace data. The [OpenTelemetry documentation](https://opentelemetry.io/docs/) contains example applications for many programming languages that can help you get started, including Java, Python, Go, and JavaScript.
|
||||
|
||||
|
|
@ -20,9 +20,9 @@ OpenSearch Trace Analytics consists of two components---Data Prepper and the Tra
|
|||
|
||||
1. The [OpenTelemetry Collector](https://opentelemetry.io/docs/collector/getting-started/) receives data from the application and formats it into OpenTelemetry data.
|
||||
|
||||
1. [Data Prepper](../data-prepper/) processes the OpenTelemetry data, transforms it for use in OpenSearch, and indexes it on an OpenSearch cluster.
|
||||
1. [Data Prepper]({{site.url}}{{site.baseurl}}/monitoring-plugins/trace/data-prepper/) processes the OpenTelemetry data, transforms it for use in OpenSearch, and indexes it on an OpenSearch cluster.
|
||||
|
||||
1. The [Trace Analytics OpenSearch Dashboards plugin](../ta-opensearch-dashboards/) displays the data in near real-time as a series of charts and tables, with an emphasis on service architecture, latency, error rate, and throughput.
|
||||
1. The [Trace Analytics OpenSearch Dashboards plugin]({{site.url}}{{site.baseurl}}/monitoring-plugins/trace/ta-dashboards/) displays the data in near real-time as a series of charts and tables, with an emphasis on service architecture, latency, error rate, and throughput.
|
||||
|
||||
|
||||
## Jaeger HotROD
|
||||
|
|
@ -39,7 +39,7 @@ Download or clone the [Data Prepper repository](https://github.com/opensearch-pr
|
|||
|
||||
Close the file and run `docker-compose up --build`. After the containers start, navigate to `http://localhost:8080` in a web browser.
|
||||
|
||||
|
||||
|
||||
|
||||
Click one of the buttons in the web interface to send a request to the application. Each request starts a series of operations across the services that make up the application. From the console logs, you can see that these operations share the same `trace-id`, which lets you track all of the operations in the request as a single *trace*:
|
||||
|
||||
|
|
@ -80,4 +80,4 @@ curl -X GET -u 'admin:admin' -k 'https://localhost:9200/otel-v1-apm-span-000001/
|
|||
|
||||
Navigate to `http://localhost:5601` in a web browser and choose **Trace Analytics**. You can see the results of your single click in the Jaeger HotROD web interface: the number of traces per API and HTTP method, latency trends, a color-coded map of the service architecture, and a list of trace IDs that you can use to drill down on individual operations.
|
||||
|
||||
If you don't see your trace, adjust the timeframe in OpenSearch Dashboards. For more information on using the plugin, see [OpenSearch Dashboards plugin](../ta-opensearch-dashboards/).
|
||||
If you don't see your trace, adjust the timeframe in OpenSearch Dashboards. For more information on using the plugin, see [OpenSearch Dashboards plugin]({{site.url}}{{site.baseurl}}/monitoring-plugins/trace/ta-dashboards/).
|
||||
|
|
|
|||
|
|
@ -14,4 +14,4 @@ A single operation, such as a user clicking a button, can trigger an extended se
|
|||
|
||||
Trace Analytics can help you visualize this flow of events and identify performance problems.
|
||||
|
||||
|
||||
|
||||
|
|
|
|||
|
|
@ -7,16 +7,16 @@ nav_order: 50
|
|||
|
||||
# Trace Analytics OpenSearch Dashboards plugin
|
||||
|
||||
The Trace Analytics plugin for OpenSearch Dashboards provides at-a-glance visibility into your application performance, along with the ability to drill down on individual traces. For installation instructions, see [Standalone OpenSearch Dashboards plugin install](../../opensearch-dashboards/plugins/).
|
||||
The Trace Analytics plugin for OpenSearch Dashboards provides at-a-glance visibility into your application performance, along with the ability to drill down on individual traces. For installation instructions, see [Standalone OpenSearch Dashboards plugin install]({{site.url}}{{site.baseurl}}/dashboards/install/plugins/).
|
||||
|
||||
The **Dashboard** view groups traces together by HTTP method and path so that you can see the average latency, error rate, and trends associated with a particular operation. For a more focused view, try filtering by trace group name.
|
||||
|
||||
|
||||
|
||||
|
||||
To drill down on the traces that make up a trace group, choose the number of traces in righthand column. Then choose an individual trace for a detailed summary.
|
||||
|
||||
|
||||
|
||||
|
||||
The **Services** view lists all services in the application, plus an interactive map that shows how the various services connect to each other. In contrast to the dashboard, which helps identify problems by operation, the service map helps identify problems by service. Try sorting by error rate or latency to get a sense of potential problem areas of your application.
|
||||
|
||||
|
||||
|
||||
|
|
|
|||
|
|
@ -14,7 +14,7 @@ To create and deploy an OpenSearch cluster according to your requirements, it’
|
|||
|
||||
There are many ways to design a cluster. The following illustration shows a basic architecture:
|
||||
|
||||
|
||||
|
||||
|
||||
This is a four-node cluster that has one dedicated master node, one dedicated coordinating node, and two data nodes that are master-eligible and also used for ingesting data.
|
||||
|
||||
|
|
@ -36,7 +36,7 @@ This page demonstrates how to work with the different node types. It assumes tha
|
|||
|
||||
## Prerequisites
|
||||
|
||||
Before you get started, you must install and configure OpenSearch on all of your nodes. For information about the available options, see [Install and configure OpenSearch](../install/).
|
||||
Before you get started, you must install and configure OpenSearch on all of your nodes. For information about the available options, see [Install and configure OpenSearch]({{site.url}}{{site.baseurl}}/opensearch/install/).
|
||||
|
||||
After you're done, use SSH to connect to each node, then open the `config/opensearch.yml` file. You can set all configurations for your cluster in this file.
|
||||
|
||||
|
|
@ -188,7 +188,7 @@ x.x.x.x 34 38 0 0.12 0.07 0.06 md - o
|
|||
x.x.x.x 23 38 0 0.12 0.07 0.06 md - opensearch-c1
|
||||
```
|
||||
|
||||
To better understand and monitor your cluster, use the [cat API](../catapis/).
|
||||
To better understand and monitor your cluster, use the [cat API]({{site.url}}{{site.baseurl}}/opensearch/catapis/).
|
||||
|
||||
|
||||
## (Advanced) Step 6: Configure shard allocation awareness or forced awareness
|
||||
|
|
@ -322,9 +322,9 @@ old_index 0 r UNASSIGNED
|
|||
|
||||
In this case, all primary shards are allocated to `opensearch-d2`. Again, all replica shards are unassigned because we only have one warm node.
|
||||
|
||||
A popular approach is to configure your [index templates](../index-templates/) to set the `index.routing.allocation.require.temp` value to `hot`. This way, OpenSearch stores your most recent data on your hot nodes.
|
||||
A popular approach is to configure your [index templates]({{site.url}}{{site.baseurl}}/opensearch/index-templates/) to set the `index.routing.allocation.require.temp` value to `hot`. This way, OpenSearch stores your most recent data on your hot nodes.
|
||||
|
||||
You can then use the [Index State Management (ISM)](../../im-plugin/) plugin to periodically check the age of an index and specify actions to take on it. For example, when the index reaches a specific age, change the `index.routing.allocation.require.temp` setting to `warm` to automatically move your data from hot nodes to warm nodes.
|
||||
You can then use the [Index State Management (ISM)]({{site.url}}{{site.baseurl}}/im-plugin/) plugin to periodically check the age of an index and specify actions to take on it. For example, when the index reaches a specific age, change the `index.routing.allocation.require.temp` setting to `warm` to automatically move your data from hot nodes to warm nodes.
|
||||
|
||||
|
||||
## Next steps
|
||||
|
|
@ -332,7 +332,7 @@ You can then use the [Index State Management (ISM)](../../im-plugin/) plugin to
|
|||
If you are using the security plugin, the previous request to `_cat/nodes?v` might have failed with an initialization error. To initialize the plugin, run `opensearch/plugins/opensearch-security/tools/securityadmin.sh`. A sample command that uses the demo certificates might look like this:
|
||||
|
||||
```bash
|
||||
sudo ./securityadmin.sh -cd ../securityconfig/ -icl -nhnv -cacert /etc/opensearch/root-ca.pem -cert /etc/opensearch/kirk.pem -key /etc/opensearch/kirk-key.pem -h <private-ip>
|
||||
sudo ./securityadmin.sh -cd {{site.url}}{{site.baseurl}}/securityconfig/ -icl -nhnv -cacert /etc/opensearch/root-ca.pem -cert /etc/opensearch/kirk.pem -key /etc/opensearch/kirk-key.pem -h <private-ip>
|
||||
```
|
||||
|
||||
For full guidance around configuration options, see [Security configuration](../../security/configuration).
|
||||
For full guidance around configuration options, see [Security configuration]({{site.url}}{{site.baseurl}}/security-plugin/configuration).
|
||||
|
|
|
|||
|
|
@ -65,4 +65,4 @@ PUT /_cluster/settings
|
|||
|
||||
You can find `opensearch.yml` in `/usr/share/opensearch/config/opensearch.yml` (Docker) or `/etc/opensearch/opensearch.yml` (RPM and DEB) on each node.
|
||||
|
||||
The demo configuration includes a number of settings for the security plugin that you should modify before using OpenSearch for a production workload. To learn more, see [Security](../../security/).
|
||||
The demo configuration includes a number of settings for the security plugin that you should modify before using OpenSearch for a production workload. To learn more, see [Security]({{site.url}}{{site.baseurl}}/security-plugin/).
|
||||
|
|
|
|||
|
|
@ -12,7 +12,7 @@ OpenSearch is a distributed search and analytics engine based on [Apache Lucene]
|
|||
|
||||
Unsurprisingly, people often use OpenSearch as the backend for a search application---think [Wikipedia](https://en.wikipedia.org/wiki/Wikipedia:FAQ/Technical#What_software_is_used_to_run_Wikipedia?) or an online store. It offers excellent performance and can scale up and down as the needs of the application grow or shrink.
|
||||
|
||||
An equally popular, but less obvious use case is log analytics, in which you take the logs from an application, feed them into OpenSearch, and use the rich search and visualization functionality to identify issues. For example, a malfunctioning web server might throw a 500 error 0.5% of the time, which can be hard to notice unless you have a real-time graph of all HTTP status codes that the server has thrown in the past four hours. You can use [OpenSearch Dashboards](../opensearch-dashboards/) to build these sorts of visualizations from data in OpenSearch.
|
||||
An equally popular, but less obvious use case is log analytics, in which you take the logs from an application, feed them into OpenSearch, and use the rich search and visualization functionality to identify issues. For example, a malfunctioning web server might throw a 500 error 0.5% of the time, which can be hard to notice unless you have a real-time graph of all HTTP status codes that the server has thrown in the past four hours. You can use [OpenSearch Dashboards]({{site.url}}{{site.baseurl}}/dashboards/) to build these sorts of visualizations from data in OpenSearch.
|
||||
|
||||
|
||||
## Clusters and nodes
|
||||
|
|
@ -21,7 +21,7 @@ Its distributed design means that you interact with OpenSearch *clusters*. Each
|
|||
|
||||
You can run OpenSearch locally on a laptop---its system requirements are minimal---but you can also scale a single cluster to hundreds of powerful machines in a data center.
|
||||
|
||||
In a single node cluster, such as a laptop, one machine has to do everything: manage the state of the cluster, index and search data, and perform any preprocessing of data prior to indexing it. As a cluster grows, however, you can subdivide responsibilities. Nodes with fast disks and plenty of RAM might be great at indexing and searching data, whereas a node with plenty of CPU power and a tiny disk could manage cluster state. For more information on setting node types, see [Cluster formation](cluster/).
|
||||
In a single node cluster, such as a laptop, one machine has to do everything: manage the state of the cluster, index and search data, and perform any preprocessing of data prior to indexing it. As a cluster grows, however, you can subdivide responsibilities. Nodes with fast disks and plenty of RAM might be great at indexing and searching data, whereas a node with plenty of CPU power and a tiny disk could manage cluster state. For more information on setting node types, see [Cluster formation]({{site.url}}{{site.baseurl}}/opensearch/cluster/).
|
||||
|
||||
|
||||
## Indices and documents
|
||||
|
|
|
|||
|
|
@ -108,7 +108,7 @@ networks:
|
|||
opensearch-net:
|
||||
```
|
||||
|
||||
Then make your changes to `opensearch.yml`. For a full list of settings, see [Security](../../../security/configuration/). This example adds (extremely) verbose audit logging:
|
||||
Then make your changes to `opensearch.yml`. For a full list of settings, see [Security]({{site.url}}{{site.baseurl}}/security-plugin/configuration/). This example adds (extremely) verbose audit logging:
|
||||
|
||||
```yml
|
||||
plugins.security.ssl.transport.pemcert_filepath: node.pem
|
||||
|
|
@ -133,7 +133,7 @@ plugins.security.audit.config.disabled_rest_categories: NONE
|
|||
plugins.security.audit.config.disabled_transport_categories: NONE
|
||||
```
|
||||
|
||||
Use this same override process to specify new [authentication settings](../../../security/configuration/configuration/) in `/usr/share/opensearch/plugins/opensearch-security/securityconfig/config.yml`, as well as new default [internal users, roles, mappings, action groups, and tenants](../../../security/configuration/yaml/).
|
||||
Use this same override process to specify new [authentication settings]({{site.url}}{{site.baseurl}}/security-plugin/configuration/configuration/) in `/usr/share/opensearch/plugins/opensearch-security/securityconfig/config.yml`, as well as new default [internal users, roles, mappings, action groups, and tenants]({{site.url}}{{site.baseurl}}/security-plugin/configuration/yaml/).
|
||||
|
||||
To start the cluster, run `docker-compose up`.
|
||||
|
||||
|
|
@ -162,7 +162,7 @@ volumes:
|
|||
- ./custom-opensearch.yml: /full/path/to/custom-opensearch.yml
|
||||
```
|
||||
|
||||
Remember that the certificates you specify in your Docker Compose file must be the same as the certificates listed in your custom `opensearch.yml` file. At a minimum, you should replace the root, admin, and node certificates with your own. For more information about adding and using certificates, see [Configure TLS certificates](../security/configuration/tls.md).
|
||||
Remember that the certificates you specify in your Docker Compose file must be the same as the certificates listed in your custom `opensearch.yml` file. At a minimum, you should replace the root, admin, and node certificates with your own. For more information about adding and using certificates, see [Configure TLS certificates]({{site.url}}{{site.baseurl}}/security-plugin/configuration/tls).
|
||||
|
||||
```yml
|
||||
plugins.security.ssl.transport.pemcert_filepath: new-node-cert.pem
|
||||
|
|
|
|||
|
|
@ -183,7 +183,7 @@ services:
|
|||
- ./custom-opensearch_dashboards.yml:/usr/share/opensearch-dashboards/config/opensearch_dashboards.yml
|
||||
```
|
||||
|
||||
You can also configure `docker-compose.yml` and `opensearch.yml` [to take your own certificates](../docker-security/) for use with the [Security](../../security/configuration/) plugin.
|
||||
You can also configure `docker-compose.yml` and `opensearch.yml` [to take your own certificates]({{site.url}}{{site.baseurl}}/opensearch/install/docker-security/) for use with the [Security]({{site.url}}{{site.baseurl}}/security-plugin/configuration/) plugin.
|
||||
|
||||
|
||||
### (Optional) Set up Performance Analyzer
|
||||
|
|
@ -298,7 +298,7 @@ docker build --tag=opensearch-custom-plugin .
|
|||
docker run -p 9200:9200 -p 9600:9600 -v /usr/share/opensearch/data opensearch-custom-plugin
|
||||
```
|
||||
|
||||
You can also use a `Dockerfile` to pass your own certificates for use with the [security](../../../security/) plugin, similar to the `-v` argument in [Configure OpenSearch](#configure-opensearch):
|
||||
You can also use a `Dockerfile` to pass your own certificates for use with the [security]({{site.url}}{{site.baseurl}}/security-plugin/) plugin, similar to the `-v` argument in [Configure OpenSearch](#configure-opensearch):
|
||||
|
||||
```
|
||||
FROM opensearchproject/opensearch:{{site.opensearch_version}}
|
||||
|
|
|
|||
|
|
@ -21,7 +21,7 @@ vm.max_map_count=262144
|
|||
|
||||
Then run `sudo sysctl -p` to reload.
|
||||
|
||||
The [sample docker-compose.yml](../docker/#sample-docker-compose-file) file also contains several key settings:
|
||||
The [sample docker-compose.yml]({{site.url}}{{site.baseurl}}/opensearch/install/docker/#sample-docker-compose-file) file also contains several key settings:
|
||||
|
||||
- `bootstrap.memory_lock=true`
|
||||
|
||||
|
|
|
|||
|
|
@ -95,9 +95,9 @@ Navigate to the OpenSearch home directory (most likely, it is `/usr/share/opense
|
|||
sudo bin/opensearch-plugin install https://d3g5vo6xdbdb9a.cloudfront.net/downloads/opensearch-plugins/opensearch-security/opensearch-security-{{site.opensearch_major_minor_version}}.1.0.zip
|
||||
```
|
||||
|
||||
After installing the security plugin, you can run `sudo sh /usr/share/opensearch/plugins/opensearch-security/tools/install_demo_configuration.sh` to quickly get started with demo certificates. Otherwise, you must configure it manually and run [securityadmin.sh](../../../security/configuration/security-admin/).
|
||||
After installing the security plugin, you can run `sudo sh /usr/share/opensearch/plugins/opensearch-security/tools/install_demo_configuration.sh` to quickly get started with demo certificates. Otherwise, you must configure it manually and run [securityadmin.sh]({{site.url}}{{site.baseurl}}/security-plugin/configuration/security-admin/).
|
||||
|
||||
The security plugin has a corresponding [OpenSearch Dashboards plugin](../../../opensearch-dashboards/install/plugins) that you probably want to install as well.
|
||||
The security plugin has a corresponding [OpenSearch Dashboards plugin]({{site.url}}{{site.baseurl}}/opensearch-dashboards/install/plugins) that you probably want to install as well.
|
||||
|
||||
|
||||
### Job scheduler
|
||||
|
|
@ -113,7 +113,7 @@ sudo bin/opensearch-plugin install https://d3g5vo6xdbdb9a.cloudfront.net/downloa
|
|||
sudo bin/opensearch-plugin install https://d3g5vo6xdbdb9a.cloudfront.net/downloads/opensearch-plugins/opensearch-alerting/opensearch-alerting-{{site.opensearch_major_minor_version}}.1.0.zip
|
||||
```
|
||||
|
||||
To install Alerting, you must first install the Job Scheduler plugin. Alerting has a corresponding [OpenSearch Dashboards plugin](../../../opensearch-dashboards/install/plugins/) that you probably want to install as well.
|
||||
To install Alerting, you must first install the Job Scheduler plugin. Alerting has a corresponding [OpenSearch Dashboards plugin]({{site.url}}{{site.baseurl}}/opensearch-dashboards/install/plugins/) that you probably want to install as well.
|
||||
|
||||
|
||||
### SQL
|
||||
|
|
@ -136,7 +136,7 @@ sudo bin/opensearch-plugin install https://d3g5vo6xdbdb9a.cloudfront.net/downloa
|
|||
sudo bin/opensearch-plugin install https://d3g5vo6xdbdb9a.cloudfront.net/downloads/opensearch-plugins/opensearch-index-management/opensearch-index-management-{{site.opensearch_major_minor_version}}.2.0.zip
|
||||
```
|
||||
|
||||
To install Index State Management, you must first install the Job Scheduler plugin. ISM has a corresponding [OpenSearch Dashboards plugin](../../../opensearch-dashboards/install/plugins/) that you probably want to install as well.
|
||||
To install Index State Management, you must first install the Job Scheduler plugin. ISM has a corresponding [OpenSearch Dashboards plugin]({{site.url}}{{site.baseurl}}/opensearch-dashboards/install/plugins/) that you probably want to install as well.
|
||||
|
||||
|
||||
### k-NN
|
||||
|
|
|
|||
|
|
@ -45,7 +45,7 @@ You can modify `config/opensearch.yml` or specify environment variables as argum
|
|||
./opensearch-tar-install.sh -Ecluster.name=opensearch-cluster -Enode.name=opensearch-node1 -Ehttp.host=0.0.0.0 -Ediscovery.type=single-node
|
||||
```
|
||||
|
||||
For other settings, see [Important settings](../important-settings/).
|
||||
For other settings, see [Important settings]({{site.url}}{{site.baseurl}}/opensearch/install/important-settings/).
|
||||
|
||||
|
||||
### (Optional) Set up Performance Analyzer
|
||||
|
|
@ -142,6 +142,6 @@ In a tarball installation, Performance Analyzer collects data when it is enabled
|
|||
|
||||
### (Optional) Removing Performance Analyzer
|
||||
|
||||
See [Clean up Performance Analyzer files](../plugins/#optional-clean-up-performance-analyzer-files).
|
||||
See [Clean up Performance Analyzer files]({{site.url}}{{site.baseurl}}/plugins/#optional-clean-up-performance-analyzer-files).
|
||||
|
||||
{% endcomment %}
|
||||
|
|
|
|||
|
|
@ -3,7 +3,9 @@ layout: default
|
|||
title: Full-text queries
|
||||
parent: Query DSL
|
||||
nav_order: 40
|
||||
redirect_from: /docs/opensearch/full-text/
|
||||
redirect_from:
|
||||
- /docs/opensearch/full-text/
|
||||
- /opensearch/full-text/
|
||||
---
|
||||
|
||||
# Full-text queries
|
||||
|
|
|
|||
|
|
@ -145,7 +145,7 @@ The search query “To be, or not to be” is analyzed and tokenized into an arr
|
|||
...
|
||||
```
|
||||
|
||||
For a list of all full-text queries, see [Full-text queries](../full-text/).
|
||||
For a list of all full-text queries, see [Full-text queries]({{site.url}}{{site.baseurl}}/opensearch/full-text/).
|
||||
|
||||
If you want to query for an exact term like “HAMLET” in the speaker field and don't need the results to be sorted by relevance scores, a term-level query is more efficient:
|
||||
|
||||
|
|
|
|||
|
|
@ -112,7 +112,7 @@ POST _reindex
|
|||
}
|
||||
```
|
||||
|
||||
For a list of all query operations, see [Full-text queries](../full-text/).
|
||||
For a list of all query operations, see [Full-text queries]({{site.url}}{{site.baseurl}}/opensearch/query-dsl/full-text/).
|
||||
|
||||
## Combine one or more indices
|
||||
|
||||
|
|
|
|||
|
|
@ -100,7 +100,7 @@ Setting | Description
|
|||
sudo ./bin/opensearch-plugin install repository-s3
|
||||
```
|
||||
|
||||
If you're using the Docker installation, see [Customize the Docker image](../../install/docker/#customize-the-docker-image). Your `Dockerfile` should look something like this:
|
||||
If you're using the Docker installation, see [Customize the Docker image]({{site.url}}{{site.baseurl}}/opensearch/install/docker/#customize-the-docker-image). Your `Dockerfile` should look something like this:
|
||||
|
||||
```
|
||||
FROM opensearchproject/opensearch:{{site.opensearch_version}}
|
||||
|
|
|
|||
|
|
@ -20,7 +20,7 @@ By including a task ID, you can get information specific to a particular task. N
|
|||
GET _tasks/<task_id>
|
||||
```
|
||||
|
||||
Note that if a task finishes running, it won't be returned as part of your request. For an example of a task that takes a little longer to finish, you can run the [`_reindex`](../reindex-data) API operation on a larger document, and then run `tasks`.
|
||||
Note that if a task finishes running, it won't be returned as part of your request. For an example of a task that takes a little longer to finish, you can run the [`_reindex`]({{site.url}}{{site.baseurl}}/opensearch/reindex-data) API operation on a larger document, and then run `tasks`.
|
||||
|
||||
**Sample Response**
|
||||
```json
|
||||
|
|
|
|||
|
|
@ -15,4 +15,4 @@ Bytes | The supported units for byte size are `b` for bytes, `kb` for kibibytes,
|
|||
Distances | The supported units for distance are `mi` for miles, `yd` for yards, `ft` for feet, `in` for inches, `km` for kilometers, `m` for meters, `cm` for centimeters, `mm` for millimeters, and `nmi` or `NM` for nautical miles. | `5mi` or `4ft`
|
||||
Quantities without units | For large values that don't have a unit, use `k` for kilo, `m` for mega, `g` for giga, `t` for tera, and `p` for peta. | `5k` for 5,000
|
||||
|
||||
To convert output units to human-readable values, see [Common REST parameters](../common-parameters/).
|
||||
To convert output units to human-readable values, see [Common REST parameters]({{site.url}}{{site.baseurl}}/opensearch/common-parameters/).
|
||||
|
|
|
|||
|
|
@ -39,7 +39,7 @@ These methods are described in the following sections.
|
|||
Prefix matching finds documents that matches the last term in the query string.
|
||||
|
||||
For example, assume that the user types “qui” into a search UI. To autocomplete this phrase, use the `match_phrase_prefix` query to search all `text_entry` fields that begin with the prefix "qui."
|
||||
To make the word order and relative positions flexible, specify a `slop` value. To learn about the `slop` option, see [Options](../full-text/#options).
|
||||
To make the word order and relative positions flexible, specify a `slop` value. To learn about the `slop` option, see [Options]({{site.url}}{{site.baseurl}}/opensearch/query-dsl/full-text/#options).
|
||||
|
||||
#### Sample Request
|
||||
|
||||
|
|
@ -59,7 +59,7 @@ GET shakespeare/_search
|
|||
|
||||
Prefix matching doesn’t require any special mappings. It works with your data as-is.
|
||||
However, it’s a fairly resource-intensive operation. A prefix of `a` could match hundreds of thousands of terms and not be useful to your user.
|
||||
To limit the impact of prefix expansion, set `max_expansions` to a reasonable number. To learn about the `max_expansions` option, see [Options](../full-text/#options).
|
||||
To limit the impact of prefix expansion, set `max_expansions` to a reasonable number. To learn about the `max_expansions` option, see [Options]({{site.url}}{{site.baseurl}}/opensearch/query-dsl/full-text/#options).
|
||||
|
||||
#### Sample Request
|
||||
|
||||
|
|
|
|||
|
|
@ -11,13 +11,13 @@ redirect_from: /docs/async/security/
|
|||
|
||||
You can use the security plugin with asynchronous searches to limit non-admin users to specific actions. For example, you might want some users to only be able to submit or delete asynchronous searches, while you might want others to only view the results.
|
||||
|
||||
All asynchronous search indices are protected as system indices. Only a super admin user or an admin user with a Transport Layer Security (TLS) certificate can access system indices. For more information, see [System indices](../../security/configuration/system-indices/).
|
||||
All asynchronous search indices are protected as system indices. Only a super admin user or an admin user with a Transport Layer Security (TLS) certificate can access system indices. For more information, see [System indices]({{site.url}}{{site.baseurl}}/security-plugin/configuration/system-indices/).
|
||||
|
||||
## Basic permissions
|
||||
|
||||
As an admin user, you can use the security plugin to assign specific permissions to users based on which API operations they need access to. For a list of supported APIs operations, see [Asynchronous search](../).
|
||||
As an admin user, you can use the security plugin to assign specific permissions to users based on which API operations they need access to. For a list of supported APIs operations, see [Asynchronous search]({{site.url}}{{site.baseurl}}/).
|
||||
|
||||
The security plugin has two built-in roles that cover most asynchronous search use cases: `asynchronous_search_full_access` and `asynchronous_search_read_access`. For descriptions of each, see [Predefined roles](../../security/access-control/users-roles/#predefined-roles).
|
||||
The security plugin has two built-in roles that cover most asynchronous search use cases: `asynchronous_search_full_access` and `asynchronous_search_read_access`. For descriptions of each, see [Predefined roles]({{site.url}}{{site.baseurl}}/security-plugin/access-control/users-roles/#predefined-roles).
|
||||
|
||||
If these roles don’t meet your needs, mix and match individual asynchronous search permissions to suit your use case. Each action corresponds to an operation in the REST API. For example, the `cluster:admin/opensearch/asynchronous_search/delete` permission lets you delete a previously submitted asynchronous search.
|
||||
|
||||
|
|
@ -25,7 +25,7 @@ If these roles don’t meet your needs, mix and match individual asynchronous se
|
|||
|
||||
Use backend roles to configure fine-grained access to asynchronous searches based on roles. For example, users of different departments in an organization can view asynchronous searches owned by their own department.
|
||||
|
||||
First, make sure your users have the appropriate [backend roles](../../security/access-control/). Backend roles usually come from an [LDAP server](../../security/configuration/ldap/) or [SAML provider](../../security/configuration/saml/). However, if you use the internal user database, you can use the REST API to [add them manually](../../security/access-control/api/#create-user).
|
||||
First, make sure your users have the appropriate [backend roles]({{site.url}}{{site.baseurl}}/security-plugin/access-control/). Backend roles usually come from an [LDAP server]({{site.url}}{{site.baseurl}}/security-plugin/configuration/ldap/) or [SAML provider]({{site.url}}{{site.baseurl}}/security-plugin/configuration/saml/). However, if you use the internal user database, you can use the REST API to [add them manually]({{site.url}}{{site.baseurl}}/security-plugin/access-control/api/#create-user).
|
||||
|
||||
Now when users view asynchronous search resources in OpenSearch Dashboards (or make REST API calls), they only see asynchronous searches submitted by users who have a subset of the backend role.
|
||||
For example, consider two users: `judy` and `elon`.
|
||||
|
|
|
|||
|
|
@ -140,7 +140,7 @@ The call doesn't return results until the warmup operation finishes or the reque
|
|||
GET /_tasks
|
||||
```
|
||||
|
||||
After the operation has finished, use the [k-NN `_stats` API operation](#Stats) to see what the k-NN plugin loaded into the graph.
|
||||
After the operation has finished, use the [k-NN `_stats` API operation](#stats) to see what the k-NN plugin loaded into the graph.
|
||||
|
||||
|
||||
### Best practices
|
||||
|
|
@ -149,6 +149,6 @@ For the warmup operation to function properly, follow these best practices:
|
|||
|
||||
* Don't run merge operations on indices that you want to warm up. During merge, the k-NN plugin creates new segments, and old segments are sometimes deleted. For example, you could encounter a situation in which the warmup API operation loads graphs A and B into native memory, but segment C is created from segments A and B being merged. The graphs for A and B would no longer be in memory, and graph C would also not be in memory. In this case, the initial penalty for loading graph C is still present.
|
||||
|
||||
* Confirm that all graphs you want to warm up can fit into native memory. For more information about the native memory limit, see the [knn.memory.circuit_breaker.limit statistic](../settings/#cluster-settings). High graph memory usage causes cache thrashing, which can lead to operations constantly failing and attempting to run again.
|
||||
* Confirm that all graphs you want to warm up can fit into native memory. For more information about the native memory limit, see the [knn.memory.circuit_breaker.limit statistic]({{site.url}}{{site.baseurl}}/search-plugins/knn/settings#cluster-settings). High graph memory usage causes cache thrashing, which can lead to operations constantly failing and attempting to run again.
|
||||
|
||||
* Don't index any documents that you want to load into the cache. Writing new information to segments prevents the warmup API operation from loading the graphs until they're searchable. This means that you would have to run the warmup operation again after indexing finishes.
|
||||
|
|
|
|||
|
|
@ -12,7 +12,7 @@ redirect_from: /docs/knn/approximate-knn/
|
|||
|
||||
The approximate k-NN method uses [nmslib's](https://github.com/nmslib/nmslib/) implementation of the Hierarchical Navigable Small World (HNSW) algorithm to power k-NN search. In this case, approximate means that for a given search, the neighbors returned are an estimate of the true k-nearest neighbors. Of the three methods, this method offers the best search scalability for large data sets. Generally speaking, once the data set gets into the hundreds of thousands of vectors, this approach is preferred.
|
||||
|
||||
The k-NN plugin builds an HNSW graph of the vectors for each "knn-vector field"/ "Lucene segment" pair during indexing that can be used to efficiently find the k-nearest neighbors to a query vector during search. To learn more about Lucene segments, please refer to [Apache Lucene's documentation](https://lucene.apache.org/core/8_7_0/core/org/apache/lucene/codecs/lucene87/package-summary.html#package.description). These graphs are loaded into native memory during search and managed by a cache. To learn more about pre-loading graphs into memory, refer to the [warmup API](../api#warmup). Additionally, you can see what graphs are already loaded in memory, which you can learn more about in the [stats API section](../api#stats).
|
||||
The k-NN plugin builds an HNSW graph of the vectors for each "knn-vector field"/ "Lucene segment" pair during indexing that can be used to efficiently find the k-nearest neighbors to a query vector during search. To learn more about Lucene segments, please refer to [Apache Lucene's documentation](https://lucene.apache.org/core/8_7_0/core/org/apache/lucene/codecs/lucene87/package-summary.html#package.description). These graphs are loaded into native memory during search and managed by a cache. To learn more about pre-loading graphs into memory, refer to the [warmup API]({{site.url}}{{site.baseurl}}/search-plugins/knn/api#warmup). Additionally, you can see what graphs are already loaded in memory, which you can learn more about in the [stats API section]({{site.url}}{{site.baseurl}}/search-plugins/knn/api#stats).
|
||||
|
||||
Because the graphs are constructed during indexing, it is not possible to apply a filter on an index and then use this search method. All filters are applied on the results produced by the approximate nearest neighbor search.
|
||||
|
||||
|
|
@ -20,7 +20,7 @@ Because the graphs are constructed during indexing, it is not possible to apply
|
|||
|
||||
To use the k-NN plugin's approximate search functionality, you must first create a k-NN index with setting `index.knn` to `true`. This setting tells the plugin to create HNSW graphs for the index.
|
||||
|
||||
Additionally, if you're using the approximate k-nearest neighbor method, specify `knn.space_type` to the space you're interested in. You can't change this setting after it's set. To see what spaces we support, see [spaces](#spaces). By default, `index.knn.space_type` is `l2`. For more information about index settings, such as algorithm parameters you can tweak to tune performance, see [Index settings](../knn-index#index-settings).
|
||||
Additionally, if you're using the approximate k-nearest neighbor method, specify `knn.space_type` to the space you're interested in. You can't change this setting after it's set. To see what spaces we support, see [spaces](#spaces). By default, `index.knn.space_type` is `l2`. For more information about index settings, such as algorithm parameters you can tweak to tune performance, see [Index settings]({{site.url}}{{site.baseurl}}/search-plugins/knn/knn-index#index-settings).
|
||||
|
||||
Next, you must add one or more fields of the `knn_vector` data type. This example creates an index with two `knn_vector` fields and uses cosine similarity:
|
||||
|
||||
|
|
|
|||
|
|
@ -21,7 +21,7 @@ This plugin supports three different methods for obtaining the k-nearest neighbo
|
|||
|
||||
Approximate k-NN is the best choice for searches over large indices (i.e. hundreds of thousands of vectors or more) that require low latency. You should not use approximate k-NN if you want to apply a filter on the index before the k-NN search, which greatly reduces the number of vectors to be searched. In this case, you should use either the script scoring method or painless extensions.
|
||||
|
||||
For more details about this method, see [Approximate k-NN search](approximate-knn).
|
||||
For more details about this method, see [Approximate k-NN search]({{site.url}}{{site.baseurl}}/search-plugins/knn/approximate-knn/).
|
||||
|
||||
2. **Script Score k-NN**
|
||||
|
||||
|
|
@ -29,7 +29,7 @@ This plugin supports three different methods for obtaining the k-nearest neighbo
|
|||
|
||||
Use this approach for searches over smaller bodies of documents or when a pre-filter is needed. Using this approach on large indices may lead to high latencies.
|
||||
|
||||
For more details about this method, see [Exact k-NN with scoring script](knn-score-script).
|
||||
For more details about this method, see [Exact k-NN with scoring script]({{site.url}}{{site.baseurl}}/search-plugins/knn/knn-score-script/).
|
||||
|
||||
3. **Painless extensions**
|
||||
|
||||
|
|
@ -37,7 +37,7 @@ This plugin supports three different methods for obtaining the k-nearest neighbo
|
|||
|
||||
This approach has slightly slower query performance compared to the k-NN Script Score. If your use case requires more customization over the final score, you should use this approach over Script Score k-NN.
|
||||
|
||||
For more details about this method, see [Painless scripting functions](painless-functions).
|
||||
For more details about this method, see [Painless scripting functions]({{site.url}}{{site.baseurl}}/search-plugins/knn/painless-functions/).
|
||||
|
||||
|
||||
Overall, for larger data sets, you should generally choose the approximate nearest neighbor method because it scales significantly better. For smaller data sets, where you may want to apply a filter, you should choose the custom scoring approach. If you have a more complex use case where you need to use a distance function as part of their scoring method, you should use the painless scripting approach.
|
||||
|
|
|
|||
|
|
@ -36,7 +36,7 @@ Mapping Pararameter | Required | Default | Updateable | Description
|
|||
`dimension` | true | n/a | false | The vector dimension for the field
|
||||
`method` | false | null | false | The configuration for the Approximate nearest neighbor method
|
||||
`method.name` | true, if `method` is specified | n/a | false | The identifier for the nearest neighbor method. Currently, "hnsw" is the only valid method.
|
||||
`method.space_type` | false | "l2" | false | The vector space used to calculate the distance between vectors. Refer to [here](../approximate-knn#spaces)) to see available spaces.
|
||||
`method.space_type` | false | "l2" | false | The vector space used to calculate the distance between vectors. Refer to [here]({{site.url}}{{site.baseurl}}/search-plugins/knn/approximate-knn#spaces)) to see available spaces.
|
||||
`method.engine` | false | "nmslib" | false | The approximate k-NN library to use for indexing and search. Currently, "nmslib" is the only valid engine.
|
||||
`method.parameters` | false | null | false | The parameters used for the nearest neighbor method.
|
||||
`method.parameters.ef_construction` | false | 512 | false | The size of the dynamic list used during k-NN graph creation. Higher values lead to a more accurate graph, but slower indexing speed. Only valid for "hnsw" method.
|
||||
|
|
|
|||
|
|
@ -12,7 +12,7 @@ redirect_from: /docs/knn/knn-score-script/
|
|||
|
||||
The k-NN plugin implements the OpenSearch score script plugin that you can use to find the exact k-nearest neighbors to a given query point. Using the k-NN score script, you can apply a filter on an index before executing the nearest neighbor search. This is useful for dynamic search cases where the index body may vary based on other conditions.
|
||||
|
||||
Because the score script approach executes a brute force search, it doesn't scale as well as the [approximate approach](../approximate-knn). In some cases, it might be better to think about refactoring your workflow or index structure to use the approximate approach instead of the score script approach.
|
||||
Because the score script approach executes a brute force search, it doesn't scale as well as the [approximate approach]({{site.url}}{{site.baseurl}}/search-plugins/knn/approximate-knn). In some cases, it might be better to think about refactoring your workflow or index structure to use the approximate approach instead of the score script approach.
|
||||
|
||||
## Getting started with the score script for vectors
|
||||
|
||||
|
|
@ -103,7 +103,7 @@ All parameters are required.
|
|||
- `query_value` is the point you want to find the nearest neighbors for. For the Euclidean and cosine similarity spaces, the value must be an array of floats that matches the dimension set in the field's mapping. For Hamming bit distance, this value can be either of type signed long or a base64-encoded string (for the long and binary field types, respectively).
|
||||
- `space_type` corresponds to the distance function. See the [spaces section](#spaces).
|
||||
|
||||
The [post filter example in the approximate approach](../approximate-knn/#using-approximate-k-nn-with-filters) shows a search that returns fewer than `k` results. If you want to avoid this situation, the score script method lets you essentially invert the order of events. In other words, you can filter down the set of documents over which to execute the k-nearest neighbor search.
|
||||
The [post filter example in the approximate approach]({{site.url}}{{site.baseurl}}/search-plugins/knn/approximate-knn/#using-approximate-k-nn-with-filters) shows a search that returns fewer than `k` results. If you want to avoid this situation, the score script method lets you essentially invert the order of events. In other words, you can filter down the set of documents over which to execute the k-nearest neighbor search.
|
||||
|
||||
This example shows a pre-filter approach to k-NN search with the score script approach. First, create the index:
|
||||
|
||||
|
|
|
|||
|
|
@ -10,11 +10,11 @@ redirect_from: /docs/knn/painless-functions/
|
|||
|
||||
# k-NN Painless Scripting extensions
|
||||
|
||||
With the k-NN plugin's Painless Scripting extensions, you can use k-NN distance functions directly in your Painless scripts to perform operations on `knn_vector` fields. Painless has a strict list of allowed functions and classes per context to ensure its scripts are secure. The k-NN plugin adds Painless Scripting extensions to a few of the distance functions used in [k-NN score script](../knn-score-script), so you can use them to customize your k-NN workload.
|
||||
With the k-NN plugin's Painless Scripting extensions, you can use k-NN distance functions directly in your Painless scripts to perform operations on `knn_vector` fields. Painless has a strict list of allowed functions and classes per context to ensure its scripts are secure. The k-NN plugin adds Painless Scripting extensions to a few of the distance functions used in [k-NN score script]({{site.url}}{{site.baseurl}}/search-plugins/knn/knn-score-script), so you can use them to customize your k-NN workload.
|
||||
|
||||
## Get started with k-NN's Painless Scripting functions
|
||||
|
||||
To use k-NN's Painless Scripting functions, first create an index with `knn_vector` fields like in [k-NN score script](../knn-score-script#Getting-started-with-the-score-script). Once the index is created and you ingest some data, you can use the painless extensions:
|
||||
To use k-NN's Painless Scripting functions, first create an index with `knn_vector` fields like in [k-NN score script]({{site.url}}{{site.baseurl}}/search-plugins/knn/knn-score-script#getting-started-with-the-score-script). Once the index is created and you ingest some data, you can use the painless extensions:
|
||||
|
||||
```json
|
||||
GET my-knn-index-2/_search
|
||||
|
|
|
|||
|
|
@ -40,7 +40,7 @@ Take the following steps to improve indexing performance, especially when you pl
|
|||
|
||||
* **Increase the number of indexing threads**
|
||||
|
||||
If the hardware you choose has multiple cores, you can allow multiple threads in graph construction by speeding up the indexing process. Determine the number of threads to allot with the [knn.algo_param.index_thread_qty](../settings/#Cluster-settings) setting.
|
||||
If the hardware you choose has multiple cores, you can allow multiple threads in graph construction by speeding up the indexing process. Determine the number of threads to allot with the [knn.algo_param.index_thread_qty]({{site.url}}{{site.baseurl}}/search-plugins/knn/settings#cluster-settings) setting.
|
||||
|
||||
Keep an eye on CPU utilization and choose the correct number of threads. Because graph construction is costly, having multiple threads can cause additional CPU load.
|
||||
|
||||
|
|
@ -89,7 +89,7 @@ Recall depends on multiple factors like number of vectors, number of dimensions,
|
|||
|
||||
To configure recall, adjust the algorithm parameters of the HNSW algorithm exposed through index settings. Algorithm parameters that control recall are `m`, `ef_construction`, and `ef_search`. For more information about how algorithm parameters influence indexing and search recall, see [HNSW algorithm parameters](https://github.com/nmslib/hnswlib/blob/master/ALGO_PARAMS.md). Increasing these values can help recall and lead to better search results, but at the cost of higher memory utilization and increased indexing time.
|
||||
|
||||
The default recall values work on a broader set of use cases, but make sure to run your own experiments on your data sets and choose the appropriate values. For index-level settings, see [Index settings](../knn-index#index-settings).
|
||||
The default recall values work on a broader set of use cases, but make sure to run your own experiments on your data sets and choose the appropriate values. For index-level settings, see [Index settings]({{site.url}}{{site.baseurl}}/search-plugins/knn/knn-index#index-settings).
|
||||
|
||||
## Estimating memory usage
|
||||
|
||||
|
|
@ -110,4 +110,4 @@ Having a replica doubles the total number of vectors.
|
|||
|
||||
The standard k-NN query and custom scoring option perform differently. Test with a representative set of documents to see if the search results and latencies match your expectations.
|
||||
|
||||
Custom scoring works best if the initial filter reduces the number of documents to no more than 20,000. Increasing shard count can improve latency, but be sure to keep shard size within the [recommended guidelines](../../opensearch/#primary-and-replica-shards).
|
||||
Custom scoring works best if the initial filter reduces the number of documents to no more than 20,000. Increasing shard count can improve latency, but be sure to keep shard size within the [recommended guidelines]({{site.url}}{{site.baseurl}}/opensearch#primary-and-replica-shards).
|
||||
|
|
|
|||
|
|
@ -34,4 +34,4 @@ array | nested | STRUCT
|
|||
In addition to this list, the PPL plugin also supports the `datetime` type, though it doesn't have a corresponding mapping with OpenSearch.
|
||||
To use a function without a corresponding mapping, you must explicitly convert the data type to one that does.
|
||||
|
||||
The PPL plugin supports all SQL date and time types. To learn more, see [SQL Data Types](../../sql/datatypes/).
|
||||
The PPL plugin supports all SQL date and time types. To learn more, see [SQL Data Types]({{site.url}}{{site.baseurl}}/search-plugins/sql/datatypes/).
|
||||
|
|
|
|||
|
|
@ -8,4 +8,4 @@ redirect_from: /docs/ppl/functions/
|
|||
|
||||
# Functions
|
||||
|
||||
The PPL plugin supports all SQL functions. To learn more, see [SQL Functions](../../sql/functions/).
|
||||
The PPL plugin supports all SQL functions. To learn more, see [SQL Functions]({{site.url}}{{site.baseurl}}/search-plugins/sql/functions/).
|
||||
|
|
|
|||
|
|
@ -11,7 +11,7 @@ redirect_from: /docs/ppl/
|
|||
|
||||
Piped Processing Language (PPL) is a query language that lets you use pipe (`|`) syntax to explore, discover, and query data stored in OpenSearch.
|
||||
|
||||
To quickly get up and running with PPL, use **Query Workbench** in OpenSearch Dashboards. To learn more, see [Workbench](../sql/workbench/).
|
||||
To quickly get up and running with PPL, use **Query Workbench** in OpenSearch Dashboards. To learn more, see [Workbench]({{site.url}}{{site.baseurl}}/search-plugins/sql/workbench/).
|
||||
|
||||
The PPL syntax consists of commands delimited by the pipe character (`|`) where data flows from left to right through each pipeline.
|
||||
|
||||
|
|
@ -56,4 +56,4 @@ Hattie | Bond
|
|||
Nanette | Bates
|
||||
Dale | Adams
|
||||
|
||||
|
||||
|
||||
|
|
|
|||
|
|
@ -35,17 +35,17 @@ With arithmetic operators and SQL functions, use literals and identifiers to bui
|
|||
|
||||
Rule `expressionAtom`:
|
||||
|
||||
|
||||
|
||||
|
||||
The expression in turn can be combined into a predicate with logical operator. Use a predicate in the `WHERE` and `HAVING` clause to filter out data by specific conditions.
|
||||
|
||||
Rule `expression`:
|
||||
|
||||
|
||||
|
||||
|
||||
Rule `predicate`:
|
||||
|
||||
|
||||
|
||||
|
||||
### Execution Order
|
||||
|
||||
|
|
@ -69,11 +69,11 @@ Specify the fields to be retrieved.
|
|||
|
||||
Rule `selectElements`:
|
||||
|
||||
|
||||
|
||||
|
||||
Rule `selectElement`:
|
||||
|
||||
|
||||
|
||||
|
||||
*Example 1*: Use `*` to retrieve all fields in an index:
|
||||
|
||||
|
|
@ -140,9 +140,9 @@ You can specify subqueries within the `FROM` clause.
|
|||
|
||||
Rule `tableName`:
|
||||
|
||||
|
||||
|
||||
|
||||
*Example 1*: Use index aliases to query across indexes. To learn about index aliases, see [Index Alias](../opensearch/index-alias/).
|
||||
*Example 1*: Use index aliases to query across indexes. To learn about index aliases, see [Index Alias]({{site.url}}{{site.baseurl}}/opensearch/index-alias/).
|
||||
In this sample query, `acc` is an alias for the `accounts` index:
|
||||
|
||||
```sql
|
||||
|
|
@ -191,8 +191,8 @@ Specify a condition to filter the results.
|
|||
`>=` | Greater than or equal to.
|
||||
`<=` | Less than or equal to.
|
||||
`IN` | Specify multiple `OR` operators.
|
||||
`BETWEEN` | Similar to a range query. For more information about range queries, see [Range query](../opensearch/term/#range).
|
||||
`LIKE` | Use for full text search. For more information about full-text queries, see [Full-text queries](../opensearch/full-text/).
|
||||
`BETWEEN` | Similar to a range query. For more information about range queries, see [Range query]({{site.url}}{{site.baseurl}}/opensearch/query-dsl/term#range).
|
||||
`LIKE` | Use for full text search. For more information about full-text queries, see [Full-text queries]({{site.url}}{{site.baseurl}}/opensearch/query-dsl/full-text/).
|
||||
`IS NULL` | Check if the field value is `NULL`.
|
||||
`IS NOT NULL` | Check if the field value is `NOT NULL`.
|
||||
|
||||
|
|
|
|||
|
|
@ -12,7 +12,7 @@ SQL CLI is a stand-alone Python application that you can launch with the `opense
|
|||
|
||||
Install the SQL plugin to your OpenSearch instance, run the CLI using MacOS or Linux, and connect to any valid OpenSearch end-point.
|
||||
|
||||
|
||||
|
||||
|
||||
## Features
|
||||
|
||||
|
|
@ -68,11 +68,11 @@ You can configure the following connection properties:
|
|||
- `-u/-w`: Supports username and password for HTTP basic authentication, such as with the security plugin or fine-grained access control for Amazon OpenSearch Service.
|
||||
- `--aws-auth`: Turns on AWS sigV4 authentication to connect to an Amazon OpenSearch endpoint. Use with the AWS CLI (`aws configure`) to retrieve the local AWS configuration to authenticate and connect.
|
||||
|
||||
For a list of all available configurations, see [clirc](https://github.com/opensearch-project/sql-cli/blob/master/src/conf/clirc).
|
||||
For a list of all available configurations, see [clirc](https://github.com/opensearch-project/sql/blob/main/sql-cli/src/opensearch_sql_cli/conf/clirc).
|
||||
|
||||
## Using the CLI
|
||||
|
||||
1. Save the sample [accounts test data](https://github.com/opensearch-project/sql/blob/master/src/test/resources/doctest/testdata/accounts.json) file.
|
||||
1. Save the sample [accounts test data](https://github.com/opensearch-project/sql/blob/main/doctest/test_data/accounts.json) file.
|
||||
|
||||
1. Index the sample data.
|
||||
```
|
||||
|
|
|
|||
|
|
@ -45,11 +45,11 @@ The `JOIN` clause combines columns from one or more indices using values common
|
|||
|
||||
Rule `tableSource`:
|
||||
|
||||
|
||||
|
||||
|
||||
Rule `joinPart`:
|
||||
|
||||
|
||||
|
||||
|
||||
### Example 1: Inner join
|
||||
|
||||
|
|
|
|||
|
|
@ -38,7 +38,7 @@ To use a function without a corresponding mapping, you must explicitly convert t
|
|||
|
||||
The date and time types represent a time period: `DATE`, `TIME`, `DATETIME`, `TIMESTAMP`, and `INTERVAL`. By default, the OpenSearch DSL uses the `date` type as the only date-time related type that contains all information of an absolute time point.
|
||||
|
||||
To integrate with SQL, each type other than the `timestamp` type holds part of the time period information. To use date-time functions, see [datetime](../functions#date-and-time). Some functions might have restrictions for the input argument type.
|
||||
To integrate with SQL, each type other than the `timestamp` type holds part of the time period information. To use date-time functions, see [datetime]({{site.url}}{{site.baseurl}}/search-plugins/sql/functions#date-and-time). Some functions might have restrictions for the input argument type.
|
||||
|
||||
|
||||
### Date
|
||||
|
|
|
|||
|
|
@ -29,7 +29,7 @@ PUT _plugins/_query/settings
|
|||
|
||||
Rule `singleDeleteStatement`:
|
||||
|
||||
|
||||
|
||||
|
||||
### Example
|
||||
|
||||
|
|
|
|||
|
|
@ -9,14 +9,14 @@ redirect_from: /docs/sql/
|
|||
|
||||
# SQL
|
||||
|
||||
OpenSearch SQL lets you write queries in SQL rather than the [OpenSearch query domain-specific language (DSL)](../opensearch/full-text). If you're already familiar with SQL and don't want to learn the query DSL, this feature is a great option.
|
||||
OpenSearch SQL lets you write queries in SQL rather than the [OpenSearch query domain-specific language (DSL)]({{site.url}}{{site.baseurl}}/opensearch/full-text). If you're already familiar with SQL and don't want to learn the query DSL, this feature is a great option.
|
||||
|
||||
|
||||
## Workbench
|
||||
|
||||
The easiest way to get familiar with the SQL plugin is to use **Query Workbench** in OpenSearch Dashboards to test various queries. To learn more, see [Workbench](workbench/).
|
||||
The easiest way to get familiar with the SQL plugin is to use **Query Workbench** in OpenSearch Dashboards to test various queries. To learn more, see [Workbench]({{site.url}}{{site.baseurl}}/search-plugins/sql/workbench/).
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
## REST API
|
||||
|
|
@ -72,4 +72,4 @@ See the rest of this guide for detailed information on request parameters, setti
|
|||
|
||||
## Contributing
|
||||
|
||||
To get involved and help us improve the SQL plugin, see the [development guide](https://github.com/opensearch-project/sql/blob/master/docs/developing.rst) for instructions on setting up your development environment and building the project.
|
||||
To get involved and help us improve the SQL plugin, see the [development guide](https://github.com/opensearch-project/sql/blob/main/DEVELOPER_GUIDE.rst) for instructions on setting up your development environment and building the project.
|
||||
|
|
|
|||
|
|
@ -15,17 +15,17 @@ The SQL plugin has the following limitations:
|
|||
### Select literal is not supported
|
||||
|
||||
The select literal expression is not supported. For example, `Select 1` is not supported.
|
||||
Here's a link to the Github issue - [Issue #256](https://github.com/opensearch-project/sql/issues/256).
|
||||
|
||||
|
||||
### Where clause does not support arithmetic operations
|
||||
|
||||
The `WHERE` clause does not support expressions. For example, `SELECT FlightNum FROM opensearch_dashboards_sample_data_flights where (AvgTicketPrice + 100) <= 1000` is not supported.
|
||||
Here's a link to the Github issue - [Issue #234](https://github.com/opensearch-project/sql/issues/234).
|
||||
|
||||
|
||||
### Aggregation over expression is not supported
|
||||
|
||||
You can only apply aggregation on fields, aggregations can't accept an expression as a parameter. For example, `avg(log(age))` is not supported.
|
||||
Here's a link to the Github issue - [Issue #288](https://github.com/opensearch-project/sql/issues/288).
|
||||
|
||||
|
||||
### Conflict type in multiple index query
|
||||
|
||||
|
|
@ -57,7 +57,6 @@ Error occurred in OpenSearch engine: Different mappings are not allowed for the
|
|||
"type": "VerificationException
|
||||
```
|
||||
|
||||
Here's a link to the Github issue - [Issue #445](https://github.com/opensearch-project/sql/issues/445).
|
||||
|
||||
## Subquery in the FROM clause
|
||||
|
||||
|
|
@ -77,7 +76,7 @@ But, if the outer query has `GROUP BY` or `ORDER BY`, then it's not supported.
|
|||
|
||||
The `join` query does not support aggregations on the joined result.
|
||||
For example, e.g. `SELECT depo.name, avg(empo.age) FROM empo JOIN depo WHERE empo.id == depo.id GROUP BY depo.name` is not supported.
|
||||
Here's a link to the Github issue - [Issue 110](https://github.com/opensearch-project/sql/issues/110).
|
||||
|
||||
|
||||
## Pagination only supports basic queries
|
||||
|
||||
|
|
|
|||
|
|
@ -14,11 +14,11 @@ To see basic metadata about your indices, use the `SHOW` and `DESCRIBE` commands
|
|||
|
||||
Rule `showStatement`:
|
||||
|
||||
|
||||
|
||||
|
||||
Rule `showFilter`:
|
||||
|
||||
|
||||
|
||||
|
||||
### Example 1: See metadata for indices
|
||||
|
||||
|
|
|
|||
|
|
@ -10,7 +10,7 @@ redirect_from: /docs/sql/sql-full-text/
|
|||
|
||||
Use SQL commands for full-text search. The SQL plugin supports a subset of the full-text queries available in OpenSearch.
|
||||
|
||||
To learn about full-text queries in OpenSearch, see [Full-text queries](../../opensearch/full-text/).
|
||||
To learn about full-text queries in OpenSearch, see [Full-text queries]({{site.url}}{{site.baseurl}}/opensearch/full-text/).
|
||||
|
||||
## Match
|
||||
|
||||
|
|
|
|||
|
|
@ -402,7 +402,7 @@ DELETE _opensearch/_security/api/internalusers/<username>
|
|||
|
||||
Creates or replaces the specified user. You must specify either `password` (plain text) or `hash` (the hashed user password). If you specify `password`, the security plugin automatically hashes the password before storing it.
|
||||
|
||||
Note that any role you supply in the `opensearch_security_roles` array must already exist for the security plugin to map the user to that role. To see predefined roles, refer to [the list of predefined roles](../users-roles/#predefined-roles). For instructions on how to create a role, refer to [creating a role](./#create-role).
|
||||
Note that any role you supply in the `opensearch_security_roles` array must already exist for the security plugin to map the user to that role. To see predefined roles, refer to [the list of predefined roles]({{site.url}}{{site.baseurl}}/security-plugin/access-control/users-roles#predefined-roles). For instructions on how to create a role, refer to [creating a role](#create-role).
|
||||
|
||||
#### Request
|
||||
|
||||
|
|
|
|||
|
|
@ -56,7 +56,7 @@ humanresources:
|
|||
|
||||
#### Sample role in OpenSearch Dashboards
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
## Walkthrough
|
||||
|
|
|
|||
|
|
@ -8,7 +8,7 @@ redirect_from: /docs/security/access-control/default-action-groups/
|
|||
|
||||
# Default action groups
|
||||
|
||||
This page catalogs all default action groups. Often, the most coherent way to create new action groups is to use a combination of these default groups and [individual permissions](../permissions).
|
||||
This page catalogs all default action groups. Often, the most coherent way to create new action groups is to use a combination of these default groups and [individual permissions]({{site.url}}{{site.baseurl}}/security-plugin/access-control/permissions/).
|
||||
|
||||
|
||||
## General
|
||||
|
|
|
|||
|
|
@ -10,7 +10,7 @@ redirect_from: /docs/security/access-control/document-level-security/
|
|||
|
||||
Document-level security lets you restrict a role to a subset of documents in an index. The easiest way to get started with document- and field-level security is open OpenSearch Dashboards and choose **Security**. Then choose **Roles**, create a new role, and review the **Index permissions** section.
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
## Simple roles
|
||||
|
|
|
|||
|
|
@ -8,7 +8,7 @@ redirect_from: /docs/security/access-control/field-level-security/
|
|||
|
||||
# Field-level security
|
||||
|
||||
Field-level security lets you control which document fields a user can see. Just like [document-level security](../document-level-security/), you control access by index within a role.
|
||||
Field-level security lets you control which document fields a user can see. Just like [document-level security]({{site.url}}{{site.baseurl}}/security-plugin/access-control/document-level-security/), you control access by index within a role.
|
||||
|
||||
The easiest way to get started with document- and field-level security is open OpenSearch Dashboards and choose **Security**. Then choose **Roles**, create a new role, and review the **Index permissions** section.
|
||||
|
||||
|
|
@ -96,7 +96,7 @@ someonerole:
|
|||
|
||||
### REST API
|
||||
|
||||
See [Create role](../api/#create-role).
|
||||
See [Create role]({{site.url}}{{site.baseurl}}/security-plugin/access-control/api#create-role).
|
||||
|
||||
|
||||
## Interaction with multiple roles
|
||||
|
|
@ -122,4 +122,4 @@ For example, in the `movies` index, if you include `actors`, `title`, and `year`
|
|||
|
||||
## Interaction with document-level security
|
||||
|
||||
[Document-level security](../document-level-security/) relies on OpenSearch queries, which means that all fields in the query must be visible in order for it to work properly. If you use field-level security in conjunction with document-level security, make sure you don't restrict access to the fields that document-level security uses.
|
||||
[Document-level security]({{site.url}}{{site.baseurl}}/security-plugin/access-control/document-level-security/) relies on OpenSearch queries, which means that all fields in the query must be visible in order for it to work properly. If you use field-level security in conjunction with document-level security, make sure you don't restrict access to the fields that document-level security uses.
|
||||
|
|
|
|||
|
|
@ -8,7 +8,7 @@ redirect_from: /docs/security/access-control/field-masking/
|
|||
|
||||
# Field masking
|
||||
|
||||
If you don't want to remove fields from a document using [field-level security](../field-level-security/), you can mask their values. Currently, field masking is only available for string-based fields and replaces the field's value with a cryptographic hash.
|
||||
If you don't want to remove fields from a document using [field-level security]({{site.url}}{{site.baseurl}}/security-plugin/access-control/field-level-security/), you can mask their values. Currently, field masking is only available for string-based fields and replaces the field's value with a cryptographic hash.
|
||||
|
||||
Field masking works alongside field-level security on the same per-role, per-index basis. You can allow certain roles to see sensitive fields in plain text and mask them for others. A search result with a masked field might look like this:
|
||||
|
||||
|
|
@ -70,7 +70,7 @@ someonerole:
|
|||
|
||||
### REST API
|
||||
|
||||
See [Create role](../api/#create-role).
|
||||
See [Create role]({{site.url}}{{site.baseurl}}/security-plugin/access-control/api/#create-role).
|
||||
|
||||
|
||||
## (Advanced) Use an alternative hash algorithm
|
||||
|
|
|
|||
|
|
@ -9,7 +9,7 @@ redirect_from: /docs/security/access-control/
|
|||
|
||||
# Access control
|
||||
|
||||
After you [configure the security plugin](../configuration/) to use your own certificates and preferred authentication backend, you can start adding users, creating roles, and mapping roles to users.
|
||||
After you [configure the security plugin]({{site.url}}{{site.baseurl}}/security-plugin/configuration/) to use your own certificates and preferred authentication backend, you can start adding users, creating roles, and mapping roles to users.
|
||||
|
||||
This section of the documentation covers what a user is allowed to see and do after successfully authenticating.
|
||||
|
||||
|
|
@ -18,11 +18,11 @@ This section of the documentation covers what a user is allowed to see and do af
|
|||
|
||||
Term | Description
|
||||
:--- | :---
|
||||
Permission | An individual action, such as creating an index (e.g. `indices:admin/create`). For a complete list, see [Permissions](permissions/).
|
||||
Permission | An individual action, such as creating an index (e.g. `indices:admin/create`). For a complete list, see [Permissions]({{site.url}}{{site.baseurl}}/security-plugin/access-control/permissions/).
|
||||
Action group | A set of permissions. For example, the predefined `SEARCH` action group authorizes roles to use the `_search` and `_msearch` APIs.
|
||||
Role | Security roles define the scope of a permission or action group: cluster, index, document, or field. For example, a role named `delivery_analyst` might have no cluster permissions, the `READ` action group for all indices that match the `delivery-data-*` pattern, access to all document types within those indices, and access to all fields except `delivery_driver_name`.
|
||||
Backend role | (Optional) Arbitrary strings that you specify *or* that come from an external authentication system (e.g. LDAP/Active Directory). Backend roles can help simplify the role mapping process. Rather than mapping a role to 100 individual users, you can map the role to a single backend role that all 100 users share.
|
||||
User | Users make requests to OpenSearch clusters. A user has credentials (e.g. a username and password), zero or more backend roles, and zero or more custom attributes.
|
||||
Role mapping | Users assume roles after they successfully authenticate. Role mappings, well, map roles to users (or backend roles). For example, a mapping of `kibana_user` (role) to `jdoe` (user) means that John Doe gains all the permissions of `kibana_user` after authenticating. Likewise, a mapping of `all_access` (role) to `admin` (backend role) means that any user with the backend role of `admin` gains all the permissions of `all_access` after authenticating. You can map each role to many users and/or backend roles.
|
||||
|
||||
The security plugin comes with a number of [predefined action groups](default-action-groups/), roles, mappings, and users. These entities serve as sensible defaults and are good examples of how to use the plugin.
|
||||
The security plugin comes with a number of [predefined action groups]({{site.url}}{{site.baseurl}}/security-plugin/access-control/default-action-groups/), roles, mappings, and users. These entities serve as sensible defaults and are good examples of how to use the plugin.
|
||||
|
|
|
|||
|
|
@ -80,7 +80,7 @@ To create tenants, use OpenSearch Dashboards, the REST API, or `tenants.yml`.
|
|||
|
||||
#### REST API
|
||||
|
||||
See [Create tenant](../api/#create-tenant).
|
||||
See [Create tenant]({{site.url}}{{site.baseurl}}/security-plugin/access-control/api#create-tenant).
|
||||
|
||||
|
||||
#### tenants.yml
|
||||
|
|
@ -114,7 +114,7 @@ After creating a tenant, give a role access to it using OpenSearch Dashboards, t
|
|||
|
||||
#### REST API
|
||||
|
||||
See [Create role](../api/#create-role).
|
||||
See [Create role]({{site.url}}{{site.baseurl}}/security-plugin/access-control/api#create-role).
|
||||
|
||||
|
||||
#### roles.yml
|
||||
|
|
@ -159,4 +159,4 @@ The open source version of OpenSearch Dashboards saves all objects to a single i
|
|||
The security plugin scrubs these index names of special characters, so they might not be a perfect match of tenant names and usernames.
|
||||
{: .tip }
|
||||
|
||||
To back up your OpenSearch Dashboards data, [take a snapshot](../../opensearch/snapshot-restore/) of all tenant indices using an index pattern such as `.opensearch-dashboards*`.
|
||||
To back up your OpenSearch Dashboards data, [take a snapshot]({{site.url}}{{site.baseurl}}/opensearch/snapshot-restore/) of all tenant indices using an index pattern such as `.opensearch-dashboards*`.
|
||||
|
|
|
|||
|
|
@ -10,7 +10,7 @@ redirect_from: /docs/security/access-control/permissions/
|
|||
|
||||
This page is a complete list of available permissions in the security plugin. Each permission controls access to a data type or API.
|
||||
|
||||
Rather than creating new action groups from individual permissions, you can often achieve your desired security posture using some combination of the default action groups. To learn more, see [Default Action Groups](../default-action-groups).
|
||||
Rather than creating new action groups from individual permissions, you can often achieve your desired security posture using some combination of the default action groups. To learn more, see [Default Action Groups]({{site.url}}{{site.baseurl}}/security-plugin/access-control/default-action-groups/).
|
||||
{: .tip }
|
||||
|
||||
|
||||
|
|
|
|||
|
|
@ -12,7 +12,7 @@ The security plugin includes an internal user database. Use this database in pla
|
|||
|
||||
Roles are the core way of controlling access to your cluster. Roles contain any combination of cluster-wide permissions, index-specific permissions, document- and field-level security, and tenants. Then you map users to these roles so that users gain those permissions.
|
||||
|
||||
Unless you need to create new [read-only or hidden users](../api/#read-only-and-hidden-resources), we **highly** recommend using OpenSearch Dashboards or the REST API to create new users, roles, and role mappings. The `.yml` files are for initial setup, not ongoing use.
|
||||
Unless you need to create new [read-only or hidden users]({{site.url}}{{site.baseurl}}/security-plugin/access-control/api#read-only-and-hidden-resources), we **highly** recommend using OpenSearch Dashboards or the REST API to create new users, roles, and role mappings. The `.yml` files are for initial setup, not ongoing use.
|
||||
{: .warning }
|
||||
|
||||
---
|
||||
|
|
@ -40,12 +40,12 @@ You can create users using OpenSearch Dashboards, `internal_users.yml`, or the R
|
|||
|
||||
### internal_users.yml
|
||||
|
||||
See [YAML files](../../configuration/yaml/#internal_usersyml).
|
||||
See [YAML files]({{site.url}}{{site.baseurl}}/security-plugin/configuration/yaml#internal_usersyml).
|
||||
|
||||
|
||||
### REST API
|
||||
|
||||
See [Create user](../api/#create-user).
|
||||
See [Create user]({{site.url}}{{site.baseurl}}/security-plugin/access-control/api#create-user).
|
||||
|
||||
|
||||
## Create roles
|
||||
|
|
@ -66,12 +66,12 @@ Just like users, you can create roles using OpenSearch Dashboards, `roles.yml`,
|
|||
|
||||
### roles.yml
|
||||
|
||||
See [YAML files](../../configuration/yaml/#rolesyml).
|
||||
See [YAML files]({{site.url}}{{site.baseurl}}/security-plugin/configuration/yaml#rolesyml).
|
||||
|
||||
|
||||
### REST API
|
||||
|
||||
See [Create role](../api/#create-role).
|
||||
See [Create role]({{site.url}}{{site.baseurl}}/security-plugin/access-control/api#create-role).
|
||||
|
||||
|
||||
## Map users to roles
|
||||
|
|
@ -90,12 +90,12 @@ Just like users and roles, you create role mappings using OpenSearch Dashboards,
|
|||
|
||||
### roles_mapping.yml
|
||||
|
||||
See [YAML files](../../configuration/yaml/#roles_mappingyml).
|
||||
See [YAML files]({{site.url}}{{site.baseurl}}/security-plugin/configuration/yaml#roles_mappingyml).
|
||||
|
||||
|
||||
### REST API
|
||||
|
||||
See [Create role mapping](../api/#create-role-mapping).
|
||||
See [Create role mapping]({{site.url}}{{site.baseurl}}/security-plugin/access-control/api#create-role-mapping).
|
||||
|
||||
|
||||
## Predefined roles
|
||||
|
|
@ -116,7 +116,7 @@ Role | Description
|
|||
`manage_snapshots` | Grants permissions to manage snapshot repositories, take snapshots, and restore snapshots.
|
||||
`readall` | Grants permissions for cluster-wide searches like `msearch` and search permissions for all indices.
|
||||
`readall_and_monitor` | Same as `readall`, but with added cluster monitoring permissions.
|
||||
`security_rest_api_access` | A special role that allows access to the REST API. See `plugins.security.restapi.roles_enabled` in `opensearch.yml` and [Access control for the API](../api/#access-control-for-the-api).
|
||||
`security_rest_api_access` | A special role that allows access to the REST API. See `plugins.security.restapi.roles_enabled` in `opensearch.yml` and [Access control for the API]({{site.url}}{{site.baseurl}}/security-plugin/access-control/api#access-control-for-the-api).
|
||||
`reports_read_access` | Grants permissions to generate on-demand reports, download existing reports, and view report definitions, but not to create report definitions.
|
||||
`reports_instances_read_access` | Grants permissions to generate on-demand reports and download existing reports, but not to view or create report definitions.
|
||||
`reports_full_access` | Grants full permissions to reports.
|
||||
|
|
@ -124,7 +124,7 @@ Role | Description
|
|||
`asynchronous_search_read_access` | Grants permissions to view asynchronous searches, but not to submit, modify, or delete async searches.
|
||||
|
||||
|
||||
For more detailed summaries of the permissions for each role, reference their action groups against the descriptions in [Default action groups](../default-action-groups/).
|
||||
For more detailed summaries of the permissions for each role, reference their action groups against the descriptions in [Default action groups]({{site.url}}{{site.baseurl}}/security-plugin/access-control/default-action-groups/).
|
||||
|
||||
|
||||
## Sample roles
|
||||
|
|
|
|||
|
|
@ -19,7 +19,7 @@ To enable audit logging:
|
|||
plugins.security.audit.type: internal_opensearch
|
||||
```
|
||||
|
||||
This setting stores audit logs on the current cluster. For other storage options, see [Audit Log Storage Types](storage-types/).
|
||||
This setting stores audit logs on the current cluster. For other storage options, see [Audit Log Storage Types]({{site.url}}{{site.baseurl}}/security-plugin/audit-logs/storage-types/).
|
||||
|
||||
2. Restart each node.
|
||||
|
||||
|
|
|
|||
|
|
@ -8,7 +8,7 @@ redirect_from: /docs/security/configuration/client-auth/
|
|||
|
||||
# Client certificate authentication
|
||||
|
||||
After obtaining your own certificates either from a certificate authority (CA) or by [generating your own certificates using OpenSSL](../generate-certificates), you can start configuring OpenSearch to authenticate a user using a client certificate.
|
||||
After obtaining your own certificates either from a certificate authority (CA) or by [generating your own certificates using OpenSSL]({{site.url}}{{site.baseurl}}/security-plugin/configuration/generate-certificates), you can start configuring OpenSearch to authenticate a user using a client certificate.
|
||||
|
||||
Client certificate authentication offers more security advantages than just using basic authentication (username and password). Because client certificate authentication requires both a client certificate and its private key, which are often in the user's possession, it is less vulnerable to brute force attacks in which malicious individuals try to guess a user's password.
|
||||
|
||||
|
|
@ -41,9 +41,9 @@ clientcert_auth_domain:
|
|||
|
||||
## Assigning roles to your common name
|
||||
|
||||
You can now assign your certificate's common name (CN) to a role. For this step, you must know your certificate's CN and the role you want to assign to. To get a list of all predefined roles in OpenSearch, refer to our [list of predefined roles](../../access-control/users-roles#predefined-roles). If you want to first create a role, refer to [how to create a role](../../access-control/users-roles#create-users), and then map your certificate's CN to that role.
|
||||
You can now assign your certificate's common name (CN) to a role. For this step, you must know your certificate's CN and the role you want to assign to. To get a list of all predefined roles in OpenSearch, refer to our [list of predefined roles]({{site.url}}{{site.baseurl}}/security-plugin/access-control/users-roles#predefined-roles). If you want to first create a role, refer to [how to create a role]({{site.url}}{{site.baseurl}}/security-plugin/access-control/users-roles#create-users), and then map your certificate's CN to that role.
|
||||
|
||||
After deciding which role you want to map your certificate's CN to, you can use [OpenSearch Dashboards](../../access-control/users-roles#map-users-to-roles), [`roles_mapping.yml`](../yaml/#roles_mappingyml), or the [REST API](../../access-control/api/#create-role-mapping) to map your certificate's CN to the role. The following example uses the `REST API` to map the common name `CLIENT1` to the role `readall`.
|
||||
After deciding which role you want to map your certificate's CN to, you can use [OpenSearch Dashboards]({{site.url}}{{site.baseurl}}/security-plugin/access-control/users-roles#map-users-to-roles), [`roles_mapping.yml`]({{site.url}}{{site.baseurl}}/security-plugin/configuration/yaml/#roles_mappingyml), or the [REST API]({{site.url}}{{site.baseurl}}/security-plugin/access-control/api/#create-role-mapping) to map your certificate's CN to the role. The following example uses the `REST API` to map the common name `CLIENT1` to the role `readall`.
|
||||
|
||||
**Sample request**
|
||||
|
||||
|
|
@ -110,4 +110,4 @@ output.opensearch:
|
|||
|
||||
## Using certificates with Docker
|
||||
|
||||
While we recommend using the [tarball](../../../install/tar) installation of ODFE to test client certificate authentication configurations, you can also use any of the other install types. For instructions on using Docker, for example, see [Docker security configuration](../../../install/docker-security).
|
||||
While we recommend using the [tarball]({{site.url}}{{site.baseurl}}/opensearch/install/tar) installation of ODFE to test client certificate authentication configurations, you can also use any of the other install types. For instructions on using Docker, for example, see [Docker security configuration]({{site.url}}{{site.baseurl}}/opensearch/install/docker-security).
|
||||
|
|
|
|||
|
|
@ -8,7 +8,7 @@ redirect_from: /docs/security/configuration/configuration/
|
|||
|
||||
# Backend configuration
|
||||
|
||||
One of the first steps to using the security plugin is to decide on an authentication backend, which handles [steps 2-3 of the authentication flow](../concepts/#authentication-flow). The plugin has an internal user database, but many people prefer to use an existing authentication backend, such as an LDAP server, or some combination of the two.
|
||||
One of the first steps to using the security plugin is to decide on an authentication backend, which handles [steps 2-3 of the authentication flow]({{site.url}}{{site.baseurl}}/security-plugin/configuration/concepts#authentication-flow). The plugin has an internal user database, but many people prefer to use an existing authentication backend, such as an LDAP server, or some combination of the two.
|
||||
|
||||
The main configuration file for authentication and authorization backends is `plugins/opensearch-security/securityconfig/config.yml`. It defines how the security plugin retrieves the user credentials, how it verifies these credentials, and how to fetch additional roles from backend systems (optional).
|
||||
|
||||
|
|
@ -96,7 +96,7 @@ These are the possible values for `type`:
|
|||
|
||||
- `noop`: No further authentication against any backend system is performed. Use `noop` if the HTTP authenticator has already authenticated the user completely, as in the case of JWT, Kerberos, or client certificate authentication.
|
||||
- `internal`: Use the users and roles defined in `internal_users.yml` for authentication.
|
||||
- `ldap`: Authenticate users against an LDAP server. This setting requires [additional, LDAP-specific configuration settings](../ldap/).
|
||||
- `ldap`: Authenticate users against an LDAP server. This setting requires [additional, LDAP-specific configuration settings]({{site.url}}{{site.baseurl}}/security-plugin/configuration/ldap/).
|
||||
|
||||
|
||||
## Authorization
|
||||
|
|
@ -119,7 +119,7 @@ You can define multiple entries in this section the same way as you can for auth
|
|||
These are the possible values for `type`:
|
||||
|
||||
- `noop`: Skip this step altogether.
|
||||
- `ldap`: Fetch additional roles from an LDAP server. This setting requires [additional, LDAP-specific configuration settings](../ldap/).
|
||||
- `ldap`: Fetch additional roles from an LDAP server. This setting requires [additional, LDAP-specific configuration settings]({{site.url}}{{site.baseurl}}/security-plugin/configuration/ldap/).
|
||||
|
||||
|
||||
## Examples
|
||||
|
|
|
|||
|
|
@ -16,7 +16,7 @@ plugins.security.disabled: true
|
|||
|
||||
A more permanent option is to remove the security plugin entirely. Delete the `plugins/opensearch-security` folder on all nodes, and delete the `plugins.security.*` configuration entries from `opensearch.yml`.
|
||||
|
||||
To perform these steps on the Docker image, see [Customize the Docker image](../../../opensearch/install/docker/#customize-the-docker-image).
|
||||
To perform these steps on the Docker image, see [Customize the Docker image]({{site.url}}{{site.baseurl}}/opensearch/install/docker/#customize-the-docker-image).
|
||||
|
||||
Disabling or removing the plugin exposes the configuration index for the security plugin. If the index contains sensitive information, be sure to protect it through some other means. If you no longer need the index, delete it.
|
||||
{: .warning }
|
||||
|
|
@ -26,7 +26,7 @@ Disabling or removing the plugin exposes the configuration index for the securit
|
|||
|
||||
The security plugin is actually two plugins: one for OpenSearch and one for OpenSearch Dashboards. You can use the OpenSearch plugin independently, but the OpenSearch Dashboards plugin depends on a secured OpenSearch cluster.
|
||||
|
||||
If you disable the security plugin in `opensearch.yml` (or delete the plugin entirely) and still want to use OpenSearch Dashboards, you must remove the corresponding OpenSearch Dashboards plugin. For more information, see [OpenSearch Dashboards plugin install](../../../opensearch-dashboards/install/plugins/).
|
||||
If you disable the security plugin in `opensearch.yml` (or delete the plugin entirely) and still want to use OpenSearch Dashboards, you must remove the corresponding OpenSearch Dashboards plugin. For more information, see [OpenSearch Dashboards plugin install]({{site.url}}{{site.baseurl}}/dashboards/install/plugins/).
|
||||
|
||||
|
||||
### Docker
|
||||
|
|
|
|||
|
|
@ -89,7 +89,7 @@ Just like the root certificate, use the `-days` option to specify an expiration
|
|||
|
||||
Follow the steps in [Generate an admin certificate](#generate-an-admin-certificate) with new file names to generate a new certificate for each node and as many client certificates as you need. Each certificate should use its own private key.
|
||||
|
||||
If you generate node certificates and have `plugins.security.ssl.transport.enforce_hostname_verification` set to `true` (default), be sure to specify a common name (CN) for the certificate that matches the hostname of the intended node. If you want to use the same node certificate on all nodes (not recommended), set hostname verification to `false`. For more information, see [Configure TLS certificates](../tls/#advanced-hostname-verification-and-dns-lookup).
|
||||
If you generate node certificates and have `plugins.security.ssl.transport.enforce_hostname_verification` set to `true` (default), be sure to specify a common name (CN) for the certificate that matches the hostname of the intended node. If you want to use the same node certificate on all nodes (not recommended), set hostname verification to `false`. For more information, see [Configure TLS certificates]({{site.url}}{{site.baseurl}}/security-plugin/configuration/tls/#advanced-hostname-verification-and-dns-lookup).
|
||||
|
||||
|
||||
### Sample script
|
||||
|
|
@ -170,7 +170,7 @@ This process generates many files, but these are the ones you need to add to you
|
|||
- (Optional) `each-node-cert.pem`
|
||||
- (Optional) `each-node-key.pem`
|
||||
|
||||
For information about adding and using these certificates in your own setup, see [Docker security configuration](../../../install/docker-security/) and [Configure TLS certificates](../tls/).
|
||||
For information about adding and using these certificates in your own setup, see [Docker security configuration]({{site.url}}{{site.baseurl}}//opensearch/install/docker-security/) and [Configure TLS certificates]({{site.url}}{{site.baseurl}}/security-plugin/configuration/tls/).
|
||||
|
||||
|
||||
## Run securityadmin.sh
|
||||
|
|
@ -178,13 +178,13 @@ For information about adding and using these certificates in your own setup, see
|
|||
After configuring your certificates and starting OpenSearch, run `securityadmin.sh` to initialize the security plugin:
|
||||
|
||||
```
|
||||
./securityadmin.sh -cd ../securityconfig/ -icl -nhnv -cacert ../../../config/root-ca.pem -cert ../../../config/admin.pem -key ../../../config/admin-key.pem
|
||||
./securityadmin.sh -cd {{site.url}}{{site.baseurl}}/securityconfig/ -icl -nhnv -cacert {{site.url}}{{site.baseurl}}/config/root-ca.pem -cert {{site.url}}{{site.baseurl}}/config/admin.pem -key {{site.url}}{{site.baseurl}}/config/admin-key.pem
|
||||
```
|
||||
|
||||
For more information about what this command does, see [Apply configuration changes](../security-admin/).
|
||||
For more information about what this command does, see [Apply configuration changes]({{site.url}}{{site.baseurl}}/security-plugin/configuration/security-admin/).
|
||||
{: .tip }
|
||||
|
||||
If you use Docker, see [Bash access to containers](../../../install/docker/#bash-access-to-containers).
|
||||
If you use Docker, see [Bash access to containers]({{site.url}}{{site.baseurl}}/opensearch/install/docker/#bash-access-to-containers).
|
||||
|
||||
|
||||
## OpenSearch Dashboards
|
||||
|
|
|
|||
|
|
@ -11,12 +11,12 @@ redirect_from: /docs/security/configuration/
|
|||
|
||||
The plugin includes demo certificates so that you can get up and running quickly, but before using OpenSearch in a production environment, you must configure it manually:
|
||||
|
||||
1. [Replace the demo certificates](../../opensearch/install/docker-security/)
|
||||
1. [Reconfigure opensearch.yml to use your certificates](tls/)
|
||||
1. [Reconfigure config.yml to use your authentication backend](configuration/) (if you don't plan to use the internal user database)
|
||||
1. [Modify the configuration YAML files](yaml/)
|
||||
1. [Apply changes using securityadmin.sh](security-admin/)
|
||||
1. [Replace the demo certificates]({{site.url}}{{site.baseurl}}/opensearch/install/docker-security/)
|
||||
1. [Reconfigure opensearch.yml to use your certificates]({{site.url}}{{site.baseurl}}/security-plugin/configuration/tls/)
|
||||
1. [Reconfigure config.yml to use your authentication backend]({{site.url}}{{site.baseurl}}/security-plugin/configuration/) (if you don't plan to use the internal user database)
|
||||
1. [Modify the configuration YAML files]({{site.url}}{{site.baseurl}}/security-plugin/configuration/yaml/)
|
||||
1. [Apply changes using securityadmin.sh]({{site.url}}{{site.baseurl}}/security-plugin/configuration/security-admin/)
|
||||
1. Start OpenSearch.
|
||||
1. [Add users, roles, role mappings, and tenants](../access-control/)
|
||||
1. [Add users, roles, role mappings, and tenants]({{site.url}}{{site.baseurl}}/security-plugin/access-control/)
|
||||
|
||||
If you don't want to use the plugin, see [Disable security](disable/).
|
||||
If you don't want to use the plugin, see [Disable security]({{site.url}}{{site.baseurl}}/security-plugin/configuration/disable/).
|
||||
|
|
|
|||
|
|
@ -133,7 +133,7 @@ Subjects (for example, user names) are usually stored in the `NameID` element of
|
|||
|
||||
If your IdP is compliant with the SAML 2.0 specification, you do not need to set anything special. If your IdP uses a different element name, you can also specify its name explicitly.
|
||||
|
||||
Role attributes are optional. However, most IdPs can be configured to add roles in the SAML assertions as well. If present, you can use these roles in your [role mappings](../concepts):
|
||||
Role attributes are optional. However, most IdPs can be configured to add roles in the SAML assertions as well. If present, you can use these roles in your [role mappings]({{site.url}}{{site.baseurl}}/security-plugin/configuration/concepts/):
|
||||
|
||||
```
|
||||
<saml2:Attribute Name='Role'>
|
||||
|
|
|
|||
|
|
@ -47,10 +47,10 @@ To print all available command line options, run the script with no arguments:
|
|||
To load configuration changes to the security plugin, you must provide your admin certificate to the tool:
|
||||
|
||||
```bash
|
||||
./securityadmin.sh -cd ../securityconfig/ -icl -nhnv \
|
||||
-cacert ../../../config/root-ca.pem \
|
||||
-cert ../../../config/kirk.pem \
|
||||
-key ../../../config/kirk-key.pem
|
||||
./securityadmin.sh -cd {{site.url}}{{site.baseurl}}/securityconfig/ -icl -nhnv \
|
||||
-cacert {{site.url}}{{site.baseurl}}/config/root-ca.pem \
|
||||
-cert {{site.url}}{{site.baseurl}}/config/kirk.pem \
|
||||
-key {{site.url}}{{site.baseurl}}/config/kirk-key.pem
|
||||
```
|
||||
|
||||
- The `-cd` option specifies where the security plugin configuration files to upload to the cluster can be found.
|
||||
|
|
@ -79,7 +79,7 @@ Apply configuration in `securityconfig` using PEM certificates:
|
|||
Apply configuration from a single file (`config.yml`) using PEM certificates:
|
||||
|
||||
```bash
|
||||
./securityadmin.sh -f ../securityconfig/config.yml -icl -nhnv -cert /etc/opensearch/kirk.pem -cacert /etc/opensearch/root-ca.pem -key /etc/opensearch/kirk-key.pem -t config
|
||||
./securityadmin.sh -f {{site.url}}{{site.baseurl}}/securityconfig/config.yml -icl -nhnv -cert /etc/opensearch/kirk.pem -cacert /etc/opensearch/root-ca.pem -key /etc/opensearch/kirk-key.pem -t config
|
||||
```
|
||||
|
||||
Apply configuration in `securityconfig` with keystore and truststore files:
|
||||
|
|
@ -101,7 +101,7 @@ Apply configuration in `securityconfig` with keystore and truststore files:
|
|||
You can also use keystore files in JKS format in conjunction with `securityadmin.sh`:
|
||||
|
||||
```bash
|
||||
./securityadmin.sh -cd ../securityconfig -icl -nhnv
|
||||
./securityadmin.sh -cd {{site.url}}{{site.baseurl}}/securityconfig -icl -nhnv
|
||||
-ts <path/to/truststore> -tspass <truststore password>
|
||||
-ks <path/to/keystore> -kspass <keystore password>
|
||||
```
|
||||
|
|
@ -159,13 +159,13 @@ Name | Description
|
|||
To upload all configuration files in a directory, use this:
|
||||
|
||||
```bash
|
||||
./securityadmin.sh -cd ../securityconfig -ts ... -tspass ... -ks ... -kspass ...
|
||||
./securityadmin.sh -cd {{site.url}}{{site.baseurl}}/securityconfig -ts ... -tspass ... -ks ... -kspass ...
|
||||
```
|
||||
|
||||
If you want to push a single configuration file, use this:
|
||||
|
||||
```bash
|
||||
./securityadmin.sh -f ../securityconfig/internal_users.yml -t internalusers \
|
||||
./securityadmin.sh -f {{site.url}}{{site.baseurl}}/securityconfig/internal_users.yml -t internalusers \
|
||||
-ts ... -tspass ... -ks ... -kspass ...
|
||||
```
|
||||
|
||||
|
|
@ -199,7 +199,7 @@ You can download all current configuration files from your cluster with the foll
|
|||
This command dumps the current security plugin configuration from your cluster to individual files in the directory you specify. You can then use these files as backups or to load the configuration into a different cluster. This command is useful when moving a proof-of-concept to production:
|
||||
|
||||
```bash
|
||||
./securityadmin.sh -backup ~ -icl -nhnv -cacert ../../../config/root-ca.pem -cert ../../../config/kirk.pem -key ../../../config/kirk-key.pem
|
||||
./securityadmin.sh -backup ~ -icl -nhnv -cacert {{site.url}}{{site.baseurl}}/config/root-ca.pem -cert {{site.url}}{{site.baseurl}}/config/kirk.pem -key {{site.url}}{{site.baseurl}}/config/kirk-key.pem
|
||||
```
|
||||
|
||||
To upload the dumped files to another cluster:
|
||||
|
|
@ -211,7 +211,7 @@ To upload the dumped files to another cluster:
|
|||
To migrate configuration YAML files from the OpenSearch 0.x.x format to the 1.x.x format:
|
||||
|
||||
```bash
|
||||
./securityadmin.sh -migrate ../securityconfig -ts ... -tspass ... -ks ... -kspass ...
|
||||
./securityadmin.sh -migrate {{site.url}}{{site.baseurl}}/securityconfig -ts ... -tspass ... -ks ... -kspass ...
|
||||
```
|
||||
|
||||
Name | Description
|
||||
|
|
|
|||
|
|
@ -8,7 +8,7 @@ redirect_from: /docs/security/configuration/system-indices/
|
|||
|
||||
# System indices
|
||||
|
||||
By default, OpenSearch has a protected system index, `.opensearch_security`, which you create using [securityadmin.sh](../security-admin/). Even if your user account has read permissions for all indices, you can't directly access the data in this system index.
|
||||
By default, OpenSearch has a protected system index, `.opensearch_security`, which you create using [securityadmin.sh]({{site.url}}{{site.baseurl}}/security-plugin/configuration/security-admin/). Even if your user account has read permissions for all indices, you can't directly access the data in this system index.
|
||||
|
||||
You can add additional system indices in in `opensearch.yml`. In addition to automatically creating `.opensearch_security`, the demo configuration adds several indices for the various OpenSearch plugins that integrate with the security plugin:
|
||||
|
||||
|
|
@ -17,7 +17,7 @@ plugins.security.system_indices.enabled: true
|
|||
plugins.security.system_indices.indices: [".opendistro-alerting-config", ".opendistro-alerting-alert*", ".opendistro-anomaly-results*", ".opendistro-anomaly-detector*", ".opendistro-anomaly-checkpoints", ".opendistro-anomaly-detection-state", ".opendistro-reports-*", ".opendistro-notifications-*", ".opendistro-notebooks", ".opendistro-asynchronous-search-response*"]
|
||||
```
|
||||
|
||||
To access these indices, you must authenticate with an [admin certificate](../tls/#configure-admin-certificates):
|
||||
To access these indices, you must authenticate with an [admin certificate]({{site.url}}{{site.baseurl}}/security-plugin/configuration/tls/#configure-admin-certificates):
|
||||
|
||||
```bash
|
||||
curl -k --cert ./kirk.pem --key ./kirk-key.pem -XGET 'https://localhost:9200/.opensearch_security/_search'
|
||||
|
|
|
|||
|
|
@ -10,7 +10,7 @@ redirect_from: /docs/security/configuration/tls/
|
|||
|
||||
TLS is configured in `opensearch.yml`. There are two main configuration sections: the transport layer and the REST layer. TLS is optional for the REST layer and mandatory for the transport layer.
|
||||
|
||||
You can find an example configuration template with all options on [GitHub](https://www.github.com/opensearch-project/security-ssl/blob/master/opensearchsecurity-ssl-config-template.yml).
|
||||
You can find an example configuration template with all options on [GitHub](https://github.com/opensearch-project/security/blob/main/securityconfig/opensearch.yml.example).
|
||||
{: .note }
|
||||
|
||||
|
||||
|
|
@ -115,7 +115,6 @@ Name | Description
|
|||
`plugins.security.ssl.transport.enable_openssl_if_available` | Enable OpenSSL on the transport layer if available. Optional. Default is true.
|
||||
`plugins.security.ssl.http.enable_openssl_if_available` | Enable OpenSSL on the REST layer if available. Optional. Default is true.
|
||||
|
||||
|
||||
{% comment %}
|
||||
1. Install [OpenSSL 1.1.0](https://www.openssl.org/community/binaries.html) on every node.
|
||||
1. Install [Apache Portable Runtime](https://apr.apache.org) on every node:
|
||||
|
|
@ -125,8 +124,6 @@ Name | Description
|
|||
```
|
||||
{% endcomment %}
|
||||
|
||||
1. Download the statically-linked JAR that includes OpenSSL, Apache Portable Runtime, and `netty-tcnative` for [RPM-based distributions](https://bintray.com/floragunncom/netty-tcnative/download_file?file_path=netty-tcnative-openssl-1.1.0j-static-2.0.20.Final-fedora-linux-x86_64.jar) or [other distributions](https://bintray.com/floragunncom/netty-tcnative/download_file?file_path=netty-tcnative-openssl-1.1.0j-static-2.0.20.Final-non-fedora-linux-x86_64.jar) and place it in `plugins/opensearch-security/` on every node.
|
||||
|
||||
|
||||
## (Advanced) Hostname verification and DNS lookup
|
||||
|
||||
|
|
|
|||
|
|
@ -10,14 +10,14 @@ redirect_from: /docs/security/configuration/yaml/
|
|||
|
||||
Before running `securityadmin.sh` to load the settings into the `.opensearch_security` index, configure the YAML files in `plugins/opensearch-security/securityconfig`. You might want to back up these files so that you can reuse them on other clusters.
|
||||
|
||||
The best use of these YAML files is to configure [reserved and hidden resources](../../access-control/api/#reserved-and-hidden-resources), such as the `admin` and `kibanaserver` users. You might find it easier to create other users, roles, mappings, action groups, and tenants using OpenSearch Dashboards or the REST API.
|
||||
The best use of these YAML files is to configure [reserved and hidden resources]({{site.url}}{{site.baseurl}}/security-plugin/access-control/api#reserved-and-hidden-resources), such as the `admin` and `kibanaserver` users. You might find it easier to create other users, roles, mappings, action groups, and tenants using OpenSearch Dashboards or the REST API.
|
||||
|
||||
|
||||
## internal_users.yml
|
||||
|
||||
This file contains any initial users that you want to add to the security plugin's internal user database.
|
||||
|
||||
The file format requires a hashed password. To generate one, run `plugins/opensearch-security/tools/hash.sh -p <new-password>`. If you decide to keep any of the demo users, *change their passwords* and re-run [securityadmin.sh](../security-admin/) to apply the new passwords.
|
||||
The file format requires a hashed password. To generate one, run `plugins/opensearch-security/tools/hash.sh -p <new-password>`. If you decide to keep any of the demo users, *change their passwords* and re-run [securityadmin.sh]({{site.url}}{{site.baseurl}}/security-plugin/configuration/security-admin/) to apply the new passwords.
|
||||
|
||||
```yml
|
||||
---
|
||||
|
|
|
|||
|
|
@ -61,7 +61,7 @@ Some IdPs do not sign the SAML documents by default. Make sure the IdP signs all
|
|||
|
||||
#### Keycloak
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
## Role settings
|
||||
|
|
|
|||
|
|
@ -92,7 +92,7 @@ Connected as CN=node-0.example.com,OU=SSL,O=Test,L=Test,C=DE
|
|||
ERR: CN=node-0.example.com,OU=SSL,O=Test,L=Test,C=DE is not an admin user
|
||||
```
|
||||
|
||||
You must use an admin certificate when executing the script. To learn more, see [Configure admin certificates](../../security/configuration/tls/#configure-admin-certificates).
|
||||
You must use an admin certificate when executing the script. To learn more, see [Configure admin certificates]({{site.url}}{{site.baseurl}}/security-plugin/configuration/tls/#configure-admin-certificates).
|
||||
|
||||
|
||||
## Use the diagnose option
|
||||
|
|
@ -100,7 +100,7 @@ You must use an admin certificate when executing the script. To learn more, see
|
|||
For more information on why `securityadmin.sh` is not executing, add the `--diagnose` option:
|
||||
|
||||
```
|
||||
./securityadmin.sh -diagnose -cd ../securityconfig/ -cacert ... -cert ... -key ... -keypass ...
|
||||
./securityadmin.sh -diagnose -cd {{site.url}}{{site.baseurl}}/securityconfig/ -cacert ... -cert ... -key ... -keypass ...
|
||||
```
|
||||
|
||||
The script prints the location of the generated diagnostic file.
|
||||
|
|
|
|||
|
|
@ -2,4 +2,4 @@
|
|||
# Run `bundle exec jekyll serve` first.
|
||||
# Uses https://github.com/stevenvachon/broken-link-checker
|
||||
# I have no idea why we have to exclude the ISM section, but that's the only way I can get this to run. - ae
|
||||
blc http://127.0.0.1:4000/ -ro
|
||||
blc http://localhost:4000 -ro --exclude "*opensearch.org/*" --exclude "*github.com/opensearch-project/documentation-website/*" --exclude "*apache.org*" --exclude "https://localhost:5601/"
|
||||
|
|
|
|||
24
index.md
24
index.md
|
|
@ -26,18 +26,18 @@ OpenSearch is well-suited to the following use cases:
|
|||
|
||||
Component | Purpose
|
||||
:--- | :---
|
||||
[OpenSearch](opensearch/) | Data store and search engine
|
||||
[OpenSearch Dashboards](opensearch-dashboards/) | Search frontend and visualizations
|
||||
[Security](security-plugin/) | Authentication and access control for your cluster
|
||||
[Alerting](monitoring-plugins/alerting/) | Receive notifications when your data meets certain conditions
|
||||
[SQL](search-plugins/sql/) | Use SQL or a piped processing language to query your data
|
||||
[Index State Management](im-plugin/) | Automate index operations
|
||||
[KNN](search-plugins/knn/) | Find “nearest neighbors” in your vector data
|
||||
[Performance Analyzer](monitoring-plugins/pa/) | Monitor and optimize your cluster
|
||||
[Anomaly Detection](monitoring-plugins/ad/) | Identify atypical data and receive automatic notifications
|
||||
[Asynchronous Search](search-plugins/async/) | Run search requests in the background
|
||||
[OpenSearch]({{site.url}}{{site.baseurl}}/opensearch/) | Data store and search engine
|
||||
[OpenSearch Dashboards]({{site.url}}{{site.baseurl}}/dashboards/) | Search frontend and visualizations
|
||||
[Security]({{site.url}}{{site.baseurl}}/security-plugin/) | Authentication and access control for your cluster
|
||||
[Alerting]({{site.url}}{{site.baseurl}}/monitoring-plugins/alerting/) | Receive notifications when your data meets certain conditions
|
||||
[SQL]({{site.url}}{{site.baseurl}}/search-plugins/sql/) | Use SQL or a piped processing language to query your data
|
||||
[Index State Management]({{site.url}}{{site.baseurl}}/im-plugin/) | Automate index operations
|
||||
[KNN]({{site.url}}{{site.baseurl}}/search-plugins/knn/) | Find “nearest neighbors” in your vector data
|
||||
[Performance Analyzer]({{site.url}}{{site.baseurl}}/monitoring-plugins/pa/) | Monitor and optimize your cluster
|
||||
[Anomaly Detection]({{site.url}}{{site.baseurl}}/monitoring-plugins/ad/) | Identify atypical data and receive automatic notifications
|
||||
[Asynchronous Search]({{site.url}}{{site.baseurl}}/search-plugins/async/) | Run search requests in the background
|
||||
|
||||
You can install OpenSearch plugins [individually](opensearch/install/plugins/) or use the [all-in-one packages](opensearch/install/). Most of these OpenSearch plugins have corresponding OpenSearch Dashboards plugins that provide a convenient, unified user interface.
|
||||
You can install OpenSearch plugins [individually]({{site.url}}{{site.baseurl}}/opensearch/install/plugins/) or use the [all-in-one packages]({{site.url}}{{site.baseurl}}/opensearch/install/). Most of these OpenSearch plugins have corresponding OpenSearch Dashboards plugins that provide a convenient, unified user interface.
|
||||
|
||||
For specifics around the project, see the [FAQ](https://opensearch.org/faq/).
|
||||
|
||||
|
|
@ -62,7 +62,7 @@ Docker
|
|||
curl -XGET --insecure https://localhost:9200 -u admin:admin
|
||||
```
|
||||
|
||||
To learn more, see [Install OpenSearch](opensearch/install/) and [Install OpenSearch Dashboards](dashboards/install/).
|
||||
To learn more, see [Install OpenSearch]({{site.url}}{{site.baseurl}}/opensearch/install/) and [Install OpenSearch Dashboards]({{site.url}}{{site.baseurl}}/dashboards/install/).
|
||||
|
||||
|
||||
---
|
||||
|
|
|
|||
Loading…
Reference in New Issue