Merge branch 'main' into ga

This commit is contained in:
aetter 2021-07-11 18:19:26 -07:00
commit 1e031e8362
16 changed files with 326 additions and 30 deletions

View File

@ -0,0 +1,41 @@
---
layout: default
title: Configure TLS
parent: Install OpenSearch Dashboards
nav_order: 40
---
# Configure TLS for OpenSearch Dashboards
By default, for ease of testing and getting started, OpenSearch Dashboards runs over HTTP. To enable TLS for HTTPS, update the following settings in `opensearch_dashboards.yml`.
Setting | Description
:--- | :---
opensearch.ssl.verificationMode | This setting is for communications between OpenSearch and OpenSearch Dashboards. Valid values are `full`, `certificate`, or `none`. We recommend `full` if you enable TLS, which enables hostname verification. `certificate` just checks the certificate, not the hostname, and `none` performs no checks (suitable for HTTP). Default is `full`.
opensearch.ssl.certificateAuthorities | If `opensearch.ssl.verificationMode` is `full` or `certificate`, specify the full path (e.g. `[ "/usr/share/opensearch-dashboards-1.0.0/config/root-ca.pem" ]` to the certificate authority for your OpenSearch cluster.
server.ssl.enabled | This setting is for communications between OpenSearch Dashboards and the web browser. Set to true for HTTPS, false for HTTP.
server.ssl.certificate | If `server.ssl.enabled` is true, specify the full path (e.g. `/usr/share/opensearch-dashboards-1.0.0/config/my-client-cert.pem` to a valid client certificate for your OpenSearch cluster. You can [generate your own]({{site.url}}{{site.baseurl}}/security-plugin/configuration/generate-certificates/) or get one from a certificate authority.
server.ssl.key | If `server.ssl.enabled` is true, specify the full path (e.g. `/usr/share/opensearch-dashboards-1.0.0/config/my-client-cert-key.pem` to the key for your client certificate. You can [generate your own]({{site.url}}{{site.baseurl}}/security-plugin/configuration/generate-certificates/) or get one from a certificate authority.
opensearch_security.cookie.secure | If you enable TLS for OpenSearch Dashboards, change this setting to `true`. For HTTP, set it to `false`.
This `opensearch_dashboards.yml` configuration shows OpenSearch and OpenSearch Dashboards running on the same machine with the demo configuration:
```yml
opensearch.hosts: ["https://localhost:9200"]
opensearch.ssl.verificationMode: full
opensearch.username: "kibanaserver"
opensearch.password: "kibanaserver"
opensearch.requestHeadersWhitelist: [ authorization,securitytenant ]
server.ssl.enabled: true
server.ssl.certificate: /usr/share/opensearch-1.0.0/config/client-cert.pem
server.ssl.key: /usr/share/opensearch-1.0.0/config/client-cert-key.pem
opensearch.ssl.certificateAuthorities: [ "/usr/share/opensearch-1.0.0/config/root-ca.pem" ]
opensearch_security.multitenancy.enabled: true
opensearch_security.multitenancy.tenants.preferred: ["Private", "Global"]
opensearch_security.readonly_mode.roles: ["kibana_read_only"]
opensearch_security.cookie.secure: true
```
If you use the Docker install, you can pass a custom `opensearch_dashboards.yml` to the container. To learn more, see the [Docker installation page]({{site.url}}{{site.baseurl}}/opensearch/install/docker/).
After enabling these settings and starting OpenSearch Dashboards, you can connect to it at `https://localhost:5601`. You might have to acknowledge a browser warning if your certificates are self-signed. To avoid this sort of warning (or outright browser incompatibility), best practice is to use certificates from trusted certificate authority.

View File

@ -1,28 +1,30 @@
---
layout: default
title: Notebooks (experimental)
title: Notebooks
nav_order: 50
redirect_from: /docs/notebooks/
has_children: false
---
# OpenSearch Dashboards notebooks (experimental)
# Notebooks
Notebooks have a known issue with [tenants]({{site.url}}{{site.baseurl}}/security-plugin/access-control/multi-tenancy/). If you open a notebook and can't see its visualizations, you might be under the wrong tenant, or you might not have access to the tenant at all.
{: .warning }
An OpenSearch Dashboards notebook is an interface that lets you easily combine live visualizations and narrative text in a single notebook interface.
An OpenSearch Dashboards notebook is an interface that lets you easily combine code snippets, live visualizations, and narrative text in a single notebook interface.
Notebooks let you interactively explore data by running different visualizations that you can share with team members to collaborate on a project.
A notebook is a document composed of two elements: OpenSearch Dashboards visualizations and paragraphs (Markdown). Choose multiple timelines to compare and contrast visualizations.
A notebook is a document composed of two elements: code blocks (Markdown/SQL/PPL) and visualizations. Choose multiple timelines to compare and contrast visualizations.
You can also generate [reports]({{site.url}}{{site.baseurl}}/dashboards/reporting/) directly from your notebooks.
Common use cases include creating postmortem reports, designing runbooks, building live infrastructure reports, and writing documentation.
Tenants in OpenSearch Dashboards are spaces for saving notebooks and other OpenSearch Dashboards objects. For more information, see [OpenSearch Dashboards multi-tenancy]({{site.url}}{{site.baseurl}}/security-plugin/access-control/multi-tenancy/).
{: .note }
## Get Started with notebooks
To get started, choose **OpenSearch Dashboards Notebooks** within OpenSearch Dashboards.
## Get started with notebooks
To get started, choose **Notebooks** within OpenSearch Dashboards.
### Step 1: Create a notebook
@ -32,28 +34,58 @@ A notebook is an interface for creating reports.
1. Choose **Create notebook** and enter a descriptive name.
1. Choose **Create**.
Choose **Notebook actions** to rename, duplicate, or delete a notebook.
Choose **Actions** to rename, duplicate, or delete a notebook.
### Step 2: Add a paragraph
Paragraphs combine text and visualizations for describing data.
Paragraphs combine code blocks and visualizations for describing data.
#### Add a code block
#### Add a markdown paragraph
Code blocks support markdown, SQL, and PPL languages.
1. To add text, choose **Add markdown paragraph**.
1. Add rich text with markdown syntax.
Specify the input language on the first line using `%[language type]` syntax.
For example, type `%md` for markdown, `%sql` for SQL, and `%ppl` for PPL.
##### Sample markdown block
```
%md
Add in text formatted in markdown.
```
![Markdown paragraph]({{site.url}}{{site.baseurl}}/images/markdown-notebook.png)
##### Sample SQL block
#### Add a visualization paragraph
```sql
%sql
Select * from opensearch_dashboards_sample_data_flights limit 20;
```
1. To add a visualization, choose **Add OpenSearch Dashboards visualization paragraph**.
![SQL paragraph]({{site.url}}{{site.baseurl}}/images/sql-notebook.png)
##### Sample PPL block
```
%ppl
source=opensearch_dashboards_sample_data_logs | head 20
```
![PPL paragraph]({{site.url}}{{site.baseurl}}/images/ppl-notebook.png)
#### Add a visualization
1. To add a visualization, choose **Add paragraph** and select **Visualization**.
1. In **Title**, select your visualization and choose a date range. You can choose multiple timelines to compare and contrast visualizations.
1. To run and save a paragraph, choose **Run**.
![Visualization paragraph]({{site.url}}{{site.baseurl}}/images/visualization-notebook.png)
## Paragraph actions
You can perform the following actions on paragraphs:
- Add a new paragraph to the top of a report.
@ -62,3 +94,25 @@ You can perform the following actions on paragraphs:
- Clear the outputs of all paragraphs.
- Delete all the paragraphs.
- Move paragraphs up and down.
## Sample notebooks
We prepared the following sample notebooks that showcase a variety of use cases:
- Using SQL to query the OpenSearch Dashboards sample flight data.
- Using PPL to query the OpenSearch Dashboards sample web logs data.
- Using PPL and visualizations to perform sample root cause event analysis on the OpenSearch Dashboards sample web logs data.
To add a sample notebook, choose **Actions** and select **Add sample notebooks**.
## Create a report
You can use notebooks to create PNG and PDF reports:
1. From the top menu bar, choose **Reporting actions**.
1. You can choose to **Download PDF** or **Download PNG**.
Reports generate asynchronously in the background and might take a few minutes, depending on the size of the report. A notification appears when your report is ready to download.
1. To create a schedule-based report, choose **Create report definition**. For steps to create a report definition, see [Create reports using a definition]({{site.url}}{{site.baseurl}}/dashboards/reporting#create-reports-using-a-definition).
1. To see all your reports, choose **View all reports**.

View File

@ -0,0 +1,19 @@
---
layout: default
title: Compatibility
parent: Install OpenSearch
nav_order: 2
---
# Operating system and JVM compatibility
- We recommend installing OpenSearch on RHEL- or Debian-based Linux distributions that use [systemd](https://en.wikipedia.org/wiki/Systemd), such as CentOS, Amazon Linux 2, and Ubuntu (LTS). OpenSearch should work on many Linux distributions, but we only test a handful.
- The OpenSearch tarball ships with a compatible version of Java in the `jdk` directory. To find its version, run `./jdk/bin/java -version`. For example, the OpenSearch 1.0.0 tarball ships with Java 15 (non-LTS).
{% comment %}`./jdk/bin/java -version` doesn't work on macOS with zsh at the moment, and I have no idea why. Maybe we need a macOS artifact. Regardless, the command works on Amazon Linux 2 with bash and presumably other distros. - aetter{% endcomment %}
To use a different Java installation, set the `OPENSEARCH_JAVA_HOME` environment variable to the Java install location. We recommend Java 11 (LTS), but OpenSearch also works with Java 8.
OpenSearch version | Compatible Java versions | Recommended operating systems
:--- | :--- | :---
1.x | 8, 11 | Red Hat Enterprise Linux 7, 8; CentOS 7, 8; Amazon Linux 2; Ubuntu 16.04, 18.04, 20.04

View File

@ -2,7 +2,7 @@
layout: default
title: Docker
parent: Install OpenSearch
nav_order: 1
nav_order: 3
---
# Docker image
@ -16,7 +16,7 @@ docker pull opensearchproject/opensearch-dashboards:{{site.opensearch_version}}
To check available versions, see [Docker Hub](https://hub.docker.com/u/opensearchproject).
OpenSearch images use `centos:7` as the base image. If you run Docker locally, we recommend allowing Docker to use at least 4 GB of RAM in **Preferences** > **Resources**.
OpenSearch images use `amazonlinux:2` as the base image. If you run Docker locally, set Docker to use at least 4 GB of RAM in **Preferences** > **Resources**.
---

View File

@ -11,7 +11,7 @@ The OpenSearch logs include valuable information for monitoring cluster operatio
- On Docker, OpenSearch writes most logs to the console and stores the remainder in `opensearch/logs/`. The tarball installation also uses `opensearch/logs/`.
- On most Linux installations, OpenSearch writes logs to `/var/log/opensearch/`.
Logs are available as `.log` (plain text) and `.json` files.
Logs are available as `.log` (plain text) and `.json` files. Permissions for the OpenSearch logs are `-rw-r--r--` by default, meaning that any user account on the node can read them. You can change this behavior _for each log type_ in `log4j2.properties` using the `filePermissions` option. For example, you might add `appender.rolling.filePermissions = rw-r-----` to change permissions for the JSON server log. For details, see the [Log4j 2 documentation](https://logging.apache.org/log4j/2.x/manual/appenders.html#RollingFileAppender).
## Application logs

View File

@ -1,8 +1,9 @@
---
layout: default
title: Bulk
parent: REST API reference
nav_order: 5
parent: Document APIs
grand_parent: REST API reference
nav_order: 20
---
# Bulk
@ -32,7 +33,7 @@ POST _bulk
POST {index}/_bulk
```
Specifying the index in the path means you don't need to include it in the [request body](#request-body).
Specifying the index in the path means you don't need to include it in the [request body]({{site.url}}{{site.baseurl}}/opensearch/rest-api/document-apis/bulk/#request-body).
OpenSearch also accepts PUT requests to the `_bulk` path, but we highly recommend using POST. The accepted usage of PUT---adding or replacing a single resource at a given path---doesn't make sense for bulk requests.
{: .note }

View File

@ -0,0 +1,77 @@
---
layout: default
title: Get document
parent: Document APIs
grand_parent: REST API reference
nav_order: 5
---
# Get document
After adding a JSON document to your index, you can use the get document API operation to retrieve the document's information and data.
## Example
```json
GET sample-index1/_doc/1
```
## Path and HTTP methods
```
GET <index>/_doc/<_id>
HEAD <index>/_doc/<_id>
```
```
GET <index>/_source/<_id>
HEAD <index>/_source/<_id>
```
## URL parameters
All get document URL parameters are optional.
Parameter | Type | Description
:--- | :--- | :---
preference | string | Specifies a preference of which shard to retrieve results from. Available options are `_local`, which tells the operation to retrieve results from a locally allocated shard replica, and a custom string value assigned to a specific shard replica. By default, OpenSearch executes get document operations on random shards.
realtime | boolean | Specifies whether the operation should run in realtime. If false, the operation waits for the index to refresh to analyze the source to retrieve data, which makes the operation near-realtime. Default is true.
refresh | boolean | If true, OpenSearch refreshes shards to make the operation visible to searching. Default is false.
routing | string | A value used to route the operation to a specific shard.
stored_fields | boolean | If true, the operation retrieves document fields stored in the index rather than the document's `_source`. Default is false.
_source | string | Whether to include the `_source` field in the response body. Default is true.
_source_excludes | string | A comma-separated list of source fields to exclude in the query response.
_source_includes | string | A comma-separated list of source fields to include in the query response.
version | integer | The version of the document to return, which must match the current version of the document.
version_type | enum | Retrieves a specifically typed document. Available options are `external` (retrieve the document if the specified version number is greater than the document's current version) and `external_gte` (retrieve the document if the specified version number is greater than or equal to the document's current verison). For example, to retrieve version 3 of a document, use `/_doc/1?version=3&version_type=external`.
## Response
```json
{
"_index": "sample-index1",
"_type": "_doc",
"_id": "1",
"_version": 1,
"_seq_no": 0,
"_primary_term": 9,
"found": true,
"_source": {
"text": "This is just some sample text."
}
}
```
## Response body fields
Field | Description
:--- | :---
_index | The name of the index.
_type | The document's type. OpenSearch only supports one type, which is `_doc`.
_id | The document's id.
_version | The document's version number. Updated whenever the document changes.
_seq_no | The sequnce number assigned when the document is indexed.
primary_term | The primary term assigned when the document is indexed.
found | Whether the document exists.
_routing | The shard that the document is routed to. If the document is not routed to a particular shard, this field is omitted.
_source | Contains the document's data if `found` is true. If `_source` is set to false or `stored_fields` is set to true in the URL parameters, this field is omitted.
_fields | Contains the document's data that's stored in the index. Only returned if both `stored_fields` and `found` are true.

View File

@ -0,0 +1,28 @@
---
layout: default
title: Document APIs
parent: REST API reference
has_children: true
nav_order: 7
---
# Document APIs
The document APIs allow you to handle documents relative to your index, such as adding, updating, and deleting documents.
Document APIs are separated into two categories: single document operations and multi-document operations. Multi-document operations offer performance advantages over submitting many individual requests, so whenever practical, we recommend that you use multi-document operations.
## Single document operations
- Index
- Get
- Delete
- Update
## Multi-document operations
- Bulk
- Multi get
- Delete by query
- Update by query
- Reindex

View File

@ -13,12 +13,13 @@ redirect_from:
The plugin includes demo certificates so that you can get up and running quickly, but before using OpenSearch in a production environment, you must configure it manually:
1. [Replace the demo certificates]({{site.url}}{{site.baseurl}}/opensearch/install/docker-security)
1. [Reconfigure opensearch.yml to use your certificates]({{site.url}}{{site.baseurl}}/security-plugin/configuration/tls)
1. [Reconfigure config.yml to use your authentication backend]({{site.url}}{{site.baseurl}}/security-plugin/configuration/configuration/) (if you don't plan to use the internal user database)
1. [Modify the configuration YAML files]({{site.url}}{{site.baseurl}}/security-plugin/configuration/yaml)
1. [Apply changes using securityadmin.sh]({{site.url}}{{site.baseurl}}/security-plugin/configuration/security-admin)
1. [Replace the demo certificates]({{site.url}}{{site.baseurl}}/opensearch/install/docker-security).
1. [Reconfigure opensearch.yml to use your certificates]({{site.url}}{{site.baseurl}}/security-plugin/configuration/tls).
1. [Reconfigure config.yml to use your authentication backend]({{site.url}}{{site.baseurl}}/security-plugin/configuration/configuration/) (if you don't plan to use the internal user database).
1. [Modify the configuration YAML files]({{site.url}}{{site.baseurl}}/security-plugin/configuration/yaml).
1. If you plan to use the internal user database, [set a password policy in opensearch.yml]({{site.url}}{{site.baseurl}}/security-plugin/configuration/yaml/#opensearchyml).
1. [Apply changes using securityadmin.sh]({{site.url}}{{site.baseurl}}/security-plugin/configuration/security-admin).
1. Start OpenSearch.
1. [Add users, roles, role mappings, and tenants]({{site.url}}{{site.baseurl}}/security-plugin/access-control/index/)
1. [Add users, roles, role mappings, and tenants]({{site.url}}{{site.baseurl}}/security-plugin/access-control/index/).
If you don't want to use the plugin, see [Disable security]({{site.url}}{{site.baseurl}}/security-plugin/configuration/disable).

View File

@ -89,6 +89,42 @@ snapshotrestore:
description: "Demo snapshotrestore user"
```
## opensearch.yml
In addition to many OpenSearch settings, this file contains paths to TLS certificates and their attributes, such as distinguished names and trusted certificate authorities.
```yml
plugins.security.ssl.transport.pemcert_filepath: esnode.pem
plugins.security.ssl.transport.pemkey_filepath: esnode-key.pem
plugins.security.ssl.transport.pemtrustedcas_filepath: root-ca.pem
plugins.security.ssl.transport.enforce_hostname_verification: false
plugins.security.ssl.http.enabled: true
plugins.security.ssl.http.pemcert_filepath: esnode.pem
plugins.security.ssl.http.pemkey_filepath: esnode-key.pem
plugins.security.ssl.http.pemtrustedcas_filepath: root-ca.pem
plugins.security.allow_unsafe_democertificates: true
plugins.security.allow_default_init_securityindex: true
plugins.security.authcz.admin_dn:
- CN=kirk,OU=client,O=client,L=test, C=de
plugins.security.audit.type: internal_opensearch
plugins.security.enable_snapshot_restore_privilege: true
plugins.security.check_snapshot_restore_write_privileges: true
plugins.security.restapi.roles_enabled: ["all_access", "security_rest_api_access"]
plugins.security.system_indices.enabled: true
plugins.security.system_indices.indices: [".opendistro-alerting-config", ".opendistro-alerting-alert*", ".opendistro-anomaly-results*", ".opendistro-anomaly-detector*", ".opendistro-anomaly-checkpoints", ".opendistro-anomaly-detection-state", ".opendistro-reports-*", ".opendistro-notifications-*", ".opendistro-notebooks", ".opendistro-asynchronous-search-response*"]
node.max_local_storage_nodes: 3
```
If you want to run your users' passwords against some validation, specify a regular expression (regex) in this file. You can also include an error message that loads when passwords don't pass validation. The following example demonstrates how to include a regex so OpenSearch requires new passwords to be a minimum of eight characters with at least one uppercase, one lowercase, one digit, and one special character.
Note that OpenSearch validates only users and passwords created through OpenSearch Dashboards or the REST API.
```yml
plugins.restapi.password_validation_regex: '(?=.*[A-Z])(?=.*[^a-zA-Z\d])(?=.*[0-9])(?=.*[a-z]).{8,}'
plugins.restapi.password_validation_error_message: "Password must be minimum 8 characters long and must contain at least one uppercase letter, one lowercase letter, one digit, and one special character."
```
## roles.yml

View File

@ -0,0 +1,36 @@
---
layout: default
title: Upgrade from Kibana OSS to OpenSearch Dashboards
nav_order: 50
redirect_from:
- /migrate/dashboards/
---
# Upgrade from Kibana OSS to OpenSearch Dashboards
Kibana OSS stores its visualizations and dashboards in one or more indices (`.kibana*`) on the Elasticsearch OSS cluster. As such, the most important step is to leave those indices intact as you upgrade from Elasticsearch OSS to OpenSearch.
Consider exporting all Kibana objects prior to starting the upgrade. In Kibana, choose **Stack Management**, **Saved Objects**, **Export objects**.
{: .tip }
1. After you upgrade your Elasticsearch OSS cluster to OpenSearch, stop Kibana.
1. Extract the OpenSearch Dashboards tarball to a new directory.
1. Port your settings from `<kibana-dir>/config/kibana.yml` to `<dashboards-dir>/config/opensearch_dashboards.yml`.
In general, settings with `elasticsearch` in their names map to `opensearch` (e.g. `elasticsearch.shardTimeout` and `opensearch.shardTimeout`) and settings with `kibana` in their names map to `opensearchDashboards` (e.g. `kibana.defaultAppId` and `opensearchDashboards.defaultAppId`). Most other settings use the same names.
For a full list of OpenSearch Dashboards settings, see [here](https://github.com/opensearch-project/OpenSearch-Dashboards/blob/main/config/opensearch_dashboards.yml){:target='\_blank'}.
1. If your OpenSearch cluster uses the security plugin, preserve and modify the default settings in `opensearch_dashboards.yml`, particularly `opensearch.username` and `opensearch.password`.
If you disabled the security plugin on your OpenSearch cluster, remove or comment out all `opensearch_security` settings. Then run `rm -rf plugins/security-dashboards/` to remove the security plugin.
1. Start OpenSearch Dashboards:
```
./bin/opensearch-dashboards
```
1. Log in, and verify that your saved searches, visualizations, and dashboards are present.

Binary file not shown.

Before

Width:  |  Height:  |  Size: 121 KiB

After

Width:  |  Height:  |  Size: 35 KiB

BIN
images/ppl-notebook.png Normal file

Binary file not shown.

After

Width:  |  Height:  |  Size: 357 KiB

BIN
images/sql-notebook.png Normal file

Binary file not shown.

After

Width:  |  Height:  |  Size: 367 KiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 313 KiB

View File

@ -37,7 +37,7 @@ Component | Purpose
[Anomaly Detection]({{site.url}}{{site.baseurl}}/monitoring-plugins/ad/) | Identify atypical data and receive automatic notifications
[Asynchronous Search]({{site.url}}{{site.baseurl}}/search-plugins/async/) | Run search requests in the background
You can install OpenSearch plugins [individually]({{site.url}}{{site.baseurl}}/opensearch/install/plugins/) or use the [all-in-one packages]({{site.url}}{{site.baseurl}}/opensearch/install/). Most of these OpenSearch plugins have corresponding OpenSearch Dashboards plugins that provide a convenient, unified user interface.
Most of OpenSearch plugins have a corresponding OpenSearch Dashboards plugin that provide a convenient, unified user interface.
For specifics around the project, see the [FAQ](https://opensearch.org/faq/).
@ -65,7 +65,10 @@ Docker
To learn more, see [Install and configure OpenSearch]({{site.url}}{{site.baseurl}}/opensearch/install/) and [Install and configure OpenSearch Dashboards]({{site.url}}{{site.baseurl}}/dashboards/install/).
---
## The secure path forward
OpenSearch includes a demo configuration so that you can get up and running quickly, but before using OpenSearch in a production environment, you must [configure the security plugin manually]({{site.url}}{{site.baseurl}}/security-plugin/configuration/index/): your own certificates, your own authentication method, your own users, and your own passwords.
## Looking for the Javadoc?