[DOC] Exploring data with Discover (#2435)

Creates new documentation website page and Discover content.
---------

Signed-off-by: vagimeli <vagimeli@amazon.com>
This commit is contained in:
Melissa Vagi 2023-02-01 11:11:43 -07:00 committed by GitHub
parent af824b3fb1
commit d67fa6bd4e
No known key found for this signature in database
GPG Key ID: 4AEE18F83AFDEB23
12 changed files with 171 additions and 72 deletions

View File

@ -0,0 +1,104 @@
---
layout: default
title: Exploring data with Discover
nav_order: 20
has_children: true
---
# Exploring data with Discover
**Discover** in OpenSearch Dashboards helps you extract insights and get value out of data assets across your organization. Discover enables you to:
1. **Explore data**. You can explore, customize, and filter data as well as search data using [Dashboards Query Language (DQL)]({{site.url}}{{site.baseurl}}/dashboards/dql/).
2. **Analyze data**. You can analyze data, view individual documents, and create tables summarizing data contents.
3. **Visualize data**. You can display findings from your saved searches in a single dashboard that combines different data visualization types.
## Try it: Exploring sample data with Discover
This tutorial shows you how to use Discover to analyze and understand a sample dataset. At the end of this tutorial, you should be ready to use Discover with your own data.
Before starting this tutorial, make sure you've added the **Sample flight data**. See [Quickstart guide for OpenSearch Dashboards]({{site.url}}{{site.baseurl}}/dashboards/quickstart/) for information about how to get started.
{: .warning}
### Setting up data
Watch the following short video or start with the tutorial steps to learn how to set up a sample dataset in Discover.
![Setting up the sample data in Discover]({{site.url}}{{site.baseurl}}/images/discover-setting-up-data.gif)
1. Verify access to OpenSearch Dashboards by connecting to [http://localhost:5601](http://localhost:5601) from a browser. The default username and password are `admin`.
1. On the **Home** page, choose **Discover** in the navigation pane.
1. On the index pattern toolbar, select the **opensearch_dashboards_sample_data_flights** dataset.
1. On the time filter toolbar, choose the calendar icon and then change the time range to **Last 7 days**.
### Exploring the data fields
In the **Discover** panel, you'll see a table that shows all the documents that match your search. The table includes a list of data fields that are available in the document table, as shown in the following image.
![Exploring data fields interface]({{site.url}}{{site.baseurl}}/images/discover-data-fields.png)
Follow these steps to explore the data fields:
1. View the list of **Available fields**.
1. Choose **Cancelled** to view the values (`true` and `false`).
1. Choose the plus (+) sign to add the field to the document table. The field will be automatically added to **Selected fields** and the document table.
1. Select **FlightDelay** from the **Available fields** list, and then choose the plus (+) sign to add the field to the document table.
1. Optional: Rearrange the table columns by selecting the table header and then choosing **Move left** or **Move right**.
## Searching data
You can use the search toolbar or enter a DQL query in the **DevTools** console to search data in Dashboards, as shown in the following image. The search toolbar is best for basic queries, such as searching by a field name. DQL is best for complex queries, such as searching data using a term, string, Boolean, date, range, or nested query.
![Searching data interface]({{site.url}}{{site.baseurl}}/images/discover-search.png)
Follow these steps to search data:
1. In the search toolbar, enter the Boolean query. For example, enter `FlightDelay:true AND FlightDelayMin >= 60` to search the data for flights delayed by 60 minutes or more.
1. Choose **Update**.
1. Optional: Choose the arrow (`>`) in a table row to expand the row and view the document table details.
## Filtering data
Filters allow you to refine sets of documents to subsets of those documents. For example, you can filter data to include or exclude certain fields, as shown in the following image.
![Filtering data interface]({{site.url}}{{site.baseurl}}/images/discover-filter.png)
Follow these steps to filter data:
1. In the filter bar, choose **Add filter**.
1. Select options from the **Field**, **Operator**, and **Value** dropdown lists. For example, `Cancelled`, `is`, and `true`.
1. Choose **Save**.
1. To remove the filter, choose the close icon (x) next to the filter name.
1. Optional: Add more filters to further explore the data.
## Analyzing data in the document table
You can view the document table fields to better understand the data and gather insights for more informed decision-making:
1. Choose the arrow icon (>) to expand a table row.
1. View the fields and details.
1. Switch between the **Table** and **JSON** tabs to view the different formats, as shown in the following image.
![Analyzing data in the document table]({{site.url}}{{site.baseurl}}/images/discover-analyze.png)
## Saving the search
Saving a search saves the query text, filters, and current data view. To save your search to use it later, generate a report, or build visualizations and dashboards:
1. Choose the save icon in the toolbar.
1. Give the search a title, and then choose **Save**.
1. Choose the save icon to access the saved search, as shown in the following image.
<img src="{{site.url}}{{site.baseurl}}/images/discover-save.png" alt= "Save search interface" width="350" height="350">
## Visualizing the search
You can quickly visualize an aggregated field from **Discover**:
1. From the **Available fields** list, select `FlightDelayType` and then choose **Visualize**, as shown in the following image.
<img src="{{site.url}}{{site.baseurl}}/images/discover-visualize.png" alt= "Visualizing search queries from Discover" width="350" height="350">
Dashboards creates a visualization for this field, which in this case is a basic bar chart, as shown in the following image.
<img src="{{site.url}}{{site.baseurl}}/images/discover-visualize-2.png" alt= "Bar chart created from Discover" width="600" height="600">

View File

@ -1,17 +0,0 @@
---
layout: default
title: Discover
nav_order: 10
has_children: true
---
# Discover
**Discover** in OpenSearch Dashboards helps you extract insights and get value out of data assets across your organization. You can quickly ingest and query your data, display that data in visualizations and interactive dashboards, and deliver insights to your organization.
<img src="{{site.url}}{{site.baseurl}}/images/discover-index.png" alt="Discover user interface">
## What's next?
- [Running queries in the console]({{site.url}}{{site.baseurl}}/dashboards/run-queries/)
- [Creating a dashboard with multiple data sources]({{site.url}}{{site.baseurl}}/)

View File

@ -1,11 +1,11 @@
---
layout: default
title: Multiple data sources
parent: Discover
title: Adding multiple data sources
parent: Exploring data with Discover
nav_order: 5
---
# Multiple data sources
# Adding multiple data sources
The multiple data sources feature is an experimental feature released in OpenSearch 2.4. It can't be used in a production environment. For updates on the features progress or to leave feedback on improving the feature, see the [OpenSearch Forum discussion](https://forum.opensearch.org/t/feedback-experimental-feature-connect-to-external-data-sources/11144).
{: .warning }
@ -14,14 +14,14 @@ You can add multiple data sources to a single dashboard. OpenSearch Dashboards a
In this tutorial we provide the steps for enabling the `data_source` setting in Dashboards; adding credentials, data source connections, and index patterns; and combining visualizations in a single dashboard.
## Try out the multiple data sources feature in your local environment
## Try it: Exploring the multiple data sources feature in your local environment
This tutorial uses a preconfigured data source and index pattern, and you arent required to configure settings. However, youll need to enable the `data_source` setting in the configuration file before before getting started with exploring this feature.
{: .note }
The multiple data sources feature is experimental and can't be deployed into production. You can try it out with a sample data source and a sample index pattern. Before getting started, you must first edit the YAML configuration. The following section provides the steps for enabling the feature.
## Edit the YAML configuration to enable the multiple data sources feature
## Modifying the multiple data sources settings
Dashboards is configured in the cluster settings, and the multiple data sources feature is disabled by default. To enable it, you need to edit the configuration in `opensearch_dashboards.yml` and then restart the cluster.
@ -33,7 +33,7 @@ To enable the feature:
4. Restart the Dashboards container.
5. Verify the feature configuration settings were created and configured properly by connecting to Dashboards through [http://localhost:5601](http://localhost:5601/) and viewing the **Stack Management** console. **Data Sources `Experimental`** will appear in the sidebar. Alternatively, you can open on [http://localhost:5601/app/management/opensearch-dashboards/dataSources](http://localhost:5601/app/management/opensearch-dashboards/dataSource).
## Create a data source connection
## Creating a data source connection
A data source connection specifies the parameters needed to connect to a data source. These parameters form a connection string for the data source. In Dashboards, you can add new data source connections or edit existing connections.
@ -41,90 +41,103 @@ To create a new data source connection:
1. Open Dashboards. If youre not running the security plugin, go to [`http://localhost:5601`](http://localhost:5601/). If youre running the security plugin, go to [`https://localhost:5601`](https://localhost:5601/) and log in with the username `admin` and password `admin`.
2. In the Dashboards console, select **Stack Management > Data Sources `Experimental` > Data Sources > Create data source connection**.
<img src="{{site.url}}{{site.baseurl}}/images/multi-data-sources-1.png" width=500 alt="Data sources user interface">
1. In the Dashboards console, choose **Stack Management** > **Data Sources `Experimental`** > **Data Sources** > **Create data source connection**, as shown in the following image.
3. Add information to each field to configure **Connection Details**, **Endpoint**, and **Authentication** to connect to a data source. For this tutorial, the **Endpoint URL** is `http://localhost:5601/app/management/opensearch-dashboards/dataSources`.
<img src="{{site.url}}{{site.baseurl}}/images/multi-data-sources-2.png" width=500 alt="Create a data source connection user interface">
<img src="{{site.url}}{{site.baseurl}}/images/multi-data-sources-1.png" width=500 alt="Data sources user interface">
4. Select **Create data source connection** to save your settings.
1. Add information to each field to configure **Connection Details**, **Endpoint**, and **Authentication** to connect to a data source, as shown in the following image. For this tutorial, the **Endpoint URL** is `http://localhost:5601/app/management/opensearch-dashboards/dataSources`.
5. Return to the **Data Sources** main page to confirm that the newly created data source is listed under **Data Sources**.
<img src="{{site.url}}{{site.baseurl}}/images/multi-data-sources-3.png" width=500 alt="Data sources list user interface">
<img src="{{site.url}}{{site.baseurl}}/images/multi-data-sources-2.png" width=500 alt="Create a data source connection user interface">
6. (Optional): Select the data source to verify that the settings are configured properly.
<img src="{{site.url}}{{site.baseurl}}/images/multi-data-sources-4.png" width=500 alt="Data sources settings verification user interface">
1. Choose **Create data source connection** to save your settings.
1. Return to the **Data Sources** main page to confirm that the newly created data source is listed under **Data Sources**, as shown in the following image.
## Create an index pattern
<img src="{{site.url}}{{site.baseurl}}/images/multi-data-sources-3.png" width=500 alt="Data sources list user interface">
1. Optional: Select the data source to verify that the settings are configured properly, as shown in the following image.
<img src="{{site.url}}{{site.baseurl}}/images/multi-data-sources-4.png" width=500 alt="Data sources settings verification user interface">
## Creating an index pattern
Index patterns allow you to access the OpenSearch data that you want to explore. An index pattern selects the data to use and allows you to define the field properties. Learn how to load your own data and create an index pattern following these steps. This tutorial uses the preconfigured index pattern `opensearch_dashboards_sample_data_ecommerce Default`.
1. In the Dashboards console, select **Index Patterns > Create index pattern**.
<img src="{{site.url}}{{site.baseurl}}/images/multi-data-sources-5.png" width=500 alt="Index pattern user interface">
1. In the Dashboards console, choose **Index Patterns** > **Create index pattern**, as shown in the following image.
<img src="{{site.url}}{{site.baseurl}}/images/multi-data-sources-5.png" width=500 alt="Index pattern user interface">
2. Select **Use external data source connection**.
3. Start typing in the Search data sources field to search for the data source you created earlier and then select the data source and **Next step**.
<img src="{{site.url}}{{site.baseurl}}/images/multi-data-sources-6.png" width=500 alt="Index pattern search user interface">
1. Choose **Use external data source connection**.
1. Start typing in the **Search data sources** field to search for the data source you created earlier and then select the data source and **Next step**, as shown in the following image.
4. Add an **Index pattern name** to define the index pattern and then select **Next step**.
<img src="{{site.url}}{{site.baseurl}}/images/multi-data-sources-7.png" width=500 alt="Index pattern define user interface">
<img src="{{site.url}}{{site.baseurl}}/images/multi-data-sources-6.png" width=500 alt="Index pattern search user interface">
5. Select an option for the **Time field** and then choose **Create index pattern**.
<img src="{{site.url}}{{site.baseurl}}/images/multi-data-sources-8.png" width=500 alt="Index pattern time field user interface">
1. Add an **Index pattern name** to define the index pattern and then choose **Next step**, as shown in the following image.
## Search data
<img src="{{site.url}}{{site.baseurl}}/images/multi-data-sources-7.png" width=500 alt="Index pattern define user interface">
Before you start searching for data, set up the time filter. The sample index pattern used for this tutorial contains time-based data. You can set a time filter that displays only the data within a specified time range, and you can select the time filter to change the time range or select a specific time range in the histogram.
1. Select an option for the **Time field** and then choose **Create index pattern**, as shown in the following image.
### Use the time filter
<img src="{{site.url}}{{site.baseurl}}/images/multi-data-sources-8.png" width=500 alt="Index pattern time field user interface">
1. In the Dashboards console, select **Discover** and confirm the index pattern being used is `opensearch_dashboards_sample_data_ecommerce`.
2. Select the calendar icon to change the time field. The default is **Last 15 minutes**.
3. Change the time field to **Last 7 days** and select **Refresh**.
<img src="{{site.url}}{{site.baseurl}}/images/multi-data-sources-9.png" alt="Time filter user interface">
## Searching data
4. To set the start and end times, select the bar next to the time filter. In the popup, select **Absolute**, **Relative**, or **Now** and then specify the required options.
<img src="{{site.url}}{{site.baseurl}}/images/multi-data-sources-10.png" width=300 alt="Start and end times user interface">
Before you start searching for data, set up the time filter. The sample index pattern used for this tutorial contains time-based data. You can set a time filter that displays only the data within a specified time range, and you can choose the time filter to change the time range or select a specific time range in the histogram.
### Select a time range from the histogram
### Adjusting the time filter
To adjust the time filter:
1. In the Dashboards console, choose **Discover** and confirm that the index pattern being used is `opensearch_dashboards_sample_data_ecommerce`.
2. Choose the calendar icon to change the time field. The default is **Last 15 minutes**.
3. Change the time field to **Last 7 days** and choose **Refresh**, as shown in the following image.
<img src="{{site.url}}{{site.baseurl}}/images/multi-data-sources-9.png" alt="Time filter user interface">
4. To set the start and end times, choose the bar next to the time filter. In the popup, select **Absolute**, **Relative**, or **Now** and then specify the required options, as shown in the following image.
<img src="{{site.url}}{{site.baseurl}}/images/multi-data-sources-10.png" width=300 alt="Start and end times user interface">
### Selecting a time range from the histogram
To select a time range for the histogram, you can do one of the following:
* Select the bar that represents the time range you want to zoom in on.
* Select the bar and drag to view a specific time range. You must start the selection with the cursor over the background of the chart—the cursor changes to a plus sign when you hover over a valid start point.
* Select the bar and drag to view a specific time range. You must start the selection with the cursor over the background of the chart (the cursor changes to a plus sign when you hover over a valid start point).
* Select the dropdown and then select an interval.
The following image shows a date histogram with an interval dropdown list.
<img src="{{site.url}}{{site.baseurl}}/images/multi-data-sources-11.jpg" alt="Histogram user interface">
## Create visualizations
## Creating data visualizations for a dashboard
Follow these steps to learn how to connect your visualizations in a single dashboard:
Follow these steps to learn how to create data visualizations for a dashboard:
1. In the Dashboards console, select **Visualize** > **Create visualization**.
2. Select the visualization type. For this tutorial, select **Line**.
3. Choose a source. For this tutorial, select the index pattern `opensearch_dashboards_sample_data_ecommerce`.
4. Under **Buckets**, select **Add > X-axis**.
5. In the **Aggregation** field, select **Date Histogram** and then **Update**.
6. Select **Save** and add the file name. This tutorial uses preconfigured visualizations, so you wont be able to save your visualization.
1. In the Dashboards console, choose **Visualize** > **Create visualization**.
2. Select the visualization type. For this tutorial, choose **Line**.
3. Select a source. For this tutorial, choose the index pattern `opensearch_dashboards_sample_data_ecommerce`.
4. Under **Buckets**, choose **Add** > **X-axis**.
5. In the **Aggregation** field, choose **Date Histogram** and then choose **Update**.
6. Optional: Choose **Save** and add the file name. This tutorial uses preconfigured data visualizations, so you can't save the file for this tutorial.
## Connect visualizations in a single dashboard
## Connecting visualizations in a single dashboard
Follow these steps to connect your visualizations in a single dashboard:
1. In the Dashboards console, select **Dashboard > Create dashboard**.
2. Select **Add an existing** and then select the data you want to add.
3. Select **Save** and add the dashboard name in the **Title field**. This tutorial uses preconfigured dashboards, so you wont be able to save your dashboard.
1. In the Dashboards console, choose **Dashboard** > **Create dashboard**.
2. Choose **Add an existing** and then select the data you want to add.
3. Choose **Save** and add the dashboard name in the **Title field**. This tutorial uses preconfigured dashboards, so you wont be able to save your dashboard.
4. Click on the white space left of **Add panels** to view the visualizations in a single dashboard.
Your dashboard might look like this:
Your dashboard might look like the one in the following image.
<img src="{{site.url}}{{site.baseurl}}/images/multi-data-sources-12.jpg" width=500 alt="Example dashboard using data visualizations from many data sources">
<img src="{{site.url}}{{site.baseurl}}/images/multi-data-sources-12.jpg" width=700 alt="Example dashboard using data visualizations from many data sources">
You have now explored the data sources experimental feature. We look forward to your feedback on how we can improve this feature ahead of its release for production use.
You have now explored the data sources experimental feature. To provide feedback on how this feature can be improved ahead of its release for production use, comment in the [OpenSearch forum](https://forum.opensearch.org/).
## Limitations
## Understanding feature limitations
The following limitations apply to the OpenSearch 2.4 release of this experimental feature:
The following limitations apply to this experimental feature:
* The multiple data sources feature is supported for index-pattern-based visualizations only.
* The visualization types Time Series Visual Builder (TSVB), Vega and Vega-Lite, and timeline are not supported.
@ -132,5 +145,4 @@ The following limitations apply to the OpenSearch 2.4 release of this experiment
## Related topics
* [OpenSearch 2.4.0 is ready for download](https://opensearch.org/blog/)
* [OpenSearch Forum](https://forum.opensearch.org/)
* [OpenSearch Forum](https://forum.opensearch.org/)

BIN
images/discover-analyze.png Normal file

Binary file not shown.

After

Width:  |  Height:  |  Size: 240 KiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 192 KiB

BIN
images/discover-filter.png Normal file

Binary file not shown.

After

Width:  |  Height:  |  Size: 309 KiB

Binary file not shown.

Before

Width:  |  Height:  |  Size: 261 KiB

BIN
images/discover-save.png Normal file

Binary file not shown.

After

Width:  |  Height:  |  Size: 32 KiB

BIN
images/discover-search.png Normal file

Binary file not shown.

After

Width:  |  Height:  |  Size: 294 KiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 12 MiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 55 KiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 57 KiB