[DOC] Add new documentation about Query Workbench (#5378)
* Rewrite Query Workbench --------- Signed-off-by: Melissa Vagi <vagimeli@amazon.com> Signed-off-by: Fanit Kolchina <kolchfa@amazon.com> Co-authored-by: kolchfa-aws <105444904+kolchfa-aws@users.noreply.github.com> Co-authored-by: Fanit Kolchina <kolchfa@amazon.com>
This commit is contained in:
Melissa Vagi
GitHub
parent
5ebf909e0c
commit
9973d8470e
|
|
@ -0,0 +1,118 @@
|
|||
---
|
||||
layout: default
|
||||
title: Query Workbench
|
||||
nav_order: 125
|
||||
redirect_from:
|
||||
- /search-plugins/sql/workbench/
|
||||
---
|
||||
|
||||
# Query Workbench
|
||||
|
||||
Query Workbench is a tool within OpenSearch Dashboards. You can use Query Workbench to run on-demand [SQL]({{site.url}}{{site.baseurl}}/search-plugins/sql/sql/index/) and [PPL]({{site.url}}{{site.baseurl}}/search-plugins/sql/ppl/index/) queries, translate queries into their equivalent REST API calls, and view and save results in different [response formats]({{site.url}}{{site.baseurl}}/search-plugins/sql/response-formats/).
|
||||
|
||||
A view of the Query Workbench interface within OpenSearch Dashboards is shown in the following image.
|
||||
|
||||
<img src="{{site.url}}{{site.baseurl}}/images/dashboards/query-workbench-ui.png" alt="Query Workbench interface within OpenSearch Dashboards" width="700">
|
||||
|
||||
## Prerequisites
|
||||
|
||||
Before getting started, make sure you have [indexed your data]({{site.url}}{{site.baseurl}}/im-plugin/index/).
|
||||
|
||||
For this tutorial, you can index the following sample documents. Alternatively, you can use the [OpenSearch Playground](https://playground.opensearch.org/app/opensearch-query-workbench#/), which has preloaded indexes that you can use to try out Query Workbench.
|
||||
|
||||
To index sample documents, send the following [Bulk API]({{site.url}}{{site.baseurl}}/api-reference/document-apis/bulk/) request:
|
||||
|
||||
```json
|
||||
PUT accounts/_bulk?refresh
|
||||
{"index":{"_id":"1"}}
|
||||
{"account_number":1,"balance":39225,"firstname":"Amber","lastname":"Duke","age":32,"gender":"M","address":"880 Holmes Lane","employer":"Pyrami","email":"amberduke@pyrami.com","city":"Brogan","state":"IL"}
|
||||
{"index":{"_id":"6"}}
|
||||
{"account_number":6,"balance":5686,"firstname":"Hattie","lastname":"Bond","age":36,"gender":"M","address":"671 Bristol Street","employer":"Netagy","email":"hattiebond@netagy.com","city":"Dante","state":"TN"}
|
||||
{"index":{"_id":"13"}}
|
||||
{"account_number":13,"balance":32838,"firstname":"Nanette","lastname":"Bates","age":28,"gender":"F","address":"789 Madison Street","employer":"Quility","email":"nanettebates@quility.com","city":"Nogal","state":"VA"}
|
||||
{"index":{"_id":"18"}}
|
||||
{"account_number":18,"balance":4180,"firstname":"Dale","lastname":"Adams","age":33,"gender":"M","address":"467 Hutchinson Court","email":"daleadams@boink.com","city":"Orick","state":"MD"}
|
||||
```
|
||||
{% include copy-curl.html %}
|
||||
|
||||
## Running SQL queries within Query Workbench
|
||||
|
||||
Follow these steps to learn how to run SQL queries against your OpenSearch data using Query Workbench:
|
||||
|
||||
1. Access Query Workbench.
|
||||
- To access Query Workbench, go to OpenSearch Dashboards and choose **OpenSearch Plugins** > **Query Workbench** from the main menu.
|
||||
|
||||
2. Run a query.
|
||||
- Select the **SQL** button. In the query editor, type a SQL expression and then select the **Run** button to run the query.
|
||||
|
||||
The following example query retrieves the first name, last name, and balance from the `accounts` index for accounts with a balance greater than 10,000 and sorts by balance in descending order:
|
||||
|
||||
```sql
|
||||
SELECT
|
||||
firstname,
|
||||
lastname,
|
||||
balance
|
||||
FROM
|
||||
accounts
|
||||
WHERE
|
||||
balance > 10000
|
||||
ORDER BY
|
||||
balance DESC;
|
||||
```
|
||||
{% include copy.html %}
|
||||
|
||||
3. View the results.
|
||||
- View the results in the **Results** pane, which presents the query output in tabular format. You can filter and download the results as needed.
|
||||
|
||||
The following image shows the query editor pane and results pane for the preceding SQL query:
|
||||
|
||||
<img src="{{site.url}}{{site.baseurl}}/images/dashboards/query-workbench-query-step2.png" alt="Query Workbench SQL query input and results output panes" width="800">
|
||||
|
||||
4. Clear the query editor.
|
||||
- Select the **Clear** button to clear the query editor and run a new query.
|
||||
|
||||
5. Examine how the query is processed.
|
||||
- Select the **Explain** button to examine how OpenSearch processes the query, including the steps involved and order of operations.
|
||||
|
||||
The following image shows the explanation of the SQL query that was run in step 2.
|
||||
|
||||
<img src="{{site.url}}{{site.baseurl}}/images/dashboards/query-explain.png" alt="Query Workbench SQL query explanation pane" width="500">
|
||||
|
||||
## Running PPL queries within Query Workbench
|
||||
|
||||
Follow these steps to learn how to run PPL queries against your OpenSearch data using Query Workbench:
|
||||
|
||||
1. Access Query Workbench.
|
||||
- To access Query Workbench, go to OpenSearch Dashboards and choose **OpenSearch Plugins** > **Query Workbench** from the main menu.
|
||||
|
||||
2. Run a query.
|
||||
- Select the **PPL** button. In the query editor, type a PPL query and then select the **Run** button to run the query.
|
||||
|
||||
The following is an example query that retrieves the `firstname` and `lastname` fields for documents in the `accounts` index with age greater than `18`:
|
||||
|
||||
```sql
|
||||
search source=accounts
|
||||
| where age > 18
|
||||
| fields firstname, lastname
|
||||
```
|
||||
{% include copy.html %}
|
||||
|
||||
3. View the results.
|
||||
- View the results in the **Results** pane, which presents the query output in tabular format.
|
||||
|
||||
The following image shows the query editor pane and results pane for the PPL query that was run in step 2:
|
||||
|
||||
<img src="{{site.url}}{{site.baseurl}}/images/dashboards/query-workbench-ppl.png" alt="Query Workbench PPL query input and results output panes">
|
||||
|
||||
4. Clear the query editor.
|
||||
- Select the **Clear** button to clear the query editor and run a new query.
|
||||
|
||||
5. Examine how the query is processed.
|
||||
- Select the **Explain** button to examine how OpenSearch processes the query, including the steps involved and order of operations.
|
||||
|
||||
The following image shows the explanation of the PPL query that was run in step 2.
|
||||
|
||||
<img src="{{site.url}}{{site.baseurl}}/images/dashboards/query-PPL-explain.png" alt="Query Workbench PPL query explanation pane" width="500">
|
||||
|
||||
Query Workbench does not support delete or update operations through SQL or PPL. Access to data is read-only.
|
||||
{: .important}
|
||||
|
|
@ -10,8 +10,12 @@ redirect_from:
|
|||
|
||||
# SQL and PPL
|
||||
|
||||
OpenSearch SQL lets you write queries in SQL rather than the [OpenSearch query domain-specific language (DSL)]({{site.url}}{{site.baseurl}}/opensearch/query-dsl/full-text). If you're already familiar with SQL and don't want to learn the query DSL, this feature is a great option.
|
||||
OpenSearch SQL lets you write queries in SQL rather than the [OpenSearch query domain-specific language (DSL)]({{site.url}}{{site.baseurl}}/opensearch/query-dsl/full-text/).
|
||||
|
||||
## Contributing
|
||||
## Next steps
|
||||
|
||||
To get involved and help us improve the SQL plugin, see the [development guide](https://github.com/opensearch-project/sql/blob/main/DEVELOPER_GUIDE.rst) for instructions on setting up your development environment and building the project.
|
||||
- Learn about the [SQL and PPL API]({{site.url}}{{site.baseurl}}/search-plugins/sql/sql-ppl-api/).
|
||||
- Learn about [using SQL within OpenSearch]({{site.url}}{{site.baseurl}}/search-plugins/sql/sql/index/).
|
||||
- Learn about [using PPL within OpenSearch]({{site.url}}{{site.baseurl}}/search-plugins/sql/ppl/index/).
|
||||
- Learn about [using Query Workbench for SQL and PPL queries within OpenSearch Dashboards]({{site.url}}{{site.baseurl}}/dashboards/query-workbench/).
|
||||
- Learn more about OpenSearch SQL in the [Developer Guide](https://github.com/opensearch-project/sql/blob/main/DEVELOPER_GUIDE.rst).
|
||||
|
|
@ -1,6 +1,6 @@
|
|||
---
|
||||
layout: default
|
||||
title: PPL – Piped Processing Language
|
||||
title: PPL
|
||||
parent: SQL and PPL
|
||||
nav_order: 5
|
||||
has_children: true
|
||||
|
|
@ -14,53 +14,34 @@ redirect_from:
|
|||
- /search-plugins/ppl/protocol/
|
||||
---
|
||||
|
||||
# PPL – Piped Processing Language
|
||||
# PPL
|
||||
|
||||
Piped Processing Language (PPL) is a query language that lets you use pipe (`|`) syntax to explore, discover, and query data stored in OpenSearch.
|
||||
Piped Processing Language (PPL) is a query language that focuses on processing data in a sequential, step-by-step manner. PPL uses the pipe (`|`) operator to combine commands to find and retrieve data. It is the primary language used with observability in OpenSearch and supports multi-data queries.
|
||||
|
||||
To quickly get up and running with PPL, use **Query Workbench** in OpenSearch Dashboards. To learn more, see [Workbench]({{site.url}}{{site.baseurl}}/search-plugins/sql/workbench/).
|
||||
## PPL syntax
|
||||
|
||||
The PPL syntax consists of commands delimited by the pipe character (`|`) where data flows from left to right through each pipeline.
|
||||
The following example shows the basic PPL syntax:
|
||||
|
||||
```sql
|
||||
search command | command 1 | command 2 ...
|
||||
search source=<index-name> | <command_1> | <command_2> | ... | <command_n>
|
||||
```
|
||||
{% include copy.html %}
|
||||
|
||||
You can only use read-only commands like `search`, `where`, `fields`, `rename`, `dedup`, `stats`, `sort`, `eval`, `head`, `top`, and `rare`.
|
||||
See [Syntax]({{site.url}}{{site.baseurl}}/search-plugins/sql/ppl/syntax/) for specific PPL syntax examples.
|
||||
|
||||
## Quick start
|
||||
## PPL commands
|
||||
|
||||
To get started with PPL, choose **Dev Tools** in OpenSearch Dashboards and use the `bulk` operation to index some sample data:
|
||||
PPL filters, transforms, and aggregates data using a series of commands. See [Commands](/search-plugins/sql/ppl/functions/) for a description and an example of each command.
|
||||
|
||||
```json
|
||||
PUT accounts/_bulk?refresh
|
||||
{"index":{"_id":"1"}}
|
||||
{"account_number":1,"balance":39225,"firstname":"Amber","lastname":"Duke","age":32,"gender":"M","address":"880 Holmes Lane","employer":"Pyrami","email":"amberduke@pyrami.com","city":"Brogan","state":"IL"}
|
||||
{"index":{"_id":"6"}}
|
||||
{"account_number":6,"balance":5686,"firstname":"Hattie","lastname":"Bond","age":36,"gender":"M","address":"671 Bristol Street","employer":"Netagy","email":"hattiebond@netagy.com","city":"Dante","state":"TN"}
|
||||
{"index":{"_id":"13"}}
|
||||
{"account_number":13,"balance":32838,"firstname":"Nanette","lastname":"Bates","age":28,"gender":"F","address":"789 Madison Street","employer":"Quility","city":"Nogal","state":"VA"}
|
||||
{"index":{"_id":"18"}}
|
||||
{"account_number":18,"balance":4180,"firstname":"Dale","lastname":"Adams","age":33,"gender":"M","address":"467 Hutchinson Court","email":"daleadams@boink.com","city":"Orick","state":"MD"}
|
||||
```
|
||||
## Using PPL within OpenSearch
|
||||
|
||||
Go to **Query Workbench** and select **PPL**.
|
||||
To use PPL, you must have installed OpenSearch Dashboards. PPL is available within the [Query Workbench tool](https://playground.opensearch.org/app/opensearch-query-workbench#/). See the [Query Workbench]({{site.url}}{{site.baseurl}}/dashboards/query-workbench/) documentation for a tutorial on using PPL within OpenSearch.
|
||||
|
||||
The following example returns `firstname` and `lastname` fields for documents in an `accounts` index with `age` greater than 18:
|
||||
## Developer documentation
|
||||
|
||||
```sql
|
||||
search source=accounts
|
||||
| where age > 18
|
||||
| fields firstname, lastname
|
||||
```
|
||||
Developers can find information in the following resources:
|
||||
|
||||
#### Example response
|
||||
|
||||
firstname | lastname |
|
||||
:--- | :--- |
|
||||
Amber | Duke
|
||||
Hattie | Bond
|
||||
Nanette | Bates
|
||||
Dale | Adams
|
||||
|
||||
|
||||
- [Piped Processing Language](https://github.com/opensearch-project/piped-processing-language) specification
|
||||
- [OpenSearch PPL Reference Manual](https://github.com/opensearch-project/sql/blob/main/docs/user/ppl/index.rst)
|
||||
- [Observability](https://github.com/opensearch-project/dashboards-observability/) using [PPL-based visualizations](https://github.com/opensearch-project/dashboards-observability#event-analytics)
|
||||
- PPL [Data Types](https://github.com/opensearch-project/sql/blob/main/docs/user/ppl/general/datatypes.rst)
|
||||
|
|
|
|||
|
|
@ -1,28 +1,16 @@
|
|||
---
|
||||
layout: default
|
||||
title: SQL/PPL API
|
||||
title: SQL and PPL API
|
||||
parent: SQL and PPL
|
||||
nav_order: 1
|
||||
---
|
||||
|
||||
# SQL/PPL API
|
||||
# SQL and PPL API
|
||||
|
||||
Use the SQL and PPL API to send queries to the SQL plugin. Use the `_sql` endpoint to send queries in SQL, and the `_ppl` endpoint to send queries in PPL. For both of these, you can also use the `_explain` endpoint to translate your query into [OpenSearch domain-specific language]({{site.url}}{{site.baseurl}}/opensearch/query-dsl/) (DSL) or to troubleshoot errors.
|
||||
|
||||
---
|
||||
|
||||
#### Table of contents
|
||||
- TOC
|
||||
{:toc}
|
||||
|
||||
|
||||
---
|
||||
|
||||
## Query API
|
||||
|
||||
Introduced 1.0
|
||||
{: .label .label-purple }
|
||||
|
||||
Sends an SQL/PPL query to the SQL plugin. You can pass the format for the response as a query parameter.
|
||||
|
||||
### Query parameters
|
||||
|
|
|
|||
|
|
@ -12,11 +12,7 @@ redirect_from:
|
|||
|
||||
# SQL
|
||||
|
||||
## Workbench
|
||||
|
||||
The easiest way to get familiar with the SQL plugin is to use **Query Workbench** in OpenSearch Dashboards to test various queries. To learn more, see [Workbench]({{site.url}}{{site.baseurl}}/search-plugins/sql/workbench/).
|
||||
|
||||
|
||||
SQL in OpenSearch bridges the gap between traditional relational database concepts and the flexibility of OpenSearch's document-oriented data storage. This integration gives you the ability to use your SQL knowledge to query, analyze, and extract insights from your OpenSearch data.
|
||||
|
||||
## SQL and OpenSearch terminology
|
||||
|
||||
|
|
@ -30,7 +26,7 @@ Column | Field
|
|||
|
||||
## REST API
|
||||
|
||||
For a complete REST API reference for the SQL plugin, see [SQL/PPL API]({{site.url}}{{site.baseurl}}/search-plugins/sql/sql-ppl-api).
|
||||
For a complete REST API reference for the SQL plugin, see [SQL/PPL API]({{site.url}}{{site.baseurl}}/search-plugins/sql/sql-ppl-api/).
|
||||
|
||||
To use the SQL plugin with your own applications, send requests to the `_plugins/_sql` endpoint:
|
||||
|
||||
|
|
@ -40,6 +36,7 @@ POST _plugins/_sql
|
|||
"query": "SELECT * FROM my-index LIMIT 50"
|
||||
}
|
||||
```
|
||||
{% include copy-curl.html %}
|
||||
|
||||
You can query multiple indexes by using a comma-separated list:
|
||||
|
||||
|
|
@ -49,8 +46,9 @@ POST _plugins/_sql
|
|||
"query": "SELECT * FROM my-index1,myindex2,myindex3 LIMIT 50"
|
||||
}
|
||||
```
|
||||
{% include copy-curl.html %}
|
||||
|
||||
You can also specify an index pattern with a wildcard expression:
|
||||
You can specify an index pattern with a wildcard expression:
|
||||
|
||||
```json
|
||||
POST _plugins/_sql
|
||||
|
|
@ -58,14 +56,16 @@ POST _plugins/_sql
|
|||
"query": "SELECT * FROM my-index* LIMIT 50"
|
||||
}
|
||||
```
|
||||
{% include copy-curl.html %}
|
||||
|
||||
To run the above query in the command line, use the [curl](https://curl.haxx.se/) command:
|
||||
To run the preceding query in the command line, use the [curl](https://curl.haxx.se/) command:
|
||||
|
||||
```bash
|
||||
curl -XPOST https://localhost:9200/_plugins/_sql -u 'admin:admin' -k -H 'Content-Type: application/json' -d '{"query": "SELECT * FROM my-index* LIMIT 50"}'
|
||||
```
|
||||
{% include copy.html %}
|
||||
|
||||
You can specify the [response format]({{site.url}}{{site.baseurl}}/search-plugins/sql/response-formats) as JDBC, standard OpenSearch JSON, CSV, or raw. By default, queries return data in JDBC format. The following query sets the format to JSON:
|
||||
You can specify the [response format]({{site.url}}{{site.baseurl}}/search-plugins/sql/response-formats/) as JDBC, standard OpenSearch JSON, CSV, or raw. By default, queries return data in JDBC format. The following query sets the format to JSON:
|
||||
|
||||
```json
|
||||
POST _plugins/_sql?format=json
|
||||
|
|
@ -73,5 +73,6 @@ POST _plugins/_sql?format=json
|
|||
"query": "SELECT * FROM my-index LIMIT 50"
|
||||
}
|
||||
```
|
||||
{% include copy-curl.html %}
|
||||
|
||||
See the rest of this guide for more information about request parameters, settings, supported operations, and tools.
|
||||
For more information about request parameters, settings, supported operations, and tools, see the related topics under [SQL]({{site.url}}{{site.baseurl}}/search-plugins/sql/sql/index/).
|
||||
|
|
|
|||
|
|
@ -1,76 +0,0 @@
|
|||
---
|
||||
layout: default
|
||||
title: Query Workbench
|
||||
parent: SQL
|
||||
grand_parent: SQL and PPL
|
||||
nav_order: 1
|
||||
redirect_from:
|
||||
- /search-plugins/sql/workbench/
|
||||
|
||||
---
|
||||
|
||||
# Query Workbench
|
||||
|
||||
Use the Query Workbench to easily run on-demand SQL queries, translate SQL into its REST equivalent, and view and save results as text, JSON, JDBC, or CSV.
|
||||
|
||||
|
||||
## Quick start
|
||||
|
||||
To get started with SQL Workbench, choose **Dev Tools** in OpenSearch Dashboards and use the `bulk` operation to index some sample data:
|
||||
|
||||
```json
|
||||
PUT accounts/_bulk?refresh
|
||||
{"index":{"_id":"1"}}
|
||||
{"account_number":1,"balance":39225,"firstname":"Amber","lastname":"Duke","age":32,"gender":"M","address":"880 Holmes Lane","employer":"Pyrami","email":"amberduke@pyrami.com","city":"Brogan","state":"IL"}
|
||||
{"index":{"_id":"6"}}
|
||||
{"account_number":6,"balance":5686,"firstname":"Hattie","lastname":"Bond","age":36,"gender":"M","address":"671 Bristol Street","employer":"Netagy","email":"hattiebond@netagy.com","city":"Dante","state":"TN"}
|
||||
{"index":{"_id":"13"}}
|
||||
{"account_number":13,"balance":32838,"firstname":"Nanette","lastname":"Bates","age":28,"gender":"F","address":"789 Madison Street","employer":"Quility","email":"nanettebates@quility.com","city":"Nogal","state":"VA"}
|
||||
{"index":{"_id":"18"}}
|
||||
{"account_number":18,"balance":4180,"firstname":"Dale","lastname":"Adams","age":33,"gender":"M","address":"467 Hutchinson Court","email":"daleadams@boink.com","city":"Orick","state":"MD"}
|
||||
```
|
||||
|
||||
Then return to SQL Workbench.
|
||||
|
||||
|
||||
### List indexes
|
||||
|
||||
To list all your indexes:
|
||||
|
||||
```sql
|
||||
SHOW TABLES LIKE %
|
||||
```
|
||||
|
||||
| TABLE_NAME
|
||||
| :---
|
||||
| accounts
|
||||
|
||||
|
||||
### Read data
|
||||
|
||||
After you index a document, retrieve it using the following SQL expression:
|
||||
|
||||
```sql
|
||||
SELECT *
|
||||
FROM accounts
|
||||
WHERE _id = 1
|
||||
```
|
||||
|
||||
| account_number | firstname | gender | city | balance | employer | state | email | address | lastname | age
|
||||
| :--- | :--- | :--- | :--- | :--- | :--- | :--- | :--- | :--- | :--- | :---
|
||||
| 1 | Amber | M | Brogan | 39225 | Pyrami | IL | amberduke@pyrami.com | 880 Holmes Lane | Duke | 32
|
||||
|
||||
|
||||
### Delete data
|
||||
|
||||
To delete a document from an index, use the `DELETE` clause:
|
||||
|
||||
```sql
|
||||
DELETE
|
||||
FROM accounts
|
||||
WHERE _id = 0
|
||||
```
|
||||
|
||||
| deleted_rows
|
||||
| :---
|
||||
| 1
|
||||
Binary file not shown.
|
After Width: | Height: | Size: 138 KiB |
Binary file not shown.
|
After Width: | Height: | Size: 153 KiB |
Binary file not shown.
|
After Width: | Height: | Size: 91 KiB |
Binary file not shown.
|
After Width: | Height: | Size: 158 KiB |
Binary file not shown.
|
After Width: | Height: | Size: 115 KiB |
Binary file not shown.
|
After Width: | Height: | Size: 273 KiB |
Binary file not shown.
|
After Width: | Height: | Size: 418 KiB |
BIN
images/sql.png
BIN
images/sql.png
Binary file not shown.
|
Before Width: | Height: | Size: 267 KiB |
Loading…
Reference in New Issue