iSharkFly-Docs
/
opensearch-docs-cn
mirror of https://github.com/iSharkFly-Docs/opensearch-docs-cn
1
0
Fork 0
Code Issues Packages Projects Releases Wiki Activity

Updated SQL and PPL doc 

Signed-off-by: chloe-zh <chloezh1102@gmail.com>
This commit is contained in:
Chloe Zhang 2021-05-24 16:50:29 -07:00 committed by chloe-zh
parent 58d5826866
commit cf3ba3219a
18 changed files with 250 additions and 274 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

35
docs/ppl/commands.md
View File

@ -46,7 +46,7 @@ search source=accounts;
``` ```
| account_number | firstname | address | balance | gender | city | employer | state | age | email | lastname | | account_number | firstname | address | balance | gender | city | employer | state | age | email | lastname |
:--- | :--- | :--- | :--- | :--- | :--- | :--- | :--- | :--- | :--- | :--- | :--- | :---
| 1 | Amber | 880 Holmes Lane | 39225 | M | Brogan | Pyrami | IL | 32 | amberduke@pyrami.com | Duke | 1 | Amber | 880 Holmes Lane | 39225 | M | Brogan | Pyrami | IL | 32 | amberduke@pyrami.com | Duke
| 6 | Hattie | 671 Bristol Street | 5686 | M | Dante | Netagy | TN | 36 | hattiebond@netagy.com | Bond | 6 | Hattie | 671 Bristol Street | 5686 | M | Dante | Netagy | TN | 36 | hattiebond@netagy.com | Bond
| 13 | Nanette | 789 Madison Street | 32838 | F | Nogal | Quility | VA | 28 | null | Bates | 13 | Nanette | 789 Madison Street | 32838 | F | Nogal | Quility | VA | 28 | null | Bates
@ -61,7 +61,7 @@ search source=accounts account_number=1 or gender="F";
``` ```
| account_number | firstname | address | balance | gender | city | employer | state | age | email | lastname | | account_number | firstname | address | balance | gender | city | employer | state | age | email | lastname |
:--- | :--- | :--- | :--- | :--- | :--- | :--- | :--- | :--- | :--- | :--- | :--- | :---
| 1 | Amber | 880 Holmes Lane | 39225 | M | Brogan | Pyrami | IL | 32 | amberduke@pyrami.com | Duke | | 1 | Amber | 880 Holmes Lane | 39225 | M | Brogan | Pyrami | IL | 32 | amberduke@pyrami.com | Duke |
| 13 | Nanette | 789 Madison Street | 32838 | F | Nogal | Quility | VA | 28 | null | Bates | | 13 | Nanette | 789 Madison Street | 32838 | F | Nogal | Quility | VA | 28 | null | Bates |
@ -513,13 +513,11 @@ Use the `head` command to return the first N number of results in a specified se
### Syntax ### Syntax
```sql ```sql
head [keeplast = (true | false)] [while "("<boolean-expression>")"] [N] head [N]
``` ```
Field | Description | Required | Default Field | Description | Required | Default
:--- | :--- |:--- :--- | :--- |:---
`keeplast` | Use along with the `while` argument to check if the last result in the result set is retained. The last result is what caused the `while` condition to evaluate to false or NULL. Set `keeplast` to true to retain the last result and false to discard it. | No | True
`while` | An expression that evaluates to either true or false. You cannot use statistical functions in this expression. | No | False
`N` | Specify the number of results to return. | No | 10 `N` | Specify the number of results to return. | No | 10
*Example 1*: Get the first 10 results *Example 1*: Get the first 10 results
@ -549,31 +547,6 @@ search source=accounts | fields firstname, age | head 2;
| Amber | 32 | Amber | 32
| Hattie | 36 | Hattie | 36
*Example 3*: Get the first N results that match a while condition
To get the first 3 results from all accounts with age less than 30:
```sql
search source=accounts | fields firstname, age | sort age | head while(age < 30) 3;
```
| firstname | age
:--- | :--- |
| Nanette | 28
| Amber | 32
*Example 4*: Get the first N results with a while condition with the last result that failed the condition
To get the first 3 results from all accounts with age less than 30 and include the last failed condition:
```sql
search source=accounts | fields firstname, age | sort age | head keeplast=false while(age < 30) 3;
```
| firstname | age
:--- | :--- |
| Nanette | 28
## rare ## rare
Use the `rare` command to find the least common values of all fields in a field list. Use the `rare` command to find the least common values of all fields in a field list.
@ -617,7 +590,7 @@ search source=accounts | rare age by gender;
| M | 32 | M | 32
| M | 33 | M | 33
## top {#top-command} ## top
Use the `top` command to find the most common values of all fields in the field list. Use the `top` command to find the most common values of all fields in the field list.

6
docs/ppl/endpoint.md
View File

@ -10,13 +10,13 @@ nav_order: 1
To send a query request to PPL plugin, use the HTTP POST request. To send a query request to PPL plugin, use the HTTP POST request.
We recommend a POST request because it doesn't have any length limit and it allows you to pass other parameters to the plugin for other functionality. We recommend a POST request because it doesn't have any length limit and it allows you to pass other parameters to the plugin for other functionality.
Use the explain endpoint for query translation and troubleshooting. Use the `_explain` endpoint for query translation and troubleshooting.
## Request Format ## Request Format
To use the PPL plugin with your own applications, send requests to `_opensearch/_ppl`, with your query in the request body: To use the PPL plugin with your own applications, send requests to `_plugins/_ppl`, with your query in the request body:
```json ```json
curl -H 'Content-Type: application/json' -X POST localhost:9200/_opensearch/_ppl \ curl -H 'Content-Type: application/json' -X POST localhost:9200/_plugins/_ppl \
... -d '{"query" : "source=accounts | fields firstname, lastname"}' ... -d '{"query" : "source=accounts | fields firstname, lastname"}'
``` ```

12
docs/ppl/index.md
View File

@ -48,11 +48,11 @@ search source=accounts
#### Sample Response #### Sample Response
| id | firstname | lastname | firstname | lastname |
:--- | :--- | :--- | :--- | :--- |
| 0 | Amber | Duke Amber | Duke
| 1 | Hattie | Bond Hattie | Bond
| 2 | Nanette | Bates Nanette | Bates
| 3 | Dale | Adams Dale | Adams
![PPL query workbench](../images/ppl.png) ![PPL query workbench](../images/ppl.png)

4
docs/ppl/protocol.md
View File

@ -14,7 +14,7 @@ The PPL plugin provides responses in JDBC format. The JDBC format is widely used
The body of HTTP POST request can take a few more additional fields with the PPL query: The body of HTTP POST request can take a few more additional fields with the PPL query:
```json ```json
curl -H 'Content-Type: application/json' -X POST localhost:9200/_opensearch/_ppl \ curl -H 'Content-Type: application/json' -X POST localhost:9200/_plugins/_ppl \
... -d '{"query" : "source=accounts | fields firstname, lastname"}' ... -d '{"query" : "source=accounts | fields firstname, lastname"}'
``` ```
@ -58,7 +58,7 @@ The following example shows a normal response where the schema includes a field
If any error occurred, error message and the cause will be returned instead: If any error occurred, error message and the cause will be returned instead:
```json ```json
curl -H 'Content-Type: application/json' -X POST localhost:9200/_opensearch/_ppl \ curl -H 'Content-Type: application/json' -X POST localhost:9200/_plugins/_ppl \
... -d '{"query" : "source=unknown | fields firstname, lastname"}' ... -d '{"query" : "source=unknown | fields firstname, lastname"}'
{ {
"error": { "error": {

24
docs/ppl/settings.md
View File

@ -15,7 +15,7 @@ You can update these settings like any other cluster setting:
PUT _cluster/settings PUT _cluster/settings
{ {
"transient": { "transient": {
"opensearch": { "plugins": {
"ppl": { "ppl": {
"enabled": "false" "enabled": "false"
} }
@ -24,12 +24,26 @@ PUT _cluster/settings
} }
``` ```
Requests to `_opensearch/_ppl` include index names in the request body, so they have the same access policy considerations as the `bulk`, `mget`, and `msearch` operations. If you set the `rest.action.multi.allow_explicit_index` parameter to `false`, the PPL plugin is disabled. Similarly, you can also update the settings by sending request to the plugin setting endpoint `_plugins/_query/settings` :
```json
PUT _plugins/_query/settings
{
"transient": {
"plugins": {
"ppl": {
"enabled": "false"
}
}
}
}
```
Requests to `_plugins/_ppl` include index names in the request body, so they have the same access policy considerations as the `bulk`, `mget`, and `msearch` operations. If you set the `rest.action.multi.allow_explicit_index` parameter to `false`, the PPL plugin is disabled.
You can specify the settings shown in the following table: You can specify the settings shown in the following table:
Setting | Description | Default Setting | Description | Default
:--- | :--- | :--- :--- | :--- | :---
`opensearch.ppl.enabled` | Change to `false` to disable the plugin. | True `plugins.ppl.enabled` | Change to `false` to disable the PPL component. | True
`opensearch.ppl.query.memory_limit` | Set heap memory usage limit. If a query crosses this limit, it's terminated. | 85% `plugins.query.memory_limit` | Set heap memory usage limit. If a query crosses this limit, it's terminated. | 85%
`opensearch.query.size_limit` | Set the maximum number of results that you want to see. This impacts the accuracy of aggregation operations. For example, if you have 1000 documents in an index, by default, only 200 documents are extracted from the index for aggregation. | 200 `plugins.query.size_limit` | Set the maximum number of results that you want to see. This impacts the accuracy of aggregation operations. For example, if you have 1000 documents in an index, by default, only 200 documents are extracted from the index for aggregation. | 200

122
docs/sql/basic.md
View File

@ -81,12 +81,12 @@ SELECT *
FROM accounts FROM accounts
``` ```
| id | account_number | firstname | gender | city | balance | employer | state | email | address | lastname | age | account_number | firstname | gender | city | balance | employer | state | email | address | lastname | age
:--- | :--- | :--- | :--- | :--- | :--- | :--- | :--- | :--- | :--- | :--- | :--- | :--- | :--- | :--- | :--- | :--- | :--- | :--- | :--- | :--- | :--- | :---
0 | 1 | Amber | M | Brogan | 39225 | Pyrami | IL | amberduke@pyrami.com | 880 Holmes Lane | Duke | 32 | 1 | Amber | M | Brogan | 39225 | Pyrami | IL | amberduke@pyrami.com | 880 Holmes Lane | Duke | 32
1 | 16 | Hattie | M | Dante | 5686 | Netagy | TN | hattiebond@netagy.com | 671 Bristol Street | Bond | 36 | 16 | Hattie | M | Dante | 5686 | Netagy | TN | hattiebond@netagy.com | 671 Bristol Street | Bond | 36
2 | 13 | Nanette | F | Nogal | 32838 | Quility | VA | nanettebates@quility.com | 789 Madison Street | Bates | 28 | 13 | Nanette | F | Nogal | 32838 | Quility | VA | nanettebates@quility.com | 789 Madison Street | Bates | 28
3 | 18 | Dale | M | Orick | 4180 | | MD | daleadams@boink.com | 467 Hutchinson Court | Adams | 33 | 18 | Dale | M | Orick | 4180 | | MD | daleadams@boink.com | 467 Hutchinson Court | Adams | 33
*Example 2*: Use field name(s) to retrieve only specific fields: *Example 2*: Use field name(s) to retrieve only specific fields:
@ -95,12 +95,12 @@ SELECT firstname, lastname
FROM accounts FROM accounts
``` ```
| id | firstname | lastname | firstname | lastname
:--- | :--- | :--- | :--- | :---
0 | Amber | Duke | Amber | Duke
1 | Hattie | Bond | Hattie | Bond
2 | Nanette | Bates | Nanette | Bates
3 | Dale | Adams | Dale | Adams
*Example 3*: Use field aliases instead of field names. Field aliases are used to make field names more readable: *Example 3*: Use field aliases instead of field names. Field aliases are used to make field names more readable:
@ -109,12 +109,12 @@ SELECT account_number AS num
FROM accounts FROM accounts
``` ```
| id | num | num
:--- | :--- :---
0 | 1 | 1
1 | 6 | 6
2 | 13 | 13
3 | 18 | 18
*Example 4*: Use the `DISTINCT` clause to get back only unique field values. You can specify one or more field names: *Example 4*: Use the `DISTINCT` clause to get back only unique field values. You can specify one or more field names:
@ -123,12 +123,12 @@ SELECT DISTINCT age
FROM accounts FROM accounts
``` ```
| id | age | age
:--- | :--- :---
0 | 28 | 28
1 | 32 | 32
2 | 33 | 33
3 | 36 | 36
## From ## From
@ -156,12 +156,12 @@ SELECT account_number, acc.age
FROM accounts acc FROM accounts acc
``` ```
| id | account_number | age | account_number | age
:--- | :--- | :--- | :--- | :---
0 | 1 | 32 | 1 | 32
1 | 6 | 36 | 6 | 36
2 | 13 | 28 | 13 | 28
3 | 18 | 33 | 18 | 33
*Example 2*: Use index patterns to query indices that match a specific pattern: *Example 2*: Use index patterns to query indices that match a specific pattern:
@ -170,12 +170,12 @@ SELECT account_number
FROM account* FROM account*
``` ```
| id | account_number | account_number
:--- | :--- :---
0 | 1 | 1
1 | 6 | 6
2 | 13 | 13
3 | 18 | 18
## Where ## Where
@ -205,11 +205,11 @@ FROM accounts
WHERE account_number = 1 WHERE account_number = 1
``` ```
| id | account_number | account_number
:--- | :--- | :---
0 | 1 | 1
*Example 2*: OpenSearch allows for flexible schema so documents in an index may have different fields. Use `IS NULL` or `IS NOT NULL` to retrieve only missing fields or existing fields. We do not differentiate between missing fields and fields explicitly set to `NULL`: *Example 2*: OpenSearch allows for flexible schema， so documents in an index may have different fields. Use `IS NULL` or `IS NOT NULL` to retrieve only missing fields or existing fields. We do not differentiate between missing fields and fields explicitly set to `NULL`:
```sql ```sql
SELECT account_number, employer SELECT account_number, employer
@ -217,9 +217,9 @@ FROM accounts
WHERE employer IS NULL WHERE employer IS NULL
``` ```
| id | account_number | employer | account_number | employer
:--- | :--- | :--- | :--- | :---
0 | 18 | | 18 |
*Example 3*: Deletes a document that satisfies the predicates in the `WHERE` clause: *Example 3*: Deletes a document that satisfies the predicates in the `WHERE` clause:
@ -307,12 +307,12 @@ FROM accounts
ORDER BY account_number DESC ORDER BY account_number DESC
``` ```
| id | account_number | account_number
:--- | :--- | :---
0 | 18 | 18
1 | 13 | 13
2 | 6 | 6
3 | 1 | 1
*Example 2*: Specify if documents with missing fields are to be put at the beginning or at the end of the results. The default behavior of OpenSearch is to return nulls or missing fields at the end. To push them before non-nulls, use the `IS NOT NULL` operator: *Example 2*: Specify if documents with missing fields are to be put at the beginning or at the end of the results. The default behavior of OpenSearch is to return nulls or missing fields at the end. To push them before non-nulls, use the `IS NOT NULL` operator:
@ -322,12 +322,12 @@ FROM accounts
ORDER BY employer IS NOT NULL ORDER BY employer IS NOT NULL
``` ```
| id | employer | employer
:--- | :--- | :---
0 | ||
1 | Netagy | Netagy
2 | Pyrami | Pyrami
3 | Quility | Quility
## Limit ## Limit
@ -341,9 +341,9 @@ FROM accounts
ORDER BY account_number LIMIT 1 ORDER BY account_number LIMIT 1
``` ```
| id | account_number | account_number
:--- | :--- | :---
0 | 1 | 1
*Example 2*: If you pass in two arguments, the first is mapped to the `from` parameter and the second to the `size` parameter in OpenSearch. You can use this for simple pagination for small indices, as it's inefficient for large indices. *Example 2*: If you pass in two arguments, the first is mapped to the `from` parameter and the second to the `size` parameter in OpenSearch. You can use this for simple pagination for small indices, as it's inefficient for large indices.
Use `ORDER BY` to ensure the same order between pages: Use `ORDER BY` to ensure the same order between pages:
@ -354,6 +354,6 @@ FROM accounts
ORDER BY account_number LIMIT 1, 1 ORDER BY account_number LIMIT 1, 1
``` ```
| id | account_number | account_number
:--- | :--- | :---
0 | 6 | 6

8
docs/sql/complex.md
View File

@ -142,7 +142,7 @@ The `explain` output is complicated, because a `JOIN` clause is associated with
Result set: Result set:
| a.account_number | a.firstname | a.lastname | e.id | e.name | a.account_number | a.firstname | a.lastname | e.id | e.name
:--- | :--- | :--- | :--- | :--- | :--- | :--- | :--- | :--- :--- | :--- | :--- | :--- | :---
6 | Hattie | Bond | 6 | Jane Smith 6 | Hattie | Bond | 6 | Jane Smith
### Example 2: Cross join ### Example 2: Cross join
@ -167,7 +167,7 @@ JOIN employees_nested e
Result set: Result set:
| a.account_number | a.firstname | a.lastname | e.id | e.name | a.account_number | a.firstname | a.lastname | e.id | e.name
:--- | :--- | :--- | :--- | :--- | :--- | :--- | :--- | :--- :--- | :--- | :--- | :--- | :---
1 | Amber | Duke | 3 | Bob Smith 1 | Amber | Duke | 3 | Bob Smith
1 | Amber | Duke | 4 | Susan Smith 1 | Amber | Duke | 4 | Susan Smith
1 | Amber | Duke | 6 | Jane Smith 1 | Amber | Duke | 6 | Jane Smith
@ -199,7 +199,7 @@ LEFT JOIN employees_nested e
Result set: Result set:
| a.account_number | a.firstname | a.lastname | e.id | e.name | a.account_number | a.firstname | a.lastname | e.id | e.name
:--- | :--- | :--- | :--- | :--- | :--- | :--- | :--- | :--- :--- | :--- | :--- | :--- | :---
1 | Amber | Duke | null | null 1 | Amber | Duke | null | null
6 | Hattie | Bond | 6 | Jane Smith 6 | Hattie | Bond | 6 | Jane Smith
13 | Nanette | Bates | null | null 13 | Nanette | Bates | null | null
@ -350,7 +350,7 @@ Explain:
Result set: Result set:
| a1.firstname | a1.lastname | a1.balance | a1.firstname | a1.lastname | a1.balance
:--- | :--- | :--- | :--- | :--- | :--- :--- | :--- | :---
Amber | Duke | 39225 Amber | Duke | 39225
Nanette | Bates | 32838 Nanette | Bates | 32838

2
docs/sql/datatypes.md
View File

@ -79,7 +79,7 @@ timestamp | `yyyy-MM-dd hh:mm:ss[.fraction]` | `0001-01-01 00:00:01.000000` UTC
The `interval` type represents a temporal duration or a period. The `interval` type represents a temporal duration or a period.
| Type | Syntax | Type | Syntax
:--- | :--- | :--- :--- | :---
interval | `INTERVAL expr unit` interval | `INTERVAL expr unit`
The `expr` unit is any expression that eventually iterates to a quantity value. It represents a unit for interpreting the quantity, including `MICROSECOND`, `SECOND`, `MINUTE`, `HOUR`, `DAY`, `WEEK`, `MONTH`, `QUARTER`, and `YEAR`. The `INTERVAL` keyword and the unit specifier are not case sensitive. The `expr` unit is any expression that eventually iterates to a quantity value. It represents a unit for interpreting the quantity, including `MICROSECOND`, `SECOND`, `MINUTE`, `HOUR`, `DAY`, `WEEK`, `MONTH`, `QUARTER`, and `YEAR`. The `INTERVAL` keyword and the unit specifier are not case sensitive.

13
docs/sql/delete.md
View File

@ -11,6 +11,19 @@ nav_order: 12
The `DELETE` statement deletes documents that satisfy the predicates in the `WHERE` clause. The `DELETE` statement deletes documents that satisfy the predicates in the `WHERE` clause.
If you don't specify the `WHERE` clause, all documents are deleted. If you don't specify the `WHERE` clause, all documents are deleted.
### Setting
The `DELETE` statement is disabled by default. To enable the `DELETE` functionality in SQL, you need to update the configuration by sending the following request:
```json
PUT _plugins/_query/settings
{
"transient": {
"plugins.sql.delete.enabled": "true"
}
}
```
### Syntax ### Syntax
Rule `singleDeleteStatement`: Rule `singleDeleteStatement`:

16
docs/sql/endpoints.md
View File

@ -26,7 +26,7 @@ You can send HTTP GET request with your query embedded in URL parameter.
SQL query: SQL query:
```console ```console
>> curl -H 'Content-Type: application/json' -X GET localhost:9200/_opensearch/_sql?sql=SELECT * FROM accounts >> curl -H 'Content-Type: application/json' -X GET localhost:9200/_plugins/_sql?sql=SELECT * FROM accounts
``` ```
## POST ## POST
@ -40,7 +40,7 @@ You can also send HTTP POST request with your query in request body.
SQL query: SQL query:
```console ```console
>> curl -H 'Content-Type: application/json' -X POST localhost:9200/_opensearch/_sql -d '{ >> curl -H 'Content-Type: application/json' -X POST localhost:9200/_plugins/_sql -d '{
"query" : "SELECT * FROM accounts" "query" : "SELECT * FROM accounts"
}' }'
``` ```
@ -59,7 +59,7 @@ directly.
Explain query: Explain query:
```console ```console
>> curl -H 'Content-Type: application/json' -X POST localhost:9200/_opensearch/_sql/_explain -d '{ >> curl -H 'Content-Type: application/json' -X POST localhost:9200/_plugins/_sql/_explain -d '{
"query" : "SELECT firstname, lastname FROM accounts WHERE age > 20" "query" : "SELECT firstname, lastname FROM accounts WHERE age > 20"
}' }'
``` ```
@ -119,7 +119,7 @@ The `fetch_size` parameter is only supported for the JDBC response format.
SQL query: SQL query:
```console ```console
>> curl -H 'Content-Type: application/json' -X POST localhost:9200/_opensearch/_sql -d '{ >> curl -H 'Content-Type: application/json' -X POST localhost:9200/_plugins/_sql -d '{
"fetch_size" : 5, "fetch_size" : 5,
"query" : "SELECT firstname, lastname FROM accounts WHERE age > 20 ORDER BY state ASC" "query" : "SELECT firstname, lastname FROM accounts WHERE age > 20 ORDER BY state ASC"
}' }'
@ -171,7 +171,7 @@ Result set:
To fetch subsequent pages, use the `cursor` from last response: To fetch subsequent pages, use the `cursor` from last response:
```console ```console
>> curl -H 'Content-Type: application/json' -X POST localhost:9200/_opensearch/_sql -d '{ >> curl -H 'Content-Type: application/json' -X POST localhost:9200/_plugins/_sql -d '{
"cursor": "d:eyJhIjp7fSwicyI6IkRYRjFaWEo1UVc1a1JtVjBZMmdCQUFBQUFBQUFBQU1XZWpkdFRFRkZUMlpTZEZkeFdsWnJkRlZoYnpaeVVRPT0iLCJjIjpbeyJuYW1lIjoiZmlyc3RuYW1lIiwidHlwZSI6InRleHQifSx7Im5hbWUiOiJsYXN0bmFtZSIsInR5cGUiOiJ0ZXh0In1dLCJmIjo1LCJpIjoiYWNjb3VudHMiLCJsIjo5NTF9" "cursor": "d:eyJhIjp7fSwicyI6IkRYRjFaWEo1UVc1a1JtVjBZMmdCQUFBQUFBQUFBQU1XZWpkdFRFRkZUMlpTZEZkeFdsWnJkRlZoYnpaeVVRPT0iLCJjIjpbeyJuYW1lIjoiZmlyc3RuYW1lIiwidHlwZSI6InRleHQifSx7Im5hbWUiOiJsYXN0bmFtZSIsInR5cGUiOiJ0ZXh0In1dLCJmIjo1LCJpIjoiYWNjb3VudHMiLCJsIjo5NTF9"
}' }'
``` ```
@ -182,7 +182,7 @@ The `datarows` can have more than the `fetch_size` number of records in case the
```json ```json
{ {
"cursor": "d:eyJhIjp7fSwicyI6IkRYRjFaWEo1UVc1a1JtVjBZMmdCQUFBQUFBQUFBQU1XZWpkdFRFRkZUMlpTZEZkeFdsWnJkRlZoYnpaeVVRPT0iLCJjIjpbeyJuYW1lIjoiZmlyc3RuYW1lIiwidHlwZSI6InRleHQifSx7Im5hbWUiOiJsYXN0bmFtZSIsInR5cGUiOiJ0ZXh0In1dLCJmIjo1LCJpIjoiYWNjb3VudHMabcde12345", "cursor": "d:eyJhIjp7fSwicyI6IkRYRjFaWEo1UVc1a1JtVjBZMmdCQUFBQUFBQUFBQU1XZWpkdFRFRkZUMlpTZEZkeFdsWnJkRlZoYnpaeVVRPT0iLCJjIjpbeyJuYW1lIjoiZmlyc3RuYW1lIiwidHlwZSI6InRleHQifSx7Im5hbWUiOiJsYXN0bmFtZSIsInR5cGUiOiJ0ZXh0In1dLCJmIjo1LCJpIjoiYWNjb3VudHMabcde12345",
"datarows": [ "datarows": [
[ [
"Abbey", "Abbey",
@ -209,10 +209,10 @@ The `datarows` can have more than the `fetch_size` number of records in case the
``` ```
The `cursor` context is automatically cleared on the last page. The `cursor` context is automatically cleared on the last page.
To explicitly clear cursor context, use the `_opensearch/_sql/close endpoint` operation. To explicitly clear cursor context, use the `_plugins/_sql/close endpoint` operation.
```console ```console
>> curl -H 'Content-Type: application/json' -X POST localhost:9200/_opensearch/_sql/close -d '{ >> curl -H 'Content-Type: application/json' -X POST localhost:9200/_plugins/_sql/close -d '{
"cursor": "d:eyJhIjp7fSwicyI6IkRYRjFaWEo1UVc1a1JtVjBZMmdCQUFBQUFBQUFBQU1XZWpkdFRFRkZUMlpTZEZkeFdsWnJkRlZoYnpaeVVRPT0iLCJjIjpbeyJuYW1lIjoiZmlyc3RuYW1lIiwidHlwZSI6InRleHQifSx7Im5hbWUiOiJsYXN0bmFtZSIsInR5cGUiOiJ0ZXh0In1dLCJmIjo1LCJpIjoiYWNjb3VudHMiLCJsIjo5NTF9" "cursor": "d:eyJhIjp7fSwicyI6IkRYRjFaWEo1UVc1a1JtVjBZMmdCQUFBQUFBQUFBQU1XZWpkdFRFRkZUMlpTZEZkeFdsWnJkRlZoYnpaeVVRPT0iLCJjIjpbeyJuYW1lIjoiZmlyc3RuYW1lIiwidHlwZSI6InRleHQifSx7Im5hbWUiOiJsYXN0bmFtZSIsInR5cGUiOiJ0ZXh0In1dLCJmIjo1LCJpIjoiYWNjb3VudHMiLCJsIjo5NTF9"
}' }'
``` ```

14
docs/sql/index.md
View File

@ -13,17 +13,17 @@ OpenSearch SQL lets you write queries in SQL rather than the [OpenSearch query d
## Workbench ## Workbench
The easiest way to get familiar with the SQL plugin is to use **SQL Workbench** in OpenSearch Dashboards to test various queries. To learn more, see [Workbench](workbench/). The easiest way to get familiar with the SQL plugin is to use **Query Workbench** in OpenSearch Dashboards to test various queries. To learn more, see [Workbench](workbench/).
![OpenSearch Dashboards SQL UI plugin](../images/sql.png) ![OpenSearch Dashboards SQL UI plugin](../images/sql.png)
## REST API ## REST API
To use the SQL plugin with your own applications, send requests to `_opensearch/_sql`: To use the SQL plugin with your own applications, send requests to `_plugins/_sql`:
```json ```json
POST _opensearch/_sql POST _plugins/_sql
{ {
"query": "SELECT * FROM my-index LIMIT 50" "query": "SELECT * FROM my-index LIMIT 50"
} }
@ -40,12 +40,12 @@ Column | Field
You can query multiple indices by listing them or using wildcards: You can query multiple indices by listing them or using wildcards:
```json ```json
POST _opensearch/_sql POST _plugins/_sql
{ {
"query": "SELECT * FROM my-index1,myindex2,myindex3 LIMIT 50" "query": "SELECT * FROM my-index1,myindex2,myindex3 LIMIT 50"
} }
POST _opensearch/_sql POST _plugins/_sql
{ {
"query": "SELECT * FROM my-index* LIMIT 50" "query": "SELECT * FROM my-index* LIMIT 50"
} }
@ -54,13 +54,13 @@ POST _opensearch/_sql
For a sample [curl](https://curl.haxx.se/) command, try: For a sample [curl](https://curl.haxx.se/) command, try:
```bash ```bash
curl -XPOST https://localhost:9200/_opensearch/_sql -u 'admin:admin' -k -H 'Content-Type: application/json' -d '{"query": "SELECT * FROM opensearch_dashboards_sample_data_flights LIMIT 10"}' curl -XPOST https://localhost:9200/_plugins/_sql -u 'admin:admin' -k -H 'Content-Type: application/json' -d '{"query": "SELECT * FROM opensearch_dashboards_sample_data_flights LIMIT 10"}'
``` ```
By default, queries return data in JDBC format, but you can also return data in standard OpenSearch JSON, CSV, or raw formats: By default, queries return data in JDBC format, but you can also return data in standard OpenSearch JSON, CSV, or raw formats:
```json ```json
POST _opensearch/_sql?format=json|csv|raw POST _plugins/_sql?format=json|csv|raw
{ {
"query": "SELECT * FROM my-index LIMIT 50" "query": "SELECT * FROM my-index LIMIT 50"
} }

2
docs/sql/limitation.md
View File

@ -84,7 +84,7 @@ The pagination query enables you to get back paginated responses.
Currently, the pagination only supports basic queries. For example, the following query returns the data with cursor id. Currently, the pagination only supports basic queries. For example, the following query returns the data with cursor id.
```json ```json
POST _opensearch/_sql/ POST _plugins/_sql/
{ {
"fetch_size" : 5, "fetch_size" : 5,
"query" : "SELECT OriginCountry, DestCountry FROM opensearch_dashboards_sample_data_flights ORDER BY OriginCountry ASC" "query" : "SELECT OriginCountry, DestCountry FROM opensearch_dashboards_sample_data_flights ORDER BY OriginCountry ASC"

2
docs/sql/monitoring.md
View File

@ -33,7 +33,7 @@ The meaning of fields in the response is as follows:
SQL query: SQL query:
```console ```console
>> curl -H 'Content-Type: application/json' -X GET localhost:9200/_opensearch/_sql/stats >> curl -H 'Content-Type: application/json' -X GET localhost:9200/_plugins/_sql/stats
``` ```
Result set: Result set:

177
docs/sql/protocol.md
View File

@ -28,7 +28,7 @@ OpenSearch DSL directly.
SQL query: SQL query:
```console ```console
>> curl -H 'Content-Type: application/json' -X POST localhost:9200/_opensearch/_sql -d '{ >> curl -H 'Content-Type: application/json' -X POST localhost:9200/_plugins/_sql -d '{
"query" : "SELECT firstname, lastname, balance FROM accounts", "query" : "SELECT firstname, lastname, balance FROM accounts",
"filter" : { "filter" : {
"range" : { "range" : {
@ -88,7 +88,7 @@ in prepared SQL query.
SQL query: SQL query:
```console ```console
>> curl -H 'Content-Type: application/json' -X POST localhost:9200/_opensearch/_sql -d '{ >> curl -H 'Content-Type: application/json' -X POST localhost:9200/_plugins/_sql -d '{
"query": "SELECT * FROM accounts WHERE age = ?", "query": "SELECT * FROM accounts WHERE age = ?",
"parameters": [{ "parameters": [{
"type": "integer", "type": "integer",
@ -124,13 +124,96 @@ Explain:
} }
} }
} }
```
## JDBC Format
### Description
By default, the plugin returns the JDBC standard format. This format
is provided for JDBC driver and clients that need both schema and
result set well formatted.
### Example 1
Here is an example for normal response. The
`schema` includes field name and its type
and `datarows` includes the result set.
SQL query:
```console
>> curl -H 'Content-Type: application/json' -X POST localhost:9200/_plugins/_sql -d '{
"query" : "SELECT firstname, lastname, age FROM accounts ORDER BY age LIMIT 2"
}'
```
Result set:
```json
{
"schema": [{
"name": "firstname",
"type": "text"
},
{
"name": "lastname",
"type": "text"
},
{
"name": "age",
"type": "long"
}
],
"total": 4,
"datarows": [
[
"Nanette",
"Bates",
28
],
[
"Amber",
"Duke",
32
]
],
"size": 2,
"status": 200
}
```
### Example 2
If any error occurred, error message and the cause will be returned
instead.
SQL query:
```console
>> curl -H 'Content-Type: application/json' -X POST localhost:9200/_plugins/_sql -d '{
"query" : "SELECT unknown FROM accounts"
}'
```
Result set:
```json
{
"error": {
"reason": "Invalid SQL query",
"details": "Field [unknown] cannot be found or used here.",
"type": "SemanticAnalysisException"
},
"status": 400
}
``` ```
## OpenSearch DSL ## OpenSearch DSL
### Description ### Description
By default the plugin returns original response from OpenSearch in The `json` format returns original response from OpenSearch in
JSON. Because this is the native response from OpenSearch, extra JSON. Because this is the native response from OpenSearch, extra
efforts are needed to parse and interpret it. efforts are needed to parse and interpret it.
@ -139,7 +222,7 @@ efforts are needed to parse and interpret it.
SQL query: SQL query:
```console ```console
>> curl -H 'Content-Type: application/json' -X POST localhost:9200/_opensearch/_sql -d '{ >> curl -H 'Content-Type: application/json' -X POST localhost:9200/_plugins/_sql?format=json -d '{
"query" : "SELECT firstname, lastname, age FROM accounts ORDER BY age LIMIT 2" "query" : "SELECT firstname, lastname, age FROM accounts ORDER BY age LIMIT 2"
}' }'
``` ```
@ -195,88 +278,6 @@ Result set:
} }
``` ```
## JDBC Format
### Description
JDBC format is provided for JDBC driver and client side that needs both
schema and result set well formatted.
### Example 1
Here is an example for normal response. The
`schema` includes field name and its type
and `datarows` includes the result set.
SQL query:
```console
>> curl -H 'Content-Type: application/json' -X POST localhost:9200/_opensearch/_sql?format=jdbc -d '{
"query" : "SELECT firstname, lastname, age FROM accounts ORDER BY age LIMIT 2"
}'
```
Result set:
```json
{
"schema": [{
"name": "firstname",
"type": "text"
},
{
"name": "lastname",
"type": "text"
},
{
"name": "age",
"type": "long"
}
],
"total": 4,
"datarows": [
[
"Nanette",
"Bates",
28
],
[
"Amber",
"Duke",
32
]
],
"size": 2,
"status": 200
}
```
### Example 2
If any error occurred, error message and the cause will be returned
instead.
SQL query:
```console
>> curl -H 'Content-Type: application/json' -X POST localhost:9200/_opensearch/_sql?format=jdbc -d '{
"query" : "SELECT unknown FROM accounts"
}'
```
Result set:
```json
{
"error": {
"reason": "Invalid SQL query",
"details": "Field [unknown] cannot be found or used here.",
"type": "SemanticAnalysisException"
},
"status": 400
}
```
## CSV Format ## CSV Format
### Description ### Description
@ -288,7 +289,7 @@ You can also use CSV format to download result set as CSV.
SQL query: SQL query:
```console ```console
>> curl -H 'Content-Type: application/json' -X POST localhost:9200/_opensearch/_sql?format=csv -d '{ >> curl -H 'Content-Type: application/json' -X POST localhost:9200/_plugins/_sql?format=csv -d '{
"query" : "SELECT firstname, lastname, age FROM accounts ORDER BY age" "query" : "SELECT firstname, lastname, age FROM accounts ORDER BY age"
}' }'
``` ```
@ -315,7 +316,7 @@ line tool for post processing.
SQL query: SQL query:
```console ```console
>> curl -H 'Content-Type: application/json' -X POST localhost:9200/_opensearch/_sql?format=raw -d '{ >> curl -H 'Content-Type: application/json' -X POST localhost:9200/_plugins/_sql?format=raw -d '{
"query" : "SELECT firstname, lastname, age FROM accounts ORDER BY age" "query" : "SELECT firstname, lastname, age FROM accounts ORDER BY age"
}' }'
``` ```

26
docs/sql/settings.md
View File

@ -15,19 +15,25 @@ You can update these settings like any other cluster setting:
PUT _cluster/settings PUT _cluster/settings
{ {
"transient" : { "transient" : {
"opendistro.sql.enabled" : false "plugins.sql.enabled" : false
}
}
```
Similarly, you can also update the settings by sending the request to the plugin setting endpoing `_plugins/_query/setting`:
```json
PUT _plugins/_query/settings
{
"transient" : {
"plugins.sql.enabled" : false
} }
} }
``` ```
Setting | Default | Description Setting | Default | Description
:--- | :--- | :--- :--- | :--- | :---
`opendistro.sql.enabled` | True | Change to `false` to disable the plugin. `plugins.sql.enabled` | True | Change to `false` to disable the plugin.
`opendistro.sql.query.slowlog` | 2 seconds | Configure the time limit (in seconds) for slow queries. The plugin logs slow queries as `Slow query: elapsed=xxx (ms)` in `opensearch.log`. `plugins.sql.slowlog` | 2 seconds | Configure the time limit (in seconds) for slow queries. The plugin logs slow queries as `Slow query: elapsed=xxx (ms)` in `opensearch.log`.
`opendistro.sql.query.analysis.enabled` | True | Enables or disables the query analyzer. Changing this setting to `false` lets you bypass strict syntactic and semantic analysis. `plugins.sql.cursor.keep_alive` | 1 minute | This value configures how long the cursor context is kept open. Cursor contexts are resource heavy, so we recommend a low value.
`opendistro.sql.query.analysis.semantic.suggestion` | False | If enabled, the query analyzer suggests correct field names for quick fixes. `plugins.query.memory_limit` | 85% | This setting configures the heap memory usage limit for the circuit breaker of the query engine.
`opendistro.sql.query.analysis.semantic.threshold` | 200 | Because query analysis needs to build semantic context in memory, indices with a large number of fields are be skipped. You can update this setting to apply analysis to smaller or larger indices as needed. `plugins.query.size_limit` | 200 | The setting sets the default size of index that the query engine fetches from OpenSearch.
`opendistro.sql.query.response.format` | JDBC | Sets the default response format for queries. The supported formats are JDBC, JSON, CSV, raw, and table.
`opendistro.sql.cursor.enabled` | False | You can enable or disable pagination for all queries that are supported.
`opendistro.sql.cursor.fetch_size` | 1,000 | You can set the default `fetch_size` for all queries that are supported by pagination. An explicit `fetch_size` passed in request overrides this value.
`opendistro.sql.cursor.keep_alive` | 1 minute | This value configures how long the cursor context is kept open. Cursor contexts are resource heavy, so we recommend a low value.

2
docs/sql/sql-full-text.md
View File

@ -113,7 +113,7 @@ ORDER BY _score
| account_number | address | score | account_number | address | score
:--- | :--- :--- | :--- | :---
1 | 880 Holmes Lane | 0.5 1 | 880 Holmes Lane | 0.5
6 | 671 Bristol Street | 100 6 | 671 Bristol Street | 100
13 | 789 Madison Street | 100 13 | 789 Madison Street | 100

35
docs/sql/troubleshoot.md
View File

@ -12,7 +12,7 @@ The SQL plugin is stateless, so troubleshooting is mostly focused on why a parti
The most common error is the dreaded null pointer exception, which can occur during parsing errors or when using the wrong HTTP method (POST vs. GET and vice versa). The POST method and HTTP request body offer the most consistent results: The most common error is the dreaded null pointer exception, which can occur during parsing errors or when using the wrong HTTP method (POST vs. GET and vice versa). The POST method and HTTP request body offer the most consistent results:
```json ```json
POST _opensearch/_sql POST _plugins/_sql
{ {
"query": "SELECT * FROM my-index WHERE ['name.firstname']='saanvi' LIMIT 5" "query": "SELECT * FROM my-index WHERE ['name.firstname']='saanvi' LIMIT 5"
} }
@ -23,7 +23,7 @@ If a query isn't behaving the way you expect, use the `_explain` API to see the
#### Sample request #### Sample request
```json ```json
POST _opensearch/_sql/_explain POST _plugins/_sql/_explain
{ {
"query": "SELECT * FROM my-index LIMIT 50" "query": "SELECT * FROM my-index LIMIT 50"
} }
@ -39,37 +39,6 @@ POST _opensearch/_sql/_explain
} }
``` ```
## Syntax analysis exception
You might receive the following error if the plugin can't parse your query:
```json
{
"reason": "Invalid SQL query",
"details": "Failed to parse query due to offending symbol [:] at: 'SELECT * FROM xxx WHERE xxx:' <--- HERE...
More details: Expecting tokens in {<EOF>, 'AND', 'BETWEEN', 'GROUP', 'HAVING', 'IN', 'IS', 'LIKE', 'LIMIT',
'NOT', 'OR', 'ORDER', 'REGEXP', '*', '/', '%', '+', '-', 'DIV', 'MOD', '=', '>', '<', '!',
'|', '&', '^', '.', DOT_ID}",
"type": "SyntaxAnalysisException"
}
```
To resolve this error:
1. Check if your syntax follows the [MySQL grammar](https://dev.mysql.com/doc/refman/8.0/en/).
2. If your syntax is correct, disable strict query analysis:
```json
PUT _cluster/settings
{
"persistent" : {
"opendistro.sql.query.analysis.enabled" : false
}
}
```
3. Run the query again to see if it works.
## Index mapping verification exception ## Index mapping verification exception
If you see the following verification exception: If you see the following verification exception:

24
docs/sql/workbench.md
View File

@ -1,14 +1,14 @@
--- ---
layout: default layout: default
title: Workbench title: Query Workbench
parent: SQL parent: SQL
nav_order: 1 nav_order: 1
--- ---
# Workbench # Query Workbench
Use the SQL workbench to easily run on-demand SQL queries, translate SQL into its REST equivalent, and view and save results as text, JSON, JDBC, or CSV. Use the Query workbench to easily run on-demand SQL queries, translate SQL into its REST equivalent, and view and save results as text, JSON, JDBC, or CSV.
## Quick start ## Quick start
@ -38,9 +38,9 @@ To list all your indices:
SHOW TABLES LIKE % SHOW TABLES LIKE %
``` ```
| id | TABLE_NAME | TABLE_NAME
:--- | :--- | :---
0 | accounts | accounts
### Read data ### Read data
@ -53,9 +53,9 @@ FROM accounts
WHERE _id = 1 WHERE _id = 1
``` ```
| id | account_number | firstname | gender | city | balance | employer | state | email | address | lastname | age | account_number | firstname | gender | city | balance | employer | state | email | address | lastname | age
:--- | :--- | :--- | :--- | :--- | :--- | :--- | :--- | :--- | :--- | :--- | :--- | :--- | :--- | :--- | :--- | :--- | :--- | :--- | :--- | :--- | :--- | :---
0 | 1 | Amber | M | Brogan | 39225 | Pyrami | IL | amberduke@pyrami.com | 880 Holmes Lane | Duke | 32 | 1 | Amber | M | Brogan | 39225 | Pyrami | IL | amberduke@pyrami.com | 880 Holmes Lane | Duke | 32
### Delete data ### Delete data
@ -68,6 +68,6 @@ FROM accounts
WHERE _id = 0 WHERE _id = 0
``` ```
| id | deleted_rows | deleted_rows
:--- | :--- | :---
0 | 1 | 1