iSharkFly-Docs
/
opensearch-docs-cn
mirror of https://github.com/iSharkFly-Docs/opensearch-docs-cn
1
0
Fork 0
Code Issues Packages Projects Releases Wiki Activity

Add DQL guide (#5684) 

* Add DQL guide

Signed-off-by: Fanit Kolchina <kolchfa@amazon.com>

* Add all language options and setup steps

Signed-off-by: Fanit Kolchina <kolchfa@amazon.com>

* Delete unused images

Signed-off-by: Fanit Kolchina <kolchfa@amazon.com>

* Revised intro sentences

Signed-off-by: Fanit Kolchina <kolchfa@amazon.com>

* Apply suggestions from code review

Co-authored-by: Melissa Vagi <vagimeli@amazon.com>
Signed-off-by: kolchfa-aws <105444904+kolchfa-aws@users.noreply.github.com>

* Minor rewording

Signed-off-by: Fanit Kolchina <kolchfa@amazon.com>

* Changed query descriptions and added example

Signed-off-by: Fanit Kolchina <kolchfa@amazon.com>

* Apply suggestions from code review

Co-authored-by: Nathan Bower <nbower@amazon.com>
Signed-off-by: kolchfa-aws <105444904+kolchfa-aws@users.noreply.github.com>

* Update _dashboards/dql.md

Co-authored-by: Nathan Bower <nbower@amazon.com>
Signed-off-by: kolchfa-aws <105444904+kolchfa-aws@users.noreply.github.com>

* More editorial comments

Signed-off-by: Fanit Kolchina <kolchfa@amazon.com>

---------

Signed-off-by: Fanit Kolchina <kolchfa@amazon.com>
Signed-off-by: kolchfa-aws <105444904+kolchfa-aws@users.noreply.github.com>
Co-authored-by: Melissa Vagi <vagimeli@amazon.com>
Co-authored-by: Nathan Bower <nbower@amazon.com>
This commit is contained in:
kolchfa-aws 2023-12-12 12:17:19 -05:00 committed by GitHub
parent 8a379826cc
commit f989581cd2
No known key found for this signature in database
GPG Key ID: 4AEE18F83AFDEB23
7 changed files with 356 additions and 125 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

8
_dashboards/discover/index-discover.md
View File

@ -78,13 +78,9 @@ You'll see a view similar to the one in the following image.
## Searching the data
You can use the search toolbar or enter a [DQL]({{site.url}}{{site.baseurl}}/dashboards/discover/dql/) query using the **DevTools** console to search data. While the search toolbar is best for basic queries, such as field name queries, DQL is best for complex queries, such as term, string, Boolean, date, range, or nested queries. DQL provides suggestions for fields and operators as you type, helping you build structured queries.
You can use the search toolbar to enter a [DQL]({{site.url}}{{site.baseurl}}/dashboards/discover/dql/) or [query string]({{site.url}}{{site.baseurl}}/query-dsl/full-text/query-string/) query. The search toolbar is best for basic queries; for full query and filter capability, use [query domain-specific language (DSL)]({{site.url}}{{site.baseurl}}/query-dsl/index/) in the [Dev Tools console]({{site.url}}{{site.baseurl}}/dashboards/dev-tools/index-dev/).
To search data, follow these steps:
1. Enter a simple query in the DQL search bar. For example, enter `FlightDelay:true`, which searches for delayed flights.
2. Select the **Update** button to the right of the search bar.
3. Enter a more complex query in the DQL search bar, and then select **Update**. For example, enter `FlightDelay:true AND FlightDelayMin >= 60`, which searches the data for flights delayed by 60 minutes or more.
For more information, see [Discover and Dashboard search toolbar]({{site.url}}{{site.baseurl}}/dashboards/index/#discover-and-dashboard-search-bar).
## Filtering the data

435
_dashboards/dql.md
View File

@ -1,164 +1,369 @@
---
layout: default
title: DQL
title: Dashboards Query Language (DQL)
nav_order: 130
redirect_from:
- /dashboards/dql/
- /dashboards/discover/dql/
---
# DQL
# Dashboards Query Language (DQL)
Dashboards Query Language (DQL) is a simple text-based query language for filtering data in OpenSearch Dashboards. Similar to [Query DSL]({{site.url}}{{site.baseurl}}/opensearch/query-dsl/index/), DQL uses an HTTP request body. For example, to display your site visitor data for a host in the United States, you would enter `geo.dest:US` in the search field, as shown in the following image.
Dashboards Query Language (DQL) is a simple text-based query language used to filter data in OpenSearch Dashboards. For example, to display your site visitor data for a host in the United States, you would enter `geo.dest:US` in the search field, as shown in the following image.
<img src="{{site.url}}{{site.baseurl}}/images/dashboards/dql-interface.png" alt="Search term using DQL toolbar in Dashboard" width="500">
Before you can search data in Dashboards, you must index it. In OpenSearch, the basic unit of data is a JSON document. Within an index, OpenSearch identifies each document using a unique ID. To learn more about indexing in OpenSearch, see [Index data]({{site.url}}{{site.baseurl}}/opensearch/index-data).
{: .note purple}
DQL and query string query (Lucene) language are the two search bar language options in Discover and Dashboards. To compare these language options, see [Discover and Dashboard search bar]({{site.url}}{{site.baseurl}}/dashboards/index/#discover-and-dashboard-search-bar).
{: .tip}
## Searching with terms queries
## Setup
The most basic query specifies the search term, for example:
To follow this tutorial in OpenSearch Dashboards, expand the following setup steps.
```
host:www.example.com
```
<details closed markdown="block">
<summary>
Setup
</summary>
{: .text-delta}
To access an object's nested field, list the complete path to the field separated by periods. For example, use the following path to retrieve the `lat` field in the `coordinates` object:
Use the following steps to prepare sample data for querying.
```
coordinates.lat:43.7102
```
**Step 1: Set up mappings for the index**
DQL supports leading and trailing wildcards, so you can search for any terms that match your pattern, for example:
```
host.keyword:*.example.com/*
```
To check whether a field exists or has any data, use a wildcard to see whether Dashboards returns any results,for example:
```
host.keyword:*
```
## Searching with Boolean queries
To mix and match or combine multiple queries for more refined results, you can use the Boolean operators `and`, `or`, and `not`. DQL is not case sensitive, so `AND` and `and` are the same, for example:
```
host.keyword:www.example.com and response.keyword:200
```
You also can use multiple Boolean operators in one query, for example:
```
geo.dest:US or response.keyword:200 and host.keyword:www.example.com
```
Remember that Boolean operators follow the logical precedence order of `not`, `and`, and `or`, so if you have an expression like the one in the preceding example, `response.keyword:200 and host.keyword:www.example.com` is evaluated first.
To avoid confusion, use parentheses to dictate the order in which you want to evaluate operands. If you want to evaluate `geo.dest:US or response.keyword:200` first, you can use an expression like the following:
```
(geo.dest:US or response.keyword:200) and host.keyword:www.example.com
```
## Querying dates and ranges
DQL supports numeric inequalities, for example, `bytes >= 15 and memory < 15`.
You can use the same method to find a date before or after the date specified in the query. `>` indicates a search for a date after the specified date, and `<` returns dates before the specified date, for example, `@timestamp > "2020-12-14T09:35:33`.
## Querying nested fields
Searching a document with [nested fields]({{site.url}}{{site.baseurl}}/opensearch/supported-field-types/nested/) requires you to specify the full path of the field to be retrieved. In the following example document, the `superheroes` field has nested objects:
On the main menu, select **Management** > **Dev Tools** to open [Dev Tools]({{site.url}}{{site.baseurl}}/dashboards/dev-tools/run-queries/). Send the following request to create index mappings:
```json
PUT testindex
{
"superheroes":[
{
"hero-name": "Superman",
"real-identity": "Clark Kent",
"age": 28
},
{
"hero-name": "Batman",
"real-identity": "Bruce Wayne",
"age": 26
},
{
"hero-name": "Flash",
"real-identity": "Barry Allen",
"age": 28
},
{
"hero-name": "Robin",
"real-identity": "Dick Grayson",
"age": 15
"mappings" : {
"properties" : {
"date" : {
"type" : "date",
"format" : "yyyy-MM-dd"
}
}
]
}
}
```
{% include copy.html %}
{% include copy-curl.html %}
To retrieve documents that match a specific field using DQL, specify the field, for example:
**Step 2: Ingest the documents into the index**
In **Dev Tools**, ingest the following documents into the index:
```json
PUT /testindex/_doc/1
{
"title": "The wind rises",
"description": "A biographical film",
"media_type": "film",
"date": "2013-07-20",
"page_views": 100
}
```
superheroes: {hero-name: Superman}
{% include copy-curl.html %}
```json
PUT /testindex/_doc/2
{
"title": "Gone with the wind",
"description": "A well-known 1939 American epic historical film",
"media_type": "film",
"date": "1939-09-09",
"page_views": 200
}
```
{% include copy-curl.html %}
```json
PUT /testindex/_doc/3
{
"title": "Chicago: the historical windy city",
"media_type": "article",
"date": "2023-07-29",
"page_views": 300
}
```
{% include copy-curl.html %}
```json
PUT /testindex/_doc/4
{
"article title": "Wind turbines",
"media_type": "article",
"format": "2*3"
}
```
{% include copy-curl.html %}
**Step 3: Create an index pattern**
Follow these steps to create an index pattern for your index:
1. On the main menu, select **Management** > **Dashboards Management**.
1. Select **Index patterns** and then **Create index pattern**.
1. In **Index pattern name**, enter `testindex*`. Select **Next step**.
1. In **Time field**, select `I don't want to use the time filter`.
1. Select **Create index pattern**.
For more information about index patterns, see [Index patterns]({{site.url}}{{site.baseurl}}/dashboards/management/index-patterns/).
**Step 4: Navigate to Discover and select the index pattern**
On the main menu, select **Discover**. In the upper-left corner, select `testindex*` from the **Index patterns** dropdown list. The main panel displays the documents in the index, and you can now try out the DQL queries described on this page.
The [Object fields](#object-fields) and [Nested fields](#nested-fields) sections provide links for additional setup needed to try queries in those sections.
{: .note}
</details>
## Search for terms
By default, DQL searches in the field set as the default field on the index. If the default field is not set, DQL searches all fields. For example, the following query searches for documents containing the words `rises` or `wind` in any of their fields:
```python
rises wind
```
{% include copy.html %}
To retrieve documents that match multiple fields, specify all the fields, for example:
The preceding query matches documents in which any search term appears regardless of the order. By default, DQL combines search terms with an `or`. To learn how to create Boolean expressions containing search terms, see [Boolean operators](#boolean-operators).
To search for a phrase (an ordered sequence of words), surround your text with quotation marks. For example, the following query searches for the exact text "wind rises":
```python
"wind rises"
```
superheroes: {hero-name: Superman} and superheroes: {hero-name: Batman}
{% include copy.html %}
Hyphens are reserved characters in Lucene, so if your search term contains hyphens, DQL might prompt you to switch to Lucene syntax. To avoid this, surround your search term with quotation marks in a phrase search or omit the hyphen in a regular search.
{: .tip}
## Reserved characters
The following is a list of reserved characters in DQL:
`\`, `(`, `)`, `:`, `<`, `>`, `"`, `*`
Use a backslash (`\`) to escape reserved characters. For example, to search for an expression `2*3`, specify the query as `2\*3`:
```plaintext
2\*3
```
{% include copy.html %}
## Search in a field
To search for text in a particular field, specify the field name before the colon:
```python
title: rises wind
```
{% include copy.html %}
The analyzer for the field you're searching parses the query text into tokens and matches documents in which any of the tokens appear.
DQL ignores white space characters, so `title:rises wind` and `title: rises wind` are the same.
{: .tip}
Use wildcards to refer to field names containing spaces. For example, `article*title` matches the `article title` field.
{: .tip}
## Field names
Specify the field name before the colon. The following table contains example queries with field names.
Query | Criterion for a document to match | Matching documents from the `testindex` index
:--- | :--- | :---
`title: wind` | The `title` field contains the word `wind`. | 1, 2
`title: (wind OR windy)` | The `title` field contains the word `wind` or the word `windy`. | 1, 2, 3
`title: "wind rises"` | The `title` field contains the phrase `wind rises`. | 1
`title.keyword: The wind rises` | The `title.keyword` field exactly matches `The wind rises`. | 1
`title*: wind` | Any field that starts with `title` (for example, `title` and `title.keyword`) contains the word `wind` | 1, 2
`article*title: wind` | The field that starts with `article` and ends with `title` contains the word `wind`. Matches the field `article title`. | 4
`description:*` | Documents in which the field `description` exists. | 1, 2
## Wildcards
DQL supports wildcards (`*` only) in both search terms and field names, for example:
```python
t*le: *wind and rise*
```
{% include copy.html %}
## Ranges
DQL supports numeric inequalities using the `>`, `<`, `>=`, and `<=` operators, for example:
```python
page_views > 100 and page_views <= 300
```
{% include copy.html %}
You can use the range operators on dates. For example, the following query searches for documents containing dates within the 2013--2023 range, inclusive:
```python
date >= "2013-01-01" and date < "2024-01-01"
```
{% include copy.html %}
You can query for "not equal to" by using `not` and the field name, for example:
```python
not page_views: 100
```
{% include copy.html %}
Note that the preceding query returns documents in which either the `page_views` field does not contain `100` or the field is not present. To filter by those documents that contain the field `page_views`, use the following query:
```python
page_views:* and not page_views: 100
```
{% include copy.html %}
## Boolean operators
DQL supports the `and`, `or`, and `not` Boolean operators. DQL is not case sensitive, so `AND` and `and` are the same. For example, the following query is a conjunction of two Boolean clauses:
```python
title: wind and description: epic
```
{% include copy.html %}
Boolean operators follow the logical precedence order of `not`, `and`, and `or`, so in the following example, `title: wind and description: epic` is evaluated first:
```python
media_type: article or title: wind and description: epic
```
{% include copy.html %}
To dictate the order of evaluation, group Boolean clauses in parentheses. For example, in the following query, the parenthesized expression is evaluated first:
```python
(media_type: article or title: wind) and description: epic
```
{% include copy.html %}
The field prefix refers to the token that immediately follows the colon. For example, the following query searches for documents in which the `title` field contains `windy` or documents containing the word `historical` in any of their fields:
```python
title: windy or historical
```
{% include copy.html %}
To search for documents in which the `title` field contains `windy` or `historical`, group the terms in parentheses:
```python
title: (windy or historical)
```
{% include copy.html %}
The preceding query is equivalent to `title: windy or title: historical`.
To negate a query, use the `not` operator. For example, the following query searches for documents that contain the word `wind` in the `title` field, are not of the `media_type` `article`, and do not contain `epic` in the `description` field:
```python
title: wind and not (media_type: article or description: epic)
```
{% include copy.html %}
Queries can contain multiple grouping levels, for example:
```python
title: ((wind or windy) and not rises)
```
{% include copy.html %}
## Object fields
To refer to an object's inner field, list the dot path of the field.
To index a document containing an object, follow the steps in the [object field type example]({{site.url}}{{site.baseurl}}/field-types/supported-field-types/object/#example). To search the `name` field of the `patient` object, use the following syntax:
```python
patient.name: john
```
{% include copy.html %}
## Nested fields
To refer to a nested object, list the JSON path of the field.
To index a document containing an object, follow the steps in the [nested field type example]({{site.url}}{{site.baseurl}}/field-types/supported-field-types/nested/#nested-field-type-1).
To search the `name` field of the `patients` object, use the following syntax:
```python
patients: {name: john}
```
{% include copy.html %}
To retrieve documents that match multiple fields, specify all the fields. For example, consider an additional `status` field in the following document:
```json
{
"status": "Discharged",
"patients": [
{"name" : "John Doe", "age" : 56, "smoker" : true},
{"name" : "Mary Major", "age" : 85, "smoker" : false}
]
}
```
To search for a discharged patient whose name is John, specify the `name` and the `status` in the query:
```python
patients: {name: john} and status: discharged
```
{% include copy.html %}
You can combine multiple Boolean and range queries to create a more refined query, for example:
```
superheroes: {hero-name: Superman and age < 50}
```python
patients: {name: john and smoker: true and age < 57}
```
{% include copy.html %}
## Querying doubly nested objects
## Doubly nested fields
If a document has doubly nested objects (objects nested inside other objects), retrieve a field value by specifying the full path to the field. In the following example document, the `superheroes` object is nested inside the `justice-league` object:
Consider a document with a doubly nested field. In this document, both the `patients` and `names` fields are of type `nested`:
```json
{
"justice-league": [
{
"superheroes":[
{
"hero-name": "Superman",
"real-identity": "Clark Kent",
"age": 28
},
{
"hero-name": "Batman",
"real-identity": "Bruce Wayne",
"age": 26
},
{
"hero-name": "Flash",
"real-identity": "Barry Allen",
"age": 28
},
{
"hero-name": "Robin",
"real-identity": "Dick Grayson",
"age": 15
}
]
}
]
"patients": [
{
"names": [
{ "name": "John Doe", "age": 56, "smoker": true },
{ "name": "Mary Major", "age": 85, "smoker": false}
]
}
]
}
```
To search the `name` field of the `patients` object, use the following syntax:
```python
patients: {names: {name: john}}
```
{% include copy.html %}
The following image shows the query result using the example notation `justice-league.superheroes: {hero-name:Superman}`.
In contrast, consider a document in which the `patients` field is of type `object` but the `names` field is of type `nested`:
<img src="{{site.url}}{{site.baseurl}}/images/dashboards/dql-query-result.png" alt="DQL query result" width="1000">
```json
{
"patients":
{
"names": [
{ "name": "John Doe", "age": 56, "smoker": true },
{ "name": "Mary Major", "age": 85, "smoker": false}
]
}
}
```
To search the `name` field of the `patients` object, use the following syntax:
```python
patients.names: {name: john}
```
{% include copy.html %}

28
_dashboards/index.md
View File

@ -20,6 +20,34 @@ OpenSearch Dashboards is the user interface that lets you visualize your OpenSea
| [Create visualizations]({{site.url}}{{site.baseurl}}/dashboards/visualize/viz-index/) | Learn about visualizing data in OpenSearch Dashboards. |
| [Explore and query data]({{site.url}}{{site.baseurl}}/dashboards/discover/index-discover/) | Learn how to explore and query data in OpenSearch. |
## Query languages
Query language | Where you can use it | Description
:--- | :--- | :---
[Query domain-specific language (DSL)]({{site.url}}{{site.baseurl}}/query-dsl/index/) | [Dev Tools]({{site.url}}{{site.baseurl}}/dashboards/dev-tools/index-dev/) | The primary OpenSearch query language that supports creating complex, fully customizable queries.
[Dashboards Query Language (DQL)]({{site.url}}{{site.baseurl}}/dashboards/discover/dql/) | [Discover]({{site.url}}{{site.baseurl}}/dashboards/discover/index-discover/) and [Dashboard]({{site.url}}{{site.baseurl}}/dashboards/dashboard/index/) search bar | A simple text-based query language used to filter data in OpenSearch Dashboards.
[Query string query language]({{site.url}}{{site.baseurl}}/query-dsl/full-text/query-string/) | [Discover]({{site.url}}{{site.baseurl}}/dashboards/discover/index-discover/) and [Dashboard]({{site.url}}{{site.baseurl}}/dashboards/dashboard/index/) search bar | A scaled-down query language whose syntax is based on the Apache Lucene query syntax.
[SQL]({{site.url}}{{site.baseurl}}/search-plugins/sql/sql/index/) | [Query Workbench]({{site.url}}{{site.baseurl}}/dashboards/query-workbench/) | A traditional query language that bridges the gap between relational database concepts and the flexibility of OpenSearch’s document-oriented data storage.
[Piped Processing Language (PPL)]({{site.url}}{{site.baseurl}}/search-plugins/sql/ppl/index/) | [Query Workbench]({{site.url}}{{site.baseurl}}/dashboards/query-workbench/) | The primary language used with observability in OpenSearch. PPL uses a pipe syntax that chains commands into a query.
### Discover and Dashboard search bar
Using the search bar in the [Discover]({{site.url}}{{site.baseurl}}/dashboards/discover/index-discover/) and [Dashboard]({{site.url}}{{site.baseurl}}/dashboards/dashboard/index/) apps, you can search data with the following two languages:
- [DQL]({{site.url}}{{site.baseurl}}/dashboards/discover/dql/)
- [Query string query (Lucene)]({{site.url}}{{site.baseurl}}/query-dsl/full-text/query-string/)
The following table compares DQL and query string query language features.
DQL and query string query language | DQL | Query string query language
:--- | :--- | :---
- Wildcard expressions (DQL supports `*` only)<br> - Ranges<br> - Boolean operations<br> | - Querying nested fields | - Regular expressions<br> - Fuzziness<br> - Proximity queries<br> - Boosting
By default, the query language in the Discover search toolbar is DQL. To switch to query string syntax, select **DQL** and then turn off **OpenSearch Dashboards Query Language**. The query language changes to `Lucene`, as shown in the following image.
![Using query string syntax in OpenSearch Dashboards Discover]({{site.url}}{{site.baseurl}}/images/discover-lucene-syntax.png)
## Observability
| Concept | Description |

2
_dashboards/quickstart.md
View File

@ -36,7 +36,7 @@ Sample datasets come with visualizations, dashboards, and other tools to help yo
In [**Discover**]({{site.url}}{{site.baseurl}}/dashboards/discover/index-discover/), you can:
- Choose data to explore, set a time range for that data, search it using [Dashboards Query Language (DQL)]({{site.url}}{{site.baseurl}}/dashboards/dql/), and filter the results.
- Explore the data, view individual documents, and create tables summarizing the data's contents.
- Explore the data, view individual documents, and create tables summarizing the data.
- Visualize your findings.
## Try it: Getting familiar with Discover

8
_query-dsl/full-text/query-string.md
View File

@ -39,9 +39,11 @@ You can use query string syntax in the following cases:
```
{% include copy-curl.html %}
1. In the Discover app of OpenSearch Dashboards, if you turn off DQL, as shown in the following image.
1. In the OpenSearch Dashboards Discover or Dashboard apps, if you turn off DQL, as shown in the following image.
![Using query string syntax in OpenSearch Dashboards Discover]({{site.url}}{{site.baseurl}}/images/discover-lucene-syntax.png)
For more information, see [Discover]({{site.url}}{{site.baseurl}}/dashboards/discover/index-discover/).
DQL and Query string query (Lucene) language are the two search bar language options in Discover and Dashboards. To compare these language options, see [Discover and Dashboard search bar]({{site.url}}{{site.baseurl}}/dashboards/index/#discover-and-dashboard-search-bar).
{: .tip}
1. If you search using the HTTP request query parameters, for example:
```json
@ -356,7 +358,7 @@ The query returns the same results as the query that uses the `+` and `-` operat
```
{% include copy-curl.html %}
## Grouping
### Grouping
Group multiple clauses or terms into subqueries using parentheses. For example, the following query searches for documents containing the words `gone` or `rises` that must contain the word `wind` in the title:

BIN
images/dashboards/dql-query-result.png
View File

Binary file not shown.
Side by Side

Before

Width:  |  Height:  |  Size: 116 KiB

BIN
images/dashboards/dql-sample-dataset.png
View File

Binary file not shown.
Side by Side

Before

Width:  |  Height:  |  Size: 134 KiB