[DOCS] Significantly improve SQL docs
Introduce SQL commands Move reserved keywords into an appendix Add section on security Introduce concepts section
This commit is contained in:
Costin Leau
parent
68ec958873
commit
872418ff94
|
|
@ -0,0 +1 @@
|
|||
include::syntax-reserved.asciidoc[]
|
||||
|
|
@ -1,5 +1,6 @@
|
|||
[[sql-spec-reserved]]
|
||||
=== Reserved Keywords
|
||||
[appendix]
|
||||
[[sql-syntax-reserved]]
|
||||
= Reserved Keywords
|
||||
|
||||
Table with reserved keywords that need to be quoted. Also provide an example to make it more obvious.
|
||||
|
||||
|
|
@ -0,0 +1,63 @@
|
|||
[[sql-concepts]]
|
||||
== Conventions and Terminology
|
||||
|
||||
For clarity, it is important to establish the meaning behind certain words as, the same wording might convey different meanings to different readers depending on one's familiarity with SQL versus {es}.
|
||||
|
||||
NOTE: This documentation while trying to be complete, does assume the reader has _basic_ understanding of {es} and/or SQL. If that is not the case, please continue reading the documentation however take notes and pursue the topics that are unclear either through the main {es} documentation or through the plethora of SQL material available in the open (there are simply too many excellent resources here to enumerate).
|
||||
|
||||
As a general rule, {es-sql} as the name indicates provides a SQL interface to {es}. As such, it follows the SQL terminology and conventions first, whenever possible. However the backing engine itself is {es} for which {es-sql} was purposely created hence why features or concepts that are not available, or cannot be mapped correctly, in SQL appear
|
||||
in {es-sql}.
|
||||
Last but not least, {es-sql} tries to obey the https://en.wikipedia.org/wiki/Principle_of_least_astonishment[principle of least suprise], though as all things in the world, everything is relative.
|
||||
|
||||
=== Mapping concepts across SQL and {es}
|
||||
|
||||
While SQL and {es} have different terms for the way the data is organized (and different semantics), essentially their purpose is the same.
|
||||
|
||||
So let's start from the bottom; these roughly are:
|
||||
|
||||
[cols="1,1,5", options="header"]
|
||||
|===
|
||||
|SQL
|
||||
|{es}
|
||||
|Description
|
||||
|
||||
|`column`
|
||||
|`field`
|
||||
|In both cases, at the lowest level, data is stored in in _named_ entries, of a variety of <<sql-data-types, data types>>, containing _one_ value. SQL calls such an entry a _column_ while {es} a _field_.
|
||||
Notice that in {es} a field can contain _multiple_ values of the same type (esentially a list) while in SQL, a _column_ can contain _exactly_ one value of said type.
|
||||
{es-sql} will do its best to preserve the SQL semantic and, depending on the query, reject those that return fields with more than one value.
|
||||
|
||||
|`row`
|
||||
|`document`
|
||||
|++Column++s and ++field++s do _not_ exist by themselves; they are part of a `row` or a `document`. The two have slightly different semantics: a `row` tends to be _strict_ (and have more enforcements) while a `document` tends to be a bit more flexible or loose (while still having a structure).
|
||||
|
||||
|`table`
|
||||
|`index`
|
||||
|The target against which queries, whether in SQL or {es} get executed against.
|
||||
|
||||
|`schema`
|
||||
|_implicit_
|
||||
|In RDBMS, `schema` is mainly a namespace of tables and typically used as a security boundary. {es} does not provide an equivalent concept for it. However when security is enabled, {es} automatically applies the security enforcement so that a role sees only the data it is allowed to (in SQL jargon, its _schema_).
|
||||
|
||||
|`catalog` or `database`
|
||||
|`cluster` instance
|
||||
|In SQL, `catalog` or `database` are used interchangebly and represent a set of schemas that is, a number of tables.
|
||||
In {es} the set of indices available are grouped in a `cluster`. The semantics also differ a bit; a `database` is essentially yet another namespace (which can have some implications on the way data is stored) while an {es} `cluster` is a runtime instance, or rather a set of at least one {es} instance (typically running distributed).
|
||||
In practice this means that while in SQL one can potentially have multiple catalogs inside an instance, in {es} one is restricted to only _one_.
|
||||
|
||||
|`cluster`
|
||||
|`cluster` (federated)
|
||||
|Traditionally in SQL, _cluster_ refers to a single RDMBS instance which contains a number of ++catalog++s or ++database++s (see above). The same word can be reused inside {es} as well however its semantic clarified a bit.
|
||||
|
||||
While RDBMS tend to have only one running instance, on a single machine (_not_ distributed), {es} goes the opposite way and by default, is distributed and multi-instance.
|
||||
|
||||
Further more, an {es} `cluster` can be connected to other ++cluster++s in a _federated_ fashion thus `cluster` means:
|
||||
|
||||
single cluster::
|
||||
Multiple {es} instances typically distributed across machines, running within the same namespace.
|
||||
multiple clusters::
|
||||
Multiple clusters, each with its own namespace, connected to each other in a federated setup (see <<modules-cross-cluster-search, Cross cluster Search>>).
|
||||
|
||||
|===
|
||||
|
||||
As one can see while the mapping between the concepts are not exactly one to one and the semantics somewhat different, there are more things in common than differences. In fact, thanks to SQL declarative nature, many concepts can move across {es} transparently and the terminology of the two likely to be used interchangebly through-out the rest of the material.
|
||||
|
|
@ -1,4 +1,3 @@
|
|||
[role="xpack"]
|
||||
[[sql-cli]]
|
||||
== SQL CLI
|
||||
|
||||
|
|
@ -37,18 +36,3 @@ James S.A. Corey |Leviathan Wakes |561 |1306972800000
|
|||
--------------------------------------------------
|
||||
// TODO it'd be lovely to be able to assert that this is correct but
|
||||
// that is probably more work then it is worth right now.
|
||||
|
||||
[[sql-cli-permissions]]
|
||||
[NOTE]
|
||||
===============================
|
||||
If you are using Security you need to add a few permissions to
|
||||
users so they can run SQL. To run SQL using the CLI a user needs
|
||||
`read`, `indices:admin/get`, and `cluster:monitor/main`. The
|
||||
following example configures a role that can run SQL in the CLI
|
||||
for the `test` and `bort` indices:
|
||||
|
||||
["source","yaml",subs="attributes,callouts,macros"]
|
||||
--------------------------------------------------
|
||||
include-tagged::{sql-tests}/security/roles.yml[cli_jdbc]
|
||||
--------------------------------------------------
|
||||
===============================
|
||||
|
|
|
|||
|
|
@ -1,4 +1,3 @@
|
|||
[role="xpack"]
|
||||
[[sql-jdbc]]
|
||||
== SQL JDBC
|
||||
|
||||
|
|
@ -36,11 +35,11 @@ from `artifacts.elastic.co/maven` by adding it to the repositories list:
|
|||
[float]
|
||||
=== Setup
|
||||
|
||||
The driver main class is `org.elasticsearch.xpack.sql.jdbc.jdbc.JdbcDriver`. Note the driver
|
||||
also implements the JDBC 4.0 +Service Provider+ mechanism meaning it is registerd automatically
|
||||
The driver main class is `org.elasticsearch.xpack.sql.jdbc.jdbc.JdbcDriver`.
|
||||
Note the driver implements the JDBC 4.0 +Service Provider+ mechanism meaning it is registerd automatically
|
||||
as long as its available in the classpath.
|
||||
|
||||
Once registered, the driver expects the following syntax as an URL:
|
||||
Once registered, the driver understands the following syntax as an URL:
|
||||
|
||||
["source","text",subs="attributes"]
|
||||
----
|
||||
|
|
@ -120,12 +119,12 @@ Query timeout (in seconds). That is the maximum amount of time waiting for a que
|
|||
|
||||
To put all of it together, the following URL:
|
||||
|
||||
["source","text",subs="attributes"]
|
||||
["source","text"]
|
||||
----
|
||||
jdbc:es://http://server:3456/timezone=UTC&page.size=250
|
||||
----
|
||||
|
||||
Opens up a {es-jdbc} connection to `server` on port `3456`, setting the JDBC timezone to `UTC` and its pagesize to `250` entries.
|
||||
Opens up a {es-sql} connection to `server` on port `3456`, setting the JDBC connection timezone to `UTC` and its pagesize to `250` entries.
|
||||
|
||||
=== API usage
|
||||
|
||||
|
|
@ -175,20 +174,4 @@ connection. For example:
|
|||
["source","java",subs="attributes,callouts,macros"]
|
||||
--------------------------------------------------
|
||||
include-tagged::{jdbc-tests}/SimpleExampleTestCase.java[simple_example]
|
||||
--------------------------------------------------
|
||||
|
||||
[[sql-jdbc-permissions]]
|
||||
[NOTE]
|
||||
===============================
|
||||
If you are using Security you need to add a few permissions to
|
||||
users so they can run SQL. To run SQL a user needs `read` and
|
||||
`indices:admin/get`. Some parts of the API require
|
||||
`cluster:monitor/main`. The following example configures a
|
||||
role that can run SQL in JDBC querying the `test` and `bort`
|
||||
indices:
|
||||
|
||||
["source","yaml",subs="attributes,callouts,macros"]
|
||||
--------------------------------------------------
|
||||
include-tagged::{sql-tests}/security/roles.yml[cli_jdbc]
|
||||
--------------------------------------------------
|
||||
===============================
|
||||
--------------------------------------------------
|
||||
|
|
@ -186,17 +186,3 @@ or fewer results though. `time_zone` is the time zone to use for date
|
|||
functions and date parsing. `time_zone` defaults to `utc` and can take
|
||||
any values documented
|
||||
http://www.joda.org/joda-time/apidocs/org/joda/time/DateTimeZone.html[here].
|
||||
|
||||
[[sql-rest-permissions]]
|
||||
[NOTE]
|
||||
===============================
|
||||
If you are using Security you need to add a few permissions to
|
||||
users so they can run SQL. To run SQL a user needs `read` and
|
||||
`indices:admin/get`. The following example configures a role
|
||||
that can run SQL against the `test` and `bort` indices:
|
||||
|
||||
["source","yaml",subs="attributes,callouts,macros"]
|
||||
--------------------------------------------------
|
||||
include-tagged::{sql-tests}/security/roles.yml[rest]
|
||||
--------------------------------------------------
|
||||
===============================
|
||||
|
|
|
|||
|
|
@ -1,4 +1,3 @@
|
|||
[role="xpack"]
|
||||
[[sql-translate]]
|
||||
== SQL Translate API
|
||||
|
||||
|
|
@ -57,18 +56,3 @@ the normal <<search-request-body,search>> API.
|
|||
|
||||
The request body accepts all of the <<sql-rest-fields,fields>> that
|
||||
the <<sql-rest,SQL REST API>> accepts except `cursor`.
|
||||
|
||||
[[sql-translate-permissions]]
|
||||
[NOTE]
|
||||
===============================
|
||||
If you are using Security you need to add a few permissions to
|
||||
users so they can run translate SQL. To translate SQL a user
|
||||
needs `read` and `indices:admin/get`. The following example
|
||||
configures a role that can run SQL against the `test` and
|
||||
`bort` indices:
|
||||
|
||||
["source","yaml",subs="attributes,callouts,macros"]
|
||||
--------------------------------------------------
|
||||
include-tagged::{sql-tests}/security/roles.yml[rest]
|
||||
--------------------------------------------------
|
||||
===============================
|
||||
|
|
|
|||
|
|
@ -348,6 +348,25 @@ include-tagged::{sql-specs}/datetime.csv-spec[minuteOfHour]
|
|||
include-tagged::{sql-specs}/datetime.csv-spec[secondOfMinute]
|
||||
--------------------------------------------------
|
||||
|
||||
* Extract
|
||||
|
||||
As an alternative, one can support `EXTRACT` to extract fields from datetimes.
|
||||
You can run any <<sql-functions-datetime,datetime function>>
|
||||
with `EXTRACT(<datetime_function> FROM <expression>)`. So
|
||||
|
||||
["source","sql",subs="attributes,callouts,macros"]
|
||||
--------------------------------------------------
|
||||
include-tagged::{sql-specs}/datetime.csv-spec[extractDayOfYear]
|
||||
--------------------------------------------------
|
||||
|
||||
is the equivalent to
|
||||
|
||||
["source","sql",subs="attributes,callouts,macros"]
|
||||
--------------------------------------------------
|
||||
include-tagged::{sql-specs}/datetime.csv-spec[dayOfYear]
|
||||
--------------------------------------------------
|
||||
|
||||
|
||||
[[sql-functions-aggregate]]
|
||||
=== Aggregate Functions
|
||||
|
||||
|
|
|
|||
|
|
@ -19,7 +19,11 @@ indices and return results in tabular format.
|
|||
<<sql-overview, Overview>>::
|
||||
Overview of {es-sql} and its features.
|
||||
<<sql-getting-started, Getting Started>>::
|
||||
Start using SQL right away in {es}
|
||||
Start using SQL right away in {es}.
|
||||
<<sql-concepts, Concepts and Terminology>>::
|
||||
Language conventions across SQL and {es}.
|
||||
<<sql-security,Security>>::
|
||||
Securing {es-sql} and {es}.
|
||||
<<sql-rest,REST API>>::
|
||||
Accepts SQL in a JSON document, executes it, and returns the
|
||||
results.
|
||||
|
|
@ -34,15 +38,17 @@ indices and return results in tabular format.
|
|||
<<sql-functions,Functions and Operators>>::
|
||||
List of functions and operators supported.
|
||||
<<sql-spec,SQL Language>>::
|
||||
Overview of the {es-sql} language, such as data types, syntax and
|
||||
reserved keywords.
|
||||
|
||||
Overview of the {es-sql} language, such as supported data types, commands and
|
||||
syntax.
|
||||
--
|
||||
|
||||
include::overview.asciidoc[]
|
||||
include::getting-started.asciidoc[]
|
||||
include::concepts.asciidoc[]
|
||||
include::security.asciidoc[]
|
||||
include::endpoints/index.asciidoc[]
|
||||
include::functions/index.asciidoc[]
|
||||
include::language/index.asciidoc[]
|
||||
include::appendix/index.asciidoc[]
|
||||
|
||||
:jdbc-tests!:
|
||||
|
|
|
|||
|
|
@ -1,5 +1,5 @@
|
|||
[[sql-data-types]]
|
||||
=== Data Types
|
||||
== Data Types
|
||||
|
||||
Most of {es} <<mapping-types, data types>> are available in {es-sql}, as indicated below:
|
||||
|
||||
|
|
@ -42,7 +42,7 @@ uses the data type _particularities_ of the former over the latter as ultimately
|
|||
|
||||
[[sql-multi-field]]
|
||||
[float]
|
||||
==== SQL and multi-fields
|
||||
=== SQL and multi-fields
|
||||
|
||||
A core concept in {es} is that of an `analyzed` field, that is a full-text value that is interpreted in order
|
||||
to be effectively indexed. These fields are of type <<text, `text`>> and are not used for sorting or aggregations as their actual value depends on the <<analyzer, `analyzer`>> used hence why {es} also offers the <<keyword, `keyword`>> type for storing the _exact_
|
||||
|
|
|
|||
|
|
@ -1,9 +1,10 @@
|
|||
[[sql-spec]]
|
||||
== SQL Language
|
||||
|
||||
This chapter describes the SQL syntax and data types supported in X-Pack.
|
||||
As a general rule, the syntax tries to adhere as much as possible to ANSI SQL to make the transition seamless.
|
||||
This chapter describes the SQL semantics supported in X-Pack namely:
|
||||
|
||||
<<sql-data-types>>:: Data types
|
||||
<<sql-commands>>:: Commands
|
||||
|
||||
include::data-types.asciidoc[]
|
||||
include::syntax.asciidoc[]
|
||||
include::reserved.asciidoc[]
|
||||
include::syntax/index.asciidoc[]
|
||||
|
|
|
|||
|
|
@ -1,123 +0,0 @@
|
|||
[[sql-spec-syntax]]
|
||||
=== SQL Statement Syntax
|
||||
|
||||
// Big list of the entire syntax in SQL
|
||||
|
||||
// Each entry might get its own file and code snippet
|
||||
|
||||
["source","sql",subs="attributes,callouts,macros"]
|
||||
--------------------------------------------------
|
||||
include-tagged::{sql-specs}/select.sql-spec[wildcardWithOrder]
|
||||
--------------------------------------------------
|
||||
|
||||
|
||||
[[sql-spec-syntax-order-by]]
|
||||
==== `ORDER BY`
|
||||
|
||||
Elasticsearch supports `ORDER BY` for consistent ordering. You add
|
||||
any field in the index that has <<doc-values,`doc_values`>> or
|
||||
`SCORE()` to sort by `_score`. By default SQL sorts on what it
|
||||
considers to be the most efficient way to get the results.
|
||||
|
||||
So sorting by a field looks like:
|
||||
|
||||
[source,js]
|
||||
--------------------------------------------------
|
||||
POST /_xpack/sql?format=txt
|
||||
{
|
||||
"query": "SELECT * FROM library ORDER BY page_count DESC LIMIT 5"
|
||||
}
|
||||
--------------------------------------------------
|
||||
// CONSOLE
|
||||
// TEST[setup:library]
|
||||
|
||||
which results in something like:
|
||||
|
||||
[source,text]
|
||||
--------------------------------------------------
|
||||
author | name | page_count | release_date
|
||||
-----------------+--------------------+---------------+------------------------
|
||||
Peter F. Hamilton|Pandora's Star |768 |2004-03-02T00:00:00.000Z
|
||||
Vernor Vinge |A Fire Upon the Deep|613 |1992-06-01T00:00:00.000Z
|
||||
Frank Herbert |Dune |604 |1965-06-01T00:00:00.000Z
|
||||
Alastair Reynolds|Revelation Space |585 |2000-03-15T00:00:00.000Z
|
||||
James S.A. Corey |Leviathan Wakes |561 |2011-06-02T00:00:00.000Z
|
||||
--------------------------------------------------
|
||||
// TESTRESPONSE[s/\|/\\|/ s/\+/\\+/]
|
||||
// TESTRESPONSE[_cat]
|
||||
|
||||
[[sql-spec-syntax-order-by-score]]
|
||||
For sorting by score to be meaningful you need to include a full
|
||||
text query in the `WHERE` clause. If you include multiple full
|
||||
text queries in the `WHERE` clause then their scores will be
|
||||
combined using the same rules as Elasticsearch's
|
||||
<<query-dsl-bool-query,bool query>>. Here is a simple example:
|
||||
|
||||
[source,js]
|
||||
--------------------------------------------------
|
||||
POST /_xpack/sql?format=txt
|
||||
{
|
||||
"query": "SELECT SCORE(), * FROM library WHERE match(name, 'dune') ORDER BY SCORE() DESC"
|
||||
}
|
||||
--------------------------------------------------
|
||||
// CONSOLE
|
||||
// TEST[setup:library]
|
||||
|
||||
Which results in something like:
|
||||
|
||||
[source,text]
|
||||
--------------------------------------------------
|
||||
SCORE() | author | name | page_count | release_date
|
||||
---------------+---------------+-------------------+---------------+------------------------
|
||||
2.288635 |Frank Herbert |Dune |604 |1965-06-01T00:00:00.000Z
|
||||
1.8893257 |Frank Herbert |Dune Messiah |331 |1969-10-15T00:00:00.000Z
|
||||
1.6086555 |Frank Herbert |Children of Dune |408 |1976-04-21T00:00:00.000Z
|
||||
1.4005898 |Frank Herbert |God Emperor of Dune|454 |1981-05-28T00:00:00.000Z
|
||||
--------------------------------------------------
|
||||
// TESTRESPONSE[s/\|/\\|/ s/\+/\\+/ s/\(/\\\(/ s/\)/\\\)/]
|
||||
// TESTRESPONSE[_cat]
|
||||
|
||||
Note that you can return `SCORE()` by adding it to the where clause. This
|
||||
is possible even if you are not sorting by `SCORE()`:
|
||||
|
||||
[source,js]
|
||||
--------------------------------------------------
|
||||
POST /_xpack/sql?format=txt
|
||||
{
|
||||
"query": "SELECT SCORE(), * FROM library WHERE match(name, 'dune') ORDER BY page_count DESC"
|
||||
}
|
||||
--------------------------------------------------
|
||||
// CONSOLE
|
||||
// TEST[setup:library]
|
||||
|
||||
[source,text]
|
||||
--------------------------------------------------
|
||||
SCORE() | author | name | page_count | release_date
|
||||
---------------+---------------+-------------------+---------------+------------------------
|
||||
2.288635 |Frank Herbert |Dune |604 |1965-06-01T00:00:00.000Z
|
||||
1.4005898 |Frank Herbert |God Emperor of Dune|454 |1981-05-28T00:00:00.000Z
|
||||
1.6086555 |Frank Herbert |Children of Dune |408 |1976-04-21T00:00:00.000Z
|
||||
1.8893257 |Frank Herbert |Dune Messiah |331 |1969-10-15T00:00:00.000Z
|
||||
--------------------------------------------------
|
||||
// TESTRESPONSE[s/\|/\\|/ s/\+/\\+/ s/\(/\\\(/ s/\)/\\\)/]
|
||||
// TESTRESPONSE[_cat]
|
||||
|
||||
|
||||
[[sql-spec-syntax-extract]]
|
||||
==== `EXTRACT`
|
||||
|
||||
Elasticsearch supports `EXTRACT` to extract fields from datetimes.
|
||||
You can run any <<sql-functions-datetime,datetime function>>
|
||||
with `EXTRACT(<datetime_function> FROM <expression>)`. So
|
||||
|
||||
["source","sql",subs="attributes,callouts,macros"]
|
||||
--------------------------------------------------
|
||||
include-tagged::{sql-specs}/datetime.csv-spec[extractDayOfYear]
|
||||
--------------------------------------------------
|
||||
|
||||
is the equivalent to
|
||||
|
||||
["source","sql",subs="attributes,callouts,macros"]
|
||||
--------------------------------------------------
|
||||
include-tagged::{sql-specs}/datetime.csv-spec[dayOfYear]
|
||||
--------------------------------------------------
|
||||
|
|
@ -0,0 +1,20 @@
|
|||
[[sql-syntax-describe-table]]
|
||||
=== DESCRIBE TABLE
|
||||
|
||||
.Synopsis
|
||||
[source, sql]
|
||||
----
|
||||
DESCRIBE table
|
||||
----
|
||||
|
||||
or
|
||||
|
||||
[source, sql]
|
||||
----
|
||||
DESC table
|
||||
----
|
||||
|
||||
|
||||
.Description
|
||||
|
||||
`DESC` and `DESCRIBE` are aliases to <<sql-syntax-show-columns>>.
|
||||
|
|
@ -0,0 +1,16 @@
|
|||
[[sql-commands]]
|
||||
== SQL Commands
|
||||
|
||||
This section contains the list of SQL commands supported by {es-sql} along with their syntax:
|
||||
|
||||
<<sql-syntax-describe-table>>:: Describe a table.
|
||||
<<sql-syntax-select>>:: Retrieve rows from zero or more tables.
|
||||
<<sql-syntax-show-columns>>:: List columns in table.
|
||||
<<sql-syntax-show-functions>>:: List supported functions.
|
||||
<<sql-syntax-show-tables>>:: List tables available.
|
||||
|
||||
include::describe-table.asciidoc[]
|
||||
include::select.asciidoc[]
|
||||
include::show-columns.asciidoc[]
|
||||
include::show-functions.asciidoc[]
|
||||
include::show-tables.asciidoc[]
|
||||
|
|
@ -0,0 +1,284 @@
|
|||
[[sql-syntax-select]]
|
||||
=== SELECT
|
||||
|
||||
.Synopsis
|
||||
[source, sql]
|
||||
----
|
||||
SELECT select_expr [, ...]
|
||||
[ FROM table_name ]
|
||||
[ WHERE condition ]
|
||||
[ GROUP BY grouping_element [, ...] ]
|
||||
[ HAVING condition]
|
||||
[ ORDER BY expression [ ASC | DESC ] [, ...] ]
|
||||
[ LIMIT [ count ] ]
|
||||
----
|
||||
|
||||
.Description
|
||||
|
||||
Retrieves rows from zero or more tables.
|
||||
|
||||
The general execution of `SELECT` is as follows:
|
||||
|
||||
. All elements in the `FROM` list are computed (each element can be base or alias table). Currently `FROM` supports exactly one table. Do note however that the table name can be a pattern (see <<sql-syntax-from, FROM Clause>> below).
|
||||
. If the `WHERE` clause is specified, all rows that do not satisfy the condition are eliminated from the output. (See <<sql-syntax-where, WHERE Clause>> below.)
|
||||
. If the `GROUP BY` clause is specified, or if there are aggregate function calls, the output is combined into groups of rows that match on one or more values, and the results of aggregate functions are computed. If the `HAVING` clause is present, it eliminates groups that do not satisfy the given condition. (See <<sql-syntax-group-by, GROUP BY Clause>> and <<sql-syntax-having, HAVING Clause>> below.)
|
||||
. The actual output rows are computed using the `SELECT` output expressions for each selected row or row group.
|
||||
. If the `ORDER BY` clause is specified, the returned rows are sorted in the specified order. If `ORDER BY` is not given, the rows are returned in whatever order the system finds fastest to produce. (See <<sql-syntax-order-by,ORDER BY Clause>> below.)
|
||||
. If the `LIMIT` is specified, the `SELECT` statement only returns a subset of the result rows. (See <<sql-syntax-limit, LIMIT Clause>> below.)
|
||||
|
||||
|
||||
[[sql-syntax-select-list]]
|
||||
==== `SELECT` List
|
||||
|
||||
`SELECT` list, namely the expressions between `SELECT` and `FROM`, represent the output rows of the `SELECT` statement.
|
||||
|
||||
As with a table, every output column of a `SELECT` has a name which can be either specified per column through the `AS` keyword :
|
||||
|
||||
[source,sql]
|
||||
----
|
||||
SELECT column AS c
|
||||
----
|
||||
|
||||
assigned by {es-sql} if no name is given:
|
||||
|
||||
[source,sql]
|
||||
----
|
||||
SELECT 1 + 1
|
||||
----
|
||||
|
||||
or if it's a simple column reference, use its name as the column name:
|
||||
|
||||
[source,sql]
|
||||
----
|
||||
SELECT col FROM table
|
||||
----
|
||||
|
||||
[[sql-syntax-select-wildcard]]
|
||||
==== Wildcard
|
||||
|
||||
To select all the columns in the source, one can use `*`:
|
||||
|
||||
["source","sql",subs="attributes,callouts,macros"]
|
||||
--------------------------------------------------
|
||||
include-tagged::{sql-specs}/select.sql-spec[wildcardWithOrder]
|
||||
--------------------------------------------------
|
||||
|
||||
which essentially returns all columsn found.
|
||||
|
||||
[[sql-syntax-from]]
|
||||
[float]
|
||||
==== FROM Clause
|
||||
|
||||
The `FROM` clause specifies one table for the `SELECT` and has the following syntax:
|
||||
|
||||
[source, sql]
|
||||
----
|
||||
FROM table_name [ [ AS ] alias ]
|
||||
----
|
||||
|
||||
where:
|
||||
|
||||
`table_name`::
|
||||
|
||||
Represents the name (optionally qualified) of an existing table, either a concrete or base one (actual index) or alias.
|
||||
If the table name contains special SQL characters (such as `.`,`-`,etc...) use double quotes to escape them:
|
||||
[source, sql]
|
||||
----
|
||||
SELECT ... FROM "some-table"
|
||||
----
|
||||
|
||||
The name can be a <<multi-index, pattern>> pointing to multiple indices (likely requiring quoting as mentioned above) with the restriction that *all* resolved concrete tables have **exact mapping**.
|
||||
|
||||
`alias`::
|
||||
A substitute name for the `FROM` item containing the alias. An alias is used for brevity or to eliminate ambiguity. When an alias is provided, it completely hides the actual name of the table and must be used in its place.
|
||||
|
||||
[[sql-syntax-where]]
|
||||
[float]
|
||||
==== WHERE Clause
|
||||
|
||||
The optional `WHERE` clause is used to filter rows from the query and has the following syntax:
|
||||
|
||||
[source, sql]
|
||||
----
|
||||
WHERE condition
|
||||
----
|
||||
|
||||
where:
|
||||
|
||||
`condition`::
|
||||
|
||||
Represents an expression that evaluates to a `boolean`. Only the rows that match the condition (to `true`) are returned.
|
||||
|
||||
[[sql-syntax-group-by]]
|
||||
[float]
|
||||
==== GROUP BY
|
||||
|
||||
The `GROUP BY` clause is used to divide the results into groups of rows on matching values from the designated columns. It has the following syntax:
|
||||
|
||||
[source, sql]
|
||||
----
|
||||
GROUP BY grouping_element [, ...]
|
||||
----
|
||||
|
||||
where:
|
||||
|
||||
`grouping_element`::
|
||||
|
||||
Represents an expression on which rows are being grouped _on_. It can be a column name, name or ordinal number of a column or an arbitrary expression of column values.
|
||||
|
||||
When a `GROUP BY` clause is used in a `SELECT`, _all_ output expressions must be either aggregate functions or expressions used for grouping or derivates of (otherwise there would be more than one possible value to return for each ungrouped column).
|
||||
|
||||
[[sql-syntax-having]]
|
||||
[float]
|
||||
==== HAVING
|
||||
|
||||
The `HAVING` clause can be used _only_ along aggregate functions (and thus `GROUP BY`) to filter what groups are kept or not and has the following syntax:
|
||||
|
||||
[source, sql]
|
||||
----
|
||||
GROUP BY condition
|
||||
----
|
||||
|
||||
where:
|
||||
|
||||
`condition`::
|
||||
|
||||
Represents an expression that evaluates to a `boolean`. Only groups that match the condition (to `true`) are returned.
|
||||
|
||||
Both `WHERE` and `HAVING` are used for filtering however there are several differences between them:
|
||||
|
||||
. `WHERE` works on individual *rows*, `HAVING` works on the *groups* created by ``GROUP BY``
|
||||
. `WHERE` is evaluated *before* grouping, `HAVING` is evaluated *after* grouping
|
||||
|
||||
Note that it is possible to have a `HAVING` clause without a ``GROUP BY``. In this case, an __implicit grouping__ is applied, meaning all selected rows are considered to form a single group and `HAVING` can be applied on any of the aggregate functions specified on this group. `
|
||||
As such a query emits only a single row (as there is only a single group), `HAVING` condition returns either one row (the group) or zero if the condition fails.
|
||||
|
||||
[[sql-syntax-order-by]]
|
||||
[float]
|
||||
==== ORDER BY
|
||||
|
||||
The `ORDER BY` clause is used to sort the results of `SELECT` by one or more expressions:
|
||||
|
||||
[source, sql]
|
||||
----
|
||||
ORDER BY expression [ ASC | DESC ] [, ...]
|
||||
----
|
||||
|
||||
where:
|
||||
|
||||
`expression`::
|
||||
|
||||
Represents an input column, an output column or an ordinal number of the position (starting from one) of an output column. Additionally, ordering can be done based on the results _score_ `
|
||||
The direction, if not specified, is by default `ASC` (ascending). `
|
||||
Regardless of the ordering specified, null values are ordered last (at the end).
|
||||
|
||||
IMPORTANT: When used along-side, `GROUP BY` expression can point _only_ to the columns used for grouping.
|
||||
|
||||
For example, the following query sorts by an arbitrary input field (`page_count`):
|
||||
|
||||
[source,js]
|
||||
--------------------------------------------------
|
||||
POST /_xpack/sql?format=txt
|
||||
{
|
||||
"query": "SELECT * FROM library ORDER BY page_count DESC LIMIT 5"
|
||||
}
|
||||
--------------------------------------------------
|
||||
// CONSOLE
|
||||
// TEST[setup:library]
|
||||
|
||||
which results in something like:
|
||||
|
||||
[source,text]
|
||||
--------------------------------------------------
|
||||
author | name | page_count | release_date
|
||||
-----------------`--------------------`---------------`------------------------
|
||||
Peter F. Hamilton|Pandora's Star |768 |2004-03-02T00:00:00.000Z
|
||||
Vernor Vinge |A Fire Upon the Deep|613 |1992-06-01T00:00:00.000Z
|
||||
Frank Herbert |Dune |604 |1965-06-01T00:00:00.000Z
|
||||
Alastair Reynolds|Revelation Space |585 |2000-03-15T00:00:00.000Z
|
||||
James S.A. Corey |Leviathan Wakes |561 |2011-06-02T00:00:00.000Z
|
||||
--------------------------------------------------
|
||||
// TESTRESPONSE[s/\|/\\|/ s/\`/\\`/]
|
||||
// TESTRESPONSE[_cat]
|
||||
|
||||
[[sql-syntax-order-by-score]]
|
||||
==== Order By Score
|
||||
|
||||
When doing full-text queries in the `WHERE` clause, results can be returned based on their
|
||||
{defguide}/relevance-intro.html[score] or _relevance_ to the given query.
|
||||
|
||||
NOTE: When doing multiple text queries in the `WHERE` clause then, their scores will be
|
||||
combined using the same rules as {es}'s
|
||||
<<query-dsl-bool-query,bool query>>.
|
||||
|
||||
To sort based on the `score`, use the special function `SCORE()`:
|
||||
|
||||
[source,js]
|
||||
--------------------------------------------------
|
||||
POST /_xpack/sql?format=txt
|
||||
{
|
||||
"query": "SELECT SCORE(), * FROM library WHERE match(name, 'dune') ORDER BY SCORE() DESC"
|
||||
}
|
||||
--------------------------------------------------
|
||||
// CONSOLE
|
||||
// TEST[setup:library]
|
||||
|
||||
Which results in something like:
|
||||
|
||||
[source,text]
|
||||
--------------------------------------------------
|
||||
SCORE() | author | name | page_count | release_date
|
||||
---------------`---------------`-------------------`---------------`------------------------
|
||||
2.288635 |Frank Herbert |Dune |604 |1965-06-01T00:00:00.000Z
|
||||
1.8893257 |Frank Herbert |Dune Messiah |331 |1969-10-15T00:00:00.000Z
|
||||
1.6086555 |Frank Herbert |Children of Dune |408 |1976-04-21T00:00:00.000Z
|
||||
1.4005898 |Frank Herbert |God Emperor of Dune|454 |1981-05-28T00:00:00.000Z
|
||||
--------------------------------------------------
|
||||
// TESTRESPONSE[s/\|/\\|/ s/\`/\\`/ s/\(/\\\(/ s/\)/\\\)/]
|
||||
// TESTRESPONSE[_cat]
|
||||
|
||||
Note that you can return `SCORE()` by adding it to the where clause. This
|
||||
is possible even if you are not sorting by `SCORE()`:
|
||||
|
||||
[source,js]
|
||||
--------------------------------------------------
|
||||
POST /_xpack/sql?format=txt
|
||||
{
|
||||
"query": "SELECT SCORE(), * FROM library WHERE match(name, 'dune') ORDER BY page_count DESC"
|
||||
}
|
||||
--------------------------------------------------
|
||||
// CONSOLE
|
||||
// TEST[setup:library]
|
||||
|
||||
[source,text]
|
||||
--------------------------------------------------
|
||||
SCORE() | author | name | page_count | release_date
|
||||
---------------`---------------`-------------------`---------------`------------------------
|
||||
2.288635 |Frank Herbert |Dune |604 |1965-06-01T00:00:00.000Z
|
||||
1.4005898 |Frank Herbert |God Emperor of Dune|454 |1981-05-28T00:00:00.000Z
|
||||
1.6086555 |Frank Herbert |Children of Dune |408 |1976-04-21T00:00:00.000Z
|
||||
1.8893257 |Frank Herbert |Dune Messiah |331 |1969-10-15T00:00:00.000Z
|
||||
--------------------------------------------------
|
||||
// TESTRESPONSE[s/\|/\\|/ s/\`/\\`/ s/\(/\\\(/ s/\)/\\\)/]
|
||||
// TESTRESPONSE[_cat]
|
||||
|
||||
NOTE:
|
||||
Trying to return `score` from a non full-text queries will return the same value for all results, as
|
||||
all are equilley relevant.
|
||||
|
||||
[[sql-syntax-limit]]
|
||||
[float]
|
||||
==== LIMIT
|
||||
|
||||
The `LIMIT` clause restricts (limits) the number of rows returns using the format:
|
||||
|
||||
[source, sql]
|
||||
----
|
||||
LIMIT ( count | ALL )
|
||||
----
|
||||
|
||||
where
|
||||
|
||||
count:: is a positive integer or zero indicating the maximum *possible* number of results being returned (as there might be less matches than the limit). If `0` is specified, no results are returned.
|
||||
|
||||
ALL:: indicates there is no limit and thus all results are being returned.
|
||||
|
|
@ -0,0 +1,12 @@
|
|||
[[sql-syntax-show-columns]]
|
||||
=== SHOW COLUMNS
|
||||
|
||||
.Synopsis
|
||||
[source, sql]
|
||||
----
|
||||
SHOW COLUMNS [ FROM | IN ] ? table
|
||||
----
|
||||
|
||||
.Description
|
||||
|
||||
List the columns in table and their data type (and other attributes).
|
||||
|
|
@ -0,0 +1,14 @@
|
|||
[[sql-syntax-show-functions]]
|
||||
=== SHOW FUNCTIONS
|
||||
|
||||
.Synopsis
|
||||
[source, sql]
|
||||
----
|
||||
SHOW FUNCTIONS [ LIKE? pattern<1>? ]?
|
||||
----
|
||||
|
||||
<1> SQL match pattern
|
||||
|
||||
.Description
|
||||
|
||||
List all the SQL functions and their type. The `LIKE` clause can be used to restrict the list of names to the given pattern.
|
||||
|
|
@ -0,0 +1,14 @@
|
|||
[[sql-syntax-show-tables]]
|
||||
=== SHOW TABLES
|
||||
|
||||
.Synopsis
|
||||
[source, sql]
|
||||
----
|
||||
SHOW TABLES [ LIKE? pattern<1>? ]?
|
||||
----
|
||||
|
||||
<1> SQL match pattern
|
||||
|
||||
.Description
|
||||
|
||||
List the tables available to the current user and their type. The `LIKE` clause can be used to restrict the list of names to the given pattern.
|
||||
|
|
@ -0,0 +1,37 @@
|
|||
[[sql-security]]
|
||||
== Security
|
||||
|
||||
{es-sql} integrates with security, if this is enabled on your cluster.
|
||||
In such a scenario, {es-sql} supports both security at the transport layer (by encrypting the communication between the consumer and the server) and authentication (for the access layer).
|
||||
|
||||
[float]
|
||||
==== SSL/TLS configuration
|
||||
|
||||
In case of an encrypted transport, the SSL/TLS support needs to be enabled in {es-sql} to properly establish communication with {es}. This is done by setting the `ssl` property to `true` or by using the `https` prefix in the URL. +
|
||||
Depending on your SSL configuration (whether the certificates are signed by a CA or not, whether they are global at JVM level or just local to one application), might require setting up the `keystore` and/or `truststore`, that is where the _credentials_ are stored (`keystore` - which typically stores private keys and certificates) and how to _verify_ them (`truststore` - which typically stores certificates from third party also known as CA - certificate authorities). +
|
||||
Typically (and again, do note that your environment might differ significantly), if the SSL setup for {es-sql} is not already done at the JVM level, one needs to setup the keystore if the {es-sql} security requires client authentication (PKI - Public Key Infrastructure), and setup `truststore` if SSL is enabled.
|
||||
|
||||
[float]
|
||||
==== Authentication
|
||||
|
||||
The authentication support in {es-sql} is of two types:
|
||||
|
||||
Username/Password:: Set these through `user` and `password` properties.
|
||||
PKI/X.509:: Use X.509 certificates to authenticate {es-sql} to {es}. For this, one would need to setup the `keystore` containing the private key and certificate to the appropriate user (configured in {es}) and the `truststore` with the CA certificate used to sign the SSL/TLS certificates in the {es} cluster. That is, one should setup the key to authenticate {es-sql} and also to verify that is the right one. To do so, one should set the `ssl.keystore.location` and `ssl.truststore.location` properties to indicate the `keystore` and `truststore` to use. It is recommended to have these secured through a password in which case `ssl.keystore.pass` and `ssl.truststore.pass` properties are required.
|
||||
|
||||
[float]
|
||||
[[sql-security-permissions]]
|
||||
==== Permissions (server-side)
|
||||
Lastly, one the server one need to add a few permissions to
|
||||
users so they can run SQL. To run SQL a user needs `read` and
|
||||
`indices:admin/get` permissions at minimum while some parts of
|
||||
the API require `cluster:monitor/main`.
|
||||
|
||||
The following example configures a role that can run SQL in JDBC querying the `test` and `bort`
|
||||
indices:
|
||||
|
||||
["source","yaml",subs="attributes,callouts,macros"]
|
||||
--------------------------------------------------
|
||||
include-tagged::{sql-tests}/security/roles.yml[cli_jdbc]
|
||||
--------------------------------------------------
|
||||
|
||||
Loading…
Reference in New Issue