mirror of
https://github.com/honeymoose/OpenSearch.git
synced 2025-02-14 17:05:36 +00:00
783c9ed372
Introduce client-side sorting of groups based on aggregate functions. To allow this, the Analyzer has been extended to push down to underlying Aggregate, aggregate function and the Querier has been extended to identify the case and consume the results in order and sort them based on the given columns. The underlying QueryContainer has been slightly modified to allow a view of the underlying values being extracted as the columns used for sorting might not be requested by the user. The PR also adds minor tweaks, mainly related to tree output. Close #35118
116 lines
5.1 KiB
Plaintext
116 lines
5.1 KiB
Plaintext
[role="xpack"]
|
|
[testenv="basic"]
|
|
[[sql-limitations]]
|
|
== SQL Limitations
|
|
|
|
beta[]
|
|
|
|
[float]
|
|
=== Nested fields in `SYS COLUMNS` and `DESCRIBE TABLE`
|
|
|
|
{es} has a special type of relationship fields called `nested` fields. In {es-sql} they can be used by referencing their inner
|
|
sub-fields. Even though `SYS COLUMNS` and `DESCRIBE TABLE` will still display them as having the type `NESTED`, they cannot
|
|
be used in a query. One can only reference its sub-fields in the form:
|
|
|
|
[source, sql]
|
|
--------------------------------------------------
|
|
[nested_field_name].[sub_field_name]
|
|
--------------------------------------------------
|
|
|
|
For example:
|
|
|
|
[source, sql]
|
|
--------------------------------------------------
|
|
SELECT dep.dep_name.keyword FROM test_emp GROUP BY languages;
|
|
--------------------------------------------------
|
|
|
|
[float]
|
|
=== Multi-nested fields
|
|
|
|
{es-sql} doesn't support multi-nested documents, so a query cannot reference more than one nested field in an index.
|
|
This applies to multi-level nested fields, but also multiple nested fields defined on the same level. For example, for this index:
|
|
|
|
[source, sql]
|
|
----------------------------------------------------
|
|
column | type | mapping
|
|
----------------------+---------------+-------------
|
|
nested_A |STRUCT |NESTED
|
|
nested_A.nested_X |STRUCT |NESTED
|
|
nested_A.nested_X.text|VARCHAR |KEYWORD
|
|
nested_A.text |VARCHAR |KEYWORD
|
|
nested_B |STRUCT |NESTED
|
|
nested_B.text |VARCHAR |KEYWORD
|
|
----------------------------------------------------
|
|
|
|
`nested_A` and `nested_B` cannot be used at the same time, nor `nested_A`/`nested_B` and `nested_A.nested_X` combination.
|
|
For such situations, {es-sql} will display an error message.
|
|
|
|
[float]
|
|
=== Paginating nested inner hits
|
|
|
|
When SELECTing a nested field, pagination will not work as expected, {es-sql} will return __at least__ the page size records.
|
|
This is because of the way nested queries work in {es}: the root nested field will be returned and it's matching inner nested fields as well,
|
|
pagination taking place on the **root nested document and not on its inner hits**.
|
|
|
|
[float]
|
|
=== Normalized `keyword` fields
|
|
|
|
`keyword` fields in {es} can be normalized by defining a `normalizer`. Such fields are not supported in {es-sql}.
|
|
|
|
[float]
|
|
=== Array type of fields
|
|
|
|
Array fields are not supported due to the "invisible" way in which {es} handles an array of values: the mapping doesn't indicate whether
|
|
a field is an array (has multiple values) or not, so without reading all the data, {es-sql} cannot know whether a field is a single or multi value.
|
|
|
|
[float]
|
|
=== Sorting by aggregation
|
|
|
|
When doing aggregations (`GROUP BY`) {es-sql} relies on {es}'s `composite` aggregation for its support for paginating results.
|
|
However this type of aggregation does come with a limitation: sorting can only be applied on the key used for the aggregation's buckets.
|
|
{es-sql} overcomes this limitation by doing client-side sorting however as a safety measure, allows only up to *512* rows.
|
|
|
|
It is recommended to use `LIMIT` for queries that use sorting by aggregation, essentially indicating the top N results that are desired:
|
|
|
|
[source, sql]
|
|
--------------------------------------------------
|
|
SELECT * FROM test GROUP BY age ORDER BY COUNT(*) LIMIT 100;
|
|
--------------------------------------------------
|
|
|
|
It is possible to run the same queries without a `LIMIT` however in that case if the maximum size (*512*) is passed, an exception will be
|
|
returned as {es-sql} is unable to track (and sort) all the results returned.
|
|
|
|
[float]
|
|
=== Using aggregation functions on top of scalar functions
|
|
|
|
Aggregation functions like <<sql-functions-aggs-min,`MIN`>>, <<sql-functions-aggs-max,`MAX`>>, etc. can only be used
|
|
directly on fields, and so queries like `SELECT MAX(abs(age)) FROM test` are not possible.
|
|
|
|
[float]
|
|
=== Using a sub-select
|
|
|
|
Using sub-selects (`SELECT X FROM (SELECT Y)`) is **supported to a small degree**: any sub-select that can be "flattened" into a single
|
|
`SELECT` is possible with {es-sql}. For example:
|
|
|
|
["source","sql",subs="attributes,macros"]
|
|
--------------------------------------------------
|
|
include-tagged::{sql-specs}/docs.csv-spec[limitationSubSelect]
|
|
--------------------------------------------------
|
|
|
|
The query above is possible because it is equivalent with:
|
|
|
|
["source","sql",subs="attributes,macros"]
|
|
--------------------------------------------------
|
|
include-tagged::{sql-specs}/docs.csv-spec[limitationSubSelectRewritten]
|
|
--------------------------------------------------
|
|
|
|
But, if the sub-select would include a `GROUP BY` or `HAVING` or the enclosing `SELECT` would be more complex than `SELECT X
|
|
FROM (SELECT ...) WHERE [simple_condition]`, this is currently **un-supported**.
|
|
|
|
[float]
|
|
=== Using <<sql-functions-aggs-first, `FIRST`>>/<<sql-functions-aggs-last,`LAST`>> aggregation functions in `HAVING` clause
|
|
|
|
Using `FIRST` and `LAST` in the `HAVING` clause is not supported. The same applies to
|
|
<<sql-functions-aggs-min,`MIN`>> and <<sql-functions-aggs-max,`MAX`>> when their target column
|
|
is of type <<keyword, `keyword`>> as they are internally translated to `FIRST` and `LAST`.
|