Commit Graph

151 Commits

Author SHA1 Message Date
Andrei Stefan 7fe2843a9e SQL: specify command to run the CLI on a remote machine without Elasticsearch (#52626)
(cherry picked from commit 477b0eda8322c5dcb6861bd262bfeec17ff133fe)
2020-02-21 13:29:58 +02:00
Bogdan Pintea db8b306085 SQL: update ODBC docs, cover Cloud ID, latest params (#52291)
* Refresh snapshots with latest look

Add new snapshots with the connection editor to reflect the latest UI.

* Document the effect of the late added params

Add details about the Cloud ID setting, as well as those on the Misc
tab.


(cherry picked from commit afa67625e847e99a22264f5dd6fa0daa37786c6f)
2020-02-19 17:42:28 +01:00
debadair 969cdfaaa4 [DOCS] Clean up links from SQL client app pages. (#52442)
* [DOCS] Clean up links from SQL client app pages.

* Linked to client apps from prereqs.
2020-02-18 12:42:20 -08:00
Andrei Stefan 4eea9c20ee SQL: document the use of a filter on _routing (#52355)
* Fix "Description"s for various sections in the functions pages.
* Added a TIP for searching using a routing key.
* Other small polishings

(cherry picked from commit 9fad0b1ac4409a42c435ed040f41cbaea18930a3)
2020-02-14 19:00:26 +02:00
Marios Trivyzas d9fd6fc90c
SQL: [Docs] Fix typo
Add missing closing "`"

Follows: c2e0552537
2020-02-12 21:50:57 +01:00
Marios Trivyzas c2e0552537 SQL: [Docs] Add limitation for sorting on aggs (#52210)
Add a section to point out that when ordering by an aggregate
only plain aggregate functions are allowed, no scalars/operators
can be used on top of them.

Fixes: #52204
(cherry picked from commit 78a1185549ff7f3229fd2d036567eb2a4f2cf230)
2020-02-12 12:56:06 +01:00
Marios Trivyzas f373020349 SQL: Fix ORDER BY YEAR() function (#51562)
Previously, if YEAR() was used as and ORDER BY argument without being
wrapped with another scalar (e.g. YEAR(birth_date) + 10), no script
ordering was used but instead the underlying field (e.g. birth_date)
was used instead as a performance optimisation. This works correctly if
YEAR() is the only ORDER BY arg but if further args are used as tie
breakers for the ordering wrong results are produced. This is because
2 rows with the different birth_date but on the same year are not tied
as the underlying ordering is on birth_date and not on the
YEAR(birth_date), and the following ORDER BY args are ignored.

Remove this optimisation for YEAR() to avoid incorrect results in
such cases.

As a consequence another bug is revealed: scalar functions on top
of nested fields produce scripted sorting/filtering which is not yet
supported. In such cases no error was thrown but instead all values for
such nested fields were null and were passed to the script implementing
the sorting/filtering, producing incorrect results.

Detect such cases and throw a validation exception.

Fixes: #51224
(cherry picked from commit f41efd6753dc3650a7eabb3e07b02b3b32c5704c)
2020-01-30 15:29:36 +01:00
Andrei Stefan 2908b7e5fc
SQL: add support for passing query parameters in REST API calls (#51029) (#51222)
* REST PreparedStatement-like query parameters are now supported in the form of an array of non-object, non-array values where ES SQL parser will try to infer the data type of the value being passed as parameter.

(cherry picked from commit 45b8bf619aecb1c03d7bc0cf06928dcc36005a66)
2020-01-20 16:40:19 +02:00
James Rodewig 0753915eed [DOCS] Update SQL REST API pages for new structure (#50690)
#43007 restructured the SQL REST API docs so they display across several pages.

This updates up a reference that assumes a single page in the "Paginating through a large response" section. It also reformats a tip for the Kibana console.

Closes #50688
2020-01-07 09:27:34 -06:00
Nik Everett 01293ebad5
Fix docs typos (#50365) (#50464)
Fixes a few typos in the docs.

Co-authored-by: Xiang Dai <764524258@qq.com>
2019-12-23 12:38:17 -05:00
Andrei Stefan e2982b2110 SQL: handle NULL arithmetic operations with INTERVALs (#49633)
(cherry picked from commit ce727615c08cf5ae422feb77f69ea24fb53cd9d1)
2019-12-02 17:31:05 +02:00
Marios Trivyzas 3c69d4d0bd
SQL: Add TRUNC alias for TRUNCATE (#49571)
Add TRUNC as alias to already implemented TRUNCATE
numeric function which is the flavour supported by
Oracle and PostgreSQL.

Relates to: #41195

(cherry picked from commit f2aa7f0779bc5cce40cc0c1f5e5cf1a5bb7d84f0)
2019-11-26 12:32:54 +01:00
Bogdan Pintea 8c2ab8bb72 SQL:Docs: add the PIVOT clause to SELECT section (#49129)
The PR adds the documentation on the PIVOT clause.

(cherry picked from commit a55b36065e6496c44b6e3191296931d477a8e5f5)
2019-11-20 18:21:06 +01:00
Marios Trivyzas fd1bb4a33a SQL: Fix issue with mins & hours for DATEDIFF (#49252)
Previously, DATEDIFF for minutes and hours was doing a
rounding calculation using all the time fields (secs, msecs/micros/nanos).
Instead it should first truncate the 2 dates to the respective field (mins or hours)
zeroing out all the more detailed time fields and then make the subtraction.

(cherry picked from commit 124cd18e20429e19d52fd8dc383827ea5132d428)
2019-11-19 14:25:28 +01:00
Daniel Andion d0cbbf9d58
SQL: [Docs] Typo in HAVING section (#48609)
`HAVING` section code states `GROUP BY`  instead of the appropriate keyword.

(cherry picked from commit 9d505dc3db51e250fdf1b44e4d952dcd97bf1bc1)
2019-10-29 16:37:39 +01:00
Marios Trivyzas 7fddf198b7 SQL: Implement DATEDIFF function (#47920)
Implement DATEDIFF/TIMESTAMPDIFF function as per the MS-SQL spec:
https://docs.microsoft.com/en-us/sql/t-sql/functions/datediff-transact-sql?view=sql-server-2017
which allows a user to substract two date/datetime fields and return the
difference in the date/time unit specified.

Closes: #47919
(cherry picked from commit 745699f38dc8222670ffd65b66df33b5da39040b)
2019-10-15 15:12:11 +02:00
Marios Trivyzas 6589617a51
SQL: Fix arg verification for DateAddProcessor (#48041)
Previously, the safety check for the 2nd argument of the DateAddProcessor was
restricting it to Integer which was wrong since we allow all non-rational
numbers, so it's changed to a Number check as it's done in other cases.

Enhanced some tests regarding the check for an integer (non-rational
argument).

(cherry picked from commit 0516b6eaf5eb98fa5bd087c3fece80139a6b118e)
2019-10-15 12:52:11 +02:00
Marios Trivyzas 59b3294bc9 SQL: Implement DATEADD function (#47747)
Implement DATEADD/TIMESTAMPADD function as per the MS-SQL spec:
https://docs.microsoft.com/en-us/sql/t-sql/functions/dateadd-transact-sql?view=sql-server-2017
which allows a user to add/subtract specified number of specified units
to/from a date/datetime field/expression.

Closes: #47746
(cherry picked from commit e624bc281bebb4bbe0b0c2e0a8cbc712e50097a8)
2019-10-10 16:22:13 +02:00
Andrei Stefan 75a7daae73 SQL: use calendar interval of 1y instead of fixed interval for grouping by YEAR and HISTOGRAMs (#47558)
(cherry picked from commit 55f5463eee4ecea3537df4b34645f1d87472a802)
2019-10-09 11:51:35 +03:00
Lisa Cawley 39ef795085
[DOCS] Cleans up links to security content (#47610) (#47703) 2019-10-07 15:23:19 -07:00
Marios Trivyzas f792dbf239 SQL: Implement DATE_PART function (#47206)
DATE_PART(<datetime unit>, <date/datetime>) is a function that allows
the user to extract the specified unit from a date/datetime field
similar to the EXTRACT (<datetime unit> FROM <date/datetime>) but
with different names and aliases for the units and it also provides more
options like `DATE_PART('tzoffset', datetimeField)`.

Implemented following the SQL server's spec: https://docs.microsoft.com/en-us/sql/t-sql/functions/datepart-transact-sql?view=sql-server-2017
with the difference that the <datetime unit> argument is either a
literal single quoted string or gets a value from a table field, whereas
in SQL server keywords are used (unquoted identifiers) and it's not
possible to use a value coming for a table column.

Closes: #46372
(cherry picked from commit ead743d3579eb753fd314d4a58fae205e465d72e)
2019-10-01 16:28:27 +03:00
Marios Trivyzas 01623f9f1c
SQL: Add alias DATETRUNC to DATE_TRUNC function (#47173)
To be on the safe side in terms of use cases also add the alias
DATETRUNC to the DATE_TRUNC function.

Follows: #46473

(cherry picked from commit 9ac223cb1fc66486f86e218fa785a32b61e9bacc)
2019-09-27 15:38:51 +03:00
Igor Motov ae202fda21 SQL: Add support for shape type (#46464)
Enables support for Cartesian geometries shape type. We still need to
decide how to handle the distance function since it is currently using
the haversine distance formula and returns results in meters, which
doesn't make any sense for Cartesian geometries.

Closes #46412
Relates to #43644
2019-09-26 09:47:42 -04:00
Costin Leau 16bed7f017 DOC: Fix typo
(cherry picked from commit d874f1139ef0335db181d932f1e1299e1c8e40b8)
2019-09-16 12:35:21 +03:00
Costin Leau e5eb4b49c9 DOC: Update section for SQuirrel SQL 4.0.0 (#46726)
The just released SQuirrel SQL 4.0.0 provides an Elasticsearch driver
definition out of the box; update the documentation to reflect that.

(cherry picked from commit 3aa417ed74947e69f0ff605b1c210a0678a3cb9f)
2019-09-16 12:27:31 +03:00
James Rodewig 2831535cf9 [DOCS] Replace "// CONSOLE" comments with [source,console] (#46679) 2019-09-13 11:44:54 -04:00
Marios Trivyzas d956509394 SQL: Implement DATE_TRUNC function (#46473)
DATE_TRUNC(<truncate field>, <date/datetime>) is a function that allows
the user to truncate a timestamp to the specified field by zeroing out
the rest of the fields. The function is implemented according to the
spec from PostgreSQL: https://www.postgresql.org/docs/current/functions-datetime.html#FUNCTIONS-DATETIME-TRUNC

Closes: #46319
(cherry picked from commit b37e96712db1aace09f17b574eb02ff6b942a297)
2019-09-11 21:41:02 +03:00
James Rodewig f04573f8e8
[DOCS] [5 of 5] Change // TESTRESPONSE comments to [source,console-results] (#46449) (#46459) 2019-09-06 16:09:09 -04:00
James Rodewig bb7bff5e30
[DOCS] Replace "// TESTRESPONSE" magic comments with "[source,console-result] (#46295) (#46418) 2019-09-06 09:22:08 -04:00
James Rodewig 1f36c4e50c
[DOCS] Replace "// CONSOLE" comments with [source,console] (#46159) (#46332) 2019-09-05 10:11:25 -04:00
Costin Leau 867cfe0223 DOC: Update SQL docs for DbVis and Workbench/J (#45981)
Refresh the setup for the new versions of DbVisualizer and SQL
Workbench/J which have Elasticsearch JDBC support out of the box.

(cherry picked from commit 6d257194c1055d060505e0faaaa37b41e21699f5)
2019-08-29 11:14:03 +03:00
James Rodewig db386617c7 [DOCS] Correct `IIF` conditional section title (#45979) 2019-08-26 11:13:45 -04:00
Bogdan Pintea 499be8398f SQL: ODBC: document newest connection string parameters (#44185)
* SQL: ODBC: document newest conn string parameters

This commit adds the documentation for two newly added connection string
parameters: AutoEscapePVA and IndexIncludeFrozen.

It also removes the recommended OSes from the prerequisites list and
places the recommendation distinctively: the unmet prerequisites will
fail the installation, while the driver would install on other OSes than
those recommended.

* address review suggestions.

- adjust phrasing for clearer message.

(cherry picked from commit e18ac10c6e163a04f5b7cf7fa72f262882ffb711)
2019-08-12 13:49:11 +02:00
Lisa Cawley 9e901c5f94
[DOCS] Edits 7.3 release highlights (#45066) 2019-07-31 15:49:27 -07:00
Andrei Stefan 2633d11eb7
Switch from using docvalue_fields to extracting values from _source (#44062) (#44804)
* Switch from using docvalue_fields to extracting values from _source
where applicable. Doing this means parsing the _source and handling the
numbers parsing just like Elasticsearch is doing it when it's indexing
a document.
* This also introduces a minor limitation: aliases type of fields that
are NOT part of a tree of sub-fields will not be able to be retrieved
anymore. field_caps API doesn't shed any light into a field being an
alias or not and at _source parsing time there is no way to know if a
root field is an alias or not. Fields of the type "a.b.c.alias" can be
extracted from docvalue_fields, only if the field they point to can be
extracted from docvalue_fields. Also, not all fields in a hierarchy of
fields can be evaluated to being an alias.

(cherry picked from commit 8bf8a055e38f00df5f49c8d97f632f69d6e00c2c)
2019-07-25 10:02:41 +03:00
James Rodewig d46545f729 [DOCS] Update anchors and links for Elasticsearch API relocation (#44500) 2019-07-19 09:18:23 -04:00
Igor Motov a19c7977ac Docs: fix WKTToSQL function example (#44377)
Fixes wrong example snippet in WKTToSQL documentation.

Closes #44299
2019-07-17 12:24:28 -04:00
Andrei Stefan bb3e5351b5 SQL: double quotes escaping bug fix (#43829)
(cherry picked from commit d589dcad18c3708913e13c757b91c846aeb35bb4)
2019-07-10 16:05:22 +03:00
Christoph Büscher fe3f9f0c6b Yet another `the the` cleanup (#43815) 2019-07-01 20:22:19 +02:00
Marios Trivyzas 9cd89c3453 SQL: Increase hard limit for sorting on aggregates (#43220)
To be consistent with the `search.max_buckets` default setting,
set the hard limit of the PriorityQueue used for in memory sorting,
when sorting on an aggregate function, to 10000.

Fixes: #43168

(cherry picked from commit 079e012fdea68ea0a7daae078359495047e9c407)
2019-06-14 13:51:38 +02:00
Andrei Stefan 8de65daa45 Rename TESTRESPONSE[_cat] to TESTRESPONSE[non_json] (#43087)
(cherry picked from commit 897b24e0563f59c03e85096fdb64cbc1dd1a5d60)
2019-06-11 12:40:00 +03:00
Andrei Stefan 5b35ec1d9b Restructure the SQL Language section to have proper sub-sections (#43007)
Rest docs page update
- have the section be on separate pages
- add an Overview page
- add other formats examples

(cherry picked from commit 309bd691ff3f8625f67ca09fc1dd8e265f7e6c92)
2019-06-11 12:39:59 +03:00
Andrei Stefan 4a3287836d SQL: Clarify that the connections the jdbc driver creates are not pooled (#42992)
(cherry picked from commit 406d5281bdfe682fb7ec9fefcdb61cce1b9e7270)
2019-06-11 12:39:58 +03:00
James Rodewig 5913723788 [DOCS] Change `// TESTRESPONSE[_cat]` to `// TESTRESPONSE[non_json]` (#43006) 2019-06-10 09:53:05 -04:00
Marios Trivyzas 6c50246a58
SQL: [Docs] Fix links syntax (#42806)
Fix a couple of wrong links because of the order of the anchor
and the usage of backquotes.

(cherry picked from commit 4e0c6525153b60a57202937c2ae57968c8e35285)
2019-06-03 17:51:19 +02:00
James Rodewig f51f8ed04c [DOCS] Remove unneeded options from `[source,sql]` code blocks (#42759)
In AsciiDoc, `subs="attributes,callouts,macros"` options were required
to render `include-tagged::` in a code block.

With elastic/docs#827, Elasticsearch Reference documentation migrated
from AsciiDoc to Asciidoctor.

In Asciidoctor, the `subs="attributes,callouts,macros"` options are no
longer needed to render `include-tagged::` in a code block. This commit
removes those unneeded options.

Resolves #41589
2019-05-31 13:05:13 -04:00
James Rodewig d521a88e19 [DOCS] Move callouts to end of line for Asciidoctor migration (#42356) 2019-05-24 15:03:46 -04:00
Costin Leau 9fdf4215dd Docs: Documentation for the upcoming SQL support of frozen indices (#41863)
(cherry picked from commit a3cc03eb1503df24c1706a721fcc9af38c3b2873)
(cherry picked from commit f42dcf2ffd7bd25f3f91aa6127515f393cd1860f)
2019-05-23 21:16:16 +03:00
Igor Motov 2f8c5ac6f8 Docs: Mark SQL Geo functionality as beta (#42138)
Adds beta marker to geosql documentation
2019-05-15 10:51:33 -04:00
Igor Motov 70ea3cf847
SQL: Add initial geo support (#42031) (#42135)
Adds an initial limited implementations of geo features to SQL. This implementation is based on the [OpenGIS® Implementation Standard for Geographic information - Simple feature access](http://www.opengeospatial.org/standards/sfs), which is the current standard for GIS system implementation. This effort is concentrate on SQL option AKA ISO 19125-2. 

Queries that are supported as a result of this initial implementation

Metadata commands

- `DESCRIBE table`  - returns the correct column types `GEOMETRY` for geo shapes and geo points.
- `SHOW FUNCTIONS` - returns a list that includes supported `ST_` functions
- `SYS TYPES` and `SYS COLUMNS` display correct types `GEO_SHAPE` and `GEO_POINT` for geo shapes and geo points accordingly. 

Returning geoshapes and geopoints from elasticsearch

- `SELECT geom FROM table` - returns the geoshapes and geo_points as libs/geo objects in JDBC or as WKT strings in console.
- `SELECT ST_AsWKT(geom) FROM table;` and `SELECT ST_AsText(geom) FROM table;`- returns the geoshapes ang geopoints in their WKT representation;

Using geopoints to elasticsearch

- The following functions will be supported for geopoints in queries, sorting and aggregations: `ST_GeomFromText`, `ST_X`, `ST_Y`, `ST_Z`, `ST_GeometryType`, and `ST_Distance`. In most cases when used in queries, sorting and aggregations, these function are translated into script. These functions can be used in the SELECT clause for both geopoints and geoshapes. 
- `SELECT * FROM table WHERE ST_Distance(ST_GeomFromText(POINT(1 2), point) < 10;` - returns all records for which `point` is located within 10m from the `POINT(1 2)`. In this case the WHERE clause is translated into a range query.

Limitations:

Geoshapes cannot be used in queries, sorting and aggregations as part of this initial effort. In order to fully take advantage of geoshapes we would need to have access to geoshape doc values, which is coming in #37206. `ST_Z` cannot be used on geopoints in queries, sorting and aggregations since we don't store altitude in geo_point doc values.

Relates to #29872
Backport of #42031
2019-05-14 18:57:12 -05:00