Commit Graph

484 Commits

Author SHA1 Message Date
Alpar Torok f3e67bdd17 Add resolution rule to allow resolving all deps (#46768)
Since the `resolveAllDependencies` task resolves all the congfigurations
it can find, this was not caught by our testing, but it's required to be
configuraed specifically.

We should probably cut-over to the new configurations at some point to
avoid problems like this.

Closes elastic/infra#14580
2019-09-18 11:09:43 +03:00
Costin Leau 92e518e789 SQL: Properly handle indices with no/empty mapping (#46775)
When encountering only indices with empty mapping, the IndexResolver
throws an exception as it expects to find at least one entry.
This commit fixes this case so that an empty mapping is returned.

Fix #46757

(cherry picked from commit 5f4f5807acb93b5fab36718c092c328977a396b6)
2019-09-17 16:01:22 +03:00
Costin Leau 683b5fdeca SQL: Support queries with HAVING over SELECT (#46709)
Handle queries with implicit GROUP BY where the aggregation is not in
the projection/SELECT but inside the filter/HAVING such as:

SELECT 1 FROM x HAVING COUNT(*) > 0

The engine now properly identifies the case and handles it accordingly.

Fix #37051

(cherry picked from commit fa53ca05d8219c27079b50b4a5b7aeb220c7cde2)
2019-09-17 11:14:39 +03:00
Costin Leau 90f4c2379b SQL: improve ResultSet behavior when no rows are available (#46753)
Improve the defensive behavior of ResultSet when dealing with incorrect
API usage. In particular handle the case of dealing with no row
available (either because the cursor is before the first entry or after
the last).

Fix #46750

(cherry picked from commit 58fa38e4606625962e879265d35eacb0960c6cdb)
2019-09-17 11:14:38 +03:00
Andrei Stefan 40e9353947 SQL: use the correct data type for types conversion (#46574)
(cherry picked from commit 3e25db2f302c3aafe27e4d8d4fb1743401d85e6d)
2019-09-16 15:36:17 +03:00
Marios Trivyzas d956509394 SQL: Implement DATE_TRUNC function (#46473)
DATE_TRUNC(<truncate field>, <date/datetime>) is a function that allows
the user to truncate a timestamp to the specified field by zeroing out
the rest of the fields. The function is implemented according to the
spec from PostgreSQL: https://www.postgresql.org/docs/current/functions-datetime.html#FUNCTIONS-DATETIME-TRUNC

Closes: #46319
(cherry picked from commit b37e96712db1aace09f17b574eb02ff6b942a297)
2019-09-11 21:41:02 +03:00
Ryan Ernst 86290cb3d9 Make reuse of sql test code explicit (#45884)
The sql project uses a common set of security tests, which are run in
subprojects. Currently these are shared through a shared directory, but
this is not setup correctly to ensure it is built before tests run. This
commit changes the test classes to be an artifact of the sql/qa/security
project and makes the test runner use the built artifact (a directory of
classes) for tests.

closes #45866
2019-09-11 10:56:07 -07:00
Marios Trivyzas 0963e78164 SQL: Fix issue with common type resolution (#46565)
Many scalar functions try to find out the common type between their
arguments in order to set it as their return time, e.g.:
for `float + double` the common type which is set as the return type
of the + operation is `double`.

Previously, for data types TEXT and KEYWORD (string data types) there
was no common data type found and null was returned causing NPEs when
the function was trying to resolve the return data type.

Fixes: #46551
(cherry picked from commit 291017d69dfc810707c3c7c692f5a50af431b790)
2019-09-11 19:10:15 +03:00
Andrei Stefan 7b26a8c041 Use `null` schema response for `SYS TABLES` command. (#46386)
(cherry picked from commit a6152f42a47a1ccd668e5892778c8bd2d3a78c4c)
2019-09-07 09:24:54 +03:00
Andrei Stefan 7cf100ba07 SQL: fix scripting for grouped by datetime functions (#46421)
* Fix issue with painless scripting not being correctly generated when
datetime functions are used for GROUPing of an INTERVAL operation.

(cherry picked from commit cb92828e8ec9d9d241bd6189e5835fd99f8b9a44)
2019-09-07 09:24:53 +03:00
Marios Trivyzas fd0affb503
SQL: Fix issue with IIF function when condition folds (#46290)
Previously, when the condition (1st argument) of the IIF function could
be evaluated (folded) to false, the `IfConditional` was eliminated which
caused `IndexOutOfBoundsException` to be thrown when `info()` and
`resolveType()` methods where called.

Fixes: #46268

(cherry picked from commit 9a885a3ac47bc8f52c07770d1d8d670ce0af1e59)
2019-09-04 10:32:49 +03:00
Marios Trivyzas 3bee647e5b
SQL: Fix issue with DataType for CASE with NULL (#46173)
Previously, if the DataType of all the WHEN conditions of a CASE
statement is NULL, then it was set to NULL even if the ELSE clause
has a non-NULL data type, e.g.:
```
CASE WHEN a = 1 THEN NULL
           WHEN a = 5 THEN NULL
ELSE 'foo'
```

Fixes: #46032
(cherry picked from commit 8c1012efbbd3a300afd0dfb9b18250f15ea753f9)
2019-09-02 11:17:24 +03:00
Alexander Reelsen ecafe4f4ad Update joda to 2.10.3 (#45495) 2019-08-23 10:39:39 +02:00
Armin Braun bfddaaa2ae
Acknowledge Indices Were Wiped Successfully in REST Tests (#45832) (#45842)
In internal test clusters tests we check that wiping all indices was acknowledged
but in REST tests we didn't.
This aligns the behavior in both kinds of tests.
Relates #45605 which might be caused by unacked deletes that were just slow.
2019-08-22 17:19:51 +02:00
Costin Leau 0f51dd69cb SQL: Improve serialization of SQL processors (#45678)
Encapsulate the serialization/deserialization of SQL client classes.
Make configuration specific parameters (such as ZoneId) generic just
like the version and remove the need for consumer classes to manage them
individually.
This is not only consistent but also provides significant savings in the
cursor.

Fix #40216

(cherry picked from commit 5c844798045d7baa0d932289d2e3d1607ba6a9a4)
2019-08-20 11:50:47 +03:00
Costin Leau 1cd58c8ea8 SQL: Break TextFormatter/Cursor dependency (#45613)
Improve the initialization and state passing of TextFormatter in CLI
and TEXT mode by leveraging the Page listener hook. Additionally
simplify the code inside RestSqlQueryAction.

(cherry picked from commit a56db2fa119cf9e8748723e19f1fc9f6a8afe5fc)
2019-08-17 00:16:08 +03:00
Costin Leau 96883dd028 SQL: Refactor away the cycle between Rowset and Cursor (#45516)
Improve encapsulation of pagination of rowsets by breaking the cycle
between cursor and associated rowset implementation, all logic now
residing inside each cursor implementation.

(cherry picked from commit be8fe0a0ce562fe732fae12a0b236b5731e4638c)
2019-08-17 00:16:05 +03:00
Igor Motov 98c850c08b
Geo: Change order of parameter in Geometries to lon, lat 7.x (#45618)
Changes the order of parameters in Geometries from lat, lon to lon, lat
and moves all Geometry classes are moved to the
org.elasticsearch.geomtery package.

Backport of #45332

Closes #45048
2019-08-16 14:42:02 -04:00
Andrei Stefan 30a0711777 Remove deprecated use of "interval" method, in favor of "fixedInterval". (#45501)
(cherry picked from commit 3fef65160f9e61883e9f8f7f345b814f945e2f4b)
2019-08-16 15:03:43 +03:00
Andrei Stefan adf8e20021
SQL: adds format parameter to range queries for constant date comparisons (#45503)
* Add format parameter to the range queries built for CURRENT_* functions
used in comparison conditions
* Use range queries for date fields equality/non-equality as well.

(cherry picked from commit c1e81e90f937ee5a002524d632bfce74d76962f9)
2019-08-13 23:04:30 +03:00
Armin Braun a9e1402189
Remove Settings from BaseRestRequest Constructor (#45418) (#45429)
* Resolving the todo, cleaning up the unused `settings` parameter
* Cleaning up some other minor dead code in affected classes
2019-08-12 05:14:45 +02:00
Andrei Stefan 740d58fd46
SQL: Uniquely named inner_hits sections for each nested field condition (#45341)
* Name each inner_hits section of nested queries differently and extract and combine the multiple values it generates into a single list.
This also introduces a limitation (its origin it's with Elasticsearch
though) on the sorting capabilities when the sorting is based on the
nested fields filtered: only one of the conditions applied to nested
documents will be used in the nested sorting.

(cherry picked from commit cfc5cf68f6e83b07bb9006986d0903d6be418ec6)
2019-08-09 00:22:49 +03:00
Ignacio Vera 821f6f893b
Upgrade to Lucene 8.2.0 release (#44859) (#44892) 2019-07-26 08:14:59 +02:00
Andrei Stefan fd74b63602 SQL: fix URI path being lost in case of hosted ES scenario (#44776)
(cherry picked from commit 06dea859e8fddada868941aaae15e83b4f64babe)
2019-07-25 10:27:51 +03:00
Andrei Stefan ee53f7e161 SQL: [Tests] Re-enable testDriverConfigurationWithSSLInURL test with more logging (#44800)
(cherry picked from commit 5b9ccd72e9a3bb65c8b7b06979a75cb795c17111)
2019-07-25 10:10:45 +03:00
Andrei Stefan 2633d11eb7
Switch from using docvalue_fields to extracting values from _source (#44062) (#44804)
* Switch from using docvalue_fields to extracting values from _source
where applicable. Doing this means parsing the _source and handling the
numbers parsing just like Elasticsearch is doing it when it's indexing
a document.
* This also introduces a minor limitation: aliases type of fields that
are NOT part of a tree of sub-fields will not be able to be retrieved
anymore. field_caps API doesn't shed any light into a field being an
alias or not and at _source parsing time there is no way to know if a
root field is an alias or not. Fields of the type "a.b.c.alias" can be
extracted from docvalue_fields, only if the field they point to can be
extracted from docvalue_fields. Also, not all fields in a hierarchy of
fields can be evaluated to being an alias.

(cherry picked from commit 8bf8a055e38f00df5f49c8d97f632f69d6e00c2c)
2019-07-25 10:02:41 +03:00
Andrei Stefan 04cb3aebd5 Use hasValue() methods from aggregations' InspectionHelpers (#44745)
Use InspectionHelper classes to decide if the aggregations should return null (in case there is no value) or the value itself.

(cherry picked from commit dafd7b039b0da072750e8f57e7572d24f7aad44a)
2019-07-24 16:14:45 +03:00
Tal Levy 7c84636029
Remove StreamOutput #writeOptionalStreamable and #writeStreamableList (#44602) (#44643)
remove usages of writeOptionalStreamable and writeStreambaleList

relates #34389.
2019-07-19 15:55:53 -07:00
Ryan Ernst f193d14764
Convert remaining Action Response/Request to writeable.reader (#44528) (#44607)
This commit converts readFrom to ctor with StreamInput on the remaining
ActionResponse and ActionRequest classes.

relates #34389
2019-07-19 13:33:38 -07:00
Ryan Ernst 60785a9fa8
Convert several direct uses of Streamable to Writeable (#44586) (#44604)
This commit converts several utility classes that implement Streamable
to have StreamInput constructors. It also adds a default version of
readFrom to Streamable so that overriding to throw UOE is not necessary.

relates #34389
2019-07-18 21:25:44 -07:00
Ryan Ernst edd26339c5
Convert remaining request classes in xpack core to writeable.reader (#44524) (#44534)
This commit converts all remaining classes extending ActionRequest
in xpack core to have a StreamInput constructor.

relates #34389
2019-07-18 01:11:45 -07:00
Ignacio Vera eb348d2593
Upgrade to lucene-8.2.0-snapshot-6413aae226 (#44480) 2019-07-17 13:28:28 +02:00
Ryan Ernst e0b82e92f3
Convert BaseNode(s) Request/Response classes to Writeable (#44301) (#44358)
This commit converts all BaseNodeResponse and BaseNodesResponse
subclasses to implement Writeable.Reader instead of Streamable.

relates #34389
2019-07-15 18:07:52 -07:00
Ryan Ernst 7e06888bae
Convert testclusters to use distro download plugin (#44253) (#44362)
Test clusters currently has its own set of logic for dealing with
finding different versions of Elasticsearch, downloading them, and
extracting them. This commit converts testclusters to use the
DistributionDownloadPlugin.
2019-07-15 17:53:05 -07:00
Andrei Stefan e9f9f00940
SQL: add pretty printing to JSON format (#43756) (#44220)
(cherry picked from commit cbd9d4c259bf5a541bc49f65f7973174a36df449)
2019-07-11 20:02:24 +03:00
Nick Knize 374030a53f
Upgrade to lucene-8.2.0-snapshot-860e0be5378 (#44171) (#44184)
Upgrades lucene library to lucene-8.2.0-snapshot-860e0be5378
2019-07-11 09:17:22 -05:00
Igor Motov df2e1fb43e Geo: add validator that only checks altitude (#43893)
By default, we don't check ranges while indexing geo_shapes. As a
result, it is possible to index geoshapes that contain contain
coordinates outside of -90 +90 and -180 +180 ranges. Such geoshapes
will currently break SQL and ML retrieval mechanism. This commit removes
these restriction from the validator is used in SQL and ML retrieval.
2019-07-10 16:55:03 -04:00
Ryan Ernst fb77d8f461 Removed writeTo from TransportResponse and ActionResponse (#44092)
The base classes for transport requests and responses currently
implement Streamable and Writeable. The writeTo method on these base
classes is implemented with an empty implementation. Not only does this
complicate subclasses to think they need to call super.writeTo, but it
also can lead to not implementing writeTo when it should have been
implemented, or extendiong one of these classes when not necessary,
since there is nothing to actually implement.

This commit removes the empty writeTo from these base classes, and fixes
subclasses to not call super and in some cases implement an empty
writeTo themselves.

relates #34389
2019-07-10 12:42:04 -07:00
Andrei Stefan bb3e5351b5 SQL: double quotes escaping bug fix (#43829)
(cherry picked from commit d589dcad18c3708913e13c757b91c846aeb35bb4)
2019-07-10 16:05:22 +03:00
Andrei Stefan 9567f337f5 SQL: handle SQL not being available in a more graceful way (#43665)
* Add test for SQL not being available error message in JDBC.
* Add a new qa sub-project that explicitly disables SQL XPack module in Gradle.

(cherry picked from commit 8a1ac8a3a88a325ec9b99963e0fa288c18ee0ee5)
2019-07-10 14:36:24 +03:00
Andrei Stefan 9957b66d2e SQL: concrete indices array size bug fix (#43878)
* The created array didn't have the correct initial size while attempting to resolve multiple indices

(cherry picked from commit 341006e9913e831408f5bbc7f8ad8c453a7f630e)
2019-07-10 14:36:23 +03:00
Christoph Büscher 8e8d7667cb [Tests] Fix type inference issue (#44063) 2019-07-08 17:34:35 +02:00
Ryan Ernst 3a2c698ce0
Rename Action to ActionType (#43778)
Action is a class that encapsulates meta information about an action
that allows it to be called remotely, specifically the action name and
response type. With recent refactoring, the action class can now be
constructed as a static constant, instead of needing to create a
subclass. This makes the old pattern of creating a singleton INSTANCE
both misnamed and lacking a common placement.

This commit renames Action to ActionType, thus allowing the old INSTANCE
naming pattern to be TYPE on the transport action itself. ActionType
also conveys that this class is also not the action itself, although
this change does not rename any concrete classes as those will be
removed organically as they are converted to TYPE constants.

relates #34389
2019-06-30 22:00:17 -07:00
Ryan Ernst 28ab77a023
Add StreamableResponseAction to aid in deprecation of Streamable (#43770)
The Action base class currently works for both Streamable and Writeable
response types. This commit intorduces StreamableResponseAction, for
which only the legacy Action implementions which provide newResponse()
will extend. This eliminates the need for overriding newResponse() with
an UnsupportedOperationException.

relates #34389
2019-06-28 21:40:00 -07:00
Alpar Torok d1a4d8866d Add missing dependencies so we can build in parallel (#43672) 2019-06-28 16:41:18 +03:00
Ryan Ernst 5b4089e57e
Remove nodeId from BaseNodeRequest (#43658)
TransportNodesAction provides a mechanism to easily broadcast a request
to many nodes, and collect the respones into a high level response. Each
node has its own request type, with a base class of BaseNodeRequest.
This base request requires passing the nodeId to which the request will
be sent. However, that nodeId is not used anywhere. It is private to the
base class, yet serialized to each node, where the node could just as
easily find the nodeId of the node it is on locally.

This commit removes passing the nodeId through to the node request
creation, and guards its serialization so that we can remove the base
request class altogether in the future.
2019-06-27 18:45:14 -07:00
Igor Motov 3607876a71 Geo: Makes coordinate validator in libs/geo plugable (#43657)
Moves coordinate validation from Geometry constructors into
parser.

Relates #43644
2019-06-27 19:53:41 -04:00
Alpar Torok ea44da6069 Testclusters: conver remaining x-pack (#43335)
Convert x-pack tests
2019-06-24 12:07:42 +03:00
Andrei Stefan fe0f9055d8 Fix NPE in case of subsequent scrolled requests for a CSV/TSV formatted response (#43365)
(cherry picked from commit 0ef7bb0f8b07cd0392d37f96ca9360821b19315a)
2019-06-20 11:26:11 +03:00
Jason Tedor 1f1a035def
Remove stale test logging annotations (#43403)
This commit removes some very old test logging annotations that appeared
to be added to investigate test failures that are long since closed. If
these are needed, they can be added back on a case-by-case basis with a
comment associating them to a test failure.
2019-06-19 22:58:22 -04:00