Commit Graph

56 Commits

Author SHA1 Message Date
markharwood 434430506b
Type removal - added deprecation warnings to _bulk apis (#36549)
Added warnings checks to existing tests
Added “defaultTypeIfNull” to DocWriteRequest interface so that Bulk requests can override a null choice of document type with any global custom choice.
Related to #35190
2019-01-10 21:35:19 +00:00
Costin Leau 83f7423cd6
SQL: Fix bug regarding alias fields with dots (#37279)
Field of types aliases that have dots in name are returned without a
hierarchy by field_caps, as oppose to the mapping api or field with
concrete types, which in turn breaks IndexResolver.
This commit fixes this by creating the backing hierarchy similar to the
mapping api.

Close #37224
2019-01-10 22:18:53 +02:00
Andrei Stefan 4a92de214a
SQL: Proper handling of COUNT(field_name) and COUNT(DISTINCT field_name) (#37254)
* provide overriden `hashCode` and toString methods to account for `DISTINCT`
* change the analyzer for scenarios where `COUNT <field_name>` and `COUNT DISTINCT` have different paths
* defined a new `filter` aggregation encapsulating an `exists` query to filter out null or missing values
2019-01-10 09:51:51 +02:00
Alpar Torok 6344e9a3ce
Testing conventions: add support for checking base classes (#36650) 2019-01-08 13:39:03 +02:00
Andrei Stefan 3fad9d25f6
SQL: fix COUNT DISTINCT filtering (#37176)
* Use `_count` aggregation value only for not-DISTINCT COUNT function calls
* COUNT DISTINCT will use the _exact_ version of a field (the `keyword` sub-field for example), if there is one
2019-01-08 08:47:35 +02:00
Alpar Torok a7c3d5842a
Split third party audit exclusions by type (#36763) 2019-01-07 17:24:19 +02:00
Andrei Stefan 39a072389c
SQL: add sub-selects to the Limitations page (#37012) 2019-01-07 10:08:51 +02:00
Marios Trivyzas e778abaac5
SQL: Improve error message when unable to translate to ES query DSL (#37129)
Improve error message returned to the client when an SQL statement
cannot be translated to a ES query DSL. Cases:

1. WHERE clause evaluates to FALSE => No results returned
1. Missing FROM clause => Local execution, e.g.: SELECT SIN(PI())
3. Special SQL command => Only valid of SQL iface, e.g.: SHOW TABLES

Fixes: #37040
2019-01-07 09:21:23 +02:00
Andrei Stefan 3578e69669
SQL: Handle the bwc Joda ZonedDateTime scripting class in Painless (#37024)
* Handle the bwc Joda ZonedDateTime scripting class in Painless

* Integrated the types checking in the already existent method
2019-01-04 09:19:24 +02:00
Andrei Stefan 9a6c61015c
Use the same format when comparing the error messages containing date fields (#37006)
Test fix: make sure the format in which we check the date is the same as the one we used to format the initial JDBC date field.
2018-12-31 13:41:54 +02:00
Andrei Stefan 09fa827adc
SQL: documentation improvements and updates (#36918)
* Added Limitations page
* Made the aggregations page follow the common template for functions
* Modified all tables to have the first row's cells content centered
* Polishing in other various sections
2018-12-21 23:25:54 +02:00
Costin Leau ac032a0b9d
SQL: Fix bug regarding histograms usage in scripting (#36866)
Allow scripts to correctly reference grouping functions
Fix bug in translation of date/time functions mixed with histograms.
Enhance Verifier to prevent histograms being nested inside other
 functions inside GROUP BY (as it implies double grouping)
Extend Histogram docs
2018-12-20 23:11:56 +02:00
Ryan Ernst cfc0a47232
Core: Deprecate negative epoch timestamps (#36793)
Negative timestamps are currently supported in joda time. These are
dates before epoch. However, it doesn't really make sense to have a
negative timestamp, since this is a modern format. Any dates before
epoch can be represented with normal date formats, like ISO8601.
Additionally, implementing negative epoch timestamp parsing in java time
has an edge case which would more than double the code required. This
commit deprecates use of negative epoch timestamps.
2018-12-20 00:17:06 -08:00
Julie Tibshirani 5185d61ad1
Make sure to use _doc for the type in SQL REST tests. (#36806) 2018-12-19 11:25:45 -08:00
Andrei Stefan d31eaf7313
SQL: protocol returns ISO 8601 String formatted dates instead of Long for JDBC/ODBC requests (#36800)
* Change the way the protocol returns date fields from Long values in case
of JDBC/ODBC, to ISO 8601 with millis String.
2018-12-19 16:36:16 +02:00
Alpar Torok e9ef5bdce8
Converting randomized testing to create a separate unitTest task instead of replacing the builtin test task (#36311)
- Create a separate unitTest task instead of Gradle's built in 
- convert all configuration to use the new task 
- the  built in task is now disabled
2018-12-19 08:25:20 +02:00
Mayya Sharipova f884b2b1cd
Deprecate types in index API (#36575)
* Deprecate types in index API

- deprecate type-based constructors of IndexRequest
- update tests to use typeless IndexRequest constructors
- no yaml tests as they have been already added in #35790

Relates to #35190
2018-12-18 08:53:49 -05:00
Andrei Stefan 4bc9bffb4d
SQL: Extend the ODBC metric by differentiating between 32 and 64bit platforms (#36753)
* The "overall" ODBC metric will now track a total of requests between 32bit and 64bit ODBC metrics, being calculated passively, upon request.
2018-12-18 13:29:49 +02:00
Costin Leau 6d9d5e397b
SQL: Fix translation of LIKE/RLIKE keywords (#36672)
* SQL: Fix translation of LIKE/RLIKE keywords

Refactor Like/RLike functions to simplify internals and improve query
 translation when chained or within a script context.

Fix #36039
Fix #36584
2018-12-17 18:58:06 +02:00
Andrei Stefan 2ed6ab9648
SQL: Concat should be always not nullable (#36601) 2018-12-17 14:00:56 +02:00
Costin Leau 3f128a89bc SQL: Disable integration test due to TZ issues
Due to the difference in TZ between H2 and SQL, disable minute test for
 NOW()

Fix #36695
2018-12-17 13:40:50 +02:00
Andrei Stefan b15f27f6a6
SQL: Scripting support for casting functions CAST and CONVERT (#36640) 2018-12-17 13:36:48 +02:00
Marios Trivyzas c0866393ee
SQL: Fix issue with complex HAVING and GROUP BY ordinal (#36594)
When trying to analyse a HAVING condition, we crate a temp Aggregate
Plan which wasn't created correctly (missing the expressions from the
SELECT clause) and as a result the ordinal (1, 2, etc) in the GROUP BY
couldn't be resolved correctly.

Also after successfully analysing the HAVING condition, still the
original plan was returned.

Fixes: #36059
2018-12-17 11:05:52 +02:00
Costin Leau 6ee6bb55e2
SQL: Introduce HISTOGRAM grouping function (#36510)
Introduce Histogram grouping function for bucketing/grouping data based
 on a given range. Both date and numeric histograms are supported using
 the appropriate range declaration (numbers vs intervals).

SELECT HISTOGRAM(number, 50) AS h FROM index GROUP BY h
SELECT HISTOGRAM(date, INTERVAL 1 YEAR) AS h FROM index GROUP BY h

In addition add multiply operator for Intervals
Add docs for intervals and histogram

Fix #36509
2018-12-14 18:20:37 +02:00
Costin Leau f502ca53d4
SQL: Introduce NOW/CURRENT_TIMESTAMP function (#36562)
Add CURRENT_TIMESTAMP as keyword as well function alongside NOW() 
 These return the current date/time for the  given query, computed when
 the statement reaches the server. For completeness, CURRENT_TIMESTAMP
 also accepts precision as an optional parameter.

Fix #36534
2018-12-14 15:42:36 +02:00
Marios Trivyzas db8f07c665
SQL: Fix translation to painless for conditionals (#36636)
Add missing `formatTemplate()` for conditional functions which
resulted in incomplete painless script. Moreover the specific
return type of Object in the painless signatures resulted in
casting exceptions when conditional functions are used in the
ORDER BY.

Fixes: #36631
2018-12-14 14:51:12 +02:00
Andrei Stefan 1270d85ae1
SQL: be lenient for tests involving comparison to H2 but strict for csv spec tests (#36498)
* Add a lenient parameter for floating point numbers and mark csv-spec
tests as not lenient vs. sql-spec tests to be lenient.
2018-12-13 23:29:32 +02:00
Christoph Büscher b33ff16d62
Remove deprecated `useDisMax` from MultiMatchQuery (#36488)
The getters and setters for useDisMax() have been deprecated since at least 6.0,
also there hasn't been any reference to the query parameter in the
documentation. Removing it from the builder and tests and replacing it with
`tieBreaker(1.0f)` where necessary.
2018-12-13 20:13:16 +01:00
Andrei Stefan de373060fb
SQL: non ISO 8601 versions of DAY_OF_WEEK and WEEK_OF_YEAR functions (#36358)
* Renamed DAY_OF_WEEK and WEEK_OF_YEAR functions to their ISO version and
added the same functions with different functionality.
* Rewritten the datetime functions documentation to follow the format of the other
functions documentation pages.
2018-12-12 02:29:02 +02:00
Andrei Stefan eead8a144b
SQL: move requests' parameters to requests JSON body (#36149) 2018-12-11 19:17:51 +02:00
Ioannis Kakavas d7c5d8049a
Deprecate /_xpack/security/* in favor of /_security/* (#36293)
* This commit is part of our plan to deprecate and ultimately remove the use of _xpack in the REST APIs.

- REST API docs
- HLRC docs and doc tests
- Handle REST actions with deprecation warnings
- Changed endpoints in rest-api-spec and relevant file names
2018-12-11 11:13:10 +02:00
Andy Bristol a6ff9f7f66
[test] generate unique user names (#36179) 2018-12-03 15:08:00 -08:00
Andrei Stefan aabff7318e
SQL: DATABASE() and USER() system functions (#35946) 2018-11-28 15:20:04 +02:00
Costin Leau 95346926ef
SQL: Lock down JDBC driver (#35798)
Move classes under the same package to avoid internal classes being
 exposed to the outside. Remove public visibility outside 3 classes:
 EsDriver, EsDataSource and EsTypes.
The driver only has one package, namely org.elasticsearch.xpack.sql.jdbc
Use Es prefix for classes to ease name conflict and indicate their
 destination

Fix #35437
2018-11-28 12:48:26 +02:00
Jason Tedor c42d9d91c9
Deprecate X-Pack centric SQL endpoints (#35964)
This commit is part of our plan to deprecate and ultimately remove the
use of _xpack in the REST APIs.
2018-11-27 22:16:21 -05:00
Marios Trivyzas 3f7cae3f0d
SQL: Implement GREATEST and LEAST functions (#35879)
Add GREATEST(expr1, expr2, ... exprN) and LEAST(expr1, expr2, exprN)
functions which are in the family of CONDITIONAL functions.

Implementation follows PostgreSQL behaviour, so the functions return
`NULL` when all of their arguments evaluate to `NULL`.

Renamed `CoalescePipe` and `CoalesceProcessor` to `ConditionalPipe` and
`ConditionalProcessor` respectively, to be able to reuse them for
`Greatest` and `Least` evaluations. To achieve that `ConditionalOperation`
has been added to differentiate between the functionalities at execution
time.

Closes: #35878
2018-11-26 18:21:36 +01:00
Marios Trivyzas b078e2970c
SQL: Implement null safe equality operator `<=>` (#35873)
This operator handles nulls in different way than the normal `=`.
If one of the operants is `null` and the other not it returns `false`.
If both operants are `null` it returns `true`. Therefore in contrary to
`=`, which returns `null` if at least one of the operants is `null`, this one
never returns `null` as a result.

Closes: #35871
2018-11-26 14:02:02 +01:00
Andrei Stefan 00e6fec718
SQL: XPack FeatureSet functionality (#35725)
* Introduced "client.id" parameter for REST requests
* Bug that made the Verifier run twice, fixed in the Analyzer
* Single node IT and unit testing
2018-11-26 07:11:43 +02:00
Marios Trivyzas 410f570d5f
SQL: Implement NULLIF(expr1, expr2) function (#35826)
NULLIF returns null if the 2 expressions are equal or the
expr1 otherwise.

Closes: #35818
2018-11-23 22:19:27 +01:00
Marios Trivyzas 92acf47c16
SQL: Implement NVL(expr1, expr2) (#35794)
Add NVL as alias to IFNULL as they have the same
behaviour. Add basic tests and docs.

Closes: #35782
2018-11-22 11:41:00 +01:00
Marios Trivyzas d95d885bae
SQL: Implement ISNULL(expr1, expr2) (#35793)
Add ISNULL as an alias of IFNULL as they have the
same behaviour. Add basic test and docs.

Closes: #35781
2018-11-21 23:15:10 +01:00
Marios Trivyzas e179bd393d
SQL: Implement IFNULL variant of COALESCE (#35762)
IFNULL is a MySQL variant (also used in other DBs) which
takes only 2 arguments and returns the first one that is not null.

Closes: #35749
2018-11-21 17:07:07 +01:00
Costin Leau f0a3d32602
SQL: Introduce INTERVAL support (#35521)
Introduce INTERVAL as a DataType
Add INTERVAL to the grammar which supports the standard SQL declaration
 (without precision):
 > INTERVAL '1 23:45:01.123456789' DAY TO SECOND
 but also number for single unit intervals:
 > INTERVAL 1 YEAR
 as well as the plurals of the units:
 > INTERVAL 2 YEARS
Interval are internally supported as just another Literal being backed
 by java.time.Period and java.time.Duration
Move JDBC away from JDBCType enum to SQLType interface
Refactor DataType by moving it into server core and adding dedicated (and
 much simpler) JDBC driver type
Improve internal JDBC conversion by normalizing on the DataType
Rename JDBC columnInfo to JdbcColumnInfo to differentiate between it and
 the SQL ColumnInfo

Fix #29990
2018-11-21 15:45:59 +02:00
Marios Trivyzas b1818dbdce
SQL: [docs] Add documentation for COALESCE (#35740)
Follows: #35253
2018-11-21 01:43:05 +01:00
Andrei Stefan eaf010cd73
SQL: clear the cursor if nested inner hits are enough to fulfill the query required limits (#35398) 2018-11-15 14:33:47 +02:00
Marios Trivyzas 36da6e1671
SQL: Introduce IsNull node to simplify expressions (#35206)
Add `IsNull` node in parser to simplify expressions so that `<value> IS NULL` is
no longer translated internally to `NOT(<value> IS NOT NULL)`

Replace `IsNotNullProcessor` with `CheckNullProcessor` to encapsulate both
isNull and isNotNull functionality.

Closes: #34876
Fixes: #35171
2018-11-09 11:32:38 +01:00
Andrei Stefan 432c518266
Upgrade jline to 3.8.2 (#35288) 2018-11-06 18:42:57 +02:00
Andrei Stefan c3e4575f61
SQL: new SQL CLI logo (#35261)
Added new SQL CLI logo representing the Elastic logo "painted" with
"Elastic" words, "SQL" under the logo and version on the last line
2018-11-06 17:51:52 +02:00
Marios Trivyzas 1be64a7b8b
SQL: Fix null handling for AND and OR in SELECT (#35277)
Override `process()` in `BinaryLogicProcessor` which doesn't immediately
return null if left or right argument is null, which is the behaviour of
`process()` of the parent class `BinaryProcessor`.

Also, add more tests for `AND` and `OR` in SELECT clause with literal.

Fixes: #35240
2018-11-06 15:40:16 +01:00
Costin Leau 75e9a639ee
SQL: Introduce Coalesce function (#35253)
Add Coalesce conditional for replacing null values

Fix #35060
2018-11-06 13:12:24 +02:00