Commit Graph

122 Commits

Author SHA1 Message Date
Hendrik Muhs e2f310b56c
Fix AggregationFactories.Builder equality and hash regarding order (#34005)
Fixes the equals and hash function to ignore the order of aggregations to ensure equality after serialization
and deserialization. This ensures storing configs with aggregation works properly.

This also addresses a potential issue in caching when the same query contains aggregations but in 
different order. 1st it will not hit in the cache, 2nd cache objects which shall be equal might end up twice in 
the cache.
2018-09-28 13:30:50 +02:00
Costin Leau 15515a616e
SQL: Internal refactoring of operators as functions (#34097)
Centralize and simplify the script generation between operators and functions which are currently decoupled. As part of this process most predicates (<, <=, etc...) were made ScalarFunction as their purpose and functionality is quite similar (see % and MOD functions).
Renamed ProcessDefinition to Pipe
Add ScriptWeaver as a mixin-in interface for script customization
Add logic for equals/lte/lt
Improve BinaryOperator/expression toString
Minimize duplication across string functions

Close #33975
2018-09-27 23:42:57 +03:00
Christoph Büscher 5f19146bac
[SQL] Clean up LogicalPlanBuilder#doJoin (#34048)
Currently the local `type` and `condition` variables are unused and can be removed.
This code can be added later again if joins are supported.
2018-09-27 16:27:49 +02:00
Christoph Büscher ba3ceeaccf
Clean up "unused variable" warnings (#31876)
This change cleans up "unused variable" warnings. There are several cases were we 
most likely want to suppress the warnings (especially in the client documentation test
where the snippets contain many unused variables). In a lot of cases the unused
variables can just be deleted though.
2018-09-26 14:09:32 +02:00
Marios Trivyzas 5840be6a6b
SQL: Prevent StackOverflowError when parsing large statements (#33902)
Implement circuit breaker logic in the parser which catches expressions
that can blow up the tree and result in StackOverflowError being thrown.

Co-authored-by: Costin Leau <costin.leau@gmail.com>
2018-09-25 19:20:25 +02:00
Marios Trivyzas 2ca177be41
SQL: Fix query translation of GroupBy with Having (#34010)
Two issues are resolved:
1. The `value_type` should be either long or double in case of numeric.
2. The key label for the aggregate filter (having) was duplicate of an aggr key label.

Fixes: #33520
2018-09-25 12:17:14 +02:00
Costin Leau 17605bf1c6
SQL: Move away internally from JDBCType to SQLType (#33913)
Replace JDBCType enum with SQLType interface for extensibility

Fix #33094
2018-09-22 11:03:38 +03:00
Andrei Stefan 6fb7e49b22
SQL: TRUNCATE and ROUND functions (#33779)
* Added TRUNCATE function, modified ROUND to accept two parameters instead of one. Made the second parameter optional for both functions.
* Added documentation for both functions.
2018-09-20 15:29:53 +03:00
Marios Trivyzas 168a0b23fe
SQL: Fix ANTL4 Grammar ambiguities. (#33854)
Removed rules in the grammar that were superfluous, as they
are already "caught" other rules in the same context.

Also switched to exact ambig detection for debug mode

Fixes: #31885
2018-09-20 14:26:07 +02:00
Marios Trivyzas d22b383b9c
SQL: Fix issue with options for QUERY() and MATCH(). (#33828)
Previously multiple comma separated lists of options where not
recognized correctly which resulted in only the last of them
to be taked into account, e.g.:

For the following query:
  SELECT * FROM test WHERE QUERY('search', 'default_field=foo', 'default_operator=and')"
only the `default_operator=and` was finally passed to the ES query.

Fixes: #32602
2018-09-19 10:16:24 +02:00
Andrei Stefan c6e3231ef3
SQL: day and month name functions tests locale providers enforcement (#33653)
DAYNAME and MONTHNAME functions tests will be skipped if the right JVM parameter (-Djava.locale.providers=COMPAT) is not used in unit testing environment
2018-09-19 07:33:15 +03:00
Jim Ferenczi 241c74efb2
upgrade to a new snapshot of Lucene 8 (7d0a7782fa) (#33812) 2018-09-18 18:16:40 +02:00
Costin Leau 91e45ca21b
SQL: Better handling of number parsing exceptions (#33776)
Add proper exceptions in case the parsing of numbers (too large, invalid
format) fails.

Close #33622
2018-09-18 18:51:48 +03:00
Costin Leau ab9c28a2b1
SQL: Grammar tweak for number declarations (#33767)
Consider plus and minus as part of a number declaration (to avoid the
minus be treated as a negation).

Close #33765
2018-09-18 11:44:52 +03:00
Costin Leau 736053c658
SQL: Return functions in JDBC driver metadata (#33672)
Update JDBC database metadata to advertised the supported functions
Add aliases to some date/time functions to match the ODBC spec

Fix #33671
2018-09-14 09:05:34 +03:00
Costin Leau 60ab4f97ab
SQL: Return correct catalog separator in JDBC (#33670)
JDBC DatabaseMetadata returns correct separator (:) for catalog/cluster names.

Fix #33654
2018-09-13 21:55:44 +03:00
Marios Trivyzas 7d3b99a9b7
SQL: Fix result column names for CAST (#33604)
Previously, when an non-pruned cast (casting as a different
data type) got applied on a table column in the `SELECT` clause,
the name of the result column didn't contain the target data type
of the cast, e.g.:

  SELECT CAST(MAX(salary) AS DOUBLE) FROM "test_emp"

returned as column name:

  CAST(MAX(salary))

instead of:

  CAST(MAX(salary) AS DOUBLE)

Closes #33571

* Added more tests for trivial casts that are pruned
2018-09-13 10:28:05 +02:00
David Turner 94f6d4560d Revert "Mark NamedDateTimeProcessorTests as @AwaitsFix"
This reverts commit 44a80d61b2.
2018-09-13 07:32:33 +02:00
David Turner 44a80d61b2 Mark NamedDateTimeProcessorTests as @AwaitsFix 2018-09-13 07:24:51 +02:00
Vladimir Dolzhenko cfc20825ab mute NamedDateTimeProcessorTests 2018-09-12 20:34:56 +02:00
Costin Leau 91bca174f5
SQL: Make Literal a NamedExpression (#33583)
* SQL: Make Literal a NamedExpression

Literal now is a NamedExpression reducing the need for Aliases for
folded expressions leading to simpler optimization rules.

Fix #33523
2018-09-11 18:47:49 +03:00
Andrei Stefan a3e1f1e46f
SQL: Adds MONTHNAME, DAYNAME and QUARTER functions (#33411)
* Added monthname, dayname and quarter functions
* Updated docs tests with the new functions
2018-09-11 14:35:34 +03:00
Andrei Stefan a55fa4fd6b
Fix Replace function. Adds more tests to all string functions. (#33478) 2018-09-11 11:00:56 +03:00
Marios Trivyzas f441bb87ad
SQL: Fix result column names for arithmetic functions (#33500)
Previously, when an arithmetic function got applied on a
table column in the `SELECT` clause, the name of the result
column contained weird characters used internally when
processing the SQL statement e.g.:

  SELECT CHAR(emp_no % 10000) FROM "test_emp"

returned:

  CHAR((emp_no{f}#14) % 10000))

as the column name instead of:

  CHAR((emp_no) % 10000))

Also, fix an issue that causes a ClassCastException to be thrown
when using functions where both arguments are literals.

Closes #31869
Closes #33461
2018-09-10 22:02:11 +02:00
Alan Woodward 39c3234c2f
Upgrade to latest Lucene snapshot (#33505)
* LeafCollector.setScorer() now takes a Scorable
* Scorers may not have null Weights
* IndexWriter.getFlushingBytes() reports how much memory is being used by IW threads writing to disk
2018-09-10 20:51:55 +01:00
Colin Goodheart-Smithe 79e7969734
Removes redundant test method in SQL tests (#33498) 2018-09-07 13:10:27 +01:00
Costin Leau 7b923ea604
SQL: Improve alias vs index resolution (#33393)
Improve pattern resolution for aliases vs indices
2018-09-07 07:24:43 +03:00
Jim Ferenczi 7ad71f906a
Upgrade to a Lucene 8 snapshot (#33310)
The main benefit of the upgrade for users is the search optimization for top scored documents when the total hit count is not needed. However this optimization is not activated in this change, there is another issue opened to discuss how it should be integrated smoothly.
Some comments about the change:
* Tests that can produce negative scores have been adapted but we need to forbid them completely: #33309

Closes #32899
2018-09-06 14:42:06 +02:00
Costin Leau d7965ba681
SQL: Align SYS TABLE for ODBC SQL_ALL_* args (#33364)
Fix a bug in SYS TABLES command that did skipped SQL_ALL_* arguments for
catalog and table types

Fix #33312
2018-09-05 10:44:18 +03:00
Costin Leau 17c7f99343
SQL: Show/desc commands now support table ids (#33363)
Extend SHOW TABLES, DESCRIBE and SHOW COLUMNS to support table
identifiers not just SQL LIKE pattern.
This allows both Elasticsearch-style multi-index patterns and SQL LIKE.
To disambiguate between the two (as the " vs ' can be easy to miss),
the grammar now requires LIKE keyword as a prefix for all LIKE-like
patterns.

Also added some docs comparing the two types of patterns.

Fix #33294
2018-09-04 16:54:10 +03:00
Andrei Stefan 0c4b3162be
SQL: test coverage for JdbcResultSet (#32813)
* Tests for JdbcResultSet
* Added VARCHAR conversion for different types
* Made error messages consistent: they now contain both the type that fails to be converted and the value itself
2018-08-31 16:12:01 +03:00
Andrei Stefan 20e1b5e2fe
Different handling for security specific errors in the CLI. Fix for https://github.com/elastic/elasticsearch/issues/33230 (#33255) 2018-08-31 13:08:32 +03:00
Costin Leau 73eb4cbbbe
SQL: Support multi-index format as table identifier (#33278)
Extend tableIdentifier to support multi-index format; not just * but
also enumeration and exclusion

Fix #33162
2018-08-31 10:45:25 +03:00
Costin Leau 83c3d7a6cf
SQL: prevent duplicate generation for repeated aggs (#33252)
Prevent generation of duplicate aggs caused by repetitive functions,
leading to invalid query.

Fix #30287
2018-08-31 00:13:03 +03:00
Alpar Torok f29f0af7bc
Consider multi release jars when running third party audit (#33206)
Exclude classes meant for newer versions than what we are auditing against, those classes won't be found. There's no reason to exclude JDK classes from newer versions, with this PR, we will not extract them in the first place.
2018-08-29 09:53:04 +03:00
Alpar Torok 2cc611604f
Run Third party audit with forbidden APIs CLI (part3/3) (#33052)
The new implementation is functional equivalent with the old, ant based one.
It parses task standard error to get the missing classes and violations in the same way.
I considered re-using ForbiddenApisCliTask but Gradle makes it hard to build inheritance with tasks that have task actions , since the order of the task actions can't be controlled.
This inheritance isn't dully desired either as the third party audit task is much more opinionated and we don't want to expose some of the configuration.
We could probably extract a common base class without any task actions, but probably more trouble than it's worth.

Closes #31715
2018-08-28 10:03:30 +03:00
Andrei Stefan 3d9ca4baee
SQL: Enable aggregations to create a separate bucket for missing values (#32832)
Enable aggregations to create a separate bucket for missing values.
2018-08-27 17:56:28 +03:00
Andrei Stefan de95dead2d
SQL: skip uppercasing/lowercasing function tests for AZ locales as well (#32910)
* Added the rest of the Locales that have different behavior for uppercasing/lowercasing scenarios to the skip list
2018-08-23 00:21:38 +03:00
Alpar Torok 82d10b484a
Run forbidden api checks with runtimeJavaVersion (#32947)
Run forbidden APIs checks with runtime hava version
2018-08-22 09:05:22 +03:00
Nik Everett 2c81d7f77e
Build: Rework shadow plugin configuration (#32409)
This reworks how we configure the `shadow` plugin in the build. The major
change is that we no longer bundle dependencies in the `compile` configuration,
instead we bundle dependencies in the new `bundle` configuration. This feels
more right because it is a little more "opt in" rather than "opt out" and the
name of the `bundle` configuration is a little more obvious.

As an neat side effect of this, the `runtimeElements` configuration used when
one project depends on another now contains exactly the dependencies needed
to run the project so you no longer need to reference projects that use the
shadow plugin like this:

```
testCompile project(path: ':client:rest-high-level', configuration: 'shadow')
```

You can instead use the much more normal:

```
testCompile "org.elasticsearch.client:elasticsearch-rest-high-level-client:${version}"
```
2018-08-21 20:03:28 -04:00
Alpar Torok 4b34b3f4aa
Set forbidden APIs target compatibility to compiler java version (#32935)
Set forbidden apis target compatibility to compiler version

Fix outstanding deprecation
2018-08-20 09:27:02 +03:00
Andrei Stefan 7b618f3db4
SQL: Bug fix for the optional "start" parameter usage inside LOCATE function (#32576)
The incorrect NodeInfo is created when the optional parameter is not used, leading to the incorrect constructor being used. Simplified LocateFunctionProcessorDefinition by using one constructor instead of two.
Fixes https://github.com/elastic/elasticsearch/issues/32554
2018-08-09 19:30:02 +03:00
Andrei Stefan 6750e15773
SQL: Ignore H2 comparative tests for uppercasing/lowercasing string functions (#32604)
Skip the comparative tests using lowercasing/uppercasing against H2 (which considers the Locale).
ES-SQL is, so far, ignoring the Locale.
Still, the same queries are executed against ES-SQL alone and results asserted to be correct.
2018-08-09 19:28:48 +03:00
Nicholas Knize e162127ff3 Upgrade to Lucene-7.5.0-snapshot-13b9e28f9d
The main feature is the inclusion of bkd backed geo_shape with
INTERSECT, DISJOINT, WITHIN bounding box and polygon query support.
2018-08-09 11:15:02 -05:00
Armin Braun 580d59e2d7
CORE: Upgrade to Jackson 2.8.11 (#32670)
* closes #30352
2018-08-08 12:04:25 +02:00
Armin Braun 0a67cb4133
LOGGING: Upgrade to Log4J 2.11.1 (#32616)
* LOGGING: Upgrade to Log4J 2.11.1
* Upgrade to `2.11.1` to fix memory leaks in slow logger when logging large requests
   * This was caused by a bug in Log4J https://issues.apache.org/jira/browse/LOG4J2-2269 and is fixed in `2.11.1` via https://git-wip-us.apache.org/repos/asf?p=logging-log4j2.git;h=9496c0c
* Fixes #32537
* Fixes #27300
2018-08-06 14:56:21 +02:00
Andrei Stefan 9e1e38ff51
Minor fix for javadoc (applicable for java 11). (#32573) 2018-08-02 23:46:08 +03:00
David Turner 2db420a81d Add @AwaitsFix for #32554 2018-08-01 17:26:45 +01:00
David Turner 621350c1b9 Remove broken @link in Javadoc 2018-08-01 17:12:55 +01:00
Andrei Stefan 4c388539a0
SQL: Added support for string manipulating functions with more than one parameter (#32356)
Added support for string manipulating functions with more than one parameter:
CONCAT, LEFT, RIGHT, REPEAT, POSITION, LOCATE, REPLACE, SUBSTRING, INSERT
2018-08-01 12:29:06 +03:00