Commit Graph

170 Commits

Author SHA1 Message Date
Igor Motov 3e14c30aa1 SQL: Add version to cursor serialization (elastic/x-pack-elasticsearch#3064)
This commits also simplifies the serialization mechanism by remove 2 ways to serialize the cursor. Adding the version there was complicating things too much otherwise.

Original commit: elastic/x-pack-elasticsearch@4f2c541e0a
2017-11-21 11:24:27 -05:00
Igor Motov 318621cdc1 SQL: Move all SQL projects into org.elasticsearch.sql package (elastic/x-pack-elasticsearch#3069)
Elasticsearch plugins are traditionally using org.elasticsearch.plugin group, clients are using org.elasticsearch.client, so it makes sense to move all sql pieces and deliverables into org.elasticsearch.sql group. Otherwise, users might get really confused about an artifact named `server` in the group `org.elasticsearch`.

Original commit: elastic/x-pack-elasticsearch@c3a4042040
2017-11-21 10:53:40 +01:00
Nik Everett 87a3e93fae SQL: Improve error message for huge error responses (elastic/x-pack-elasticsearch#3049)
If the user has a funky proxy or something that throws back a massive
error message to the CLI or JDBC then return a sensible error message.

Original commit: elastic/x-pack-elasticsearch@94219969ea
2017-11-17 12:29:07 -05:00
Nik Everett fce5b494be SQL: Stop serializing errors for CLI and JDBC (elastic/x-pack-elasticsearch#3034)
Now that we can parse Elasticsearch's standard error messages in the CLI
and JDBC client we can just let those standard error messages bubble out
of Elasticsearch rather than catch and encode them.

In a followup we can remove the encoding entirely.

Original commit: elastic/x-pack-elasticsearch@bad043b6f7
2017-11-17 12:16:37 -05:00
Igor Motov de087bd859 SQL: Replace NOCOMMIT with TODO
Original commit: elastic/x-pack-elasticsearch@299f82db5c
2017-11-15 18:05:50 -05:00
Nik Everett adae4019dd SQL: improve exception when error can't be parsed (elastic/x-pack-elasticsearch#3024)
When an error response wasn't parseable we would through fairly obscure
error messages about what is wrong with it. Great if you are fixing a
bug in Elasticsearch, no great if a proxy eats the response. This adds
the response to the error messages so you can see what was returned.

Original commit: elastic/x-pack-elasticsearch@00e542afc8
2017-11-15 15:33:53 -05:00
Nik Everett 89e80e0cba Teach SQL to parse Elasticsearch's standard error responses (elastic/x-pack-elasticsearch#2764)
This teaches SQL to parse Elasticsearch's standard error responses
but doesn't change SQL to general Elasticsearch's standard error responses
in all cases. That can come in a followup. We do this parsing with
jackson-core, the same dependency Elasticsearch uses for parsing
json. We shade jackson-core in the JDBC driver so that users don't have to worry about
dependency clashes. We do not do so in the CLI because it is a standalone
application.

We get a few "bonus" changes along the way:
1. We save a copy operation. Before this change responses were spooled
into memory and then parsed. After this change they are parsed directly
from the response stream.
2. We had a few classes entirely to support the spooling operation that we
no longer need: `BytesArray`, `FastByteArrayInputStream`, and
`BasicByteArrayOutputStream`.
3. SQL's `Version` was incorrectly parsing the version from the jar manifest.
We didn't notice because the test was rigged to return `UNKNOWN` because
we *were* running the test from the compiled classes directory instead of the
jar. As part of shading jackson we moved running the tests to running against
the shaded jar. Now we can actually assert that we parse the version correctly.
It turns out we weren't. So I fixed it.

Original commit: elastic/x-pack-elasticsearch@2e8f397bf4
2017-11-14 21:31:35 -05:00
Costin Leau 94d0a2d1ee Polishing for handling subtleties in the JDBC behavior: (elastic/x-pack-elasticsearch#2967)
1. decouple JdbcDriver from other classes to not trigger static
initialization (this happens through JDBC service discovery)
2. reduce visibility of JdbcDriver#close so only on jar unloading it
gets triggered
3. mark 3 methods introduced in Jdbc 4.1 as unsupported (their semantics
are somewhat weird)
4. Move versioning info in one class
5. Hook Version class in both JDBC entry points to perform cp sanity
checks
6. Remove JdbcDataSource#close (DebugLog are closed when the Driver gets
unloaded by the DriverManager) as there can be multiple instances of DS
but only one for Driver known by the DriverManager

Replace Strings with constants
Properly set TZ in security tests as well
JdbcDataSource is more defensive with its internal properties
JdbcConfiguration password parameter is aligned with JDBC DriverManager
Remove usage of JdbcConnection API
Removed JdbcConnection#setTimeZone - this encourages folks to use our
private API which would tie us down.
It is somewhat limiting for folks but it has less downsides overall and
does not trip debugging (which adds a proxy unaware of this method).
Update docs
Add JdbcDataSource into the Jdbc suite

Original commit: elastic/x-pack-elasticsearch@c713665d53
2017-11-15 00:29:41 +02:00
Costin Leau c8eb883c47 Fix incorrect assertions and javadoc
Original commit: elastic/x-pack-elasticsearch@0e6ebc0a69
2017-11-14 09:46:26 +02:00
Costin Leau 9a0b43cd17 Fix several NOCOMMITS (elastic/x-pack-elasticsearch#2968)
* Fix several NOCOMMITS

- renamed Assert to Check to make the intent clear
- clarify esMajor/Minor inside connection (thse are actually our own
methods, not part of JDBC API)
- wire pageTimeout into Cursor#nextPage


Original commit: elastic/x-pack-elasticsearch@7626c0a44a
2017-11-14 01:08:10 +02:00
Igor Motov ea0e58f971 SQL: introduce setting to disable SQL (elastic/x-pack-elasticsearch#2966)
Adds xpack.sql.enabled setting to provide ability to disable SQL on elasticsearch nodes.

relates elastic/x-pack-elasticsearch#2872

Original commit: elastic/x-pack-elasticsearch@d13b72e9ea
2017-11-13 15:10:47 -05:00
Igor Motov 9b96c7eb33 SQL: clean unused classes and empty close() methods (elastic/x-pack-elasticsearch#2956)
relates elastic/x-pack-elasticsearch#2876

Original commit: elastic/x-pack-elasticsearch@c32c6a0058
2017-11-10 14:22:58 -05:00
Igor Motov 3c444c4719 SQL: Fix test when random JDK TZ doesn't exist in Joda (elastic/x-pack-elasticsearch#2903)
JodaTime timezone db can be out of date compared to that of the JDK which causes the JDBC Connection to fail when the randomized tests pick a timezone that's available in the JDK but not in Joda, like SystemV/PST8. This is happening because JdbcConnection configuration is using system default timezone and tries to pass it to Elasticsearch that is using joda. This commit, explicitly sets the time zone on JdbcConnection to a time zone randomly selected from a list of timezones that are known to both JDK and Joda.

relates elastic/x-pack-elasticsearch#2812

Original commit: elastic/x-pack-elasticsearch@b02e9794a8
2017-11-10 14:16:11 -05:00
Costin Leau 8f55a4192c Improve grammar so keywords have priority over identifiers
Original commit: elastic/x-pack-elasticsearch@8222e30060
2017-11-09 21:40:09 +02:00
Costin Leau c48d2b14dc Define structure for docs and improve grammar
The keywords inside SqlBase are now sorted alphabetically - much easier
to read and update the docs

Original commit: elastic/x-pack-elasticsearch@5aa89c5950
2017-11-09 17:49:10 +02:00
Luca Cavanna 753a8d6a6d Build: Run assemble automatically before running cli from gradle (elastic/x-pack-elasticsearch#2918)
Original commit: elastic/x-pack-elasticsearch@43f7396764
2017-11-07 20:51:07 +01:00
Nik Everett a211077554 Switch JDBC metadata to indicate all columns nullable (elastic/x-pack-elasticsearch#2835)
We were returning "nullability unknown" but in Elasticsearch all columns
are nullable.

Original commit: elastic/x-pack-elasticsearch@6ceae418ea
2017-11-04 23:28:49 +00:00
Nik Everett 6ce140cf0f SQL: Make schema optional in RowSet (elastic/x-pack-elasticsearch#2834)
This prevents us from having to hack a fake schema together on the
second (and third and fourth, etc) page of results.

Original commit: elastic/x-pack-elasticsearch@7ba3119daa
2017-11-04 22:52:13 +00:00
Nik Everett 44dc98385a SQL: Reuse a registry meant to be shared (elastic/x-pack-elasticsearch#2838)
The CLI and JDBC were meant to share the same named xcontent registry
but the cli action didn't reuse it by mistake.

Original commit: elastic/x-pack-elasticsearch@6c9af5a22a
2017-11-04 13:10:35 +00:00
Nik Everett 68779a81e4 SQL: Remove now unused class (elastic/x-pack-elasticsearch#2839)
The `AbstractSqlServer` class was used when SQL's CLI and JDBC were
transport actions. They are REST layer concepts now and it is unused so
it should go.

Original commit: elastic/x-pack-elasticsearch@25ec699564
2017-11-04 00:01:15 +00:00
Nik Everett 41284cae93 SQL: Fix the name of the indices lookup action (elastic/x-pack-elasticsearch#2840)
Fix the name of the action the SQL uses to lookup index information from
the cluster state. The old name was silly.

Original commit: elastic/x-pack-elasticsearch@805fb29662
2017-11-03 23:37:22 +00:00
Nik Everett 25a0ec42f6 SQL: Reorganize the translate action (elastic/x-pack-elasticsearch#2841)
Organizes the SQL translate action to match the way that x-pack has been
organizing new actions for a while. All of the pieces are put into the
same class file.

Original commit: elastic/x-pack-elasticsearch@def911c0ab
2017-11-03 23:25:52 +00:00
Nik Everett 43b3a7d7aa Fix import
I missed it in the last PR.

Original commit: elastic/x-pack-elasticsearch@58b2047165
2017-10-30 19:01:15 -04:00
Nik Everett c13f0a8272 Rename SQL's net-client project to shared-client (elastic/x-pack-elasticsearch#2756)
The `net-client` project contained more then just the `net-client`.
It contains stuff like `SuppressForbidden` and `Strings` and `IOUtil`
and other things shared between the CLI and JDBC. It also does contain
the http client. Anyway, it makes more sense to call it `shared-client`,
I think.

Alos updated the copywrite dates on the files that I touched because
they are all 2017 files.

Removed some uses of `String.EMPTY` because they don't buy us anything
and require an extra import. `""` is just one less step.

Original commit: elastic/x-pack-elasticsearch@465c6445c4
2017-10-30 22:59:01 +00:00
Nik Everett d933b1b48b Initial docs for SQL (elastic/x-pack-elasticsearch#2810)
Adds docs for the REST API, translate API, the CLI, and JDBC.

Next we need to add more example queries and documentation for our
extensions.

Original commit: elastic/x-pack-elasticsearch@ed6d1360d2
2017-10-30 17:23:27 +00:00
Costin Leau 66719f7a92 Improve DateTime script template
When dealing with TZ, the script is reused across extractors by
externalizing the tz & chronofield

Original commit: elastic/x-pack-elasticsearch@9e63ea8cd8
2017-10-28 13:35:16 +03:00
Nik Everett f92684c436 Fix SQL test
Original commit: elastic/x-pack-elasticsearch@14d07c2607
2017-10-27 17:38:34 -04:00
Costin Leau 660cb3b814 Upgrade ANTLR to 4.5.3
Relates to elastic/x-pack-elasticsearch#26834

Original commit: elastic/x-pack-elasticsearch@57ba8f1d5f
2017-10-27 23:29:00 +03:00
Costin Leau 8d5572a77b SQL: improve validation of properties (both names and values) (elastic/x-pack-elasticsearch#2759)
* refactor Configuration class

move away from Properties
perform validation of settings names and values at startup
better exception handling

Original commit: elastic/x-pack-elasticsearch@d8a9edeccf
2017-10-27 22:54:03 +03:00
Costin Leau af591b9edd SQL: Remove usage of Settings inside SqlSettings (elastic/x-pack-elasticsearch#2757)
* Remove usage of Settings inside SqlSettings

Also hook client timeouts to the backend
Set UTC as default timezone when using CSV
As the JVM timezone changes, make sure to pin it to UTC since this is what the results are computed against

Original commit: elastic/x-pack-elasticsearch@3e7aad8c1f
2017-10-27 18:55:59 +03:00
Costin Leau 6b1d0d1c8e SQL: Remove time threshold for rule executor (elastic/x-pack-elasticsearch#2750)
Refactor RuleExecutor to eliminate time-based thresholds

Original commit: elastic/x-pack-elasticsearch@ba131f8058
2017-10-27 18:43:13 +03:00
Igor Motov 387944d176 SQL: Move special joda time handling into DocValueExtractor (elastic/x-pack-elasticsearch#2758)
Moves joda time handling into DocValueExtractor, that's the only place where it occurs at the moment.

Original commit: elastic/x-pack-elasticsearch@205e82990a
2017-10-22 15:54:00 +02:00
Costin Leau 353c0c500b Fix and enable datetime tests (elastic/x-pack-elasticsearch#2680)
Firstly, data in H2 is now stored in TIMESTAMP WITH TIME ZONE since H2
does not allow a global TZ to be set and picks the JVM TZ when a record
is read.
JdbcAssert is now aware of this allows TIMESTAMP with TZ == TIMESTAMP

Discovered a serious bug in DateTimeFunction - unfortunately date
histogram is not useful except for year since most extract functions
avoid ordering which a histogram preserves.
Thus most DTF are now terms aggs with scripting.

Improved a bug that caused duplicate functions to not be detected because
of aliasing.

Moved some datetime tests to CSV but the aggs tests now are in sync with
H2

Fixed bug that caused arithmetic on aggs to not be properly resolved by
splitting the processor definition tree to aggName (unresolved) and
aggPath (resolved)

Original commit: elastic/x-pack-elasticsearch@e75ada68f1
2017-10-13 13:52:48 +03:00
Costin Leau 2bbb86eff7 Eliminate NOCOMMITS per discussion
Original commit: elastic/x-pack-elasticsearch@bd091a6608
2017-10-13 00:33:16 +03:00
Nik Everett 852af7de57 Fix error handling in SQL's CLI (elastic/x-pack-elasticsearch#2730)
We weren't returning errors correctly from the server
or catching them correctly in the CLI. This fixes that
and adds simple integration tests.

Original commit: elastic/x-pack-elasticsearch@259da0da6f
2017-10-12 16:32:15 +00:00
Nik Everett 6478713304 Add support for username and password in SQL CLI (elastic/x-pack-elasticsearch#2718)
Add support for username and password in SQL CLI and adds tests that CLI works with security.

Original commit: elastic/x-pack-elasticsearch@39c8dbfc97
2017-10-12 15:55:29 +00:00
Costin Leau fa4504ed28 Fix some NOCOMMITs
Original commit: elastic/x-pack-elasticsearch@1a6ac1e6c6
2017-10-12 14:24:56 +03:00
Costin Leau e3d072aeea Make JDBC driver throw only SQLException (elastic/x-pack-elasticsearch#2708)
A JDBC driver should throw only checked SQLExceptions.
Introduce JdbcSQLException and fix some no-commits along the way.

Original commit: elastic/x-pack-elasticsearch@299fcf9ace
2017-10-12 12:27:19 +03:00
Igor Motov 7f0c44f138 SQL: Fix string to boolean conversion (elastic/x-pack-elasticsearch#2728)
This commit also adds tests for conversion to boolean

Original commit: elastic/x-pack-elasticsearch@d7740929a2
2017-10-11 19:56:36 -04:00
Igor Motov 06b4c043b9 SQL: Add toXcontent test for SqlResponse (elastic/x-pack-elasticsearch#2726)
Original commit: elastic/x-pack-elasticsearch@aa7b8f1a91
2017-10-11 19:55:59 -04:00
Costin Leau 2026198dd4 Add size for column tests (elastic/x-pack-elasticsearch#2685)
Add displaySize to columnInfo

Original commit: elastic/x-pack-elasticsearch@ed1d265e98
2017-10-12 00:03:41 +03:00
Costin Leau c9a41e111e Improve Sql exception reporting (elastic/x-pack-elasticsearch#2679)
Better handling of SQL exceptions (result of incorrect queries) vs
unexpected ones (engine failure, ES...)

Original commit: elastic/x-pack-elasticsearch@2698402cdb
2017-10-11 23:23:52 +03:00
Costin Leau 57fcbb81cb SQL: Improve JDBC communication (elastic/x-pack-elasticsearch#2660)
* Improve JDBC communication
Jdbc HTTP client uses only one url for messages and relies on / for ping
Fixed ES prefix being discarded (missing /)
Add HEAD handler for JDBC endpoint

Original commit: elastic/x-pack-elasticsearch@389f82262e
2017-10-11 23:03:03 +03:00
Costin Leau 1cd6fb23ec Make SearchCursor limit aware
Original commit: elastic/x-pack-elasticsearch@c4839bc293
2017-10-10 19:22:42 +03:00
Igor Motov ed712d0e3f SQL: clean unused Exceptions and replace Util functions with core versions (elastic/x-pack-elasticsearch#2654)
This commit removes ThrowableConsumer, WrappingException, ActionUtils and ObjectUtils by replacing them with core equivalents when needed.

Original commit: elastic/x-pack-elasticsearch@5a68418a3d
2017-10-09 18:00:29 -04:00
Igor Motov fa410095ce SQL: remove synchronous execute method from Command (elastic/x-pack-elasticsearch#2653)
We switched most of the commands to asynchronous mode, this commit removes the synchronous version of exec to prevent accidental use of it in the future.

Original commit: elastic/x-pack-elasticsearch@03d0a6350d
2017-10-09 16:01:56 -04:00
Nik Everett 8675f6998b Use LF line endings in SQL generated files (elastic/x-pack-elasticsearch#26822) (elastic/x-pack-elasticsearch#2706)
Right now if you run `gradle regen` on Windows you'll get `CRLF` line
endings on all the ANTLR generated files because we run
```
ant.fixcrlf(srcdir: outputPath) {
    patternset(includes: 'SqlBase*.java')
}
```

The docs for fixcrlf say that the default line endings that it
corrects to is based on the OS:
https://ant.apache.org/manual/Tasks/fixcrlf.html

This change locks it to `LF`.

Original commit: elastic/x-pack-elasticsearch@4396729e04
2017-10-07 07:54:21 -04:00
Costin Leau 00405ddace Fix incorrect getColumnClassName
Original commit: elastic/x-pack-elasticsearch@a11bfd477f
2017-10-05 14:12:10 +03:00
Costin Leau e1a7c59d15 Rework StringUtilsTest from plain JUnit to ESTestCase
Original commit: elastic/x-pack-elasticsearch@cd16043f23
2017-09-30 01:17:33 +03:00
Costin Leau e0d02033de More JDBC improvements
properly return the precision for VARCHAR
ignore type when specified in index pattern

Original commit: elastic/x-pack-elasticsearch@71a5ac1812
2017-09-30 00:40:57 +03:00