Commit Graph

67 Commits

Author SHA1 Message Date
Nik Everett d1b0067e47 SQL: Ban PrintWriter#println in CLI (elastic/x-pack-elasticsearch#4118)
`PrintWriter#println` doesn't pay attention to the terminal
configuration that we specify so it breaks tests on Windows. If we
instead always use `PrintWriter#print('\n')` then the tests work
properly on windows *and* the CLI actually works properly on Windows.

relates elastic/x-pack-elasticsearch#4109

Original commit: elastic/x-pack-elasticsearch@ac17e691c8
2018-03-14 16:15:32 -04:00
Lee Hinman b14baf4a6f Decouple XContentBuilder from BytesReference (elastic/x-pack-elasticsearch#4119)
* Decouple XContentBuilder from BytesReference

This commit handles the removal of all mentions of BytesReference from
XContentBuilder. This is needed so that we can completely decouple the XContent
code and move it into its own dependency.

This is the x-pack side of https://github.com/elastic/elasticsearch/pull/28972

Original commit: elastic/x-pack-elasticsearch@8ba2e97b26
2018-03-14 13:48:05 -06:00
Jason Tedor 5c48fc4eaa Adjust main response constructor arguments
This commit adjusts the invocation of the main response constructor as
these were changed upstream to remove a parameter.

Original commit: elastic/x-pack-elasticsearch@846b33c9e9
2018-03-14 07:44:40 -04:00
Nik Everett 5c3b69fe57 SQL: Clean up catch
Use a mutli-catch because it is a little cleaner.

Original commit: elastic/x-pack-elasticsearch@8555e82bca
2018-03-13 10:05:58 -04:00
Nik Everett 47a2f63d5e SQL: Be more careful with break and eof (elastic/x-pack-elasticsearch#4092)
The SQL CLI was being a bit cavalier about `null`, `ctrl-c`, and
`ctrl-d` while reading passwords to the point where it'd halt with
an exception if the user hit `ctrl-d` while typing a password. This
changes it so that the CLI will instead shut down if the user
`ctrl-c`s or `ctrl-d`s while on the password prompt with an
ENOPERM error code.

This also fixes a packaging test failure I caused by a copy and paste
error where the CLI was always enforcing things as though it was reading
a password all the time. This error was causing packaging test failures.

Original commit: elastic/x-pack-elasticsearch@a882c50fc7
2018-03-13 09:00:53 -04:00
Yannick Welsch 4bce53a1ad Disallow logger methods with Object parameter
Relates to elastic/elasticsearch#28969

Original commit: elastic/x-pack-elasticsearch@1eff5eecd9
2018-03-12 11:03:23 +01:00
Igor Motov a0800c0b09 SQL: Move parameter processing from the client to the server (elastic/x-pack-elasticsearch#3822)
Moves the `?` parameter substitution in prepared from the client to 
the server side, where it can be reused by the ODBC driver and can 
be implemented in a safer manner.

relates elastic/x-pack-elasticsearch#3699

Original commit: elastic/x-pack-elasticsearch@b876ccd8ae
2018-03-07 09:25:25 -05:00
Lee Hinman 2147d217df Wrap stream passed to createParser in try-with-resources (elastic/x-pack-elasticsearch#4055)
This wraps the stream (`.streamInput()`) that is passed to many of the
`createParser` instances in the enclosing (or a new) try-with-resources block.
This ensures the `BytesReference.streamInput()` is closed.

Relates to elastic/x-pack-elasticsearch#28504

Original commit: elastic/x-pack-elasticsearch@7546e3b4d4
2018-03-04 16:48:15 -07:00
Costin Leau 58f43ad4f0 SQL: Re-enable matrix aggregations (elastic/x-pack-elasticsearch#3977)
Remove functions without a backing matrix agg

MatrixAgg works across multiple fields and exposing it directly in SQL
does not work. Instead isolated functions are exposed which get folded
and optimized into one matrix agg per field. Thus not all matrix 
functions can be exposed in SQL, at least at this time.

Instead of depending on the plugin directly, depend on the plugin client
jar (matrix-agg-client)

Remove outdated test

Original commit: elastic/x-pack-elasticsearch@ec9b31bf59
2018-03-04 01:53:56 +02:00
Costin Leau 8fd361ba83 SQL: Fix type and value in SYS TYPES (elastic/x-pack-elasticsearch#3979)
The type of BUFFER_LENGTH needs to be an integer (not NULL) and unsigned
indicate the opposite of signed.
Change isSigned from Object to primitive
Since all the consumer of isSigned expect a primitive, an Object is just causing troubles by being null.
Update description table

Original commit: elastic/x-pack-elasticsearch@8e1960bdb5
2018-03-03 22:12:22 +02:00
Nik Everett d962d33a2a SQL: Replace the cli fixture with in-process testing (elastic/x-pack-elasticsearch#3889)
I'm really really sad to be removing the cli-fixture but I've had
trouble with it leaking recently it is pretty slow. Beyond that, we'd
prefer that our test fixture only fixture things that are external
depndencies.

So, yeah, I'm removing it. So we get faster tests and no chance of
leaking processes. We lose some "realness" in the tests. Instead of
interacting with the CLI like a real user we embed it in the test
process. That means we don't test the forking, we don't test the
executable jar, and we don't test the jLine console detection stuff. On
the other hand we were kind of forcing the jLine console detection stuff
in a funky way with the fixture anyway. And we test the executable jar
in the packaging tests. And that'll have to do.

I haven't renamed `RemoteCli` because it'd bloat this commit with
mechanical changes that'd make it hard to review. I'll rename it in a
followup commit.

This also updates jLine so we can disable blinking to matching
parentheses during testing. I have no clue why, but this wasn't
happening when we used the fixture. The trouble with the blinking is
that it is based on *time* so it slows things down. Worse, it works
inconsistently! Sometimes it spits out sensible ascii codes and
sometimes it, well, spits out weird garbage. When you use it in person
it works fine though. So we keep it on when not testing.

Cleans up some redundancy in when testing CLI errors. Less copy and
paste good.

I was tempted to disable the xterm emulation entirely while working on
this because upgrading jLine changed a few things and it was a real pain
to update. But If we turned that off then we'd have *nothing* testing
the colors and such. That'd be a shame because we use color in the
output to commicate stuff. I like it so I don't want to break it.

While I was there, I replaces the cli connector's `PrintWriter` with a
`BufferedWriter`. The `PrintWriter` was kind of a trap because `println`
would fail to work properly on windows because we force the terminal
into xterm mode and it doesn't know what to do with windows line
endings. Windows.....

Additionally I fixed a race condition between disabling echo when
reading passwords and fast writers. We were disabling the echo shortly
after sending the prompt. A fast enough writer could send us text before
the echo disable kicked in. Now I delegate to `LineReader#readLine`
with a special echo mask that disables echo. This is both easier to test
and doesn't seem to have the race condition. This race condition was
failing the tests because they are so much faster now. Yay!

Original commit: elastic/x-pack-elasticsearch@d0ec027396
2018-02-27 09:24:16 -08:00
Costin Leau b8c9c5325c SQL: add catalog filter to SYS COLUMNS command (elastic/x-pack-elasticsearch#3978)
Add basic support for catalog parameters in SYS COLUMN
Pass an empty string instead of a null inside the prepared statement
Don't use pattern for catalog in getColumns

Original commit: elastic/x-pack-elasticsearch@17e9e851a0
2018-02-25 05:08:48 +02:00
Costin Leau b2631f9ac8 SQL: Extend field resolution to deal with unquoted qualified columns (elastic/x-pack-elasticsearch#3894)
* foo.bar can mean either: no table field foo.bar or table foo, field bar
The resolution rule has been extended to include the latter case as
fallback.
* Always check field ambiguity
* Since field with dots can create confusion (when not qualified), the
analyzer always now always checks for both qualified and not-qualified
fields and throws an ambiguity message with the potential candidates.
This forces the use of qualifiers or quotes to better indicate the
desired field.
* Add example of aliasing the table to remove ambiguity

Original commit: elastic/x-pack-elasticsearch@8b70b9c4ee
2018-02-25 04:39:48 +02:00
Lee Hinman c763a14132 Pass InputStream when creating XContent parser (elastic/x-pack-elasticsearch#4001)
* Pass InputStream when creating XContent parser

Rather than passing the raw `BytesReference` in when creating the xcontent
parser, this passes the StreamInput (which is an InputStream), this allows us to
decouple XContent from BytesReference.

This is the x-pack side of https://github.com/elastic/elasticsearch/pull/28754

* Use the streamInput variant, not sourceAsString

Original commit: elastic/x-pack-elasticsearch@dd5d8b1654
2018-02-23 15:22:28 -07:00
Luca Cavanna df8dcb22b8 [TEST] share code between streamable/writeable/xcontent base test classes (elastic/x-pack-elasticsearch#4021)
This is the xpack side of https://github.com/elastic/elasticsearch/pull/28785

Original commit: elastic/x-pack-elasticsearch@f1202e23af
2018-02-23 10:49:04 +01:00
Costin Leau 749d0e3880 SQL: Regen grammar
Original commit: elastic/x-pack-elasticsearch@7cac9f5ea0
2018-02-15 22:34:12 +02:00
Costin Leau ae8152b55b SQL: add catalog and type arguments to SYS TABLES (elastic/x-pack-elasticsearch#3920)
Extend the SYS TABLES command to allow filtering by catalog and types
Add creation of EnumSet inside node tests
Handle EnumSet inside test for replaceChildren

Original commit: elastic/x-pack-elasticsearch@407ce7b006
2018-02-15 22:07:51 +02:00
Costin Leau 844c485e0d SQL: introduce SYS CATALOGS and TABLE TYPES commands (elastic/x-pack-elasticsearch#3941)
These are the equivalent of JDBC/ODBC getCatalogs and getTableTypes on
the server side.

Original commit: elastic/x-pack-elasticsearch@a6fa733680
2018-02-15 21:51:23 +02:00
Tanguy Leroux 00dec27d9f SQL: Relax RemoteFailure parsing so that it also parses error metadata (elastic/x-pack-elasticsearch#3938)
The current parsing of errors in the RemoteFailure class is strict and
fails on any field that is not one of [caused_by, reason, root_cause,
stack_trace, type]. Sadly some exceptions adds more headers or
metadata when they are printed as XContent and such fields can't be
easily ignored at parsing time.

This commit changes the RemoteFailure.parseFromResponse() method so that
it parses errors using the same behavior as the high level rest client:
it parses any unknown field as a metadata if it's string value or an
array of string and just ignores and skips everything else without
throwing an exception.

Original commit: elastic/x-pack-elasticsearch@1348706807
2018-02-14 21:05:23 +01:00
Lee Hinman affc9e3563 Move more XContent.createParser calls to non-deprecated version (elastic/x-pack-elasticsearch#3928)
Part 2

This moves more of the callers to pass in the DeprecationHandler.

Relates to elastic/x-pack-elasticsearch#28504

Original commit: elastic/x-pack-elasticsearch@e0b52bf050
2018-02-14 09:41:26 -07:00
Costin Leau a5b129c10c SQL: Introduce the catalog concept inside the grammar (elastic/x-pack-elasticsearch#3911)
Indices can now specify the catalog (the cluster) in which they belong.
This is a first step in supporting both CCS and BI tools which tend to
be explicit

Original commit: elastic/x-pack-elasticsearch@0b4633facc
2018-02-14 12:30:32 +02:00
Andy Bristol e520453551 Fix location of sql cli jar
The `elasticsearch` directory in the plugin folder structure was
removed, so make sure the CLI jar ends up in $ES_HOME/bin/x-pack as
intended

Follow up from elastic/x-pack-elasticsearch#28589

Original commit: elastic/x-pack-elasticsearch@6580b38fe0
2018-02-13 11:30:56 -08:00
Igor Motov c82fdad41d SQL: Consolidate more type information into DataType (elastic/x-pack-elasticsearch#3850)
Consolidates type handling into DataType, makes DataType available to 
JDBC by moving to sql-proto and removes support for all parameter types 
that cannot be handled by the server.

Original commit: elastic/x-pack-elasticsearch@b8024f5c46
2018-02-12 16:40:09 -05:00
Nik Everett 45ac81127a SQL: Drop requirement to have keystore (elastic/x-pack-elasticsearch#3896)
SQL doesn't need the keystore. We required it by mistake.

Original commit: elastic/x-pack-elasticsearch@6840637feb
2018-02-09 17:17:46 -05:00
Ryan Ernst 4e578fe7cc Build: Replace provided configuration with compileOnly (elastic/x-pack-elasticsearch#3868)
This is the xpack side of elastic/elasticsearch#28564

Original commit: elastic/x-pack-elasticsearch@60033824a4
2018-02-09 11:30:43 -08:00
Costin Leau 3c6f9d4d29 SQL: fix name of Arithmetic functions (elastic/x-pack-elasticsearch#3884)
* SQL: fix name of Arithmetic functions

Remove id from name of Arithmetic function and improve name of Neg

Original commit: elastic/x-pack-elasticsearch@4f3e8d6a2d
2018-02-09 19:48:22 +02:00
Lee Hinman dc833f2448 Remove all instances of the deprecated `ParseField.match` method (elastic/x-pack-elasticsearch#3874)
* Remove all instances of the deprecated `ParseField.match` method

This removes all the server references to the deprecated `ParseField.match`
method in favor of the method that passes in the deprecation logger.

Relates to https://github.com/elastic/elasticsearch/issues/28504

* Fix line-length

Original commit: elastic/x-pack-elasticsearch@e7cb2611f7
2018-02-09 09:01:18 -07:00
Ryan Ernst 7d71c3f1cc Build: Remove refs to x-pack-elasticsearch (elastic/x-pack-elasticsearch#3863)
These were removed before, but sql added a bunch of new hard references
to the project name.

Original commit: elastic/x-pack-elasticsearch@2eb156fd67
2018-02-07 20:50:12 -08:00
Ryan Ernst f360d2d8d7 Build: Cleanup gradle files (elastic/x-pack-elasticsearch#3867)
There were a number of leftover unnecessary elements in the module
build.gradle files that were holdovers from copying the original plugin
build.gradle. This commit removes these elements.

Original commit: elastic/x-pack-elasticsearch@08babbd520
2018-02-07 16:58:09 -08:00
Nik Everett 10efd82694 SQL: Shift sql-cli into its proper home
The old home made sense before x-pack was split. The new home lines up
with where security keeps its scripts.

The jar file wasn't being included any more so this re-adds that.

Original commit: elastic/x-pack-elasticsearch@d3ec941397
2018-02-06 16:17:18 -05:00
Nik Everett a2f1ed3f62 SQL: Fix javadoc in JdbcConfiguration
It wasn't building....

Original commit: elastic/x-pack-elasticsearch@059c6397a2
2018-02-06 14:41:06 -05:00
Lee Hinman 5a3004300c SQL: Add some Javadocs (elastic/x-pack-elasticsearch#3816)
* SQL: Add some Javadocs

* Reflow javadoc for fewer columns

Original commit: elastic/x-pack-elasticsearch@d23239b2e5
2018-02-05 09:03:06 -07:00
Nik Everett 876aebf7e0 SQL: Make extract work for any datetime function (elastic/x-pack-elasticsearch#3756)
This allows any datetime function to be present in `EXTRACT` which feels
more consistent. `EXTRACT(FOO FROM bar)` is now just sugar for
`FOO(bar)`. This is *much* simpler to explain in the documentation then
"these 10 fields are supported by extract and they are the same as this
subset of the datetime functions."

The implementation of this is a little simpler then the old way. Instead
of resolving the function in the parser we create an
`UnresolvedFunction` that looks *almost* just like what we'd create for
a single argument function and resolve the function in the `Analyzer`.
This feels like a net positive as it allows us to group `EXTRACT`
resolution failures with other function resolution failures.

This also creates `UnresolvedFunctionTests` and
`UnresolvedAttributeTests`. I had to create `UnresolvedFunctionTests`
because `UnreolvedFunction` now has three boolean parameters which is
incompatible with the generic `NodeSubclassTests`'s requirement that all
ctor parameters be unique. I created `UnresolvedAttributeTests` because
I didn't want `UnresolvedFunctionTests` to call `NodeSubclassTests` and
figured that we'd want `UnresolvedAttributeTest` eventually and now felt
like as good a time as any.

Added a 

Original commit: elastic/x-pack-elasticsearch@358aada308
2018-02-03 16:10:09 -05:00
Costin Leau 34fe0beb30 SQL: add comments on JDBC thread-safety (or rather lack of)
Original commit: elastic/x-pack-elasticsearch@6ea5adc6a8
2018-02-02 18:11:24 +02:00
Costin Leau bda97c6e59 SQL: More polish
Original commit: elastic/x-pack-elasticsearch@cedbad466b
2018-02-02 18:01:51 +02:00
Costin Leau b5dd4c649a SQL: Minor fixes
Original commit: elastic/x-pack-elasticsearch@14ea747e20
2018-02-02 17:50:33 +02:00
Costin Leau 817c4759c6 Remove unused classes
Original commit: elastic/x-pack-elasticsearch@f7aee26821
2018-02-02 17:14:54 +02:00
Nik Everett 116807ef54 SQL: Handle rename in core
Original commit: elastic/x-pack-elasticsearch@32cd41e6c8
2018-01-30 12:50:32 -05:00
Lee Hinman 91cb994a83 [BUILD] use xpackModule for more things in SQL
Original commit: elastic/x-pack-elasticsearch@6139d949e5
2018-01-29 16:11:19 -07:00
Lee Hinman 7f3ba5c0ae Fix javadoc syntax in DataTypeConversion
Original commit: elastic/x-pack-elasticsearch@d7a2bbbe0f
2018-01-29 15:47:27 -07:00
Lee Hinman a789944b8d Merge remote-tracking branch 'origin/master' into feature/sql_2
Original commit: elastic/x-pack-elasticsearch@f96f80f0d0
2018-01-29 15:42:01 -07:00
Igor Motov 1627ec1378 SQL: Consolidate JDBC type handling (elastic/x-pack-elasticsearch#3761)
Consolidates handling of JDBC types conversion into a single file that should simplify maintaining consistency between type handling. Also separates the types that are handled as part of Elasticsearch output and types that are handled as user-supplied parameters.

relates elastic/x-pack-elasticsearch#3556

Original commit: elastic/x-pack-elasticsearch@d251fce66b
2018-01-29 16:35:42 -05:00
Costin Leau 07658cc04f SQL: replace JDBC transport meta columns with SQL call (elastic/x-pack-elasticsearch#3740)
Replace meta column endpoint with SYS COLUMNS command

Original commit: elastic/x-pack-elasticsearch@819874bc5b
2018-01-29 19:18:24 +02:00
Costin Leau e929d16d76 SQL: Replace ListTable transport with SQL command (elastic/x-pack-elasticsearch#3675)
Introduce system commands as alternative to meta HTTP endpoints
Pass in cluster name
Use 'BASE TABLE' instead of 'INDEX' when describing a table to stick
with the SQL terminology

Original commit: elastic/x-pack-elasticsearch@600312b8f7
2018-01-27 11:54:19 +02:00
Ryan Ernst 5a86450df7 Build: Replace references to x-pack-elasticsearch paths with helper methods (elastic/x-pack-elasticsearch#3748)
In order to more easily integrate xpack once it moves into the
elasticsearch repo, references to the existing x-pack-elasticsearch need
to be reduced. This commit introduces a few helper "methods" available
to any project within xpack (through gradle project extension
properties, as closures). All refeerences to project paths now use these
helper methods, except for those pertaining to bwc, which will be
handled in a followup.

Original commit: elastic/x-pack-elasticsearch@850668744c
2018-01-26 21:48:30 -08:00
Lee Hinman dc40562be6 SQL: Remove cli.debug system parameter (elastic/x-pack-elasticsearch#3722)
This was requested for removal in
https://github.com/elastic/x-pack-elasticsearch/pull/3686#discussion_r163286953
and is a followup for that.

Original commit: elastic/x-pack-elasticsearch@6134693a34
2018-01-25 10:51:12 -07:00
Lee Hinman 4d213666d5 SQL: Move Exception classes to follow ES format (elastic/x-pack-elasticsearch#3721)
This commit moves the exception classes that SQL uses to follow the
Elasticsearch convention. In the places where varargs were used, the
`LoggerMessageFormat` (`{}`) standard is used instead.

In also removes on Exception - `ExecutionException` since it seemed to not ever
be beefed up, it can be re-added later if needed.

This removes the varargs version of `ClientException` to push the formatting
back on the caller, since `ClientException` cannot depend on Elasticsearch for
formatting the arguments.

There were also a couple of places where we incorrectly passed the Throwable
cause as a var-arg and were unintentionally swallowing it since `String.format`
discards unused arguments.

Relates to elastic/x-pack-elasticsearch#2880

Original commit: elastic/x-pack-elasticsearch@5f5d580e57
2018-01-24 13:02:01 -07:00
Igor Motov 36bd849cd3 SQL: Refactor the type handling (elastic/x-pack-elasticsearch#3694)
Separates ES types and field metadata.

Relates to elastic/x-pack-elasticsearch#3556

Original commit: elastic/x-pack-elasticsearch@c9601f3390
2018-01-24 14:43:23 -05:00
Lee Hinman 9f8c8ef3e9 Parse CLI's `-d` or `--debug` flag as turning on debug (elastic/x-pack-elasticsearch#3686)
Previously in order to turn on debug mode for the CLI, a user needed to specify
`-d true`. You now only need to specify `-d` or `--debug` to turn on debug mode.

Resolves elastic/x-pack-elasticsearch#3315

Original commit: elastic/x-pack-elasticsearch@d5052059e6
2018-01-23 15:40:43 -07:00
Lee Hinman 0256f736e1 [TEST] Ensure reflectively created Lists always have at least one element
Fixes an issue in the test where the class can be constructed but fails to
replace its children due to validation of the number of elements in the list.

Original commit: elastic/x-pack-elasticsearch@3199318d9c
2018-01-23 15:22:14 -07:00