This renames that `write` and `read` methods in SQL to `writeTo` and
`readFrom` to line up with the names used in core. I don't have a
strong opinion whether or not any name is better than any other but
I figure there isn't a good reason for SQL to be different from the
rest of Elasticsearch.
Original commit: elastic/x-pack-elasticsearch@e5de9a4b81
* TimeoutInfo - This is now tracked in the SQL tracker github issue
* AbstractProto - Convert to a TODO as we *can* handle it after
release. I've added it to the SQL tracker github issue in a special
section for low priority protocol stuff. Protocol stuff is special
because if we can make the change before release we don't have to
worry about backwards compatibility.
Original commit: elastic/x-pack-elasticsearch@dbef9db5f8
* Move CLI to TransportSqlAction
* Moves REST endpoint from `/_cli` to `/_sql/cli`
* Removes the special purpose CLI transport action instead
implements the CLI entirely on the REST layer, delegating
all SQL stuff to the same action that backs the `/_sql` REST
API.
* Reworks "embedded testing mode" to use a `FilterClient` to
bounce capture the sql transport action and execute in embedded.
* Switches CLI formatting from consuming the entire response
to consuming just the first page of the response and returning
a `cursor` that can be used to read the next page. That read is
not yet implemented.
* Switch CLI formatting from the consuming the `RowSetCursor` to
consuming the `SqlResponse` object.
* Adds tests for CLI formatting.
* Support next page in the cli
* Rename cli's CommandRequest/CommandResponse to
QueryInitRequest/QueryInitResponse to line up with jdbc
* Implement QueryPageRequest/QueryPageResponse in cli
* Use `byte[]` to represent the cursor in the cli. Those bytes
mean something, but only to the server. The only reasonint that
the client does about them is "if length == 0 then there isn't a
next page."
* Pull common code from jdbc's QueryInitRequest, QueryPageRequest,
QueryInitResponse, and QueryPageResponse into the shared-proto
project
* By implication this switches jdbc's QueryPageRequest to using
the same cursor implementation as the cli
Original commit: elastic/x-pack-elasticsearch@193586f1ee
Instead of throwing and catching an exception for invalid
indices this returns *why* they are invalid in a convenient
object form that can be thrown as an exception when the index
is required or the entire index can be ignored when listing
indices.
Original commit: elastic/x-pack-elasticsearch@f45cbce647
This integrates SQL's metadata calls with security by creating
`SqlIndicesAction` and routing all of SQL's metadata calls through
it. Since it *does* know up from which indices it is working against
it can be an `IndicesRequest.Replaceable` and integrate with the
existing security infrastructure for filtering indices.
This request is implemented fairly similarly to the `GetIndexAction`
with the option to read from the master or from a local copy of
cluster state. Currently SQL forces it to run on the local copy
because the request doesn't properly support serialization. I'd
like to implement that in a followup.
Original commit: elastic/x-pack-elasticsearch@15f9512820
In core we prefer not to extend `AbstractLifecycleComponent` because
the reasons for the tradeoffs that it makes are lost to the sands of
time.
Original commit: elastic/x-pack-elasticsearch@ec1a32bbb3
Removes the line length checkstyle suppression for `sql/jdbc*` and
fixes all the checkstyle violations. Removes a few files that had
violations that were not used.
Original commit: elastic/x-pack-elasticsearch@031c2ba8e3
Core doesn't go in for fancy collection utils in general and just
manipulates the required collections in line. In an effort to keep
SQL "more like the rest of Elasticsearch" I'm starting to remove
SQL's `CollectionUtils`.
Original commit: elastic/x-pack-elasticsearch@878ee181cb
It is used for debugging tests and should be kept abstract or
else it violates the naming conventions. It violates the naming
conventions because we do not wish to run it with the normal build.
Original commit: elastic/x-pack-elasticsearch@ce5810b15a
Scrolling was only implemented for the `SqlAction` (not jdbc or cli)
and it was implemented by keeping request state on the server. On
principle we try to avoid adding extra state to elasticsearch where
possible because it creates extra points of failure and tends to
have lots of hidden complexity.
This replaces the state on the server with serializing state to the
client. This looks to the user like a "next_page" key with fairly
opaque content. It actually consists of an identifier for the *kind*
of scroll, the scroll id, and a base64 string containing the field
extractors.
Right now this only implements scrolling for `SqlAction`. The plan
is to reuse the same implementation for jdbc and cli in a followup.
This also doesn't implement all of the required serialization.
Specifically it doesn't implement serialization of
`ProcessingHitExtractor` because I haven't implemented serialization
for *any* `ColumnProcessors`.
Original commit: elastic/x-pack-elasticsearch@a8567bc5ec
Remove some duplicated methods, add some templating plus logging of
ES resultset (for easier debugging)
Rename debug test for CSV plus add one for Sql spec
Original commit: elastic/x-pack-elasticsearch@d2c46a2ed2
Adds a granular licensing support to SQL. JDBC now requires a platinum license, everything else work with any non-expired license.
Original commit: elastic/x-pack-elasticsearch@a30470e2c9
Most tests worked fine. The datetime tests are broken for some time
zones. The csv tests were broken because they accepted the default
fetch size which looks like it is broken.
Original commit: elastic/x-pack-elasticsearch@e034c2f102
This moves validating that each index contains only a single type
into EsCatalog which removes a race condition and removes a method
from the Catalog interface. Removing methods from the Catalog
interface is nice because the fewer methods that the interface has
the fewer have to be secured.
Original commit: elastic/x-pack-elasticsearch@85cd089e47
Adds a test that shows *how* SQL fails to address an index with two types
to the full cluster restart tests. Because we're writing this code
against 7.0 don't actually execute the test, but we will execute it when
we merge to 6.x and it *should* work.
Original commit: elastic/x-pack-elasticsearch@b536e9a142
This adds support for field level security to SQL by creating a new type of flow for securing requests that look like sql requests. `AuthorizationService` verifies that the user can execute the request but doesn't check the indices in the request because they are not yet ready. Instead, it adds a `BiFunction` to the context that can be used to check permissions for an index while servicing the request. This allows requests to cooperatively secure themselves. SQL does this by implementing filtering on top of its `Catalog` abstraction and backing that filtering with security's filters. This minimizes the touch points between security and SQL.
Stuff I'd like to do in followups:
What doesn't work at all still:
1. `SHOW TABLES` is still totally unsecured
2. `DESCRIBE TABLE` is still totally unsecured
3. JDBC's metadata APIs are still totally unsecured
What kind of works but not well:
1. The audit trail doesn't show the index being authorized for SQL.
Original commit: elastic/x-pack-elasticsearch@86f88ba2f5
Indices discovery actively ignores indices with more than one type.
However queries made such indices throw an exception (assuming the user
by mistake or not, selects such an index).
Original commit: elastic/x-pack-elasticsearch@16855c7b8f
Nik Everett
Igor Motov
Costin Leau