2017-12-13 10:19:31 -05:00
|
|
|
description = 'Integration tests for SQL'
|
|
|
|
apply plugin: 'elasticsearch.build'
|
2018-01-17 15:48:58 -05:00
|
|
|
archivesBaseName = 'qa-sql'
|
2018-07-06 11:24:12 -04:00
|
|
|
group = "org.elasticsearch.x-pack.qa.sql"
|
2017-12-13 10:19:31 -05:00
|
|
|
|
|
|
|
dependencies {
|
2019-06-04 16:50:23 -04:00
|
|
|
compile project(":test:framework")
|
2017-12-13 10:19:31 -05:00
|
|
|
|
SQL: Replace the cli fixture with in-process testing (elastic/x-pack-elasticsearch#3889)
I'm really really sad to be removing the cli-fixture but I've had
trouble with it leaking recently it is pretty slow. Beyond that, we'd
prefer that our test fixture only fixture things that are external
depndencies.
So, yeah, I'm removing it. So we get faster tests and no chance of
leaking processes. We lose some "realness" in the tests. Instead of
interacting with the CLI like a real user we embed it in the test
process. That means we don't test the forking, we don't test the
executable jar, and we don't test the jLine console detection stuff. On
the other hand we were kind of forcing the jLine console detection stuff
in a funky way with the fixture anyway. And we test the executable jar
in the packaging tests. And that'll have to do.
I haven't renamed `RemoteCli` because it'd bloat this commit with
mechanical changes that'd make it hard to review. I'll rename it in a
followup commit.
This also updates jLine so we can disable blinking to matching
parentheses during testing. I have no clue why, but this wasn't
happening when we used the fixture. The trouble with the blinking is
that it is based on *time* so it slows things down. Worse, it works
inconsistently! Sometimes it spits out sensible ascii codes and
sometimes it, well, spits out weird garbage. When you use it in person
it works fine though. So we keep it on when not testing.
Cleans up some redundancy in when testing CLI errors. Less copy and
paste good.
I was tempted to disable the xterm emulation entirely while working on
this because upgrading jLine changed a few things and it was a real pain
to update. But If we turned that off then we'd have *nothing* testing
the colors and such. That'd be a shame because we use color in the
output to commicate stuff. I like it so I don't want to break it.
While I was there, I replaces the cli connector's `PrintWriter` with a
`BufferedWriter`. The `PrintWriter` was kind of a trap because `println`
would fail to work properly on windows because we force the terminal
into xterm mode and it doesn't know what to do with windows line
endings. Windows.....
Additionally I fixed a race condition between disabling echo when
reading passwords and fast writers. We were disabling the echo shortly
after sending the prompt. A fast enough writer could send us text before
the echo disable kicked in. Now I delegate to `LineReader#readLine`
with a special echo mask that disables echo. This is both easier to test
and doesn't seem to have the race condition. This race condition was
failing the tests because they are so much faster now. Yay!
Original commit: elastic/x-pack-elasticsearch@d0ec0273964630a3a113ab27009fdff6a182ecb2
2018-02-27 12:24:16 -05:00
|
|
|
// JDBC testing dependencies
|
2019-10-28 15:11:55 -04:00
|
|
|
compile project(path: xpackModule('sql:jdbc'))
|
2018-07-03 09:56:31 -04:00
|
|
|
|
2019-03-28 09:21:59 -04:00
|
|
|
compile "net.sourceforge.csvjdbc:csvjdbc:${csvjdbcVersion}"
|
2017-12-13 10:19:31 -05:00
|
|
|
|
2018-04-12 15:43:35 -04:00
|
|
|
// CLI testing dependencies
|
2019-10-28 15:11:55 -04:00
|
|
|
compile project(path: xpackModule('sql:sql-cli'))
|
2019-11-14 06:01:23 -05:00
|
|
|
|
SQL: Add initial geo support (#42031) (#42135)
Adds an initial limited implementations of geo features to SQL. This implementation is based on the [OpenGIS® Implementation Standard for Geographic information - Simple feature access](http://www.opengeospatial.org/standards/sfs), which is the current standard for GIS system implementation. This effort is concentrate on SQL option AKA ISO 19125-2.
Queries that are supported as a result of this initial implementation
Metadata commands
- `DESCRIBE table` - returns the correct column types `GEOMETRY` for geo shapes and geo points.
- `SHOW FUNCTIONS` - returns a list that includes supported `ST_` functions
- `SYS TYPES` and `SYS COLUMNS` display correct types `GEO_SHAPE` and `GEO_POINT` for geo shapes and geo points accordingly.
Returning geoshapes and geopoints from elasticsearch
- `SELECT geom FROM table` - returns the geoshapes and geo_points as libs/geo objects in JDBC or as WKT strings in console.
- `SELECT ST_AsWKT(geom) FROM table;` and `SELECT ST_AsText(geom) FROM table;`- returns the geoshapes ang geopoints in their WKT representation;
Using geopoints to elasticsearch
- The following functions will be supported for geopoints in queries, sorting and aggregations: `ST_GeomFromText`, `ST_X`, `ST_Y`, `ST_Z`, `ST_GeometryType`, and `ST_Distance`. In most cases when used in queries, sorting and aggregations, these function are translated into script. These functions can be used in the SELECT clause for both geopoints and geoshapes.
- `SELECT * FROM table WHERE ST_Distance(ST_GeomFromText(POINT(1 2), point) < 10;` - returns all records for which `point` is located within 10m from the `POINT(1 2)`. In this case the WHERE clause is translated into a range query.
Limitations:
Geoshapes cannot be used in queries, sorting and aggregations as part of this initial effort. In order to fully take advantage of geoshapes we would need to have access to geoshape doc values, which is coming in #37206. `ST_Z` cannot be used on geopoints in queries, sorting and aggregations since we don't store altitude in geo_point doc values.
Relates to #29872
Backport of #42031
2019-05-14 19:57:12 -04:00
|
|
|
// H2GIS testing dependencies
|
2019-11-14 06:01:23 -05:00
|
|
|
compile("org.orbisgis:h2gis:${h2gisVersion}") {
|
SQL: Add initial geo support (#42031) (#42135)
Adds an initial limited implementations of geo features to SQL. This implementation is based on the [OpenGIS® Implementation Standard for Geographic information - Simple feature access](http://www.opengeospatial.org/standards/sfs), which is the current standard for GIS system implementation. This effort is concentrate on SQL option AKA ISO 19125-2.
Queries that are supported as a result of this initial implementation
Metadata commands
- `DESCRIBE table` - returns the correct column types `GEOMETRY` for geo shapes and geo points.
- `SHOW FUNCTIONS` - returns a list that includes supported `ST_` functions
- `SYS TYPES` and `SYS COLUMNS` display correct types `GEO_SHAPE` and `GEO_POINT` for geo shapes and geo points accordingly.
Returning geoshapes and geopoints from elasticsearch
- `SELECT geom FROM table` - returns the geoshapes and geo_points as libs/geo objects in JDBC or as WKT strings in console.
- `SELECT ST_AsWKT(geom) FROM table;` and `SELECT ST_AsText(geom) FROM table;`- returns the geoshapes ang geopoints in their WKT representation;
Using geopoints to elasticsearch
- The following functions will be supported for geopoints in queries, sorting and aggregations: `ST_GeomFromText`, `ST_X`, `ST_Y`, `ST_Z`, `ST_GeometryType`, and `ST_Distance`. In most cases when used in queries, sorting and aggregations, these function are translated into script. These functions can be used in the SELECT clause for both geopoints and geoshapes.
- `SELECT * FROM table WHERE ST_Distance(ST_GeomFromText(POINT(1 2), point) < 10;` - returns all records for which `point` is located within 10m from the `POINT(1 2)`. In this case the WHERE clause is translated into a range query.
Limitations:
Geoshapes cannot be used in queries, sorting and aggregations as part of this initial effort. In order to fully take advantage of geoshapes we would need to have access to geoshape doc values, which is coming in #37206. `ST_Z` cannot be used on geopoints in queries, sorting and aggregations since we don't store altitude in geo_point doc values.
Relates to #29872
Backport of #42031
2019-05-14 19:57:12 -04:00
|
|
|
exclude group: "org.locationtech.jts"
|
|
|
|
}
|
|
|
|
|
2019-03-22 17:20:41 -04:00
|
|
|
// select just the parts of JLine that are needed
|
|
|
|
compile("org.jline:jline-terminal-jna:${jlineVersion}") {
|
|
|
|
exclude group: "net.java.dev.jna"
|
|
|
|
}
|
|
|
|
compile "org.jline:jline-terminal:${jlineVersion}"
|
|
|
|
compile "org.jline:jline-reader:${jlineVersion}"
|
|
|
|
compile "org.jline:jline-style:${jlineVersion}"
|
|
|
|
|
|
|
|
testRuntime "org.elasticsearch:jna:${versions.jna}"
|
2017-12-13 10:19:31 -05:00
|
|
|
}
|
|
|
|
|
|
|
|
/* disable unit tests because these are all integration tests used
|
|
|
|
* other qa projects. */
|
2019-04-09 14:52:50 -04:00
|
|
|
test.enabled = false
|
2017-12-13 10:19:31 -05:00
|
|
|
|
|
|
|
dependencyLicenses.enabled = false
|
2018-06-09 07:28:41 -04:00
|
|
|
dependenciesInfo.enabled = false
|
2017-12-13 10:19:31 -05:00
|
|
|
|
|
|
|
// the main files are actually test files, so use the appropriate forbidden api sigs
|
2020-04-15 16:23:55 -04:00
|
|
|
tasks.named('forbiddenApisMain').configure {
|
2018-08-22 02:05:22 -04:00
|
|
|
replaceSignatureFiles 'es-all-signatures', 'es-test-signatures'
|
2017-12-13 10:19:31 -05:00
|
|
|
}
|
|
|
|
|
SQL: Add initial geo support (#42031) (#42135)
Adds an initial limited implementations of geo features to SQL. This implementation is based on the [OpenGIS® Implementation Standard for Geographic information - Simple feature access](http://www.opengeospatial.org/standards/sfs), which is the current standard for GIS system implementation. This effort is concentrate on SQL option AKA ISO 19125-2.
Queries that are supported as a result of this initial implementation
Metadata commands
- `DESCRIBE table` - returns the correct column types `GEOMETRY` for geo shapes and geo points.
- `SHOW FUNCTIONS` - returns a list that includes supported `ST_` functions
- `SYS TYPES` and `SYS COLUMNS` display correct types `GEO_SHAPE` and `GEO_POINT` for geo shapes and geo points accordingly.
Returning geoshapes and geopoints from elasticsearch
- `SELECT geom FROM table` - returns the geoshapes and geo_points as libs/geo objects in JDBC or as WKT strings in console.
- `SELECT ST_AsWKT(geom) FROM table;` and `SELECT ST_AsText(geom) FROM table;`- returns the geoshapes ang geopoints in their WKT representation;
Using geopoints to elasticsearch
- The following functions will be supported for geopoints in queries, sorting and aggregations: `ST_GeomFromText`, `ST_X`, `ST_Y`, `ST_Z`, `ST_GeometryType`, and `ST_Distance`. In most cases when used in queries, sorting and aggregations, these function are translated into script. These functions can be used in the SELECT clause for both geopoints and geoshapes.
- `SELECT * FROM table WHERE ST_Distance(ST_GeomFromText(POINT(1 2), point) < 10;` - returns all records for which `point` is located within 10m from the `POINT(1 2)`. In this case the WHERE clause is translated into a range query.
Limitations:
Geoshapes cannot be used in queries, sorting and aggregations as part of this initial effort. In order to fully take advantage of geoshapes we would need to have access to geoshape doc values, which is coming in #37206. `ST_Z` cannot be used on geopoints in queries, sorting and aggregations since we don't store altitude in geo_point doc values.
Relates to #29872
Backport of #42031
2019-05-14 19:57:12 -04:00
|
|
|
// just a test fixture: we aren't using this jars in releases and H2GIS requires disabling a lot of checks
|
|
|
|
thirdPartyAudit.enabled = false
|
|
|
|
|
2017-12-13 10:19:31 -05:00
|
|
|
subprojects {
|
2019-09-11 13:55:31 -04:00
|
|
|
if (subprojects.isEmpty()) {
|
|
|
|
// leaf project
|
|
|
|
apply plugin: 'elasticsearch.standalone-rest-test'
|
|
|
|
} else {
|
|
|
|
apply plugin: 'elasticsearch.build'
|
|
|
|
}
|
|
|
|
|
|
|
|
configurations.testRuntimeClasspath {
|
|
|
|
resolutionStrategy.force "org.slf4j:slf4j-api:1.7.25"
|
|
|
|
}
|
2019-09-18 03:39:48 -04:00
|
|
|
configurations.testRuntime {
|
|
|
|
// This is also required to make resolveAllDependencies work
|
|
|
|
resolutionStrategy.force "org.slf4j:slf4j-api:1.7.25"
|
|
|
|
}
|
2017-12-13 10:19:31 -05:00
|
|
|
dependencies {
|
2019-11-14 06:01:23 -05:00
|
|
|
|
2017-12-13 10:19:31 -05:00
|
|
|
/* Since we're a standalone rest test we actually get transitive
|
|
|
|
* dependencies but we don't really want them because they cause
|
|
|
|
* all kinds of trouble with the jar hell checks. So we suppress
|
|
|
|
* them explicitly for non-es projects. */
|
2018-10-30 14:54:15 -04:00
|
|
|
testCompile(xpackProject('plugin:sql:qa')) {
|
2017-12-13 10:19:31 -05:00
|
|
|
transitive = false
|
|
|
|
}
|
2019-06-04 16:50:23 -04:00
|
|
|
testCompile project(":test:framework")
|
2017-12-13 10:19:31 -05:00
|
|
|
|
|
|
|
// JDBC testing dependencies
|
2019-03-28 09:21:59 -04:00
|
|
|
testRuntime "net.sourceforge.csvjdbc:csvjdbc:${csvjdbcVersion}"
|
|
|
|
testRuntime "com.h2database:h2:${h2Version}"
|
SQL: Add initial geo support (#42031) (#42135)
Adds an initial limited implementations of geo features to SQL. This implementation is based on the [OpenGIS® Implementation Standard for Geographic information - Simple feature access](http://www.opengeospatial.org/standards/sfs), which is the current standard for GIS system implementation. This effort is concentrate on SQL option AKA ISO 19125-2.
Queries that are supported as a result of this initial implementation
Metadata commands
- `DESCRIBE table` - returns the correct column types `GEOMETRY` for geo shapes and geo points.
- `SHOW FUNCTIONS` - returns a list that includes supported `ST_` functions
- `SYS TYPES` and `SYS COLUMNS` display correct types `GEO_SHAPE` and `GEO_POINT` for geo shapes and geo points accordingly.
Returning geoshapes and geopoints from elasticsearch
- `SELECT geom FROM table` - returns the geoshapes and geo_points as libs/geo objects in JDBC or as WKT strings in console.
- `SELECT ST_AsWKT(geom) FROM table;` and `SELECT ST_AsText(geom) FROM table;`- returns the geoshapes ang geopoints in their WKT representation;
Using geopoints to elasticsearch
- The following functions will be supported for geopoints in queries, sorting and aggregations: `ST_GeomFromText`, `ST_X`, `ST_Y`, `ST_Z`, `ST_GeometryType`, and `ST_Distance`. In most cases when used in queries, sorting and aggregations, these function are translated into script. These functions can be used in the SELECT clause for both geopoints and geoshapes.
- `SELECT * FROM table WHERE ST_Distance(ST_GeomFromText(POINT(1 2), point) < 10;` - returns all records for which `point` is located within 10m from the `POINT(1 2)`. In this case the WHERE clause is translated into a range query.
Limitations:
Geoshapes cannot be used in queries, sorting and aggregations as part of this initial effort. In order to fully take advantage of geoshapes we would need to have access to geoshape doc values, which is coming in #37206. `ST_Z` cannot be used on geopoints in queries, sorting and aggregations since we don't store altitude in geo_point doc values.
Relates to #29872
Backport of #42031
2019-05-14 19:57:12 -04:00
|
|
|
|
|
|
|
// H2GIS testing dependencies
|
2019-11-14 06:01:23 -05:00
|
|
|
testRuntime("org.orbisgis:h2gis:${h2gisVersion}") {
|
2019-09-11 13:55:31 -04:00
|
|
|
exclude group: "org.locationtech.jts"
|
|
|
|
exclude group: "com.fasterxml.jackson.core"
|
SQL: Add initial geo support (#42031) (#42135)
Adds an initial limited implementations of geo features to SQL. This implementation is based on the [OpenGIS® Implementation Standard for Geographic information - Simple feature access](http://www.opengeospatial.org/standards/sfs), which is the current standard for GIS system implementation. This effort is concentrate on SQL option AKA ISO 19125-2.
Queries that are supported as a result of this initial implementation
Metadata commands
- `DESCRIBE table` - returns the correct column types `GEOMETRY` for geo shapes and geo points.
- `SHOW FUNCTIONS` - returns a list that includes supported `ST_` functions
- `SYS TYPES` and `SYS COLUMNS` display correct types `GEO_SHAPE` and `GEO_POINT` for geo shapes and geo points accordingly.
Returning geoshapes and geopoints from elasticsearch
- `SELECT geom FROM table` - returns the geoshapes and geo_points as libs/geo objects in JDBC or as WKT strings in console.
- `SELECT ST_AsWKT(geom) FROM table;` and `SELECT ST_AsText(geom) FROM table;`- returns the geoshapes ang geopoints in their WKT representation;
Using geopoints to elasticsearch
- The following functions will be supported for geopoints in queries, sorting and aggregations: `ST_GeomFromText`, `ST_X`, `ST_Y`, `ST_Z`, `ST_GeometryType`, and `ST_Distance`. In most cases when used in queries, sorting and aggregations, these function are translated into script. These functions can be used in the SELECT clause for both geopoints and geoshapes.
- `SELECT * FROM table WHERE ST_Distance(ST_GeomFromText(POINT(1 2), point) < 10;` - returns all records for which `point` is located within 10m from the `POINT(1 2)`. In this case the WHERE clause is translated into a range query.
Limitations:
Geoshapes cannot be used in queries, sorting and aggregations as part of this initial effort. In order to fully take advantage of geoshapes we would need to have access to geoshape doc values, which is coming in #37206. `ST_Z` cannot be used on geopoints in queries, sorting and aggregations since we don't store altitude in geo_point doc values.
Relates to #29872
Backport of #42031
2019-05-14 19:57:12 -04:00
|
|
|
}
|
2019-11-14 06:01:23 -05:00
|
|
|
|
2019-10-28 15:11:55 -04:00
|
|
|
testRuntime project(path: xpackModule('sql:jdbc'))
|
2018-06-28 08:56:16 -04:00
|
|
|
testRuntime xpackProject('plugin:sql:sql-client')
|
2018-06-08 10:15:28 -04:00
|
|
|
|
SQL: Replace the cli fixture with in-process testing (elastic/x-pack-elasticsearch#3889)
I'm really really sad to be removing the cli-fixture but I've had
trouble with it leaking recently it is pretty slow. Beyond that, we'd
prefer that our test fixture only fixture things that are external
depndencies.
So, yeah, I'm removing it. So we get faster tests and no chance of
leaking processes. We lose some "realness" in the tests. Instead of
interacting with the CLI like a real user we embed it in the test
process. That means we don't test the forking, we don't test the
executable jar, and we don't test the jLine console detection stuff. On
the other hand we were kind of forcing the jLine console detection stuff
in a funky way with the fixture anyway. And we test the executable jar
in the packaging tests. And that'll have to do.
I haven't renamed `RemoteCli` because it'd bloat this commit with
mechanical changes that'd make it hard to review. I'll rename it in a
followup commit.
This also updates jLine so we can disable blinking to matching
parentheses during testing. I have no clue why, but this wasn't
happening when we used the fixture. The trouble with the blinking is
that it is based on *time* so it slows things down. Worse, it works
inconsistently! Sometimes it spits out sensible ascii codes and
sometimes it, well, spits out weird garbage. When you use it in person
it works fine though. So we keep it on when not testing.
Cleans up some redundancy in when testing CLI errors. Less copy and
paste good.
I was tempted to disable the xterm emulation entirely while working on
this because upgrading jLine changed a few things and it was a real pain
to update. But If we turned that off then we'd have *nothing* testing
the colors and such. That'd be a shame because we use color in the
output to commicate stuff. I like it so I don't want to break it.
While I was there, I replaces the cli connector's `PrintWriter` with a
`BufferedWriter`. The `PrintWriter` was kind of a trap because `println`
would fail to work properly on windows because we force the terminal
into xterm mode and it doesn't know what to do with windows line
endings. Windows.....
Additionally I fixed a race condition between disabling echo when
reading passwords and fast writers. We were disabling the echo shortly
after sending the prompt. A fast enough writer could send us text before
the echo disable kicked in. Now I delegate to `LineReader#readLine`
with a special echo mask that disables echo. This is both easier to test
and doesn't seem to have the race condition. This race condition was
failing the tests because they are so much faster now. Yay!
Original commit: elastic/x-pack-elasticsearch@d0ec0273964630a3a113ab27009fdff6a182ecb2
2018-02-27 12:24:16 -05:00
|
|
|
// TODO check if needed
|
2019-03-22 17:20:41 -04:00
|
|
|
testRuntime("org.antlr:antlr4-runtime:${antlrVersion}") {
|
2017-12-13 10:19:31 -05:00
|
|
|
transitive = false
|
|
|
|
}
|
|
|
|
|
2018-04-12 15:43:35 -04:00
|
|
|
// CLI testing dependencies
|
2019-10-28 15:11:55 -04:00
|
|
|
testRuntime project(path: xpackModule('sql:sql-cli'))
|
2019-11-14 06:01:23 -05:00
|
|
|
testRuntime(xpackProject('plugin:sql:sql-action')) {
|
2018-05-25 15:41:41 -04:00
|
|
|
transitive = false
|
|
|
|
}
|
2019-03-22 17:20:41 -04:00
|
|
|
|
|
|
|
testRuntime("org.jline:jline-terminal-jna:${jlineVersion}") {
|
|
|
|
exclude group: "net.java.dev.jna"
|
|
|
|
}
|
|
|
|
testRuntime "org.jline:jline-terminal:${jlineVersion}"
|
|
|
|
testRuntime "org.jline:jline-reader:${jlineVersion}"
|
|
|
|
testRuntime "org.jline:jline-style:${jlineVersion}"
|
|
|
|
|
|
|
|
testRuntime "org.elasticsearch:jna:${versions.jna}"
|
2020-04-22 11:12:54 -04:00
|
|
|
|
|
|
|
// spatial dependency
|
|
|
|
testRuntime project(path: xpackModule('spatial'))
|
2017-12-13 10:19:31 -05:00
|
|
|
}
|
|
|
|
|
|
|
|
if (project.name != 'security') {
|
2018-01-17 15:48:58 -05:00
|
|
|
// The security project just configures its subprojects
|
2019-06-20 02:21:29 -04:00
|
|
|
apply plugin: 'elasticsearch.testclusters'
|
2017-12-13 10:19:31 -05:00
|
|
|
apply plugin: 'elasticsearch.rest-test'
|
|
|
|
|
2019-06-20 02:21:29 -04:00
|
|
|
testClusters.integTest {
|
2019-07-15 20:53:05 -04:00
|
|
|
testDistribution = 'DEFAULT'
|
2017-12-13 10:19:31 -05:00
|
|
|
setting 'xpack.ml.enabled', 'false'
|
|
|
|
setting 'xpack.watcher.enabled', 'false'
|
|
|
|
}
|
|
|
|
|
2019-06-20 02:21:29 -04:00
|
|
|
task runqa {
|
|
|
|
doFirst {
|
|
|
|
println "Run with `-Dtestclusters.inspect.failure=true integTest` to leave the cluster running after failure"
|
|
|
|
}
|
2017-12-13 10:19:31 -05:00
|
|
|
}
|
|
|
|
}
|
|
|
|
}
|