OpenSearch/x-pack/plugin/sql/qa/build.gradle

138 lines
5.0 KiB
Groovy
Raw Normal View History

import org.elasticsearch.gradle.test.RunTask
description = 'Integration tests for SQL'
apply plugin: 'elasticsearch.build'
archivesBaseName = 'qa-sql'
group = "org.elasticsearch.x-pack.qa.sql"
dependencies {
compile "org.elasticsearch.test:framework:${version}"
SQL: Replace the cli fixture with in-process testing (elastic/x-pack-elasticsearch#3889) I'm really really sad to be removing the cli-fixture but I've had trouble with it leaking recently it is pretty slow. Beyond that, we'd prefer that our test fixture only fixture things that are external depndencies. So, yeah, I'm removing it. So we get faster tests and no chance of leaking processes. We lose some "realness" in the tests. Instead of interacting with the CLI like a real user we embed it in the test process. That means we don't test the forking, we don't test the executable jar, and we don't test the jLine console detection stuff. On the other hand we were kind of forcing the jLine console detection stuff in a funky way with the fixture anyway. And we test the executable jar in the packaging tests. And that'll have to do. I haven't renamed `RemoteCli` because it'd bloat this commit with mechanical changes that'd make it hard to review. I'll rename it in a followup commit. This also updates jLine so we can disable blinking to matching parentheses during testing. I have no clue why, but this wasn't happening when we used the fixture. The trouble with the blinking is that it is based on *time* so it slows things down. Worse, it works inconsistently! Sometimes it spits out sensible ascii codes and sometimes it, well, spits out weird garbage. When you use it in person it works fine though. So we keep it on when not testing. Cleans up some redundancy in when testing CLI errors. Less copy and paste good. I was tempted to disable the xterm emulation entirely while working on this because upgrading jLine changed a few things and it was a real pain to update. But If we turned that off then we'd have *nothing* testing the colors and such. That'd be a shame because we use color in the output to commicate stuff. I like it so I don't want to break it. While I was there, I replaces the cli connector's `PrintWriter` with a `BufferedWriter`. The `PrintWriter` was kind of a trap because `println` would fail to work properly on windows because we force the terminal into xterm mode and it doesn't know what to do with windows line endings. Windows..... Additionally I fixed a race condition between disabling echo when reading passwords and fast writers. We were disabling the echo shortly after sending the prompt. A fast enough writer could send us text before the echo disable kicked in. Now I delegate to `LineReader#readLine` with a special echo mask that disables echo. This is both easier to test and doesn't seem to have the race condition. This race condition was failing the tests because they are so much faster now. Yay! Original commit: elastic/x-pack-elasticsearch@d0ec0273964630a3a113ab27009fdff6a182ecb2
2018-02-27 12:24:16 -05:00
// JDBC testing dependencies
compile project(path: xpackModule('sql:jdbc'), configuration: 'nodeps')
compile project(path: xpackModule('sql:sql-action'))
compile "net.sourceforge.csvjdbc:csvjdbc:1.0.34"
// CLI testing dependencies
SQL: Replace the cli fixture with in-process testing (elastic/x-pack-elasticsearch#3889) I'm really really sad to be removing the cli-fixture but I've had trouble with it leaking recently it is pretty slow. Beyond that, we'd prefer that our test fixture only fixture things that are external depndencies. So, yeah, I'm removing it. So we get faster tests and no chance of leaking processes. We lose some "realness" in the tests. Instead of interacting with the CLI like a real user we embed it in the test process. That means we don't test the forking, we don't test the executable jar, and we don't test the jLine console detection stuff. On the other hand we were kind of forcing the jLine console detection stuff in a funky way with the fixture anyway. And we test the executable jar in the packaging tests. And that'll have to do. I haven't renamed `RemoteCli` because it'd bloat this commit with mechanical changes that'd make it hard to review. I'll rename it in a followup commit. This also updates jLine so we can disable blinking to matching parentheses during testing. I have no clue why, but this wasn't happening when we used the fixture. The trouble with the blinking is that it is based on *time* so it slows things down. Worse, it works inconsistently! Sometimes it spits out sensible ascii codes and sometimes it, well, spits out weird garbage. When you use it in person it works fine though. So we keep it on when not testing. Cleans up some redundancy in when testing CLI errors. Less copy and paste good. I was tempted to disable the xterm emulation entirely while working on this because upgrading jLine changed a few things and it was a real pain to update. But If we turned that off then we'd have *nothing* testing the colors and such. That'd be a shame because we use color in the output to commicate stuff. I like it so I don't want to break it. While I was there, I replaces the cli connector's `PrintWriter` with a `BufferedWriter`. The `PrintWriter` was kind of a trap because `println` would fail to work properly on windows because we force the terminal into xterm mode and it doesn't know what to do with windows line endings. Windows..... Additionally I fixed a race condition between disabling echo when reading passwords and fast writers. We were disabling the echo shortly after sending the prompt. A fast enough writer could send us text before the echo disable kicked in. Now I delegate to `LineReader#readLine` with a special echo mask that disables echo. This is both easier to test and doesn't seem to have the race condition. This race condition was failing the tests because they are so much faster now. Yay! Original commit: elastic/x-pack-elasticsearch@d0ec0273964630a3a113ab27009fdff6a182ecb2
2018-02-27 12:24:16 -05:00
compile project(path: xpackModule('sql:sql-cli'), configuration: 'nodeps')
compile "org.jline:jline:3.6.0"
}
/* disable unit tests because these are all integration tests used
* other qa projects. */
test.enabled = false
dependencyLicenses.enabled = false
Fix unknown licenses (#31223) The goal of this commit is to address unknown licenses when producing the dependencies info report. We have two different checks that we run on licenses. The first check is whether or not we have stashed a copy of the license text for a dependency in the repository. The second is to map every dependency to a license type (e.g., BSD 3-clause). The problem here is that the way we were handling licenses in the second check differs from how we handle licenses in the first check. The first check works by finding a license file with the name of the artifact followed by the text -LICENSE.txt. Yet in some cases we allow mapping an artifact name to another name used to check for the license (e.g., we map lucene-.* to lucene, and opensaml-.* to shibboleth. The second check understood the first way of looking for a license file but not the second way. So in this commit we teach the second check about the mappings from artifact names to license names. We do this by copying the configuration from the dependencyLicenses task to the dependenciesInfo task and then reusing the code from the first check in the second check. There were some other challenges here though. For example, dependenciesInfo was checking too many dependencies. For now, we should only be checking direct dependencies and leaving transitive dependencies from another org.elasticsearch artifact to that artifact (we want to do this differently in a follow-up). We also want to disable dependenciesInfo for projects that we do not publish, users only care about licenses they might be exposed to if they use our assembled products. With all of the changes in this commit we have eliminated all unknown licenses. A follow-up will enforce that when we add a new dependency it does not get mapped to unknown, these will be forbidden in the future. Therefore, with this change and earlier changes are left having no unknown licenses and two custom licenses; custom here means it does not map to an SPDX license type. Those two licenses are xz and ldapsdk. A future change will not allow additional custom licenses unless they are explicitly whitelisted. This ensures that if a new dependency is added it is mapped to an SPDX license or mapped to custom because it does not have an SPDX license.
2018-06-09 07:28:41 -04:00
dependenciesInfo.enabled = false
// the main files are actually test files, so use the appropriate forbidden api sigs
forbiddenApisMain {
replaceSignatureFiles 'es-all-signatures', 'es-test-signatures'
}
thirdPartyAudit.excludes = [
SQL: Replace the cli fixture with in-process testing (elastic/x-pack-elasticsearch#3889) I'm really really sad to be removing the cli-fixture but I've had trouble with it leaking recently it is pretty slow. Beyond that, we'd prefer that our test fixture only fixture things that are external depndencies. So, yeah, I'm removing it. So we get faster tests and no chance of leaking processes. We lose some "realness" in the tests. Instead of interacting with the CLI like a real user we embed it in the test process. That means we don't test the forking, we don't test the executable jar, and we don't test the jLine console detection stuff. On the other hand we were kind of forcing the jLine console detection stuff in a funky way with the fixture anyway. And we test the executable jar in the packaging tests. And that'll have to do. I haven't renamed `RemoteCli` because it'd bloat this commit with mechanical changes that'd make it hard to review. I'll rename it in a followup commit. This also updates jLine so we can disable blinking to matching parentheses during testing. I have no clue why, but this wasn't happening when we used the fixture. The trouble with the blinking is that it is based on *time* so it slows things down. Worse, it works inconsistently! Sometimes it spits out sensible ascii codes and sometimes it, well, spits out weird garbage. When you use it in person it works fine though. So we keep it on when not testing. Cleans up some redundancy in when testing CLI errors. Less copy and paste good. I was tempted to disable the xterm emulation entirely while working on this because upgrading jLine changed a few things and it was a real pain to update. But If we turned that off then we'd have *nothing* testing the colors and such. That'd be a shame because we use color in the output to commicate stuff. I like it so I don't want to break it. While I was there, I replaces the cli connector's `PrintWriter` with a `BufferedWriter`. The `PrintWriter` was kind of a trap because `println` would fail to work properly on windows because we force the terminal into xterm mode and it doesn't know what to do with windows line endings. Windows..... Additionally I fixed a race condition between disabling echo when reading passwords and fast writers. We were disabling the echo shortly after sending the prompt. A fast enough writer could send us text before the echo disable kicked in. Now I delegate to `LineReader#readLine` with a special echo mask that disables echo. This is both easier to test and doesn't seem to have the race condition. This race condition was failing the tests because they are so much faster now. Yay! Original commit: elastic/x-pack-elasticsearch@d0ec0273964630a3a113ab27009fdff6a182ecb2
2018-02-27 12:24:16 -05:00
// jLine's optional dependencies
'org.apache.sshd.client.SshClient',
'org.apache.sshd.client.auth.keyboard.UserInteraction',
'org.apache.sshd.client.channel.ChannelShell',
'org.apache.sshd.client.channel.ClientChannel',
'org.apache.sshd.client.channel.ClientChannelEvent',
'org.apache.sshd.client.future.AuthFuture',
'org.apache.sshd.client.future.ConnectFuture',
'org.apache.sshd.client.future.OpenFuture',
'org.apache.sshd.client.session.ClientSession',
'org.apache.sshd.common.Factory',
'org.apache.sshd.common.channel.PtyMode',
'org.apache.sshd.common.config.keys.FilePasswordProvider',
'org.apache.sshd.common.util.io.NoCloseInputStream',
'org.apache.sshd.common.util.io.NoCloseOutputStream',
'org.apache.sshd.server.Command',
'org.apache.sshd.server.Environment',
'org.apache.sshd.server.ExitCallback',
'org.apache.sshd.server.SessionAware',
'org.apache.sshd.server.Signal',
'org.apache.sshd.server.SshServer',
'org.apache.sshd.server.keyprovider.SimpleGeneratorHostKeyProvider',
'org.apache.sshd.server.scp.ScpCommandFactory$Builder',
'org.apache.sshd.server.session.ServerSession',
'org.apache.sshd.server.subsystem.sftp.SftpSubsystemFactory$Builder',
'org.fusesource.jansi.Ansi',
'org.fusesource.jansi.internal.CLibrary$Termios',
'org.fusesource.jansi.internal.CLibrary$WinSize',
'org.fusesource.jansi.internal.CLibrary',
'org.fusesource.jansi.internal.Kernel32$CHAR_INFO',
'org.fusesource.jansi.internal.Kernel32$CONSOLE_SCREEN_BUFFER_INFO',
'org.fusesource.jansi.internal.Kernel32$COORD',
'org.fusesource.jansi.internal.Kernel32$FOCUS_EVENT_RECORD',
'org.fusesource.jansi.internal.Kernel32$INPUT_RECORD',
'org.fusesource.jansi.internal.Kernel32$KEY_EVENT_RECORD',
'org.fusesource.jansi.internal.Kernel32$MOUSE_EVENT_RECORD',
'org.fusesource.jansi.internal.Kernel32$SMALL_RECT',
'org.fusesource.jansi.internal.Kernel32',
'org.fusesource.jansi.internal.WindowsSupport',
'org.mozilla.universalchardet.UniversalDetector',
]
subprojects {
apply plugin: 'elasticsearch.standalone-rest-test'
dependencies {
/* Since we're a standalone rest test we actually get transitive
* dependencies but we don't really want them because they cause
* all kinds of trouble with the jar hell checks. So we suppress
* them explicitly for non-es projects. */
testCompile(xpackProject('plugin:sql:qa')) {
transitive = false
}
testCompile "org.elasticsearch.test:framework:${version}"
// JDBC testing dependencies
testRuntime "net.sourceforge.csvjdbc:csvjdbc:1.0.34"
testRuntime "com.h2database:h2:1.4.197"
testRuntime project(path: xpackModule('sql:jdbc'), configuration: 'nodeps')
testRuntime xpackProject('plugin:sql:sql-client')
SQL: Replace the cli fixture with in-process testing (elastic/x-pack-elasticsearch#3889) I'm really really sad to be removing the cli-fixture but I've had trouble with it leaking recently it is pretty slow. Beyond that, we'd prefer that our test fixture only fixture things that are external depndencies. So, yeah, I'm removing it. So we get faster tests and no chance of leaking processes. We lose some "realness" in the tests. Instead of interacting with the CLI like a real user we embed it in the test process. That means we don't test the forking, we don't test the executable jar, and we don't test the jLine console detection stuff. On the other hand we were kind of forcing the jLine console detection stuff in a funky way with the fixture anyway. And we test the executable jar in the packaging tests. And that'll have to do. I haven't renamed `RemoteCli` because it'd bloat this commit with mechanical changes that'd make it hard to review. I'll rename it in a followup commit. This also updates jLine so we can disable blinking to matching parentheses during testing. I have no clue why, but this wasn't happening when we used the fixture. The trouble with the blinking is that it is based on *time* so it slows things down. Worse, it works inconsistently! Sometimes it spits out sensible ascii codes and sometimes it, well, spits out weird garbage. When you use it in person it works fine though. So we keep it on when not testing. Cleans up some redundancy in when testing CLI errors. Less copy and paste good. I was tempted to disable the xterm emulation entirely while working on this because upgrading jLine changed a few things and it was a real pain to update. But If we turned that off then we'd have *nothing* testing the colors and such. That'd be a shame because we use color in the output to commicate stuff. I like it so I don't want to break it. While I was there, I replaces the cli connector's `PrintWriter` with a `BufferedWriter`. The `PrintWriter` was kind of a trap because `println` would fail to work properly on windows because we force the terminal into xterm mode and it doesn't know what to do with windows line endings. Windows..... Additionally I fixed a race condition between disabling echo when reading passwords and fast writers. We were disabling the echo shortly after sending the prompt. A fast enough writer could send us text before the echo disable kicked in. Now I delegate to `LineReader#readLine` with a special echo mask that disables echo. This is both easier to test and doesn't seem to have the race condition. This race condition was failing the tests because they are so much faster now. Yay! Original commit: elastic/x-pack-elasticsearch@d0ec0273964630a3a113ab27009fdff6a182ecb2
2018-02-27 12:24:16 -05:00
// TODO check if needed
testRuntime("org.antlr:antlr4-runtime:4.5.3") {
transitive = false
}
// CLI testing dependencies
SQL: Replace the cli fixture with in-process testing (elastic/x-pack-elasticsearch#3889) I'm really really sad to be removing the cli-fixture but I've had trouble with it leaking recently it is pretty slow. Beyond that, we'd prefer that our test fixture only fixture things that are external depndencies. So, yeah, I'm removing it. So we get faster tests and no chance of leaking processes. We lose some "realness" in the tests. Instead of interacting with the CLI like a real user we embed it in the test process. That means we don't test the forking, we don't test the executable jar, and we don't test the jLine console detection stuff. On the other hand we were kind of forcing the jLine console detection stuff in a funky way with the fixture anyway. And we test the executable jar in the packaging tests. And that'll have to do. I haven't renamed `RemoteCli` because it'd bloat this commit with mechanical changes that'd make it hard to review. I'll rename it in a followup commit. This also updates jLine so we can disable blinking to matching parentheses during testing. I have no clue why, but this wasn't happening when we used the fixture. The trouble with the blinking is that it is based on *time* so it slows things down. Worse, it works inconsistently! Sometimes it spits out sensible ascii codes and sometimes it, well, spits out weird garbage. When you use it in person it works fine though. So we keep it on when not testing. Cleans up some redundancy in when testing CLI errors. Less copy and paste good. I was tempted to disable the xterm emulation entirely while working on this because upgrading jLine changed a few things and it was a real pain to update. But If we turned that off then we'd have *nothing* testing the colors and such. That'd be a shame because we use color in the output to commicate stuff. I like it so I don't want to break it. While I was there, I replaces the cli connector's `PrintWriter` with a `BufferedWriter`. The `PrintWriter` was kind of a trap because `println` would fail to work properly on windows because we force the terminal into xterm mode and it doesn't know what to do with windows line endings. Windows..... Additionally I fixed a race condition between disabling echo when reading passwords and fast writers. We were disabling the echo shortly after sending the prompt. A fast enough writer could send us text before the echo disable kicked in. Now I delegate to `LineReader#readLine` with a special echo mask that disables echo. This is both easier to test and doesn't seem to have the race condition. This race condition was failing the tests because they are so much faster now. Yay! Original commit: elastic/x-pack-elasticsearch@d0ec0273964630a3a113ab27009fdff6a182ecb2
2018-02-27 12:24:16 -05:00
testRuntime project(path: xpackModule('sql:sql-cli'), configuration: 'nodeps')
testRuntime (xpackProject('plugin:sql:sql-action')) {
transitive = false
}
SQL: Replace the cli fixture with in-process testing (elastic/x-pack-elasticsearch#3889) I'm really really sad to be removing the cli-fixture but I've had trouble with it leaking recently it is pretty slow. Beyond that, we'd prefer that our test fixture only fixture things that are external depndencies. So, yeah, I'm removing it. So we get faster tests and no chance of leaking processes. We lose some "realness" in the tests. Instead of interacting with the CLI like a real user we embed it in the test process. That means we don't test the forking, we don't test the executable jar, and we don't test the jLine console detection stuff. On the other hand we were kind of forcing the jLine console detection stuff in a funky way with the fixture anyway. And we test the executable jar in the packaging tests. And that'll have to do. I haven't renamed `RemoteCli` because it'd bloat this commit with mechanical changes that'd make it hard to review. I'll rename it in a followup commit. This also updates jLine so we can disable blinking to matching parentheses during testing. I have no clue why, but this wasn't happening when we used the fixture. The trouble with the blinking is that it is based on *time* so it slows things down. Worse, it works inconsistently! Sometimes it spits out sensible ascii codes and sometimes it, well, spits out weird garbage. When you use it in person it works fine though. So we keep it on when not testing. Cleans up some redundancy in when testing CLI errors. Less copy and paste good. I was tempted to disable the xterm emulation entirely while working on this because upgrading jLine changed a few things and it was a real pain to update. But If we turned that off then we'd have *nothing* testing the colors and such. That'd be a shame because we use color in the output to commicate stuff. I like it so I don't want to break it. While I was there, I replaces the cli connector's `PrintWriter` with a `BufferedWriter`. The `PrintWriter` was kind of a trap because `println` would fail to work properly on windows because we force the terminal into xterm mode and it doesn't know what to do with windows line endings. Windows..... Additionally I fixed a race condition between disabling echo when reading passwords and fast writers. We were disabling the echo shortly after sending the prompt. A fast enough writer could send us text before the echo disable kicked in. Now I delegate to `LineReader#readLine` with a special echo mask that disables echo. This is both easier to test and doesn't seem to have the race condition. This race condition was failing the tests because they are so much faster now. Yay! Original commit: elastic/x-pack-elasticsearch@d0ec0273964630a3a113ab27009fdff6a182ecb2
2018-02-27 12:24:16 -05:00
testRuntime "org.jline:jline:3.6.0"
}
if (project.name != 'security') {
// The security project just configures its subprojects
apply plugin: 'elasticsearch.rest-test'
integTestCluster {
setting 'xpack.monitoring.enabled', 'false'
setting 'xpack.ml.enabled', 'false'
setting 'xpack.watcher.enabled', 'false'
setting 'script.max_compilations_rate', '1000/1m'
}
task runqa(type: RunTask) {
setting 'xpack.monitoring.enabled', 'false'
setting 'xpack.ml.enabled', 'false'
setting 'xpack.watcher.enabled', 'false'
setting 'script.max_compilations_rate', '1000/1m'
}
}
Task assemble = project.tasks.findByName('assemble')
if (assemble) {
assemble.enabled = false
}
Task dependenciesInfo = project.tasks.findByName('dependenciesInfo')
if (dependenciesInfo) {
dependenciesInfo.enabled = false
}
}