Commit Graph

42 Commits

Author SHA1 Message Date
Nik Everett df802b40c8 SQL: Rework SSL testing (elastic/x-pack-elasticsearch#3126)
Drop the ssl tests against the java builtin https server. They were
failing and the failures were undebuggable. I still don't know what was
happening because you can't get any logging out of the server.

Add SSL tests against Elasticsearch because that is what actually needs
to work.

relates elastic/x-pack-elasticsearch#2870

Original commit: elastic/x-pack-elasticsearch@284cf7fb58
2017-11-27 18:52:16 -05:00
Igor Motov a4915a5714 SQL: remove all remaining NOCOMMITs
relates elastic/x-pack-elasticsearch#2873

Original commit: elastic/x-pack-elasticsearch@68b206efd2
2017-11-21 14:37:59 -05:00
Nik Everett 3d0f57d976 Add remaining security tests (elastic/x-pack-elasticsearch#2797)
This adds all of the security tests I think SQL is going to need for the initial release. SQL is still missing an entire scenario though: SSL enabled. Either way, this removes some `NOCOMMIT`s in `qa/sql/security`. Adding the SSL testing can come later.

Original commit: elastic/x-pack-elasticsearch@851620b606
2017-10-26 17:23:35 +00:00
Nik Everett 269b656cf2 Prune the forbiddenPatterns tasks that are skipped (elastic/x-pack-elasticsearch#2754)
We don't need to blindly skip all of these tasks. We don't have
NOCOMMITs everywhere, just lots of places....

Original commit: elastic/x-pack-elasticsearch@d772cb1b96
2017-10-13 16:21:33 +00:00
Nik Everett 3e6714e205 Sql: Fix ci script
Merge error caused us to use the wrong command in ci.

Original commit: elastic/x-pack-elasticsearch@09d0a66258
2017-09-20 14:38:03 -04:00
Nik Everett bac9afee7e Merge branch 'master' into feature/sql
Original commit: elastic/x-pack-elasticsearch@270ab71b19
2017-09-14 11:43:48 -04:00
Aaron Bull Schaefer 447f224677 Add releaseTest option to CI script (elastic/x-pack-elasticsearch#2482)
This option runs a normal check but with `-Dsnapshot=false` (the flag
used to indicate a release build).

Related to https://github.com/elastic/infra/issues/2759 and
https://github.com/elastic/infra/issues/2739 from the ES side.

Original commit: elastic/x-pack-elasticsearch@e674e68905
2017-09-13 16:01:21 -07:00
Igor Motov f3193aca5d Merge remote-tracking branch 'elastic/master' into sql-remove-no-commits
Original commit: elastic/x-pack-elasticsearch@f712c08cf1
2017-09-07 12:40:33 -04:00
Jason Tedor d9dce1afcd Fix typo in CI script
This commit fixes a typo in the CI script, the command-line flag for
setting tests.badapples had a typo.

Original commit: elastic/x-pack-elasticsearch@c67a620485
2017-09-03 18:38:40 -04:00
Jason Tedor bf68b5a907 Add intake option to CI script
This commit adds an option for running the intake build from the CI
scripts. The intake build differs from check in that it runs compilation
and pre-commit checks before running tests so that common build failures
causes will happen earlier than otherwise.

Relates elastic/x-pack-elasticsearch#2420

Original commit: elastic/x-pack-elasticsearch@b179ce1087
2017-09-03 16:43:15 -04:00
Nik Everett 97ddee8d1b Shuffle SQL's integration tests some (elastic/x-pack-elasticsearch#2403)
This shuffles all of SQL's QA tests into the `qa/sql` directory, moving
some shared resources into the new `qa:sql` project. It also rigs up
testing of the rest SQL interface in all the sql qa configurations:
without security, with security, and against multiple nodes.

I've had to make some modifications to how we handle the audit log
because it has gotten pretty slow. If these modifications turn out to
not be fast enough then I'll change the test to querying the log files
and drop the audit log index entirely but the index seems to be holding
out for now.

Original commit: elastic/x-pack-elasticsearch@ff3b5a74c1
2017-08-30 17:22:46 -04:00
Ryan Ernst 3ef93c55b0 Build: Add build metadata generation and usage to xpack (elastic/x-pack-elasticsearch#2368)
This commit adds generation of build_metadata files from bwc checkouts
as well as the sibling ES checkout of the xpack ci script.

relates elastic/elasticsearch#26397

Original commit: elastic/x-pack-elasticsearch@e17d904beb
2017-08-28 14:03:02 -07:00
Nik Everett 0605802d22 Add a multi-node test to sql (elastic/x-pack-elasticsearch#2136)
SQL relies on being able to fetch information about fields from
the cluster state and it'd be disasterous if that information
wasn't available. This should catch that.

Original commit: elastic/x-pack-elasticsearch@1a62747332
2017-08-02 14:39:25 -04:00
Nik Everett 767a43ca44 Move sql rest test into qa (elastic/x-pack-elasticsearch#2149)
Running the sql rest action test inside the server caused a dependency
loop which was failing the build.

Original commit: elastic/x-pack-elasticsearch@43283671d8
2017-08-01 17:23:07 -04:00
Nik Everett ec36875872 Sql security take 1 (elastic/x-pack-elasticsearch#2106)
Add some basic security testing/integration.

The good news:
1. Basic security now works. Users without access to an index can't run sql queries against it. Without this change they could.
2. Document level security works! At least so far as I can tell.

The work left to do:
1. Field level security doesn't work properly. I mean, it kind of works in that the field's values don't leak but it just looks like they all have null values.
2. We will need to test scrolling.
3. I've only added tests for the rest sql action. I'll need to add tests for jdbc and the CLI as well.
4. I've only added tests for `SELECT` and have ignored stuff like `DESCRIBE` and `SHOW TABLES`.

Original commit: elastic/x-pack-elasticsearch@b9909bbda0
2017-07-27 17:40:14 -04:00
Nik Everett 23c76e31e5 Add a smoke test sql option to the ci script (elastic/x-pack-elasticsearch#2101)
This should allow creation of a ci build to smoke test sql which
should be faster but less complete than the normal build on the
sql branch.

Original commit: elastic/x-pack-elasticsearch@87748b37f9
2017-07-27 09:59:57 -04:00
Nik Everett e15c9e66b0 Checkout *elastic*'s master branch
Needed because I hacked around branch selection because we don't have
a `feature/sql` branch in the public elasticsearch repo.

Original commit: elastic/x-pack-elasticsearch@3fc1084588
2017-07-25 13:48:58 -04:00
Nik Everett 71beda43ff Use master branch in CI
Original commit: elastic/x-pack-elasticsearch@bbdf8b5a40
2017-07-25 13:43:33 -04:00
Nik Everett 823149e456 Skip forbiddenPatterns in CI for now
Original commit: elastic/x-pack-elasticsearch@3788e155ab
2017-07-25 09:50:45 -04:00
Ryan Ernst 404ba010ad Test: Fix ci script to use pr author's fork when looking for source branch (elastic/x-pack-elasticsearch#1548)
Original commit: elastic/x-pack-elasticsearch@faf72043be
2017-05-25 12:03:40 -07:00
Ryan Ernst 5a3c85cd57 Build: Add bwcTest target to ci script (elastic/x-pack-elasticsearch#1488)
This is to allow bwcTest to be run by ci. This commit also fixes the
existing packagingTest target, which was never updated after the xpack
repo split.

Original commit: elastic/x-pack-elasticsearch@f4696c1899
2017-05-22 08:53:36 -07:00
Ryan Ernst 0255c45f1f Build: Use a deep clone of ES for ci (elastic/x-pack-elasticsearch#840)
WIth the addition of building the backcompat version in elasticsearch, a
shallow clone no longer works, as we need to have eg 5.x branch
available. This change removes the depth argument from cloning
elasticsearch.

Original commit: elastic/x-pack-elasticsearch@3d89fe92ec
2017-03-24 11:50:02 -07:00
Ryan Ernst 89f996eed2 Adding new build files after split
Original commit: elastic/x-pack-elasticsearch@ddd5a5bfa6
2017-02-10 11:02:42 -08:00
Tim Vernum 8820902f0c [ci] Print git revision of elasticsearch to ci log (elastic/elasticsearch#4782)
The output `Checked out Elasticsearch revision: ...` shows which revision of elasticsearch was included in the CI build. This makes it easier to diagnose and resolve x-pack build failures.

Original commit: elastic/x-pack-elasticsearch@8c39aafc2d
2017-02-07 10:36:49 +11:00
Dimitrios Liappis 17c3164979 [ci] Fix bug with -Dtests.jvm.argline
The special `-Dtests.jvm.argline` params needed for jdk-9 builds do not
get passed correctly if enclosed within single quotes.

Fix jdk9 target for `dev-tools/ci` script to correctly pass the
-Dtests.jvm.argline parameters.

Relates:  elastic/elasticsearch#4428 

Original commit: elastic/x-pack-elasticsearch@6cd329b8da
2016-12-20 16:35:18 +02:00
Dimitrios Liappis 8a3181ceed [ci] Add jdk9 target to the dev-tools/ci script
To add support for jdk9 tests for x-pack we need a number of parameters
for the jvm, passed in the gradle command. To resolve issues with arg
passing through bash variables and improve readability, switch to an
array construct for storing the arguments.

Add new jdk9 target to allow execution of jdk-9-ea based tests in the CI
infra.

Relates: elastic/elasticsearch#4415 
Relates: https://github.com/elastic/infra/issues/1573#issuecomment-267832307

Original commit: elastic/x-pack-elasticsearch@b41e5f4b98
2016-12-19 19:53:00 +02:00
Ryan Ernst 6d4d599f91 Build: Convert xplugins to use new extra projects setup (elastic/elasticsearch#4175)
* Build: Convert xplugins to use new extra projects setup

This change makes the gradle initialization for xplugins look in the
correct location for elasticsearch, which is now as a sibling of an
elasticsearch-extra directory, with x-plugins as a child of the extra
directory.

The elasticsearch side of this change is
elastic/elasticsearchelastic/elasticsearch#21773. This change will enable renaming x-plugins
to x-pack, see elastic/elasticsearch#3643.

Original commit: elastic/x-pack-elasticsearch@09398aea5a
2016-12-14 15:02:28 -08:00
Boaz Leskes 7a996da86d ci: don't use ssh to check elasticsearch repo for a sibling branch
ssh keys are not always available

Original commit: elastic/x-pack-elasticsearch@5c974724ea
2016-12-06 11:37:50 +01:00
Boaz Leskes 7b044a6e16 ci: remove unneeded AWAITSFIX comment
Original commit: elastic/x-pack-elasticsearch@726d26fcdb
2016-12-06 10:00:22 +01:00
Boaz Leskes 11ae30cf90 ci: properly accommodate for repo name to be part of `GIT_BRANCH`
previous commit missed a `*`

Original commit: elastic/x-pack-elasticsearch@33b9f32c62
2016-12-06 09:52:03 +01:00
Boaz Leskes 373c350920 ci: accommodate for repo name to be part of `GIT_BRANCH`
It seems that that variable is set by the git jenkins plugin and it's format is `origin/master`. See https://wiki.jenkins-ci.org/display/JENKINS/Git+Plugin#GitPlugin-Environmentvariables

Original commit: elastic/x-pack-elasticsearch@17f8ec1a9e
2016-12-06 09:43:28 +01:00
Ryan Ernst 7483abb226 Add fallback branch until CI configuration is fixed to pass GIT_BRANCH
Original commit: elastic/x-pack-elasticsearch@68d12b33e9
2016-12-05 16:34:08 -08:00
Ryan Ernst 75a72b34cb Build: Checkout relevant branch of elasticsearch for ci (elastic/elasticsearch#4270)
The ci script checks out elasticsearch for x-plugins to use. However, it
always checks out the master branch. This change makes the script look
for the source branch of a PR (falling back to GIT_BRANCH which is provided by jenkins for non PR CI jobs) and checks if that branch exists in elasticsearch. If it does not, it falls back to the target branch for the PR.

Original commit: elastic/x-pack-elasticsearch@73146bb3b3
2016-12-05 16:10:33 -08:00
Dimitrios Liappis 9294827a91 Switch to /bin/bash
Since there is no hard requirement for a strict POSIX shell
implementation, we can switch to /bin/bash.

Clean up left over break statements and consistently use $() for command
substitutions.

Original commit: elastic/x-pack-elasticsearch@0a33dfed56
2016-11-29 14:44:56 +02:00
Dimitrios Liappis 589e880a9c Make cli arg parsing compliant with requirements.
Remove duplicate definition of GRADLE_OPT_TEST and rename arguments as
discussed in
https://github.com/elastic/x-plugins/pull/4137#discussion_r89660620

Default to `check` if no argument is passed, but exit with error in an
unsupported parameter is provided.

Also ensure the pipefail option is only set when the executing shell
is *really* bash as opposed to POSIX sh which doesn't support it.

Original commit: elastic/x-pack-elasticsearch@64bd740295
2016-11-28 13:19:48 +02:00
Dimitrios Liappis 1fea75cc4f Refactor ci script to choose test based on single cli arg
We prefer just passing a single cli argument that allows the script to
choose the corresponding gradle command from a preset list of
choices. For now, valid options are `packagingtests` or `tests`. The
script defaults to `tests` if no argument is supplied.

Original commit: elastic/x-pack-elasticsearch@f6315b1ad5
2016-11-23 13:44:15 +02:00
Dimitrios Liappis 46055d0430 Ensure ci script fails properly with invalid parameters
Prevent the ci script from going to infinite loop if the user provides a
non POSIX style command line argument (not prefixed with - or --)

Original commit: elastic/x-pack-elasticsearch@e0eb0e6df7
2016-11-21 17:18:48 +02:00
Dimitrios Liappis d19c13d2b5 Allow defining gradle test params via cli argument
For Elasticsearch x-plugin packaging tests we need a different gradle
command.

This commit introduces an optional cli argument (-g|--gradleoptstring)
that can be used to set the gradle parameters to execute tests.

Relates https://github.com/elastic/infra/issues/1518

Original commit: elastic/x-pack-elasticsearch@3a32e323ae
2016-11-21 16:36:55 +02:00
Dimitrios Liappis 28f1bc5d28 Honor gradle options for no Kibana builds+tests (elastic/elasticsearch#3448)
https://github.com/elastic/x-plugins/pull/3229 broke the possibility of
building+testing x-pack without Kibana via dev-tools/ci by setting an
option in ~/.gradle/gradle.properties as described in
https://github.com/dliappis/x-plugins/blob/master/README.asciidoc,
unless nvm is present.

Conditionally execute the nvm/nodejs code only if
`xpack.kibana.build=false` is not set in `~/.gradle/gradle.properties`

Original commit: elastic/x-pack-elasticsearch@affb4ac711
2016-09-13 08:34:22 -04:00
Court Ewing 877d5feff7 ci: install the correct node.js version via nvm (elastic/elasticsearch#3229)
This will ensure that CI runs the Kibana plugin tests with the
appropriate version of node.js.

Original commit: elastic/x-pack-elasticsearch@aeb7399f91
2016-09-12 14:32:12 -04:00
jaymode 9e2635631c test: update the ci script
This updates the CI script to be POSIX shell compatible so that it can run under
both dash and bash. Also, no arguments should be needed; this should run what
is run in CI.

Original commit: elastic/x-pack-elasticsearch@4cd87e48a6
2016-04-07 08:34:58 -04:00
Lee Hinman da3d7177be Add Shield HTTP APIs for users and roles
Relates to elastic/elasticsearch#33

Original commit: elastic/x-pack-elasticsearch@a0942c9334
2016-01-18 16:21:22 -07:00