[test] Gradle-ify vagrant tests
This gets the tar and tar_plugins tests working in gradle. It does so by
adding a subproject, qa/vagrant, which adds the following tasks:
Verification
------------
checkPackages - Check the packages against a representative sample of the
linux distributions we have in our Vagrantfile
checkPackagesAllDistros - Check the packages against all the linux
distributions we have in our Vagrantfile
Package Verification
--------------------
checkCentos6 - Run packaging tests against centos-6
checkCentos7 - Run packaging tests against centos-7
checkDebian8 - Run packaging tests against debian-8
checkFedora22 - Run packaging tests against fedora-22
checkOel7 - Run packaging tests against oel-7
checkOpensuse13 - Run packaging tests against opensuse-13
checkSles12 - Run packaging tests against sles-12
checkUbuntu1204 - Run packaging tests against ubuntu-1204
checkUbuntu1404 - Run packaging tests against ubuntu-1404
checkUbuntu1504 - Run packaging tests against ubuntu-1504
Vagrant
-------
smokeTestCentos6 - Smoke test the centos-6 VM
smokeTestCentos7 - Smoke test the centos-7 VM
smokeTestDebian8 - Smoke test the debian-8 VM
smokeTestFedora22 - Smoke test the fedora-22 VM
smokeTestOel7 - Smoke test the oel-7 VM
smokeTestOpensuse13 - Smoke test the opensuse-13 VM
smokeTestSles12 - Smoke test the sles-12 VM
smokeTestUbuntu1204 - Smoke test the ubuntu-1204 VM
smokeTestUbuntu1404 - Smoke test the ubuntu-1404 VM
smokeTestUbuntu1504 - Smoke test the ubuntu-1504 VM
vagrantHaltCentos6 - Shutdown the vagrant VM running centos-6
vagrantHaltCentos7 - Shutdown the vagrant VM running centos-7
vagrantHaltDebian8 - Shutdown the vagrant VM running debian-8
vagrantHaltFedora22 - Shutdown the vagrant VM running fedora-22
vagrantHaltOel7 - Shutdown the vagrant VM running oel-7
vagrantHaltOpensuse13 - Shutdown the vagrant VM running opensuse-13
vagrantHaltSles12 - Shutdown the vagrant VM running sles-12
vagrantHaltUbuntu1204 - Shutdown the vagrant VM running ubuntu-1204
vagrantHaltUbuntu1404 - Shutdown the vagrant VM running ubuntu-1404
vagrantHaltUbuntu1504 - Shutdown the vagrant VM running ubuntu-1504
vagrantSmokeTest - Smoke test some representative distros from the Vagrantfile
vagrantSmokeTestAllDistros - Smoke test all distros from the Vagrantfile
vagrantUpCentos6 - Startup a vagrant VM running centos-6
vagrantUpCentos7 - Startup a vagrant VM running centos-7
vagrantUpDebian8 - Startup a vagrant VM running debian-8
vagrantUpFedora22 - Startup a vagrant VM running fedora-22
vagrantUpOel7 - Startup a vagrant VM running oel-7
vagrantUpOpensuse13 - Startup a vagrant VM running opensuse-13
vagrantUpSles12 - Startup a vagrant VM running sles-12
vagrantUpUbuntu1204 - Startup a vagrant VM running ubuntu-1204
vagrantUpUbuntu1404 - Startup a vagrant VM running ubuntu-1404
vagrantUpUbuntu1504 - Startup a vagrant VM running ubuntu-1504
It does not make the "check" task depend on "checkPackages" so running the
vagrant tests is still optional. They are slow and depend on vagrant and
virtualbox.
The Package Verification tasks are useful for testing individual distros.
The Vagrant tasks are listed in `gradle tasks` primarily for discoverability.
This commit is contained in:
Nik Everett
parent
efde8e8a26
commit
1dd00dddd5
155
TESTING.asciidoc
155
TESTING.asciidoc
|
|
@ -326,152 +326,63 @@ vagrant plugin install vagrant-cachier
|
|||
. Validate your installed dependencies:
|
||||
|
||||
-------------------------------------
|
||||
gradle :qa:vagrant:validate
|
||||
gradle :qa:vagrant:checkVagrantVersion
|
||||
-------------------------------------
|
||||
|
||||
. Download the VMs. Since Maven or ant or something eats the progress reports
|
||||
from Vagrant when you run it inside mvn its probably best if you run this one
|
||||
time to setup all the VMs one at a time. Run this to download and setup the VMs
|
||||
we use for testing by default:
|
||||
. Download and smoke test the VMs with `gradle vagrantSmokeTest` or
|
||||
`gradle vagrantSmokeTestAllDistros`. The first time you run this it will
|
||||
download the base images and provision the boxes and immediately quit. If you
|
||||
you this again it'll skip the download step.
|
||||
|
||||
--------------------------------------------------------
|
||||
vagrant up --provision trusty --provider virtualbox && vagrant halt trusty
|
||||
vagrant up --provision centos-7 --provider virtualbox && vagrant halt centos-7
|
||||
--------------------------------------------------------
|
||||
. Run the tests with `gradle checkPackages`. This will cause gradle to build
|
||||
the tar, zip, and deb packages and all the plugins. It will then run the tests
|
||||
on ubuntu-1404 and centos-7. We chose those two distributions as the default
|
||||
because they cover deb and rpm packaging and SyvVinit and systemd.
|
||||
|
||||
or run this to download and setup all the VMs:
|
||||
You can run on all the VMs by running `gradle checkPackagesAllDistros`. You can
|
||||
run a particular VM with a command like `gradle checkOel7`. See `gradle tasks`
|
||||
for a list. Its important to know that if you ctrl-c any of these `gradle`
|
||||
commands then the boxes will remain running and you'll have to terminate them
|
||||
with `vagrant halt`.
|
||||
|
||||
-------------------------------------------------------------------------------
|
||||
vagrant halt
|
||||
for box in $(vagrant status | grep 'poweroff\|not created' | cut -f1 -d' '); do
|
||||
vagrant up --provision $box --provider virtualbox
|
||||
vagrant halt $box
|
||||
done
|
||||
-------------------------------------------------------------------------------
|
||||
|
||||
. Smoke test the maven/ant dance that we use to get vagrant involved in
|
||||
integration testing is working:
|
||||
|
||||
---------------------------------------------
|
||||
mvn -Dtests.vagrant -Psmoke-vms -pl qa/vagrant verify
|
||||
---------------------------------------------
|
||||
|
||||
or this to validate all the VMs:
|
||||
|
||||
-------------------------------------------------
|
||||
mvn -Dtests.vagrant=all -Psmoke-vms -pl qa/vagrant verify
|
||||
-------------------------------------------------
|
||||
|
||||
That will start up the VMs and then immediate quit.
|
||||
|
||||
. Finally run the tests. The fastest way to get this started is to run:
|
||||
|
||||
-----------------------------------
|
||||
mvn clean install -DskipTests
|
||||
mvn -Dtests.vagrant -pl qa/vagrant verify
|
||||
-----------------------------------
|
||||
|
||||
You could just run:
|
||||
|
||||
--------------------
|
||||
mvn -Dtests.vagrant verify
|
||||
--------------------
|
||||
|
||||
but that will run all the tests. Which is probably a good thing, but not always
|
||||
what you want.
|
||||
|
||||
Whichever snippet you run mvn will build the tar, zip and deb packages. If you
|
||||
have rpmbuild installed it'll build the rpm package as well. Then mvn will
|
||||
spin up trusty and verify the tar, zip, and deb package. If you have rpmbuild
|
||||
installed it'll spin up centos-7 and verify the tar, zip and rpm packages. We
|
||||
chose those two distributions as the default because they cover deb and rpm
|
||||
packaging and SyvVinit and systemd.
|
||||
|
||||
You can control the boxes that are used for testing like so. Run just
|
||||
fedora-22 with:
|
||||
|
||||
--------------------------------------------
|
||||
mvn -Dtests.vagrant -pl qa/vagrant verify -DboxesToTest=fedora-22
|
||||
--------------------------------------------
|
||||
|
||||
or run jessie and trusty:
|
||||
|
||||
------------------------------------------------------------------
|
||||
mvn -Dtests.vagrant -pl qa/vagrant verify -DboxesToTest='jessie, trusty'
|
||||
------------------------------------------------------------------
|
||||
|
||||
or run all the boxes:
|
||||
|
||||
---------------------------------------
|
||||
mvn -Dtests.vagrant=all -pl qa/vagrant verify
|
||||
---------------------------------------
|
||||
|
||||
If you want to run a specific test on several boxes you can do:
|
||||
|
||||
---------------------------------------
|
||||
mvn -Dtests.vagrant=all -pl qa/vagrant verify -DtestScripts=*tar*.bats
|
||||
---------------------------------------
|
||||
|
||||
Its important to know that if you ctrl-c any of these `mvn` runs that you'll
|
||||
probably leave a VM up. You can terminate it by running:
|
||||
|
||||
------------
|
||||
vagrant halt
|
||||
------------
|
||||
|
||||
This is just regular vagrant so you can run normal multi box vagrant commands
|
||||
to test things manually. Just run:
|
||||
|
||||
---------------------------------------
|
||||
vagrant up trusty --provider virtualbox && vagrant ssh trusty
|
||||
---------------------------------------
|
||||
|
||||
to get an Ubuntu or
|
||||
|
||||
-------------------------------------------
|
||||
vagrant up centos-7 --provider virtualbox && vagrant ssh centos-7
|
||||
-------------------------------------------
|
||||
|
||||
to get a CentOS. Once you are done with them you should halt them:
|
||||
|
||||
-------------------
|
||||
vagrant halt trusty
|
||||
-------------------
|
||||
All the regular vagrant commands should just work so you can get a shell in a
|
||||
VM running trusty by running
|
||||
`vagrant up ubuntu-1404 --provider virtualbox && vagrant ssh ubuntu-1404`.
|
||||
|
||||
These are the linux flavors the Vagrantfile currently supports:
|
||||
|
||||
* precise aka Ubuntu 12.04
|
||||
* trusty aka Ubuntu 14.04
|
||||
* vivid aka Ubuntun 15.04
|
||||
* jessie aka Debian 8, the current debina stable distribution
|
||||
* ubuntu-1204 aka precise
|
||||
* ubuntu-1404 aka trusty
|
||||
* ubuntu-1504 aka vivid
|
||||
* debian-8 aka jessie, the current debian stable distribution
|
||||
* centos-6
|
||||
* centos-7
|
||||
* fedora-22
|
||||
* oel-7 aka Oracle Enterprise Linux 7
|
||||
* sles-12
|
||||
* opensuse-13
|
||||
|
||||
We're missing the following from the support matrix because there aren't high
|
||||
quality boxes available in vagrant atlas:
|
||||
|
||||
* sles-11
|
||||
* opensuse-13
|
||||
* oel-6
|
||||
|
||||
We're missing the follow because our tests are very linux/bash centric:
|
||||
|
||||
* Windows Server 2012
|
||||
|
||||
Its important to think of VMs like cattle: if they become lame you just shoot
|
||||
Its important to think of VMs like cattle. If they become lame you just shoot
|
||||
them and let vagrant reprovision them. Say you've hosed your precise VM:
|
||||
|
||||
----------------------------------------------------
|
||||
vagrant ssh precise -c 'sudo rm -rf /bin'; echo oops
|
||||
vagrant ssh ubuntu-1404 -c 'sudo rm -rf /bin'; echo oops
|
||||
----------------------------------------------------
|
||||
|
||||
All you've got to do to get another one is
|
||||
|
||||
----------------------------------------------
|
||||
vagrant destroy -f trusty && vagrant up trusty --provider virtualbox
|
||||
vagrant destroy -f ubuntu-1404 && vagrant up ubuntu-1404 --provider virtualbox
|
||||
----------------------------------------------
|
||||
|
||||
The whole process takes a minute and a half on a modern laptop, two and a half
|
||||
|
|
@ -489,13 +400,8 @@ vagrant halt
|
|||
vagrant destroy -f
|
||||
------------------
|
||||
|
||||
|
||||
----------
|
||||
vagrant up
|
||||
----------
|
||||
|
||||
would normally start all the VMs but we've prevented that because that'd
|
||||
consume a ton of ram.
|
||||
`vagrant up` would normally start all the VMs but we've prevented that because
|
||||
that'd consume a ton of ram.
|
||||
|
||||
== Testing scripts more directly
|
||||
|
||||
|
|
@ -504,7 +410,7 @@ destructive. When working with a single package its generally faster to run its
|
|||
tests in a tighter loop than maven provides. In one window:
|
||||
|
||||
--------------------------------
|
||||
mvn -pl distribution/rpm package
|
||||
gradle :distribution:rpm:assemble
|
||||
--------------------------------
|
||||
|
||||
and in another window:
|
||||
|
|
@ -518,10 +424,7 @@ sudo bats $BATS/*rpm*.bats
|
|||
If you wanted to retest all the release artifacts on a single VM you could:
|
||||
|
||||
-------------------------------------------------
|
||||
# Build all the distributions fresh but skip recompiling elasticsearch:
|
||||
mvn -amd -pl distribution install -DskipTests
|
||||
# Copy them all the testroot
|
||||
mvn -Dtests.vagrant -pl qa/vagrant pre-integration-test
|
||||
gradle copyDepsToTestRoot
|
||||
vagrant up trusty --provider virtualbox && vagrant ssh trusty
|
||||
cd $TESTROOT
|
||||
sudo bats $BATS/*.bats
|
||||
|
|
|
|||
|
|
@ -22,15 +22,15 @@
|
|||
# under the License.
|
||||
|
||||
Vagrant.configure(2) do |config|
|
||||
config.vm.define "precise" do |config|
|
||||
config.vm.define "ubuntu-1204" do |config|
|
||||
config.vm.box = "ubuntu/precise64"
|
||||
ubuntu_common config
|
||||
end
|
||||
config.vm.define "trusty" do |config|
|
||||
config.vm.define "ubuntu-1404" do |config|
|
||||
config.vm.box = "ubuntu/trusty64"
|
||||
ubuntu_common config
|
||||
end
|
||||
config.vm.define "vivid" do |config|
|
||||
config.vm.define "ubuntu-1504" do |config|
|
||||
config.vm.box = "ubuntu/vivid64"
|
||||
ubuntu_common config, extra: <<-SHELL
|
||||
# Install Jayatana so we can work around it being present.
|
||||
|
|
@ -40,7 +40,7 @@ Vagrant.configure(2) do |config|
|
|||
# Wheezy's backports don't contain Openjdk 8 and the backflips required to
|
||||
# get the sun jdk on there just aren't worth it. We have jessie for testing
|
||||
# debian and it works fine.
|
||||
config.vm.define "jessie" do |config|
|
||||
config.vm.define "debian-8" do |config|
|
||||
config.vm.box = "debian/jessie64"
|
||||
deb_common config,
|
||||
'echo deb http://http.debian.net/debian jessie-backports main > /etc/apt/sources.list.d/backports.list', 'backports'
|
||||
|
|
@ -137,7 +137,7 @@ def deb_common(config, add_openjdk_repository_command, openjdk_list, extra: '')
|
|||
extra: <<-SHELL
|
||||
export DEBIAN_FRONTEND=noninteractive
|
||||
ls /etc/apt/sources.list.d/#{openjdk_list}.list > /dev/null 2>&1 ||
|
||||
(echo "Importing java-8 ppa" &&
|
||||
(echo "==> Importing java-8 ppa" &&
|
||||
#{add_openjdk_repository_command} &&
|
||||
apt-get update)
|
||||
#{extra}
|
||||
|
|
@ -223,9 +223,11 @@ def provision(config,
|
|||
install() {
|
||||
# Only apt-get update if we haven't in the last day
|
||||
if [ ! -f #{update_tracking_file} ] || [ "x$(find #{update_tracking_file} -mtime +0)" == "x#{update_tracking_file}" ]; then
|
||||
#{update_command} || true
|
||||
touch #{update_tracking_file}
|
||||
echo "==> Updating repository"
|
||||
#{update_command} || true
|
||||
touch #{update_tracking_file}
|
||||
fi
|
||||
echo "==> Installing $1"
|
||||
#{install_command} $1
|
||||
}
|
||||
ensure() {
|
||||
|
|
@ -242,17 +244,18 @@ def provision(config,
|
|||
installed bats || {
|
||||
# Bats lives in a git repository....
|
||||
ensure git
|
||||
echo "==> Installing bats"
|
||||
git clone https://github.com/sstephenson/bats /tmp/bats
|
||||
# Centos doesn't add /usr/local/bin to the path....
|
||||
/tmp/bats/install.sh /usr
|
||||
rm -rf /tmp/bats
|
||||
}
|
||||
cat \<\<VARS > /etc/profile.d/elasticsearch_vars.sh
|
||||
export ZIP=/elasticsearch/distribution/zip/target/releases
|
||||
export TAR=/elasticsearch/distribution/tar/target/releases
|
||||
export RPM=/elasticsearch/distribution/rpm/target/releases
|
||||
export DEB=/elasticsearch/distribution/deb/target/releases
|
||||
export TESTROOT=/elasticsearch/qa/vagrant/target/testroot
|
||||
export ZIP=/elasticsearch/distribution/zip/build/releases
|
||||
export TAR=/elasticsearch/distribution/tar/build/releases
|
||||
export RPM=/elasticsearch/distribution/rpm/build/releases
|
||||
export DEB=/elasticsearch/distribution/deb/build/releases
|
||||
export TESTROOT=/elasticsearch/qa/vagrant/build/testroot
|
||||
export BATS=/elasticsearch/qa/vagrant/src/test/resources/packaging/scripts
|
||||
VARS
|
||||
SHELL
|
||||
|
|
|
|||
|
|
@ -63,4 +63,3 @@ eclipse {
|
|||
defaultOutputDir = new File(file('build'), 'eclipse')
|
||||
}
|
||||
}
|
||||
|
||||
|
|
|
|||
|
|
@ -0,0 +1,71 @@
|
|||
/*
|
||||
* Licensed to Elasticsearch under one or more contributor
|
||||
* license agreements. See the NOTICE file distributed with
|
||||
* this work for additional information regarding copyright
|
||||
* ownership. Elasticsearch licenses this file to you under
|
||||
* the Apache License, Version 2.0 (the "License"); you may
|
||||
* not use this file except in compliance with the License.
|
||||
* You may obtain a copy of the License at
|
||||
*
|
||||
* http://www.apache.org/licenses/LICENSE-2.0
|
||||
*
|
||||
* Unless required by applicable law or agreed to in writing,
|
||||
* software distributed under the License is distributed on an
|
||||
* "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
|
||||
* KIND, either express or implied. See the License for the
|
||||
* specific language governing permissions and limitations
|
||||
* under the License.
|
||||
*/
|
||||
package org.elasticsearch.gradle.vagrant
|
||||
|
||||
import org.gradle.api.DefaultTask
|
||||
import org.gradle.api.tasks.*
|
||||
import org.gradle.logging.ProgressLogger
|
||||
import org.gradle.logging.ProgressLoggerFactory
|
||||
import org.gradle.process.internal.ExecAction
|
||||
import org.gradle.process.internal.ExecActionFactory
|
||||
|
||||
import javax.inject.Inject
|
||||
|
||||
/**
|
||||
* Runs bats over vagrant. Pretty much like running it using Exec but with a
|
||||
* nicer output formatter.
|
||||
*/
|
||||
class BatsOverVagrantTask extends DefaultTask {
|
||||
String command
|
||||
String boxName
|
||||
ExecAction execAction
|
||||
|
||||
BatsOverVagrantTask() {
|
||||
execAction = getExecActionFactory().newExecAction()
|
||||
}
|
||||
|
||||
@Inject
|
||||
ProgressLoggerFactory getProgressLoggerFactory() {
|
||||
throw new UnsupportedOperationException();
|
||||
}
|
||||
|
||||
@Inject
|
||||
ExecActionFactory getExecActionFactory() {
|
||||
throw new UnsupportedOperationException();
|
||||
}
|
||||
|
||||
void boxName(String boxName) {
|
||||
this.boxName = boxName
|
||||
}
|
||||
|
||||
void command(String command) {
|
||||
this.command = command
|
||||
}
|
||||
|
||||
@TaskAction
|
||||
void exec() {
|
||||
// It'd be nice if --machine-readable were, well, nice
|
||||
execAction.commandLine(['vagrant', 'ssh', boxName, '--command', command])
|
||||
execAction.setStandardOutput(new TapLoggerOutputStream(
|
||||
command: command,
|
||||
factory: getProgressLoggerFactory(),
|
||||
logger: logger))
|
||||
execAction.execute();
|
||||
}
|
||||
}
|
||||
|
|
@ -0,0 +1,108 @@
|
|||
/*
|
||||
* Licensed to Elasticsearch under one or more contributor
|
||||
* license agreements. See the NOTICE file distributed with
|
||||
* this work for additional information regarding copyright
|
||||
* ownership. Elasticsearch licenses this file to you under
|
||||
* the Apache License, Version 2.0 (the "License"); you may
|
||||
* not use this file except in compliance with the License.
|
||||
* You may obtain a copy of the License at
|
||||
*
|
||||
* http://www.apache.org/licenses/LICENSE-2.0
|
||||
*
|
||||
* Unless required by applicable law or agreed to in writing,
|
||||
* software distributed under the License is distributed on an
|
||||
* "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
|
||||
* KIND, either express or implied. See the License for the
|
||||
* specific language governing permissions and limitations
|
||||
* under the License.
|
||||
*/
|
||||
package org.elasticsearch.gradle.vagrant
|
||||
|
||||
import com.carrotsearch.gradle.junit4.LoggingOutputStream
|
||||
import org.gradle.api.GradleScriptException
|
||||
import org.gradle.api.InvalidUserDataException
|
||||
import org.gradle.api.logging.Logger
|
||||
import org.gradle.logging.ProgressLogger
|
||||
import org.gradle.logging.ProgressLoggerFactory
|
||||
import java.util.regex.Matcher
|
||||
|
||||
/**
|
||||
* Adapts an OutputStream containing output from bats into a ProgressLogger
|
||||
* and a Logger. Every test output goes to the ProgressLogger and all failures
|
||||
* and non-test output goes to the Logger. That means you can always glance
|
||||
* at the result of the last test and the cumulative pass/fail/skip stats and
|
||||
* the failures are all logged.
|
||||
*
|
||||
* There is a Tap4j project but we can't use it because it wants to parse the
|
||||
* entire TAP stream at once and won't parse it stream-wise.
|
||||
*/
|
||||
class TapLoggerOutputStream extends LoggingOutputStream {
|
||||
ProgressLogger progressLogger
|
||||
Logger logger
|
||||
int testsCompleted = 0
|
||||
int testsFailed = 0
|
||||
int testsSkipped = 0
|
||||
Integer testCount
|
||||
String countsFormat
|
||||
|
||||
TapLoggerOutputStream(Map args) {
|
||||
logger = args.logger
|
||||
progressLogger = args.factory.newOperation(VagrantLoggerOutputStream)
|
||||
progressLogger.setDescription("TAP output for $args.command")
|
||||
progressLogger.started()
|
||||
progressLogger.progress("Starting $args.command...")
|
||||
}
|
||||
|
||||
void flush() {
|
||||
if (end == start) return
|
||||
line(new String(buffer, start, end - start))
|
||||
start = end
|
||||
}
|
||||
|
||||
void line(String line) {
|
||||
// System.out.print "===> $line\n"
|
||||
if (testCount == null) {
|
||||
try {
|
||||
testCount = line.split('\\.').last().toInteger()
|
||||
def length = (testCount as String).length()
|
||||
countsFormat = "%0${length}d"
|
||||
countsFormat = "[$countsFormat|$countsFormat|$countsFormat/$countsFormat]"
|
||||
return
|
||||
} catch (Exception e) {
|
||||
throw new GradleScriptException(
|
||||
'Error parsing first line of TAP stream!!', e)
|
||||
}
|
||||
}
|
||||
Matcher m = line =~ /(?<status>ok|not ok) \d+(?<skip> # skip (?<skipReason>\(.+\))?)? \[(?<suite>.+)\] (?<test>.+)/
|
||||
if (!m.matches()) {
|
||||
/* These might be failure report lines or comments or whatever. Its hard
|
||||
to tell and it doesn't matter. */
|
||||
logger.warn(line)
|
||||
return
|
||||
}
|
||||
boolean skipped = m.group('skip') != null
|
||||
boolean success = !skipped && m.group('status') == 'ok'
|
||||
String skipReason = m.group('skipReason')
|
||||
String suiteName = m.group('suite')
|
||||
String testName = m.group('test')
|
||||
|
||||
String status
|
||||
if (skipped) {
|
||||
status = "SKIPPED"
|
||||
testsSkipped++
|
||||
} else if (success) {
|
||||
status = " OK"
|
||||
testsCompleted++
|
||||
} else {
|
||||
status = " FAILED"
|
||||
testsFailed++
|
||||
}
|
||||
|
||||
String counts = sprintf(countsFormat,
|
||||
[testsCompleted, testsFailed, testsSkipped, testCount])
|
||||
progressLogger.progress("Tests $counts, $status [$suiteName] $testName")
|
||||
if (!success) {
|
||||
logger.warn(line)
|
||||
}
|
||||
}
|
||||
}
|
||||
|
|
@ -0,0 +1,73 @@
|
|||
/*
|
||||
* Licensed to Elasticsearch under one or more contributor
|
||||
* license agreements. See the NOTICE file distributed with
|
||||
* this work for additional information regarding copyright
|
||||
* ownership. Elasticsearch licenses this file to you under
|
||||
* the Apache License, Version 2.0 (the "License"); you may
|
||||
* not use this file except in compliance with the License.
|
||||
* You may obtain a copy of the License at
|
||||
*
|
||||
* http://www.apache.org/licenses/LICENSE-2.0
|
||||
*
|
||||
* Unless required by applicable law or agreed to in writing,
|
||||
* software distributed under the License is distributed on an
|
||||
* "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
|
||||
* KIND, either express or implied. See the License for the
|
||||
* specific language governing permissions and limitations
|
||||
* under the License.
|
||||
*/
|
||||
package org.elasticsearch.gradle.vagrant
|
||||
|
||||
import org.gradle.api.DefaultTask
|
||||
import org.gradle.api.tasks.*
|
||||
import org.gradle.logging.ProgressLogger
|
||||
import org.gradle.logging.ProgressLoggerFactory
|
||||
import org.gradle.process.internal.ExecAction
|
||||
import org.gradle.process.internal.ExecActionFactory
|
||||
|
||||
import javax.inject.Inject
|
||||
|
||||
/**
|
||||
* Runs a vagrant command. Pretty much like Exec task but with a nicer output
|
||||
* formatter and defaults to `vagrant` as first part of commandLine.
|
||||
*/
|
||||
class VagrantCommandTask extends DefaultTask {
|
||||
List<Object> commandLine
|
||||
String boxName
|
||||
ExecAction execAction
|
||||
|
||||
VagrantCommandTask() {
|
||||
execAction = getExecActionFactory().newExecAction()
|
||||
}
|
||||
|
||||
@Inject
|
||||
ProgressLoggerFactory getProgressLoggerFactory() {
|
||||
throw new UnsupportedOperationException();
|
||||
}
|
||||
|
||||
@Inject
|
||||
ExecActionFactory getExecActionFactory() {
|
||||
throw new UnsupportedOperationException();
|
||||
}
|
||||
|
||||
void boxName(String boxName) {
|
||||
this.boxName = boxName
|
||||
}
|
||||
|
||||
void commandLine(Object... commandLine) {
|
||||
this.commandLine = commandLine
|
||||
}
|
||||
|
||||
@TaskAction
|
||||
void exec() {
|
||||
// It'd be nice if --machine-readable were, well, nice
|
||||
execAction.commandLine(['vagrant'] + commandLine)
|
||||
execAction.setStandardOutput(new VagrantLoggerOutputStream(
|
||||
command: commandLine.join(' '),
|
||||
factory: getProgressLoggerFactory(),
|
||||
/* Vagrant tends to output a lot of stuff, but most of the important
|
||||
stuff starts with ==> $box */
|
||||
squashedPrefix: "==> $boxName: "))
|
||||
execAction.execute();
|
||||
}
|
||||
}
|
||||
|
|
@ -0,0 +1,121 @@
|
|||
/*
|
||||
* Licensed to Elasticsearch under one or more contributor
|
||||
* license agreements. See the NOTICE file distributed with
|
||||
* this work for additional information regarding copyright
|
||||
* ownership. Elasticsearch licenses this file to you under
|
||||
* the Apache License, Version 2.0 (the "License"); you may
|
||||
* not use this file except in compliance with the License.
|
||||
* You may obtain a copy of the License at
|
||||
*
|
||||
* http://www.apache.org/licenses/LICENSE-2.0
|
||||
*
|
||||
* Unless required by applicable law or agreed to in writing,
|
||||
* software distributed under the License is distributed on an
|
||||
* "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
|
||||
* KIND, either express or implied. See the License for the
|
||||
* specific language governing permissions and limitations
|
||||
* under the License.
|
||||
*/
|
||||
package org.elasticsearch.gradle.vagrant
|
||||
|
||||
import com.carrotsearch.gradle.junit4.LoggingOutputStream
|
||||
import org.gradle.logging.ProgressLogger
|
||||
import org.gradle.logging.ProgressLoggerFactory
|
||||
|
||||
/**
|
||||
* Adapts an OutputStream being written to by vagrant into a ProcessLogger. It
|
||||
* has three hacks to make the output nice:
|
||||
*
|
||||
* 1. Attempt to filter out the "unimportant" output from vagrant. Usually
|
||||
* vagrant prefixes its more important output with "==> $boxname: ". The stuff
|
||||
* that isn't prefixed that way can just be thrown out.
|
||||
*
|
||||
* 2. It also attempts to detect when vagrant does tricks assuming its writing
|
||||
* to a terminal emulator and renders the output more like gradle users expect.
|
||||
* This means that progress indicators for things like box downloading work and
|
||||
* box importing look pretty good.
|
||||
*
|
||||
* 3. It catches lines that look like "==> $boxName ==> Heading text" and stores
|
||||
* the text after the second arrow as a "heading" for use in annotating
|
||||
* provisioning. It does this because provisioning can spit out _lots_ of text
|
||||
* and its very easy to lose context when there isn't a scrollback. So we've
|
||||
* sprinkled `echo "==> Heading text"` into the provisioning scripts for this
|
||||
* to catch so it can render the output like
|
||||
* "Heading text > stdout from the provisioner".
|
||||
*/
|
||||
class VagrantLoggerOutputStream extends LoggingOutputStream {
|
||||
static final String HEADING_PREFIX = '==> '
|
||||
|
||||
ProgressLogger progressLogger
|
||||
String squashedPrefix
|
||||
String lastLine = ''
|
||||
boolean inProgressReport = false
|
||||
String heading = ''
|
||||
|
||||
VagrantLoggerOutputStream(Map args) {
|
||||
progressLogger = args.factory.newOperation(VagrantLoggerOutputStream)
|
||||
progressLogger.setDescription("Vagrant $args.command")
|
||||
progressLogger.started()
|
||||
progressLogger.progress("Starting vagrant $args.command...")
|
||||
squashedPrefix = args.squashedPrefix
|
||||
}
|
||||
|
||||
void flush() {
|
||||
if (end == start) return
|
||||
line(new String(buffer, start, end - start))
|
||||
start = end
|
||||
}
|
||||
|
||||
void line(String line) {
|
||||
// debugPrintLine(line) // Uncomment me to log every incoming line
|
||||
if (line.startsWith('\r\u001b')) {
|
||||
/* We don't want to try to be a full terminal emulator but we want to
|
||||
keep the escape sequences from leaking and catch _some_ of the
|
||||
meaning. */
|
||||
line = line.substring(2)
|
||||
if ('[K' == line) {
|
||||
inProgressReport = true
|
||||
}
|
||||
return
|
||||
}
|
||||
if (line.startsWith(squashedPrefix)) {
|
||||
line = line.substring(squashedPrefix.length())
|
||||
inProgressReport = false
|
||||
lastLine = line
|
||||
if (line.startsWith(HEADING_PREFIX)) {
|
||||
line = line.substring(HEADING_PREFIX.length())
|
||||
heading = line + ' > '
|
||||
} else {
|
||||
line = heading + line
|
||||
}
|
||||
} else if (inProgressReport) {
|
||||
inProgressReport = false
|
||||
line = lastLine + line
|
||||
} else {
|
||||
return
|
||||
}
|
||||
// debugLogLine(line) // Uncomment me to log every line we add to the logger
|
||||
progressLogger.progress(line)
|
||||
}
|
||||
|
||||
void debugPrintLine(line) {
|
||||
System.out.print '----------> '
|
||||
for (int i = start; i < end; i++) {
|
||||
switch (buffer[i] as char) {
|
||||
case ' '..'~':
|
||||
System.out.print buffer[i] as char
|
||||
break
|
||||
default:
|
||||
System.out.print '%'
|
||||
System.out.print Integer.toHexString(buffer[i])
|
||||
}
|
||||
}
|
||||
System.out.print '\n'
|
||||
}
|
||||
|
||||
void debugLogLine(line) {
|
||||
System.out.print '>>>>>>>>>>> '
|
||||
System.out.print line
|
||||
System.out.print '\n'
|
||||
}
|
||||
}
|
||||
|
|
@ -0,0 +1,156 @@
|
|||
/*
|
||||
* Licensed to Elasticsearch under one or more contributor
|
||||
* license agreements. See the NOTICE file distributed with
|
||||
* this work for additional information regarding copyright
|
||||
* ownership. Elasticsearch licenses this file to you under
|
||||
* the Apache License, Version 2.0 (the "License"); you may
|
||||
* not use this file except in compliance with the License.
|
||||
* You may obtain a copy of the License at
|
||||
*
|
||||
* http://www.apache.org/licenses/LICENSE-2.0
|
||||
*
|
||||
* Unless required by applicable law or agreed to in writing,
|
||||
* software distributed under the License is distributed on an
|
||||
* "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
|
||||
* KIND, either express or implied. See the License for the
|
||||
* specific language governing permissions and limitations
|
||||
* under the License.
|
||||
*/
|
||||
|
||||
import org.elasticsearch.gradle.vagrant.VagrantCommandTask
|
||||
import org.elasticsearch.gradle.vagrant.BatsOverVagrantTask
|
||||
import org.gradle.api.InvalidUserDataException
|
||||
|
||||
def testScripts = '2*.bats' // Once the rpm and deb are build this should be *.bats
|
||||
def testCommand = "cd \$TESTROOT && sudo bats --tap \$BATS/$testScripts"
|
||||
def smokeTestCommand = 'echo I work'
|
||||
def representativeBoxes = ['ubuntu-1404', 'centos-7']
|
||||
def boxes = representativeBoxes + ['ubuntu-1204', 'ubuntu-1504', 'debian-8',
|
||||
'centos-6', 'oel-7', 'fedora-22', 'opensuse-13', 'sles-12']
|
||||
|
||||
configurations {
|
||||
test
|
||||
}
|
||||
|
||||
dependencies {
|
||||
test project(path: ':distribution:tar', configuration: 'archives')
|
||||
// NOCOMMMIT: we need the rpm and deb!
|
||||
// test project(path: ':distribution:rpm', configuration: 'archives')
|
||||
// test project(path: ':distribution:deb', configuration: 'archives')
|
||||
// Collect all the plugins
|
||||
for (Project subproj : project.rootProject.subprojects) {
|
||||
if (subproj.path.startsWith(':plugins:')) {
|
||||
test project(subproj.path)
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
task checkPackages {
|
||||
group 'Verification'
|
||||
description 'Check the packages against a representative sample of the ' +
|
||||
'linux distributions we have in our Vagrantfile'
|
||||
}
|
||||
|
||||
task checkPackagesAllDistros {
|
||||
group 'Verification'
|
||||
description 'Check the packages against all the linux distributions we ' +
|
||||
'have in our Vagrantfile'
|
||||
}
|
||||
|
||||
task clean(type: Delete) {
|
||||
group = 'Build'
|
||||
delete buildDir
|
||||
}
|
||||
|
||||
task copyDepsToTestRoot(type: Copy) {
|
||||
description "Dump bats test dependencies into the \$TESTROOT."
|
||||
into "$buildDir/testroot"
|
||||
from configurations.test
|
||||
}
|
||||
|
||||
task checkVagrantVersion(type: Exec) {
|
||||
group 'Package Verification'
|
||||
description 'Check that the version of vagrant is ok'
|
||||
commandLine 'vagrant', '--version'
|
||||
standardOutput = new ByteArrayOutputStream()
|
||||
doLast {
|
||||
def version = standardOutput.toString().trim()
|
||||
if ((version ==~ /Vagrant 1\.[789]\..+/) == false) {
|
||||
throw new InvalidUserDataException('Illegal version of vagrant [' +
|
||||
version + ']. Need [Vagrant 1.7+]')
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
task vagrantSmokeTest {
|
||||
group 'Vagrant'
|
||||
description 'Smoke test some representative distros from the Vagrantfile'
|
||||
}
|
||||
|
||||
task vagrantSmokeTestAllDistros {
|
||||
group 'Vagrant'
|
||||
description 'Smoke test all distros from the Vagrantfile'
|
||||
}
|
||||
|
||||
// Each box gets it own set of tasks
|
||||
boxes.each { box ->
|
||||
def boxTask = taskifyBoxName box
|
||||
task "vagrantUp$boxTask"(type: VagrantCommandTask) {
|
||||
group 'Vagrant'
|
||||
description "Startup a vagrant VM running $box"
|
||||
boxName box
|
||||
/* Its important that we try to reprovision the box even if it already
|
||||
exists. That way updates to the vagrant configuration take automatically.
|
||||
That isn't to say that the updates will always be compatible. Its ok to
|
||||
just destroy the boxes if they get busted but that is a manual step
|
||||
because its slow-ish. */
|
||||
/* We lock the provider to virtualbox because the Vagrantfile specifies
|
||||
lots of boxes that only work properly in virtualbox. Virtualbox is
|
||||
vagrant's default but its possible to change that default and folks do.
|
||||
But the boxes that we use are unlikely to work properly with other
|
||||
virtualization providers. Thus the lock. */
|
||||
commandLine 'up', box, '--provision', '--provider', 'virtualbox'
|
||||
/* It'd be possible to check if the box is already up here and output
|
||||
SKIPPED but that would require running vagrant status which is slow! */
|
||||
dependsOn checkVagrantVersion
|
||||
}
|
||||
task "vagrantHalt$boxTask"(type: VagrantCommandTask) {
|
||||
group 'Vagrant'
|
||||
description "Shutdown the vagrant VM running $box"
|
||||
boxName box
|
||||
commandLine 'halt', box
|
||||
}
|
||||
|
||||
task "smokeTest$boxTask"(type: Exec) {
|
||||
group 'Vagrant'
|
||||
description "Smoke test the $box VM"
|
||||
dependsOn "vagrantUp$boxTask"
|
||||
finalizedBy "vagrantHalt$boxTask"
|
||||
commandLine 'vagrant', 'ssh', box, '--command',
|
||||
"set -o pipefail && $smokeTestCommand | sed -ue \'s/^/ $box: /'"
|
||||
vagrantSmokeTestAllDistros.dependsOn name
|
||||
if (representativeBoxes.contains(box)) {
|
||||
vagrantSmokeTest.dependsOn name
|
||||
}
|
||||
}
|
||||
|
||||
task "check$boxTask"(type: BatsOverVagrantTask) {
|
||||
group 'Package Verification'
|
||||
description "Run packaging tests against $box"
|
||||
dependsOn "vagrantUp$boxTask"
|
||||
finalizedBy "vagrantHalt$boxTask"
|
||||
boxName box
|
||||
command testCommand
|
||||
dependsOn copyDepsToTestRoot
|
||||
checkPackagesAllDistros.dependsOn name
|
||||
if (representativeBoxes.contains(box)) {
|
||||
checkPackages.dependsOn name
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
|
||||
// Twists the box name into a sensible task name
|
||||
def taskifyBoxName(box) {
|
||||
box.capitalize().replace('-', '')
|
||||
}
|
||||
|
|
@ -1,413 +0,0 @@
|
|||
<?xml version="1.0" encoding="UTF-8"?>
|
||||
<project xmlns="http://maven.apache.org/POM/4.0.0"
|
||||
xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
|
||||
xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/xsd/maven-4.0.0.xsd">
|
||||
<modelVersion>4.0.0</modelVersion>
|
||||
<parent>
|
||||
<groupId>org.elasticsearch.qa</groupId>
|
||||
<artifactId>elasticsearch-qa</artifactId>
|
||||
<version>3.0.0-SNAPSHOT</version>
|
||||
</parent>
|
||||
|
||||
<artifactId>qa-vagrant</artifactId>
|
||||
<name>QA: Elasticsearch Vagrant Tests</name>
|
||||
<description>Tests the Elasticsearch distribution artifacts on virtual
|
||||
machines using vagrant and bats.</description>
|
||||
<packaging>pom</packaging>
|
||||
|
||||
<!-- The documentation for how to run this is in ../../Vagrantfile -->
|
||||
<properties>
|
||||
<testScripts>*.bats</testScripts>
|
||||
<testCommand>sudo bats $BATS/${testScripts}</testCommand>
|
||||
|
||||
<allDebBoxes>precise, trusty, vivid, jessie</allDebBoxes>
|
||||
<allRpmBoxes>centos-6, centos-7, oel-7, fedora-22, opensuse-13, sles-12</allRpmBoxes>
|
||||
|
||||
<defaultDebBoxes>trusty</defaultDebBoxes>
|
||||
<defaultRpmBoxes>centos-7</defaultRpmBoxes>
|
||||
|
||||
<!-- Unless rpmbuild is available on the host we can't test rpm based
|
||||
boxes because we can't build the rpm and they fail without the rpm.
|
||||
So to get good coverage you'll need to run this on a system with
|
||||
rpmbuild installed - either osx via homebrew or fedora/centos/rhel.
|
||||
-->
|
||||
|
||||
<!-- rpmbuild location : default to /usr/bin/rpmbuild -->
|
||||
<packaging.rpm.rpmbuild>/usr/bin/rpmbuild</packaging.rpm.rpmbuild>
|
||||
|
||||
<!-- This version is consistently available in maven central so its useful
|
||||
for now but it doesn't have any guarantee of compatibility so it
|
||||
should be switched out with 2.0.0 once that is released. -->
|
||||
<upgrade.from.version>2.0.0-beta1</upgrade.from.version>
|
||||
</properties>
|
||||
|
||||
<build>
|
||||
<plugins>
|
||||
<!-- Clean the location where we keep the distribution artifacts
|
||||
to make sure that there aren't any old versions in there. -->
|
||||
<plugin>
|
||||
<artifactId>maven-clean-plugin</artifactId>
|
||||
<executions>
|
||||
<execution>
|
||||
<id>clean-testroot</id>
|
||||
<phase>pre-integration-test</phase>
|
||||
<goals>
|
||||
<goal>clean</goal>
|
||||
</goals>
|
||||
<configuration>
|
||||
<excludeDefaultDirectories>true</excludeDefaultDirectories>
|
||||
<filesets>
|
||||
<fileset>
|
||||
<directory>${project.build.directory}/testroot</directory>
|
||||
</fileset>
|
||||
</filesets>
|
||||
</configuration>
|
||||
</execution>
|
||||
</executions>
|
||||
</plugin>
|
||||
<!-- Put the distribution artifacts some place the test can get at
|
||||
them -->
|
||||
<plugin>
|
||||
<artifactId>maven-dependency-plugin</artifactId>
|
||||
<executions>
|
||||
<execution>
|
||||
<id>copy-common-to-testroot</id>
|
||||
<phase>pre-integration-test</phase>
|
||||
<goals>
|
||||
<goal>copy</goal>
|
||||
</goals>
|
||||
<configuration>
|
||||
<outputDirectory>${project.build.directory}/testroot</outputDirectory>
|
||||
<artifactItems>
|
||||
<artifactItem>
|
||||
<groupId>org.elasticsearch.distribution.zip</groupId>
|
||||
<artifactId>elasticsearch</artifactId>
|
||||
<version>${elasticsearch.version}</version>
|
||||
<type>zip</type>
|
||||
</artifactItem>
|
||||
<artifactItem>
|
||||
<groupId>org.elasticsearch.distribution.tar</groupId>
|
||||
<artifactId>elasticsearch</artifactId>
|
||||
<version>${elasticsearch.version}</version>
|
||||
<type>tar.gz</type>
|
||||
</artifactItem>
|
||||
<artifactItem>
|
||||
<groupId>org.elasticsearch.distribution.deb</groupId>
|
||||
<artifactId>elasticsearch</artifactId>
|
||||
<version>${elasticsearch.version}</version>
|
||||
<type>deb</type>
|
||||
</artifactItem>
|
||||
<artifactItem>
|
||||
<!-- We also use an old version of elasticsearch for
|
||||
testing upgrades -->
|
||||
<groupId>org.elasticsearch.distribution.deb</groupId>
|
||||
<artifactId>elasticsearch</artifactId>
|
||||
<version>${upgrade.from.version}</version>
|
||||
<type>deb</type>
|
||||
</artifactItem>
|
||||
<artifactItem>
|
||||
<groupId>org.elasticsearch.plugin</groupId>
|
||||
<artifactId>jvm-example</artifactId>
|
||||
<version>${elasticsearch.version}</version>
|
||||
<type>zip</type>
|
||||
</artifactItem>
|
||||
<artifactItem>
|
||||
<groupId>org.elasticsearch.plugin</groupId>
|
||||
<artifactId>analysis-icu</artifactId>
|
||||
<version>${elasticsearch.version}</version>
|
||||
<type>zip</type>
|
||||
</artifactItem>
|
||||
<artifactItem>
|
||||
<groupId>org.elasticsearch.plugin</groupId>
|
||||
<artifactId>analysis-kuromoji</artifactId>
|
||||
<version>${elasticsearch.version}</version>
|
||||
<type>zip</type>
|
||||
</artifactItem>
|
||||
<artifactItem>
|
||||
<groupId>org.elasticsearch.plugin</groupId>
|
||||
<artifactId>analysis-phonetic</artifactId>
|
||||
<version>${elasticsearch.version}</version>
|
||||
<type>zip</type>
|
||||
</artifactItem>
|
||||
<artifactItem>
|
||||
<groupId>org.elasticsearch.plugin</groupId>
|
||||
<artifactId>analysis-smartcn</artifactId>
|
||||
<version>${elasticsearch.version}</version>
|
||||
<type>zip</type>
|
||||
</artifactItem>
|
||||
<artifactItem>
|
||||
<groupId>org.elasticsearch.plugin</groupId>
|
||||
<artifactId>analysis-stempel</artifactId>
|
||||
<version>${elasticsearch.version}</version>
|
||||
<type>zip</type>
|
||||
</artifactItem>
|
||||
<artifactItem>
|
||||
<groupId>org.elasticsearch.plugin</groupId>
|
||||
<artifactId>discovery-gce</artifactId>
|
||||
<version>${elasticsearch.version}</version>
|
||||
<type>zip</type>
|
||||
</artifactItem>
|
||||
<artifactItem>
|
||||
<groupId>org.elasticsearch.plugin</groupId>
|
||||
<artifactId>delete-by-query</artifactId>
|
||||
<version>${elasticsearch.version}</version>
|
||||
<type>zip</type>
|
||||
</artifactItem>
|
||||
<artifactItem>
|
||||
<groupId>org.elasticsearch.plugin</groupId>
|
||||
<artifactId>discovery-azure</artifactId>
|
||||
<version>${elasticsearch.version}</version>
|
||||
<type>zip</type>
|
||||
</artifactItem>
|
||||
<artifactItem>
|
||||
<groupId>org.elasticsearch.plugin</groupId>
|
||||
<artifactId>discovery-ec2</artifactId>
|
||||
<version>${elasticsearch.version}</version>
|
||||
<type>zip</type>
|
||||
</artifactItem>
|
||||
<artifactItem>
|
||||
<groupId>org.elasticsearch.plugin</groupId>
|
||||
<artifactId>discovery-multicast</artifactId>
|
||||
<version>${elasticsearch.version}</version>
|
||||
<type>zip</type>
|
||||
</artifactItem>
|
||||
<artifactItem>
|
||||
<groupId>org.elasticsearch.plugin</groupId>
|
||||
<artifactId>lang-expression</artifactId>
|
||||
<version>${elasticsearch.version}</version>
|
||||
<type>zip</type>
|
||||
</artifactItem>
|
||||
<artifactItem>
|
||||
<groupId>org.elasticsearch.plugin</groupId>
|
||||
<artifactId>lang-groovy</artifactId>
|
||||
<version>${elasticsearch.version}</version>
|
||||
<type>zip</type>
|
||||
</artifactItem>
|
||||
<artifactItem>
|
||||
<groupId>org.elasticsearch.plugin</groupId>
|
||||
<artifactId>lang-javascript</artifactId>
|
||||
<version>${elasticsearch.version}</version>
|
||||
<type>zip</type>
|
||||
</artifactItem>
|
||||
<artifactItem>
|
||||
<groupId>org.elasticsearch.plugin</groupId>
|
||||
<artifactId>lang-python</artifactId>
|
||||
<version>${elasticsearch.version}</version>
|
||||
<type>zip</type>
|
||||
</artifactItem>
|
||||
<artifactItem>
|
||||
<groupId>org.elasticsearch.plugin</groupId>
|
||||
<artifactId>mapper-murmur3</artifactId>
|
||||
<version>${elasticsearch.version}</version>
|
||||
<type>zip</type>
|
||||
</artifactItem>
|
||||
<artifactItem>
|
||||
<groupId>org.elasticsearch.plugin</groupId>
|
||||
<artifactId>mapper-size</artifactId>
|
||||
<version>${elasticsearch.version}</version>
|
||||
<type>zip</type>
|
||||
</artifactItem>
|
||||
<artifactItem>
|
||||
<groupId>org.elasticsearch.plugin</groupId>
|
||||
<artifactId>repository-azure</artifactId>
|
||||
<version>${elasticsearch.version}</version>
|
||||
<type>zip</type>
|
||||
</artifactItem>
|
||||
<artifactItem>
|
||||
<groupId>org.elasticsearch.plugin</groupId>
|
||||
<artifactId>repository-s3</artifactId>
|
||||
<version>${elasticsearch.version}</version>
|
||||
<type>zip</type>
|
||||
</artifactItem>
|
||||
<artifactItem>
|
||||
<groupId>org.elasticsearch.plugin</groupId>
|
||||
<artifactId>store-smb</artifactId>
|
||||
<version>${elasticsearch.version}</version>
|
||||
<type>zip</type>
|
||||
</artifactItem>
|
||||
<artifactItem>
|
||||
<groupId>org.elasticsearch.plugin</groupId>
|
||||
<artifactId>site-example</artifactId>
|
||||
<version>${elasticsearch.version}</version>
|
||||
<type>zip</type>
|
||||
</artifactItem>
|
||||
</artifactItems>
|
||||
</configuration>
|
||||
</execution>
|
||||
</executions>
|
||||
</plugin>
|
||||
<plugin>
|
||||
<groupId>org.apache.maven.plugins</groupId>
|
||||
<artifactId>maven-antrun-plugin</artifactId>
|
||||
<dependencies>
|
||||
<dependency>
|
||||
<groupId>ant-contrib</groupId>
|
||||
<artifactId>ant-contrib</artifactId>
|
||||
<version>1.0b3</version>
|
||||
<exclusions>
|
||||
<exclusion>
|
||||
<groupId>ant</groupId>
|
||||
<artifactId>ant</artifactId>
|
||||
</exclusion>
|
||||
</exclusions>
|
||||
</dependency>
|
||||
</dependencies>
|
||||
<executions>
|
||||
<execution>
|
||||
<id>check-vagrant-version</id>
|
||||
<phase>validate</phase>
|
||||
<goals>
|
||||
<goal>run</goal>
|
||||
</goals>
|
||||
<configuration>
|
||||
<target>
|
||||
<taskdef resource="net/sf/antcontrib/antlib.xml"
|
||||
classpathref="maven.dependency.classpath" />
|
||||
<ant antfile="src/dev/ant/vagrant-integration-tests.xml"
|
||||
target="check-vagrant-version"/>
|
||||
</target>
|
||||
</configuration>
|
||||
</execution>
|
||||
<execution>
|
||||
<id>setup-version-files</id>
|
||||
<phase>pre-integration-test</phase>
|
||||
<goals>
|
||||
<goal>run</goal>
|
||||
</goals>
|
||||
<configuration>
|
||||
<target>
|
||||
<echo message="${project.version}"
|
||||
file="${project.build.directory}/testroot/version"/>
|
||||
<echo message="${upgrade.from.version}"
|
||||
file="${project.build.directory}/testroot/upgrade_from_version"/>
|
||||
</target>
|
||||
</configuration>
|
||||
</execution>
|
||||
<execution>
|
||||
<id>test-vms</id>
|
||||
<phase>integration-test</phase>
|
||||
<goals>
|
||||
<goal>run</goal>
|
||||
</goals>
|
||||
<configuration>
|
||||
<target unless="${skipTests}">
|
||||
<taskdef resource="net/sf/antcontrib/antlib.xml"
|
||||
classpathref="maven.dependency.classpath" />
|
||||
<ant antfile="src/dev/ant/vagrant-integration-tests.xml"
|
||||
target="vagrant-test-all-boxes"/>
|
||||
</target>
|
||||
</configuration>
|
||||
</execution>
|
||||
</executions>
|
||||
</plugin>
|
||||
</plugins>
|
||||
</build>
|
||||
|
||||
<profiles>
|
||||
<profile>
|
||||
<!-- Enable the rpm artifact and rpm-boxes because we're on an
|
||||
rpm-based distribution. -->
|
||||
<id>rpm</id>
|
||||
<activation>
|
||||
<file>
|
||||
<exists>${packaging.rpm.rpmbuild}</exists>
|
||||
</file>
|
||||
</activation>
|
||||
<build>
|
||||
<plugins>
|
||||
<plugin>
|
||||
<artifactId>maven-dependency-plugin</artifactId>
|
||||
<executions>
|
||||
<execution>
|
||||
<id>copy-rpm-to-testroot</id>
|
||||
<phase>pre-integration-test</phase>
|
||||
<goals>
|
||||
<goal>copy</goal>
|
||||
</goals>
|
||||
<configuration>
|
||||
<outputDirectory>${project.build.directory}/testroot</outputDirectory>
|
||||
<artifactItems>
|
||||
<artifactItem>
|
||||
<groupId>org.elasticsearch.distribution.rpm</groupId>
|
||||
<artifactId>elasticsearch</artifactId>
|
||||
<version>${elasticsearch.version}</version>
|
||||
<type>rpm</type>
|
||||
</artifactItem>
|
||||
<artifactItem>
|
||||
<!-- We also use an old version of elasticsearch for
|
||||
testing upgrades -->
|
||||
<groupId>org.elasticsearch.distribution.rpm</groupId>
|
||||
<artifactId>elasticsearch</artifactId>
|
||||
<version>${upgrade.from.version}</version>
|
||||
<type>rpm</type>
|
||||
</artifactItem>
|
||||
</artifactItems>
|
||||
</configuration>
|
||||
</execution>
|
||||
</executions>
|
||||
</plugin>
|
||||
</plugins>
|
||||
</build>
|
||||
<properties>
|
||||
<rpmOk>ok</rpmOk>
|
||||
</properties>
|
||||
</profile>
|
||||
<profile>
|
||||
<!-- Enable the rpm artifact and rpm-boxes because we're on an
|
||||
rpm-based distribution. -->
|
||||
<id>rpm-via-homebrew</id>
|
||||
<activation>
|
||||
<file>
|
||||
<exists>/usr/local/bin/rpmbuild</exists>
|
||||
</file>
|
||||
</activation>
|
||||
<build>
|
||||
<plugins>
|
||||
<plugin>
|
||||
<artifactId>maven-dependency-plugin</artifactId>
|
||||
<executions>
|
||||
<execution>
|
||||
<id>copy-rpm-to-testroot</id>
|
||||
<phase>pre-integration-test</phase>
|
||||
<goals>
|
||||
<goal>copy</goal>
|
||||
</goals>
|
||||
<configuration>
|
||||
<outputDirectory>${project.build.directory}/testroot</outputDirectory>
|
||||
<artifactItems>
|
||||
<artifactItem>
|
||||
<groupId>org.elasticsearch.distribution.rpm</groupId>
|
||||
<artifactId>elasticsearch</artifactId>
|
||||
<version>${elasticsearch.version}</version>
|
||||
<type>rpm</type>
|
||||
</artifactItem>
|
||||
<artifactItem>
|
||||
<!-- We also use an old version of elasticsearch for
|
||||
testing upgrades -->
|
||||
<groupId>org.elasticsearch.distribution.rpm</groupId>
|
||||
<artifactId>elasticsearch</artifactId>
|
||||
<version>${upgrade.from.version}</version>
|
||||
<type>rpm</type>
|
||||
</artifactItem>
|
||||
</artifactItems>
|
||||
</configuration>
|
||||
</execution>
|
||||
</executions>
|
||||
</plugin>
|
||||
</plugins>
|
||||
</build>
|
||||
<properties>
|
||||
<rpmOk>ok</rpmOk>
|
||||
</properties>
|
||||
</profile>
|
||||
|
||||
<!-- This profile manipulates what is run. -->
|
||||
<profile>
|
||||
<!-- Smoke tests the VMs but doesn't actually run the bats tests -->
|
||||
<id>smoke-vms</id>
|
||||
<properties>
|
||||
<testCommand>echo skipping tests</testCommand>
|
||||
</properties>
|
||||
</profile>
|
||||
</profiles>
|
||||
</project>
|
||||
|
|
@ -1,112 +0,0 @@
|
|||
<?xml version="1.0"?>
|
||||
<project name="elasticsearch-integration-tests">
|
||||
<target name="vagrant-test-all-boxes" depends="set-boxesToTest">
|
||||
<echo message="Running package tests on ${boxesToTest}"/>
|
||||
<foreach list="${boxesToTest}" trim="true" param="box"
|
||||
target="vagrant-test" inheritall="true" inheritrefs="true"/>
|
||||
</target>
|
||||
|
||||
<target name="set-boxesToTest">
|
||||
<!-- If the user specifies the vagrant=all then try on all boxes. -->
|
||||
<condition property="debBoxes" value="${allDebBoxes}"
|
||||
else="${defaultDebBoxes}">
|
||||
<equals arg1="${tests.vagrant}" arg2="all"/>
|
||||
</condition>
|
||||
<condition property="rpmBoxes" value="${allRpmBoxes}"
|
||||
else="${defaultRpmBoxes}">
|
||||
<equals arg1="${tests.vagrant}" arg2="all"/>
|
||||
</condition>
|
||||
<!-- By defaut only test on debian boxes because we always can build debs.
|
||||
Only test on rpm based boxes if we can build RPM. -->
|
||||
<condition property="proposedBoxesToTest" value="${debBoxes}, ${rpmBoxes}"
|
||||
else="${debBoxes}">
|
||||
<equals arg1="${rpmOk}" arg2="ok"/>
|
||||
</condition>
|
||||
<!-- Only use the proposedBoxesToTest list if the user did -Dtests.vagrant
|
||||
or -Dtests.vagrant=all. Otherwise we assume they did
|
||||
-Dtests.vagrant=somelistofboxes and we just set the boxesToTest to
|
||||
that. If they set an rpm based box and don't have the rpm available
|
||||
its on them. -->
|
||||
<condition property="boxesToTest" value="${proposedBoxesToTest}"
|
||||
else="${tests.vagrant}">
|
||||
<or>
|
||||
<equals arg1="${tests.vagrant}" arg2="true"/>
|
||||
<equals arg1="${tests.vagrant}" arg2="all"/>
|
||||
</or>
|
||||
</condition>
|
||||
</target>
|
||||
|
||||
<target name="vagrant-test" depends="vagrant-up">
|
||||
<trycatch>
|
||||
<try>
|
||||
<exec executable="vagrant" failonerror="true">
|
||||
<arg value="ssh"/>
|
||||
<arg value="${box}"/>
|
||||
<arg value="--command"/>
|
||||
<arg value="
|
||||
set -o pipefail;
|
||||
cd $TESTROOT;
|
||||
${testCommand} | sed -ue 's/^/${box}: /'
|
||||
"/>
|
||||
</exec>
|
||||
</try>
|
||||
<finally>
|
||||
<exec executable="vagrant" failonerror="true">
|
||||
<arg value="halt"/>
|
||||
<arg value="${box}"/>
|
||||
</exec>
|
||||
</finally>
|
||||
</trycatch>
|
||||
</target>
|
||||
|
||||
<target name="vagrant-up">
|
||||
<exec executable="vagrant" failonerror="true">
|
||||
<arg value="up"/>
|
||||
<arg value="${box}"/>
|
||||
<!-- Its important that we try to reprovision the box even if it already
|
||||
exists. That way updates to the vagrant configuration take automatically.
|
||||
That isn't to say that the updates will always be compatible. Its ok to
|
||||
just destroy the boxes if they get busted. -->
|
||||
<arg value="--provision"/>
|
||||
<!-- We lock the provider to virtualbox because the Vagrantfile specifies
|
||||
lots of boxes that only work properly in virtualbox. Virtualbox is
|
||||
vagrant's default but its possible to change that default and folks do.
|
||||
But the boxes that we use are unlikely to work properly with other
|
||||
virtualization providers. Thus the lock. -->
|
||||
<arg value="--provider"/>
|
||||
<arg value="virtualbox"/>
|
||||
</exec>
|
||||
</target>
|
||||
|
||||
<target name="check-vagrant-version">
|
||||
<check-version executable="vagrant" ok="^1\.[789]\..+$"
|
||||
message="Only known to work with Vagrant 1.7+"/>
|
||||
</target>
|
||||
|
||||
<macrodef name="check-version">
|
||||
<attribute name="executable" description="The executable to check."/>
|
||||
<attribute name="rewrite" default="(?:\S*\s)*(.+)"
|
||||
description="Regex extracting the version from the output of the executable. Defaults to everything after the last space."/>
|
||||
<attribute name="ok" description="The regex to check the version against."/>
|
||||
<attribute name="message" description="The message to report on failure."/>
|
||||
<sequential>
|
||||
<exec executable="@{executable}" failonerror="true"
|
||||
outputproperty="versionOutput">
|
||||
<arg value="--version" />
|
||||
</exec>
|
||||
<propertyregex property="version" input="${versionOutput}"
|
||||
regexp="@{rewrite}" select="\1" />
|
||||
<echo message="The @{executable} version is ${version}"/>
|
||||
<fail message="@{message}">
|
||||
<condition>
|
||||
<not>
|
||||
<!-- Very simple version checking.... -->
|
||||
<matches string="${version}" pattern="@{ok}"/>
|
||||
</not>
|
||||
</condition>
|
||||
</fail>
|
||||
</sequential>
|
||||
</macrodef>
|
||||
|
||||
|
||||
</project>
|
||||
|
|
@ -417,9 +417,7 @@ run_elasticsearch_tests() {
|
|||
}' | grep \"count\"\ :\ 1
|
||||
|
||||
curl -s -XGET 'http://localhost:9200/library/book/_search/template?pretty' -d '{
|
||||
"template": {
|
||||
"file": "is_guide"
|
||||
}
|
||||
"file": "is_guide"
|
||||
}' | grep \"total\"\ :\ 1
|
||||
|
||||
curl -s -XDELETE 'http://localhost:9200/_all'
|
||||
|
|
|
|||
|
|
@ -149,7 +149,7 @@ install_and_check_plugin() {
|
|||
# $2 description of the source of the plugin list
|
||||
compare_plugins_list() {
|
||||
cat $1 | sort > /tmp/plugins
|
||||
ls /elasticsearch/plugins/*/pom.xml | cut -d '/' -f 4 |
|
||||
ls /elasticsearch/plugins/*/build.gradle | cut -d '/' -f 4 |
|
||||
sort > /tmp/expected
|
||||
echo "Checking plugins from $2 (<) against expected plugins (>):"
|
||||
diff /tmp/expected /tmp/plugins
|
||||
|
|
|
|||
|
|
@ -30,7 +30,8 @@ String[] projects = [
|
|||
'plugins:site-example',
|
||||
'plugins:store-smb',
|
||||
'qa:smoke-test-client',
|
||||
'qa:smoke-test-plugins'
|
||||
'qa:smoke-test-plugins',
|
||||
'qa:vagrant',
|
||||
]
|
||||
|
||||
if (hasProperty('elasticsearch.projectsPrefix')) {
|
||||
|
|
@ -39,4 +40,3 @@ if (hasProperty('elasticsearch.projectsPrefix')) {
|
|||
}
|
||||
|
||||
include projects
|
||||
|
||||
|
|
|
|||
Loading…
Reference in New Issue