Commit Graph

41 Commits

Author SHA1 Message Date
Jason Tedor f999d639be
Update platforms tests to use Gradle wrapper
Gradle will no longer be needed in the test VMs as the Gradle wrapper
can be used to run the platform tests. This commit updates the platform
tests to use the Gradle wrapper, and removes installing Gradle in the VM
which will speed up VM provisioning.

Relates 
2018-01-05 17:25:10 -05:00
David Roberts d21167e0c2
[TEST] Remove leftover ES temp directories before Vagrant tests ()
Some of the Vagrant tests were failing due to ES temp directories
left over from previous uses of the same VM confusing subsequent
tests into thinking there were multiple ES installs present.

This change wipes all ES temp directories when the test VMs are
brought up.
2017-12-08 17:37:05 +00:00
Dimitrios Liappis d3e3bc8656
Tests: Add Fedora-27 to packaging tests
Replace Fedora-25 with Fedora-27 and fix old references in
`TESTING.asciidoc`.

Relates 
2017-11-20 16:59:00 +02:00
Dimitrios Liappis 5079afd0bb Tests: Add Fedora-26 to packaging tests
Add Fedora-26 to packaging tests. Don't retire Fedora-25 yet, as it
will be EOL, one month after Fedora-27 is out.

Relates 
2017-10-03 12:01:20 +03:00
Jason Tedor 9aa42a438b Unzip quietly while provisioning virtual machines
When provisioning the virtual machines used for packaging, we download
the Gradle zip archive and unzip. This unzip is noisy produing a lot of
unnecessary output. This commit silences this output.

Relates 
2017-07-20 12:45:56 +09:00
Dimitrios Liappis 39d42292dd Tests: Add Debian 9 (Stretch) to the packaging tests
Debian 9 aka Stretch is the current stable[1].

Add Debian-9 to the packaging tests.

[1] https://wiki.debian.org/DebianStretch

Relates 
2017-07-03 11:02:17 +03:00
Dimitrios Liappis 8f8fc74b2e Tests: Switch to openSUSE 42 (Leap) for packaging tests
openSUSE-13 has reached [EOL](https://en.opensuse.org/Lifetime).

Replace openSUSE-13 with openSUSE-42 (Leap) for packaging tests and
update docs.

Relates 
2017-06-01 15:40:47 +03:00
Dimitrios Liappis 9a3ab3e800 Tests: Switch to Fedora-25 for packaging tests
Switch the Fedora Vagrant box for packaging tests to Fedora-25 as
Fedora-24 will become EOL one month after Fedora-26 is out.

Relates 
2017-05-04 12:04:53 +03:00
Nik Everett 0c12d0ce37 Make bats tests refuse to start on non-VMs ()
The bats tests are descructive and must be run as root. This is a
horrible combination on any sane system but perfectly fine to do
in a VM. This change modifies the tests so they revuse to start
unless they are in an environment with an `/etc/is_vagrant_vm`
file. The Vagrantfile creates it on startup.

Closes 
2017-04-26 10:40:35 -04:00
Tanguy Leroux e81bbc288a Remove Ubuntu 12.04 ()
Ubuntu 12.04 will be EOL on April 28, 2017.
2017-04-19 09:39:42 +02:00
Ryan Ernst 2666ecd76f Build: Set vagrant group for gradle install in vagrant ()
We previously removed setting the vagrant group because sles-12 and
opensuse-13 did not have this group. Now that those images have the
group, we can go back to setting both user and group to vagrant.
2017-03-09 12:05:36 -08:00
Ryan Ernst 0a2aa0a8c4 Fix sles vagrant image to no longer use custom repositories.
These are no longer necessary now that the jdk is packaged with the
image.
2017-03-03 02:22:00 -08:00
Jason Tedor 98d3949a98 Install Gradle quietly
When downloading Gradle to install inside the VMs used for testing, the
download progress logs do not play well with the Gradle progress logger
so we see garbage like:

==> centos-6: ==> Installing gradle
==> centos-6:
==> centos-6:
==> centos-6: %
==> centos-6:
==> centos-6: T
==> centos-6: o
==> centos-6: t
==> centos-6: a
==> centos-6: l
==> centos-6:
==> centos-6:
==> centos-6:
==> centos-6:
==> centos-6: %
==> centos-6:
==> centos-6: R
==> centos-6: e
==> centos-6: c
==> centos-6: e
==> centos-6: i
==> centos-6: v
==> centos-6: e
==> centos-6: d
==> centos-6:
==> centos-6: %
==> centos-6:
==> centos-6: X
==> centos-6: f
==> centos-6: e
==> centos-6: r
==> centos-6: d
==> centos-6:
==> centos-6:
==> centos-6: A
==> centos-6: v
==> centos-6: e
==> centos-6: r
==> centos-6: a
==> centos-6: g
==> centos-6: e
==> centos-6:
==> centos-6: S
==> centos-6: p
==> centos-6: e
==> centos-6: e
==> centos-6: d
==> centos-6:
==> centos-6:

This commit addresses this by setting curl to be silent and only show
errors. This instead gives:

==> centos-6: ==> Installing gradle
==> centos-6: Archive:  /tmp/gradle.zip

Relates 
2017-03-02 12:42:51 -05:00
Ryan Ernst e866da580c Only set vagrant user for gradle installation
The vagrant group is not available on all systems the packaging and
platform tests run on.
2017-02-27 16:13:00 -08:00
Ryan Ernst 48280a9403 Tests: Add platformTest to vagrant ()
This change adds a new test task, platformTest, which runs `gradle test
integTest` within a vagrant host. This will allow us to still test on
all the supported platforms, but be able to standardize on the tools
provided in the host system, for example, with a modern version of git
that can allow .

In order to have sufficient memory and cpu to run the tests, the
vagrantfile has been updated to use 8GB and 4 cpus by default. This can
be customized with the `VAGRANT_MEMORY` and `VAGRANT_CPUS` environment
variables.  Also, to save time to show this can work, it currently uses
the same Vagrantfile the packaging tests do. There are a lot of cleanups
we can do to how the gradle-vagrant tasks work, including generating
Vagrantfile altogether, but I think this is fine for now as the same
machines that would run platformTest run packagingTest, and they are
relatively beefy machines, so the higher memory and cpu for them, with
either task, should not be an issue.
2017-02-27 09:21:28 -08:00
Ryan Ernst 37f7a70e22 Tests: Remove java installation from vagrant provisioning ()
The elastic images used for testing different systems now have java
installed in the base image. This commit removes the installation of
java, which should make the packagingTest runs more stable, as they will
not depend on flaky system repository mirrors.
2017-02-16 19:32:12 -08:00
Dimitrios Liappis 7e2ec3460b Prepare Vagrantfile tests for SLES12-SP2
SLES-12 SP2 has a system package available for
java-1_8_0-openjdk and thus we can deprecate the OpenSUSE additional
repo.

Also remove the `src rpm` repo which is not needed as we don't build
packages from source and complains about missing `media.2` directory.

Relates 
2017-02-03 18:52:48 +02:00
Tanguy Leroux 6cafe688b3 [TEST] Ensure file permission for /etc/sudoers.d/elasticsearch_vars
Ubuntu 12.04 checks the file permission for /etc/sudoers.d/elasticsearch_vars is mode 0440. This commit adds a `chmod` before the file is used by the `sudo` command.
2016-11-15 17:42:57 +01:00
Tanguy Leroux aa871f839a [Tests] Do not pass all env vars to sudo ()
In  the command executed to run the packaging tests has been changed to "sudo -E bats ...", forcing all environment variables from the vagrant user to be passed to the `sudo` command. This breaks a test on opensuse-13 (the one where it checks that elasticsearch cannot be started when `java` is not found) because all the PATH from the user is passed to the sudo command.

This commit restores the previous behavior while allowing only necessary testing environment variables to be passed using a /etc/sudoers.d file.
2016-11-15 15:19:34 +01:00
Tanguy Leroux 9cade1cc18 Add Vagrant Gradle plugin
This commit changes the current :elactisearch:qa:vagrant build file and transforms it into a Gradle plugin in order to reuse it in other projects.

Most of the code from the build.gradle file has been moved into the VagrantTestPlugin class. To avoid duplicated VMs when running vagrant tests, the Gradle plugin sets the following environment variables before running vagrant commands:
VAGRANT_CWD: absolute path to the folder that contains the Vagrantfile
VAGRANT_PROJECT_DIR: absolute path to the Gradle project that use the VagrantTestPlugin

The VAGRANT_PROJECT_DIR is used to share project folders and files with the vagrant VM. These folders and files are exported when running the task `gradle vagrantSetUp` which:
- collects all project archives dependencies and copies them into `${project.buildDir}/bats/archives`
- copy all project bats testing files from 'src/test/resources/packaging/tests' into `${project.buildDir}/bats/tests`
- copy all project bats utils files from 'src/test/resources/packaging/utils' into `${project.buildDir}/bats/utils`

It is also possible to inherit and grab the archives/tests/utils files from project dependencies using the plugin configuration:

apply plugin: 'elasticsearch.vagrant'
esvagrant {
    inheritTestUtils true|false
    inheritTestArchives true|false
    inheritTests true|false
}
dependencies {
    // Inherit Bats test utils from :qa:vagrant project
    bats project(path: ':qa:vagrant', configuration: 'bats')
}

The folders `${project.buildDir}/bats/archives`, `${project.buildDir}/bats/tests` and `${project.buildDir}/bats/utils` are then exported to the vagrant VMs and mapped to the BATS_ARCHIVES, BATS_TESTS and BATS_UTILS environnement variables.

The following Gradle tasks have also be renamed:

* gradle vagrantSetUp
This task copies all the necessary files to the project build directory (was `prepareTestRoot`)

* gradle vagrantSmokeTest
This task starts the VMs and echoes a "Hello world" within each VM (was: `smokeTest`)
2016-11-14 10:33:05 +01:00
Dimitrios Liappis b0bb72ccd7 Deprecate EOL'ed Ubuntu 15.04 from Vagrantfile
* Deprecate EOL'ed Ubuntu 15.04 from Vagrantfile

Relates pr 
2016-10-17 17:27:01 +03:00
Dimitrios Liappis 02dd1ba61a [vagrant] packaging tests: add parametrized retries for dnf install ()
* Add parametrized retries for dnf install

Given that dnf doesn't do retries installation of openjdk can sometimes
be affected by checksum or network issues with mirrors offered by
metalink.

Allow setting number of retries through the parameter
`install_command_retries`

* Insert delay between package install retries

Fedora's metalink occasionally returns broken mirrors. Pausing for a few
seconds between retries increases the chance of receiving a different
list of mirrors from metalink and success with package installation.
2016-10-05 11:42:25 -04:00
Tanguy Leroux 567093cf78 Add Ubuntu-16.04 to Vagrant VMs () 2016-09-13 09:26:56 +02:00
Jason Tedor 5e6eb987bc Correct Vagrant boxes memory allocation
This commit corrects the memory allocation to the boxes uses for the
Vagrant tests to actually be 3 GB (really, this is just an OCD-driven
commit).
2016-08-23 23:12:24 -04:00
Nik Everett e55581bb67 Give vagrant test boxes 3gb of ram
Now that Elasticsearch defaults to 2gb heaps those boxes need more ram,
especially opensuse-13 for some reason. It has failed to run the
packaging tests ever since the change.....
2016-08-23 14:54:40 -04:00
Jason Tedor 68676f8cf1 Migrate Vagrant tests for Fedora to Fedora 24
This commit migrates the Vagrant box for Fedora for the packaging tests
from Fedora 22 to Fedora 24 as Fedora 22 reached end-of-line upon the
release of Fedora 24.

Relates 
2016-07-07 13:31:30 -04:00
Jason Tedor d7b478cd04 Change Debian repository for Vagrant debian-8 box
This commit modifies the Debian repository to work around some upstream
reliability issues with the previous repository.
2016-07-05 07:13:43 -04:00
Jason Tedor a7793f7271 Cutover to elastic Vagrant boxes
This commit cuts the Vagrant tests over to the elastic Vagrant boxes.
2016-03-20 22:11:30 -04:00
Nik Everett 9b0a47d8e3 Fix rpm and deb distributions
and test them with vagrant
2015-11-18 14:16:42 -05:00
Nik Everett 1dd00dddd5 [test] Gradle-ify vagrant tests
This gets the tar and tar_plugins tests working in gradle. It does so by
adding a subproject, qa/vagrant, which adds the following tasks:

Verification
------------
checkPackages - Check the packages against a representative sample of the
                linux distributions we have in our Vagrantfile
checkPackagesAllDistros - Check the packages against all the linux
                          distributions we have in our Vagrantfile

Package Verification
--------------------
checkCentos6 - Run packaging tests against centos-6
checkCentos7 - Run packaging tests against centos-7
checkDebian8 - Run packaging tests against debian-8
checkFedora22 - Run packaging tests against fedora-22
checkOel7 - Run packaging tests against oel-7
checkOpensuse13 - Run packaging tests against opensuse-13
checkSles12 - Run packaging tests against sles-12
checkUbuntu1204 - Run packaging tests against ubuntu-1204
checkUbuntu1404 - Run packaging tests against ubuntu-1404
checkUbuntu1504 - Run packaging tests against ubuntu-1504

Vagrant
-------
smokeTestCentos6 - Smoke test the centos-6 VM
smokeTestCentos7 - Smoke test the centos-7 VM
smokeTestDebian8 - Smoke test the debian-8 VM
smokeTestFedora22 - Smoke test the fedora-22 VM
smokeTestOel7 - Smoke test the oel-7 VM
smokeTestOpensuse13 - Smoke test the opensuse-13 VM
smokeTestSles12 - Smoke test the sles-12 VM
smokeTestUbuntu1204 - Smoke test the ubuntu-1204 VM
smokeTestUbuntu1404 - Smoke test the ubuntu-1404 VM
smokeTestUbuntu1504 - Smoke test the ubuntu-1504 VM
vagrantHaltCentos6 - Shutdown the vagrant VM running centos-6
vagrantHaltCentos7 - Shutdown the vagrant VM running centos-7
vagrantHaltDebian8 - Shutdown the vagrant VM running debian-8
vagrantHaltFedora22 - Shutdown the vagrant VM running fedora-22
vagrantHaltOel7 - Shutdown the vagrant VM running oel-7
vagrantHaltOpensuse13 - Shutdown the vagrant VM running opensuse-13
vagrantHaltSles12 - Shutdown the vagrant VM running sles-12
vagrantHaltUbuntu1204 - Shutdown the vagrant VM running ubuntu-1204
vagrantHaltUbuntu1404 - Shutdown the vagrant VM running ubuntu-1404
vagrantHaltUbuntu1504 - Shutdown the vagrant VM running ubuntu-1504
vagrantSmokeTest - Smoke test some representative distros from the Vagrantfile
vagrantSmokeTestAllDistros - Smoke test all distros from the Vagrantfile
vagrantUpCentos6 - Startup a vagrant VM running centos-6
vagrantUpCentos7 - Startup a vagrant VM running centos-7
vagrantUpDebian8 - Startup a vagrant VM running debian-8
vagrantUpFedora22 - Startup a vagrant VM running fedora-22
vagrantUpOel7 - Startup a vagrant VM running oel-7
vagrantUpOpensuse13 - Startup a vagrant VM running opensuse-13
vagrantUpSles12 - Startup a vagrant VM running sles-12
vagrantUpUbuntu1204 - Startup a vagrant VM running ubuntu-1204
vagrantUpUbuntu1404 - Startup a vagrant VM running ubuntu-1404
vagrantUpUbuntu1504 - Startup a vagrant VM running ubuntu-1504

It does not make the "check" task depend on "checkPackages" so running the
vagrant tests is still optional. They are slow and depend on vagrant and
virtualbox.

The Package Verification tasks are useful for testing individual distros.

The Vagrant tasks are listed in `gradle tasks` primarily for discoverability.
2015-11-05 14:28:30 -05:00
Nik Everett e45b2f7f0c [test] Jayatana is ignored
Installs javatana in vivid, emulates its on-login actions when starting
elasticsearch and verifies that elasticsearch turns off javatana.

Relates to 
2015-09-28 03:33:05 +02:00
Nik Everett 3d72ebd876 [test] Give vagrant boxes 2gb of ram
This lets them run our tests without oomkiller getting angry.
2015-09-24 10:04:59 -04:00
Nik Everett a08cd26d5e Merge pull request from nik9000/fully_update_on_repo_add
Remove esoteric apt-get in Vagrantfile
2015-09-22 09:07:20 -04:00
Alexander Reelsen 2b46a72a6c BATS testing: Add SLES-12 to list of tested virtual machines
This adds SuSe Linux Enterprise Server 12 to the list of tested VMs.
SLES 12 is using systemd, so that the current RPM works
out of the box.

SLES12 however is already quite old and does not ship with java8, so this
required adding an opensuse repo.
2015-09-22 15:02:22 +02:00
Nik Everett c427b3e9c2 [test] Remove esoteric apt-get in Vagrantfile
Removes an esoteric `apt-get update` variant used in Vagrantfile that was
causing only parts of the apt repository to update. That was the point of
the command but when it would leave the repository only half built which
made installing anything but Java difficult. The speed isn't worth the
complexity.
2015-09-17 15:48:10 -04:00
Nik Everett 058d385942 [test] Add opensuse-13 to packaging tests
This gets opensuse-13 working with vagrant and the packaging tests. They pass
with some minor tweaks.

Closes 
2015-09-16 10:09:39 -04:00
Nik Everett 4cabe39096 [test] Packaging tests use Java 8
To do this we:
1. All the rpm based distros we test support Java 8. We just ask to install
it.
2. There is a ppa that works for the Ubuntus. We just add that for them.
3. Debian Jessie has Java 8 in its backports. We just add that repository.
4. Debian Wheezy doesn't have Java 8 easily accessible so we drop it. We
could add it back with Orache Java 8 at a later date but that will take a
few more backflips and won't support things like vagrant-cachier.

This required a ton of rebuilding of vagrant boxes so it also fixes:
1. apt-get update is run too frequently
2. Lots of weird warning messages are spit out of apt-get
3. Switch from the chef provided based images to those provided by boxcutter.
The chef images has left vagrant atlas!

Closes 
2015-09-09 08:04:40 -04:00
Nik Everett f5735e8571 Merge pull request from nik9000/prompts_2
Set prompts to be more consistent
2015-08-17 11:33:38 -07:00
Nik Everett 391ea379e2 Test: Use jvm-example for testing bin/plugin
Related to 
2015-08-17 10:53:16 -07:00
Nik Everett b6f2cebf8b Vagrant tests: set prompts to be more consistent
Also a small amount of cleanup in the way we create VMs - just a bit less
repetition.

Prompts are always of the form "box:cwd$ ". Even for root. Which is ok because
you don't have to be that careful with root because these are VMs that you
can destroy and recreate quickly.
2015-08-10 13:49:54 -07:00
Nik Everett f84552dc98 [Packaging] Run tests in vagrant
This creates a module in qa called vagrant that can be run if you have
vagrant and virtualbox installed and will run the packaging tests in trusty
and centos-7.0. You can ask it to run tests in other linuxes. This is the full
list:
* precise aka Ubuntu 12.04
* trusty aka Ubuntu 14.04
* vivid aka Ubuntun 15.04
* wheezy aka Debian 7, the current debian oldstable distribution
* jessie aka Debian 8, the current debina stable distribution
* centos-6
* centos-7
* fedora-22
* oel-7

There is lots of documentation on how to do this in the TESTING.asciidoc.

Closes 
2015-08-10 05:31:42 -04:00