47 Commits

Author SHA1 Message Date
Ryan Ernst
9cb4c82c58 Build: Add fixture capabilities to integ tests
This change adds a Fixture class for use by gradle. A Fixture is an
external process that integration tests will use. It can be added as a
dependsOn for integTest, and will automatically be shutdown upon success
or failure, as well as relevant information dumped on failure. There is
also an example fixture in this change.
2015-12-19 15:46:21 -08:00
Ryan Ernst
6a99796b02 Build: Move test framework under a "test" top level dir
This allows adding more test projects, eg integ test fixtures that will
be coming soon.
2015-12-17 21:25:06 -08:00
Costin Leau
7bca97bba6 HDFS Snapshot/Restore plugin
Migrated from ES-Hadoop. Contains several improvements regarding:

* Security
Takes advantage of the pluggable security in ES 2.2 and uses that in order
to grant the necessary permissions to the Hadoop libs. It relies on a
dedicated DomainCombiner to grant permissions only when needed only to the
libraries installed in the plugin folder
Add security checks for SpecialPermission/scripting and provides out of
the box permissions for the latest Hadoop 1.x (1.2.1) and 2.x (2.7.1)

* Testing
Uses a customized Local FS to perform actual integration testing of the
Hadoop stack (and thus to make sure the proper permissions and ACC blocks
are in place) however without requiring extra permissions for testing.
If needed, a MiniDFS cluster is provided (though it requires extra
permissions to bind ports)
Provides a RestIT test

* Build system
Picks the build system used in ES (still Gradle)
2015-12-14 21:50:09 +02:00
Robert Muir
e454fadc22 Merge branch 'master' into shave_mustache 2015-12-10 07:58:24 -05:00
Jack Conradson
da5b07ae13 Added a new scripting language (PlanA).
Closes #15136
2015-12-09 16:32:37 -08:00
Robert Muir
2e299860e5 refactor mustache to lang-mustache plugin. No rest tests yet. 2015-12-08 22:45:21 -05:00
Ryan Ernst
caf77f7eea Add integ-test-zip distribution 2015-12-03 14:52:51 -08:00
Ryan Ernst
0a4a81afaf Added modules, distributions now include them (just plugins installed in
a diff dir)
2015-12-03 14:18:26 -08:00
Ryan Ernst
21b42e2a42 Fix xplugins to not omit the first level project name 2015-11-23 15:15:40 -08:00
Ryan Ernst
113a026e1f Change extra plugins to x-plugins directory instead of extra-plugin- prefix 2015-11-23 14:44:37 -08:00
Ryan Ernst
5a62282150 Build: Rework extra plugins support to be through sibling directories
This change removes the subdirectory support for extra-plugins, and
replaces it with an iteration of sibling directories with the prefix
"extra-plugin-".
2015-11-23 14:15:45 -08:00
Ryan Ernst
7d3da7e6ee Build: Get multi node smoke tests working
This change adds back the multi node smoke test, as well as making the
cluster formation for any test allow multiple nodes. The main changes in
cluster formation are abstracting out the node specific configuration to
a helper struct, as well as making a single wait task that waits for all
nodes after their start tasks have run. The output on failure was also
improved to log which node's info is being printed.
2015-11-22 14:49:05 -08:00
Ryan Ernst
b8e462b934 Build: Change project attachment into special extra-plugins dir
Currently we use the "gradle project attachment plugin" to support
building elasticsearch as part of another project. However, this plugin
has a number of issues, a large part of which is requiring consistent
use of the projectsPrefix.

This change removes projectsPrefix, and adds support for a special
extra-plugins directory in the root of elasticsearch. Any projects
checked out within this directory will be automatically added to
elasticsearch.
2015-11-22 08:44:33 -08:00
Robert Muir
a164586796 Merge branch 'mapper_migrate' into migrate-mapper-attachments 2015-11-08 02:19:50 -05:00
Ryan Ernst
ee691e547a Remove outdated projectsPrefix setup 2015-11-06 11:45:32 -08:00
Ryan Ernst
548430072f Build: Fix eclipse generation to add a core-tests projects
Eclipse does not have the ability to differentiate test dependencies
from main dependencies. This causes what looks like a circular
dependency through test-framework. This change sets up an additional
core-tests project for eclipse only, which removes this problem.
2015-11-06 11:41:18 -08:00
Nik Everett
047e3b9b69 Merge pull request #14531 from nik9000/gradle_vagrant
Gradle-ify vagrant tests
2015-11-05 14:29:06 -05:00
Nik Everett
1dd00dddd5 [test] Gradle-ify vagrant tests
This gets the tar and tar_plugins tests working in gradle. It does so by
adding a subproject, qa/vagrant, which adds the following tasks:

Verification
------------
checkPackages - Check the packages against a representative sample of the
                linux distributions we have in our Vagrantfile
checkPackagesAllDistros - Check the packages against all the linux
                          distributions we have in our Vagrantfile

Package Verification
--------------------
checkCentos6 - Run packaging tests against centos-6
checkCentos7 - Run packaging tests against centos-7
checkDebian8 - Run packaging tests against debian-8
checkFedora22 - Run packaging tests against fedora-22
checkOel7 - Run packaging tests against oel-7
checkOpensuse13 - Run packaging tests against opensuse-13
checkSles12 - Run packaging tests against sles-12
checkUbuntu1204 - Run packaging tests against ubuntu-1204
checkUbuntu1404 - Run packaging tests against ubuntu-1404
checkUbuntu1504 - Run packaging tests against ubuntu-1504

Vagrant
-------
smokeTestCentos6 - Smoke test the centos-6 VM
smokeTestCentos7 - Smoke test the centos-7 VM
smokeTestDebian8 - Smoke test the debian-8 VM
smokeTestFedora22 - Smoke test the fedora-22 VM
smokeTestOel7 - Smoke test the oel-7 VM
smokeTestOpensuse13 - Smoke test the opensuse-13 VM
smokeTestSles12 - Smoke test the sles-12 VM
smokeTestUbuntu1204 - Smoke test the ubuntu-1204 VM
smokeTestUbuntu1404 - Smoke test the ubuntu-1404 VM
smokeTestUbuntu1504 - Smoke test the ubuntu-1504 VM
vagrantHaltCentos6 - Shutdown the vagrant VM running centos-6
vagrantHaltCentos7 - Shutdown the vagrant VM running centos-7
vagrantHaltDebian8 - Shutdown the vagrant VM running debian-8
vagrantHaltFedora22 - Shutdown the vagrant VM running fedora-22
vagrantHaltOel7 - Shutdown the vagrant VM running oel-7
vagrantHaltOpensuse13 - Shutdown the vagrant VM running opensuse-13
vagrantHaltSles12 - Shutdown the vagrant VM running sles-12
vagrantHaltUbuntu1204 - Shutdown the vagrant VM running ubuntu-1204
vagrantHaltUbuntu1404 - Shutdown the vagrant VM running ubuntu-1404
vagrantHaltUbuntu1504 - Shutdown the vagrant VM running ubuntu-1504
vagrantSmokeTest - Smoke test some representative distros from the Vagrantfile
vagrantSmokeTestAllDistros - Smoke test all distros from the Vagrantfile
vagrantUpCentos6 - Startup a vagrant VM running centos-6
vagrantUpCentos7 - Startup a vagrant VM running centos-7
vagrantUpDebian8 - Startup a vagrant VM running debian-8
vagrantUpFedora22 - Startup a vagrant VM running fedora-22
vagrantUpOel7 - Startup a vagrant VM running oel-7
vagrantUpOpensuse13 - Startup a vagrant VM running opensuse-13
vagrantUpSles12 - Startup a vagrant VM running sles-12
vagrantUpUbuntu1204 - Startup a vagrant VM running ubuntu-1204
vagrantUpUbuntu1404 - Startup a vagrant VM running ubuntu-1404
vagrantUpUbuntu1504 - Startup a vagrant VM running ubuntu-1504

It does not make the "check" task depend on "checkPackages" so running the
vagrant tests is still optional. They are slow and depend on vagrant and
virtualbox.

The Package Verification tasks are useful for testing individual distros.

The Vagrant tasks are listed in `gradle tasks` primarily for discoverability.
2015-11-05 14:28:30 -05:00
Robert Muir
602feac915 move tests never running in jenkins to new evil tests module 2015-11-03 21:42:22 -05:00
Ryan Ernst
c86100f636 Switch build system to Gradle
See #13930
2015-10-29 11:40:19 -07:00
Shay Banon
bf0caafa04 remove old modules and gradle 2011-12-06 01:01:04 +02:00
Shay Banon
473b0f4dff Migrate built in plugins to their own repos, closes #1520. 2011-12-05 21:00:12 +02:00
kimchy
8d867dc24c explicitly set the cat under which jdk logging is logged 2011-02-18 05:00:08 +02:00
kimchy
503b023eac remove attempt to get ruby in scripting, problems embedding it... 2010-10-14 14:11:13 +02:00
kimchy
a754ebacd4 Python Plugin: Allow to use python for scripts, closes #418. 2010-10-09 20:59:41 +02:00
kimchy
c4d17860a1 Wikipedia River: A river to index wikipedia, closes #403. 2010-10-03 22:22:45 +02:00
kimchy
6371ed7d78 build structure for ruby support, problems in figuring out how to execute the ruby scripts in the most optimal manner 2010-10-03 15:52:44 +02:00
kimchy
5b8bc333bd JavaScript Plugin: Allow to use javascript for scripts, closes #401. 2010-10-03 02:20:37 +02:00
kimchy
efe5b5734c rename client-groovy to lang-groovy 2010-09-26 18:13:45 +02:00
kimchy
ecaaeb5250 River CouchDB Plugin, closes #382. 2010-09-23 02:26:12 +02:00
kimchy
aa68667d63 rabbitmq river 2010-09-21 17:32:34 +02:00
kimchy
4fe7cbccca rename indexer to river 2010-09-21 13:35:09 +02:00
kimchy
94a77b69d6 add a twitter indexer 2010-09-20 18:17:02 +02:00
kimchy
4f08801bb9 Thrift Plugin: Expose REST interface using thrift, closes #354. 2010-09-04 02:45:37 +03:00
kimchy
77b6d1d8b8 remove cloud plugin, reimplement an AWS specific cloud plugin with S3 gateway and EC2 discovery 2010-07-24 23:39:43 +03:00
kimchy
96d08cc641 remove the jgroups plugin 2010-06-21 22:51:54 +03:00
kimchy
636bfe8466 big refactoring thanks to proper jarjar built from source with asm 3.3 allowing to jarjar guice and others, includes package relocations 2010-06-15 16:51:38 +03:00
kimchy
5c6864e8b3 inital work on hadoop plugin - gateway hdfs support 2010-05-23 08:17:52 +03:00
kimchy
1323fe8b5e minor restructering of plugins, create the cloud plugin 2010-05-02 01:28:28 +03:00
kimchy
11e4ad9bd6 Analysis ICU Plugin, closes #151 2010-04-27 23:54:30 +03:00
kimchy
10f0eaad68 refactor plugins into sub groups 2010-04-25 23:01:00 +03:00
kimchy
b09f6c7b6b move client-groovy to an updated location 2010-04-25 22:20:18 +03:00
kimchy
6975f5d15a move zen discovery to be the default discovery, and move jgroups to a plugin 2010-04-25 21:52:28 +03:00
kimchy
51aac0cdf7 memcached initial work 2010-04-16 22:01:16 +03:00
kimchy
cb7e92b0f8 start work on the groovy plugin 2010-04-14 08:34:47 +03:00
kimchy
b799b7a9d7 plugins support + start work on attachments plugin 2010-03-27 23:43:59 +03:00
kimchy
b3337c3127 initial commit 2010-02-08 15:30:06 +02:00