Commit Graph

140 Commits

Author SHA1 Message Date
Nick Knize 755367f1c9 [PURIFY] Remove x-pack feature flag from yaml test (#68)
This commit removes the xpack no_xpack feature flag from the yaml test suite.

Signed-off-by: Peter Nied <petern@amazon.com>
2021-03-13 10:36:13 -06:00
Harold Wang 96bda527b1 Remove license option in gradlew command (#41)
* Remove license option in gradlew command

* Remove "This username and password is part of trial license. Let's remove this too" from TESTING.asciidoc

Signed-off-by: Peter Nied <petern@amazon.com>
2021-03-13 10:36:10 -06:00
Mark Vieira f8f5d27f6b Add option to preserve data in test clusters (#65400)
(cherry picked from commit 1ce323e1368cf5231181f1efaba1c4e425066e37)
2020-11-24 11:56:56 -08:00
James Rodewig 68d72f1e01
[DOCS] Fix "the the" typos (#64344) (#64354) 2020-10-29 10:47:42 -04:00
Jake Landis 294f40de72
[7.xUpdate TESTING.asciidoc for recent REST test changes (#62841) (#62895)
* Remove reference to Runner (no longer valid)
* Remove tests.rest (no longer valid)
* Add reference to javaRestTest
* Brief mention of qa tests
2020-10-01 11:02:29 -05:00
Qaiser Abbasi e2400dcd8e
Remove legacy debugger instructions (#60583)
In the past, the only way to run a local Elasticsearch build with a remote debugger was by extracting elasticsearch and passing ES_JAVA_OPTS. However, since switching to gradle, a convenience flag was added, `--debug-jvm` (which is documented elsewhere in the testings docs), when running a local elasticsearch build through gradle. This commit removes the old documentation.
2020-08-21 14:08:12 -07:00
Lee Hinman 01fb7272bb Add blurb about --data-dir to TESTING.asciidoc (#58963)
This used to be -Dtests.es.path.data=/whatever but changed some time in the past, this updates it
so that it references the correct method.
2020-07-07 15:40:17 -06:00
Jake Landis 604c6dd528
7.x - Create plugin for yamlTest task (#56841) (#59090)
This commit creates a new Gradle plugin to provide a separate task name
and source set for running YAML based REST tests. The only project
converted to use the new plugin in this PR is distribution/archives/integ-test-zip.
For which the testing has been moved to :rest-api-spec since it makes the most
sense and it avoids a small but awkward change to the distribution plugin.

The remaining cases in modules, plugins, and x-pack will be handled in followups.

This plugin is distinctly different from the plugin introduced in #55896 since
the YAML REST tests are intended to be black box tests over HTTP. As such they
should not (by default) have access to the classpath for that which they are testing.

The YAML based REST tests will be moved to separate source sets (yamlRestTest).
The which source is the target for the test resources is dependent on if this
new plugin is applied. If it is not applied, it will default to the test source
set.

Further, this introduces a breaking change for plugin developers that
use the YAML testing framework. They will now need to either use the new source set
and matching task, or configure the rest resources to use the old "test" source set that
matches the old integTest task. (The former should be preferred).

As part of this change (which is also breaking for plugin developers) the
rest resources plugin has been removed from the build plugin and now requires
either explicit application or application via the new YAML REST test plugin.

Plugin developers should be able to fix the breaking changes to the YAML tests
by adding apply plugin: 'elasticsearch.yaml-rest-test' and moving the YAML tests
under a yamlRestTest folder (instead of test)
2020-07-06 14:16:26 -05:00
Ryan Ernst 33d6a55d1d
Create plugin for internalClusterTest task (#56067)
This commit creates a new gradle plugin to provide a separate task name
and source set for running ESIntegTestCase tests. The only project
converted to use the new plugin in this PR is server, as an example. The
remaining cases in x-pack will be handled in followups.

backport of #55896
2020-05-06 17:20:52 -07:00
Mark Vieira 3b2b564c91
Improve IntelliJ IDE integration (#53747)
This commit makes a number of improvements when importing the
Elasticsearch project into IntelliJ IDEA. Specifically:

- Contributing documentation has been updated to reflect that the
  'idea' task should no long be used and Gradle project import is
  instead the officially supported way of setting up the project.
- Attempts to run the 'idea' task will result in a failure with a
  message directing folks to our CONTRIBUTING.md document.
- The project JDK is explicit set rather that using whatever JAVA_HOME
  is.
- Gradle build operation delegation is disabled, and test execution is
  configured to 'choose per test'.
- Gradle is configured to inherit the project JDK.
- Some code style conventions are automatically configured.
- File encoding is explicitly set to UTF-8.
- Parallel module compilation is enabled and deprecated feature
  warnings are disabled.
- A remote debug run configuration using listen mode is created.
- JUnit runner is configured with required system properties.
- License headers are configured such that Apache 2 is the default
  notice added to all source files with exception of source in /x-pack
  which will use the Elastic license.
2020-03-19 11:43:33 -07:00
Jake Landis 710a9ead69
[7.x] Updates to TESTING.asciidoc for REST testing (#52862) (#52898)
Update documentation for:
* restResources config (related #52114)
* call out YAML vs. Java based Rest tests
* update example to use newer syntax 
* update example to target a test that is not skipped 
* provide example for bwcRest test (related #52383)
2020-02-27 12:45:02 -06:00
James Rodewig fc964643bd [DOCS] Add docs build info to TESTING.asciidoc (#52271)
Adds a brief section about Elasticsearch docs and how users can
test/build them locally.
2020-02-12 13:00:45 -05:00
Andrei Stefan 9a89ed891e Detail the IDEs options for configuring the debug step (#48507)
(cherry picked from commit 451d9cf06f1331ecfb6b4fa4acaed6fb43a1fcbb)
2019-10-25 17:27:48 +03:00
Marios Trivyzas 2751a4ff1b [Docs] Fix testing docs regarding --debug-jvm (#48293)
Remove extra whitespace and highlight the starting port number.

Follows: #48188
(cherry picked from commit b84b7e13c888c9a4b110c436b4eec1ffd0aebd52)
2019-10-21 15:21:18 +02:00
Alpar Torok 7d085ffbd9 Switch to debug with server=n (#48188)
Before this change one needed to re-start debugging several times, as we
launched multiple JVMs in debug mode.
With this option the IDE has the option to re-launch and listen for
connections again leading for to a more pleasant experience.
2019-10-21 15:07:04 +03:00
William Brafford ac1ed6e740 Update TESTING.asciidoc for new :qa:os test style (#48023)
* Update testing document for new packaging tests

The TESTING.asciidoc document had gotten out of date due to some new and
wonderful changes in our vagrant testing code. I've removed all of the
instructions that no longer work, and added working examples and descriptions
in their place.
2019-10-16 17:26:06 -04:00
Mark Vieira 529946aa15
Rename system property to change bwc checkout behavior (#45574) 2019-08-16 08:54:04 -07:00
Toby McLaughlin 81163455a8 Run packaging tests on RHEL 8 (#41662) 2019-05-02 09:23:12 +10:00
Jason Tedor 60000400db
Drop support for Ubuntu 14.04 (#40709)
With the 7.0.0 release, we are dropping support for Ubuntu 14.04. This
commit removes this OS from our testing infrastructure.
2019-04-02 08:51:40 -04:00
Ryan Ernst 8f09c77777 Add no-jdk distributions (#39882)
This commit adds a variant for every official distribution that omits
the bundled jdk. The "no-jdk" naming is conveyed through the package
classifier, alongside the platform. Package tests are also added for
each new distribution.
2019-03-15 00:55:57 -07:00
Adrien Grand b665d62192 Fix formatting of TESTING.asciidoc. 2019-03-04 08:35:58 +01:00
Adrien Grand 976f988358 Add guidance for writing tests. (#39318) 2019-03-01 15:14:21 +01:00
Lee Hinman cf0bdf3b28 Update TESTING.asciidoc with platform specific instructions (#38802)
This adds the instructions for building a platform-specific distribution.
2019-02-14 08:13:52 -07:00
Alpar Torok 745d0c1bba Add packaging tests for Fedora 29, remove 27 (#38726) 2019-02-12 14:31:17 +02:00
Daniel Mitterdorfer 1de286bfb3
Add a note how to benchmark Elasticsearch
With this commit we add a note that explains when to use benchmarks and
point the reader to the macrobenchmarking tool Rally for further
information.

Relates #37694
2019-01-23 12:15:00 +01:00
Ryan Ernst 9a34b20233
Simplify integ test distribution types (#37618)
The integ tests currently use the raw zip project name as the
distribution type. This commit simplifies this specification to be
"default" or "oss". Whether zip or tar is used should be an internal
implementation detail of the integ test setup, which can (in the future)
be platform specific.
2019-01-21 12:37:17 -08:00
Alpar Torok 45db829039
Replace fixtures with docker-compose (#35651)
Creates a new plugin to manage docker-compose based test fixtures. 
Convert the smb-fixture as a first example.
2018-11-29 09:43:16 +02:00
Alpar Torok fda59ff969
The default of tests.jvms should not be capped (#35259)
Default to the number of physical cores across all sockets instead.
2018-11-19 14:01:32 +02:00
Alpar Torok 598b0a0eb4
Upadte TESTING.ascidoc BWC section (#35126)
* Upadte BWC section
2018-11-02 06:34:29 +02:00
lipsill 4bda9bdab0 TESTING.asciidoc fix examples using forbidden annotation (#34515)
Clean up examples not to use forbidden test annotation `@Nightly`.
remove references to unused annotations
2018-10-23 12:36:42 +03:00
lipsill 305b5845eb Update TESTING.asciidoc title (#34401)
Display `Testing` as h1
2018-10-15 06:54:18 +01:00
Kazuhiro Sera d45fe43a68 Fix a variety of typos and misspelled words (#32792) 2018-10-03 18:11:38 +01:00
Jason Tedor 128ea1b7b5
Add Ubuntu 18.04 to packaging tests (#34139)
We intend to start supporting Ubuntu 18.04 with the 6.5.0 release. This
commit adds Ubuntu 18.04 to the packaging tests.
2018-09-28 17:33:29 -04:00
lipsill 6646bcb065 [DOCS] Fix list formatting in TESTING.asciidoc (#33889)
Fix the steps listed  in `Testing packaging` to
```
1. step1
2. step2
--------------------------------------
vagrant plugin install vagrant-cachier
--------------------------------------
3. step3
```
instead of
```
instead of:
1. step1
2. step2
--------------------------------------
vagrant plugin install vagrant-cachier
--------------------------------------
1. step3
```
2018-09-20 16:14:56 -04:00
Andy Bristol a1b538122c
[test] java tests for archive packaging (#30734)
Ports the first couple tests for archive distributions from the old bats
project to the new java project that includes windows platforms,
consolidating them into one test method that tests that the
distributions can be extracted and their contents verified. Includes the
zip distributions which were not tested in the bats project.
2018-05-23 10:37:57 -07:00
Alpar Torok 8b7ff69479
Remove fedora 26, add 28 (#30683)
* Remove fedora 26, add 28

Closes #30579

* Update testing docs with new fedora 28 image
2018-05-22 07:22:00 +03:00
Martijn van Groningen 33cba44d0a
[Build] Add test admin when starting gradle run with trial license and
also add more documentation around gradle run task. (#30671)
2018-05-18 14:11:11 +02:00
Andy Bristol 890afadb37
[test] packaging: add windows boxes (#30402)
Adds windows server 2012r2 and 2016 vagrant boxes to packaging tests.
They can only be used if IDs for their images are specified, which are
passed to gradle and then to vagrant via env variables. Adds options
to the project property `vagrant.boxes` to choose between linux and
windows boxes.

Bats tests are run only on linux boxes, and portable packaging tests run
on all boxes. Platform tests are only run on linux boxes since they are
not being maintained.

For #26741
2018-05-16 16:42:08 -07:00
Jason Tedor 03ce3dd4a4 Enable skipping fetching latest for BWC builds (#29497)
The BWC builds always fetch the latest from the elastic/elasticsearch
repository for the BWC branches. Yet, there are use-cases for using the
local checkout without fetching the latest. This commit enables these
use-cases by adding a tests.bwc.git.fetch.latest property to skip the
fetches.
2018-04-13 09:31:06 -04:00
Ryan Ernst a75a7d22b2
Build: Use branch specific refspec sysprop for bwc builds (#29299)
This commit changes the sysprop for overriding the branch bwc builds use
to be branch specific. There are 3 different bwc branches built, but all
of them currently read the exact same sysprop. For example, with this change
and current branches, you can now specify eg `-Dtests.bwc.refspec.6.x=my_6x`
and it will build only next-minor-snapshot with that branch, while
next-bugfix-snapshot will continue to use 5.6.
2018-03-29 11:59:52 -07:00
Jason Tedor 4417580d05
Remove leftover tests.rest.spec property from docs (#29279)
We previously had a property to specify the location of the REST test
spec files but this was removed in a previous refactoring yet left
behind in the docs. This commit removes the last remaining vestige of
this parameter.
2018-03-28 08:22:06 -04:00
Bart van Oort 67a6a76aad Docs: Update generating test coverage reports (#29255)
Old docs said to use maven. That doesn't work. We can't generate the
reports right now.
2018-03-27 17:29:19 -04:00
Andy Bristol 7bf9091942
[test] packaging: gradle tasks for groovy tests (#29046)
The vagrant test plugin adds tasks for the groovy packaging tests,
which run after the bats packaging test tasks.Rename the 'bats'
configuration to 'packaging' and remove the option to inherit
archives from this configuration.
2018-03-26 13:43:09 -07:00
Paul Sanwald 9f7cd14f15
Update Gradle Testing Docs (#28970)
Attempting to run the REST tests, I noticed the testing instructions
  in the `TESTING.asciidoc` were outdated, so I fixed the paths.
  Steps I took to test:

  * Ran `./gradlew :distribution:packages:rpm:assemble` and make sure
  RPM is created in `./distribution/packages/rpm/build/distributions/`

  * Ran testing commands and verified the REST tests ran.
2018-03-09 11:54:59 -08:00
Ivan Brusic 5f6ce1ae75 Highlight the "hidden" REST API test suite (#28381) 2018-02-09 15:05:19 -08:00
Jason Tedor 79a2c25387
Update testing docs to reflect Gradle wrapper
This commit updates the testing docs to reflect the usage of the Gradle
wrapper.

Relates #28107
2018-01-05 17:19:34 -05:00
Dimitrios Liappis d3e3bc8656
Tests: Add Fedora-27 to packaging tests
Replace Fedora-25 with Fedora-27 and fix old references in
`TESTING.asciidoc`.

Relates #27434
2017-11-20 16:59:00 +02:00
Jason Tedor bef3180146 Enable BWC testing against other remotes
This commit enables BWC testing against remotes on GitHub other than
elastic/elasticsearch.git.

Relates #26918
2017-10-07 13:40:18 -04:00
Dimitrios Liappis 5079afd0bb Tests: Add Fedora-26 to packaging tests
Add Fedora-26 to packaging tests. Don't retire Fedora-25 yet, as it
will be EOL, one month after Fedora-27 is out.

Relates #26726
2017-10-03 12:01:20 +03:00
Albert Zaharovits c1452ff9ea [DOC] macOS localhost resolve for integ tests 2017-08-23 12:02:57 +03:00