Commit Graph

33 Commits

Author SHA1 Message Date
Harold Wang a011c15bc8 Rename files under qa folder (#212)
* Rename directory elasticsearch to opensearch
Rename EvilElasticsearchCliTests EvilOpenSearchCliTests

Signed-off-by: Harold Wang <harowang@amazon.com>

* Rename org.elasticsearch to org.opensearch

Signed-off-by: Harold Wang <harowang@amazon.com>

* Rename waitForElasticsearch to waitForOpenSearch

Signed-off-by: Harold Wang <harowang@amazon.com>

* Rename OpensearchNode to OpenSearchNode

Signed-off-by: Harold Wang <harowang@amazon.com>

* Rename "elasticsearch.version" to "opensearch.version"

Signed-off-by: Harold Wang <harowang@amazon.com>

* Rename elasticsearchVersionString to opensearchVersionString

Signed-off-by: Harold Wang <harowang@amazon.com>

* Rename elasticsearch.yml to opensearch.yml

Signed-off-by: Harold Wang <harowang@amazon.com>

* Rename runElasticsearchTests to runOpenSearchTests

Signed-off-by: Harold Wang <harowang@amazon.com>

* Rename elasticsearchVersion to opensearchVersion

Signed-off-by: Harold Wang <harowang@amazon.com>

* Rename elasticsearch to opensearch in gradle files

Signed-off-by: Harold Wang <harowang@amazon.com>

* Rename ElasticsearchAssertions to OpenSearchAssertions

Signed-off-by: Harold Wang <harowang@amazon.com>

* Rename folder share/elasticsearch to share/opensearch

Signed-off-by: Harold Wang <harowang@amazon.com>

* Rename elasticsearch to opensearch

Signed-off-by: Harold Wang <harowang@amazon.com>

* Rename elasticsearch-service-x64 to opensearch-service-x64

Signed-off-by: Harold Wang <harowang@amazon.com>

* Rename elasticsearch-service.bat to opensearch-service.bat

Signed-off-by: Harold Wang <harowang@amazon.com>

* Rename Elasticsearch to Opensearch
Rename elasticsearch to opensearch

Signed-off-by: Harold Wang <harowang@amazon.com>

* Rename ELASTIC_PASSWORD_FILE to OPENSEARCH_PASSWORD_FILE

Signed-off-by: Harold Wang <harowang@amazon.com>

* Rename ELASTICSEARCH_PASSWORD to OPENSEARCH_PASSWORD
Rename elasticsearch to opensearch

Signed-off-by: Harold Wang <harowang@amazon.com>

* Rename elasticsearch to opensearch

Signed-off-by: Harold Wang <harowang@amazon.com>

* Rename elasticsearch.log to opensearch.log

Signed-off-by: Harold Wang <harowang@amazon.com>

* Rename es-repo to opensearch-repo

Signed-off-by: Harold Wang <harowang@amazon.com>

* Rename ESTestCase to OpenSearchTestCase

Signed-off-by: Harold Wang <harowang@amazon.com>

* Rename ESRestTestCase to OpenSearchRestTestCase

Signed-off-by: Harold Wang <harowang@amazon.com>

* Rename elasticsearch to opensearch
Rename "Starts ElasticSearch" to "Starts OpenSearch"

Signed-off-by: Harold Wang <harowang@amazon.com>

* Rename ESElasticsearchCliTestCase to BaseOpenSearchCliTestCase

Signed-off-by: Harold Wang <harowang@amazon.com>

* Rename "elasticsearch:test" to "opensearch:test"

Signed-off-by: Harold Wang <harowang@amazon.com>

* Rename test91ElasticsearchShardCliPackaging to test91OpenSearchShardCliPackaging

Signed-off-by: Harold Wang <harowang@amazon.com>

* Rename elasticsearch.toString to opensearch.toString

Signed-off-by: Harold Wang <harowang@amazon.com>

* Rename elasticsearch.pid to opensearch.pid

Signed-off-by: Harold Wang <harowang@amazon.com>

* Rename "Opensearch" to "OpenSearch"
Rename "elasticsearch" to "opensearch"

Signed-off-by: Harold Wang <harowang@amazon.com>

* Rename Elasticsearch to OpenSearch
Remove unecessary dot after opensearch.

Signed-off-by: Harold Wang <harowang@amazon.com>
2021-03-21 20:56:34 -05:00
Rene Groeschke f58ebe58ee
Use services for archive and file operations in tasks (#62968) (#63201)
Referencing a project instance during task execution is discouraged by
Gradle and should be avoided. E.g. It is incompatible with Gradles
incubating configuration cache. Instead there are services available to handle
archive and filesystem operations in task actions.

Brings us one step closer to #57918
2020-10-05 15:52:15 +02:00
Rene Groeschke bdd7347bbf
Merge test runner task into RestIntegTest (7.x backport) (#60600)
* Merge test runner task into RestIntegTest (#60261)
* Merge test runner task into RestIntegTest
* Reorganizing Standalone runner and RestIntegTest task
* Rework general test task configuration and extension
* Fix merge issues
* use former 7.x common test configuration
2020-08-04 14:46:32 +02:00
Rene Groeschke ed4b70190b
Replace immediate task creations by using task avoidance api (#60071) (#60504)
- Replace immediate task creations by using task avoidance api
- One step closer to #56610
- Still many tasks are created during configuration phase. Tackled in separate steps
2020-07-31 13:09:04 +02:00
Rene Groeschke 01e9126588
Remove deprecated usage of testCompile configuration (#57921) (#58083)
* Remove usage of deprecated testCompile configuration
* Replace testCompile usage by testImplementation
* Make testImplementation non transitive by default (as we did for testCompile)
* Update CONTRIBUTING about using testImplementation for test dependencies
* Fail on testCompile configuration usage
2020-06-14 22:30:44 +02:00
Rene Groeschke c29bc87040
Move bwcVersions extension property to BuildParams (back port) (#56381)
* Move bwcVersions extension property to BuildParams (#56206)
* Fix :qa Task Using Broken BwC Versions Resolution (#56332)

Co-authored-by: Armin Braun <me@obrown.io>
2020-05-11 09:39:13 +02:00
Mark Vieira dd73a14d11
Improve total build configuration time (#54611) (#54994)
This commit includes a number of changes to reduce overall build
configuration time. These optimizations include:

- Removing the usage of the 'nebula.info-scm' plugin. This plugin
   leverages jgit to load read various pieces of VCS information. This
   is mostly overkill and we have our own minimal implementation for
   determining the current commit id.
- Removing unnecessary build dependencies such as perforce and jgit
   now that we don't need them. This reduces our classpath considerably.
- Expanding the usage lazy task creation, particularly in our
   distribution projects. The archives and packages projects create
   lots of tasks with very complex configuration. Avoiding the creation
   of these tasks at configuration time gives us a nice boost.
2020-04-08 16:47:02 -07:00
Mark Vieira 4b528d97ad
Consolidate duplication of BWC testing task setup in script plugin (#53079)
(cherry picked from commit 33fc8e7ebfac8d47a5f9f026b3836bb47bea141a)
2020-03-03 14:43:02 -08:00
Mark Vieira 8d2370bf00
Always use bundled JDK for external cluster nodes when BWC testing (#51505) (#51701) 2020-01-30 14:35:43 -08:00
Nhat Nguyen 69ef9b05cd Increase shard inactive time to 1h in upgrade tests (#51651)
testRecovery relies on the fact that shards are not flushed on inactive. 
Our CI recently was too slow. It took more than 20 minutes to complete
the full cluster restart suite. This slowness caused some shards of
testRecovery were flushed on inactive.

This commit increases the inactive time to 1h to reduce this noise.

Closes #51640
2020-01-30 14:48:56 -05:00
Rory Hunter c46a0e8708
Apply 2-space indent to all gradle scripts (#49071)
Backport of #48849. Update `.editorconfig` to make the Java settings the
default for all files, and then apply a 2-space indent to all `*.gradle`
files. Then reformat all the files.
2019-11-14 11:01:23 +00:00
Mark Vieira 6ab4645f4e
[7.x] Introduce type-safe and consistent pattern for handling build globals (#48818)
This commit introduces a consistent, and type-safe manner for handling
global build parameters through out our build logic. Primarily this
replaces the existing usages of extra properties with static accessors.
It also introduces and explicit API for initialization and mutation of
any such parameters, as well as better error handling for uninitialized
or eager access of parameter values.

Closes #42042
2019-11-01 11:33:11 -07:00
Alpar Torok 2b16d7bcf8
Backport testclusters all (#47565)
* Bwc testclusters all (#46265)

Convert all bwc projects to testclusters

* Fix bwc versions config

* WIP fix rolling upgrade

* Fix bwc tests on old versions

* Fix rolling upgrade
2019-10-04 16:12:53 +03:00
Alpar Torok 67bf3a4f51 Fix default distro bwc tests 2019-10-04 09:44:17 +03:00
Alpar Torok a032f9b2d5
Backport testclusters fix bwc (#47363)
* Add support for bwc for testclusters and convert full cluster restart (#45374)

* Testclusters fix bwc (#46740)

Additions to make testclsuters work with lather versions of ES

* Do common node config on bwc tests

Before this PR we always ever ran `ElasticsearchCluster.start` once, and
the common node config was never done.
This becomes apparent in upgrading from `6.x` to `7.x` as the new config
is missing preventing the cluster from starting.

* Do common node config on bwc tests

Before this PR we always ever ran `ElasticsearchCluster.start` once, and
the common node config was never done.
This becomes apparent in upgrading from `6.x` to `7.x` as the new config
is missing preventing the cluster from starting.

* Fix logic to pick up snapshot from 6.x

* Make sure ports are cleared

* Fix test

* Don't clear all the config as we rely on it

* Fix removal of keys
2019-10-02 14:37:00 +03:00
Mark Vieira 1287c7d91f
[Backport] Replace usages RandomizedTestingTask with built-in Gradle Test (#40978) (#40993)
* Replace usages RandomizedTestingTask with built-in Gradle Test (#40978)

This commit replaces the existing RandomizedTestingTask and supporting code with Gradle's built-in JUnit support via the Test task type. Additionally, the previous workaround to disable all tasks named "test" and create new unit testing tasks named "unitTest" has been removed such that the "test" task now runs unit tests as per the normal Gradle Java plugin conventions.

(cherry picked from commit 323f312bbc829a63056a79ebe45adced5099f6e6)

* Fix forking JVM runner

* Don't bump shadow plugin version
2019-04-09 11:52:50 -07:00
Mark Vieira 2569fb60de Avoid sharing source directories as it breaks intellij (#40877)
* Avoid sharing source directories as it breaks intellij
* Subprojects share main project output classes directory
* Fix jar hell
* Fix sql security with ssl integ tests
* Relax dependency ordering rule so we don't explode on cycles
2019-04-08 17:26:46 +03:00
Alpar Torok bd4ca4c702 Rename integTest to bwcTestSample for bwc test projects (#38433)
* Rename integTest to bwcTestSample for bwc test projects

This change renames the `integTest` task to `bwcTestSample` for projects
testing bwc to make it possible to run all the bwc tests that check
would run without running on bwc tests.

This change makes it possible to add a new PR check on backports to make
sure these don't break BWC tests in master.

* Rename task as per PR
2019-02-11 15:05:16 +02:00
Martijn van Groningen 445db97867
each full cluster restart round should use its own repository,
otherwise snapshots from e.g. 6.5.5 to current and 6.6.0 to current
full cluster restart round collides.
2019-01-28 13:56:02 +01:00
Armin Braun 31c33fdb9b
MINOR: Remove some Deadcode in Gradle (#37160) 2019-01-07 09:21:25 +01:00
Alpar Torok e9ef5bdce8
Converting randomized testing to create a separate unitTest task instead of replacing the builtin test task (#36311)
- Create a separate unitTest task instead of Gradle's built in 
- convert all configuration to use the new task 
- the  built in task is now disabled
2018-12-19 08:25:20 +02:00
Alpar Torok f8378d91a6
Implement VersionCollection in Java (#34050) 2018-11-01 17:43:57 +02:00
Jason Tedor 9f8dff9281
Remove debug logging in full cluster restart tests (#33612)
These logs are incredibly verbose, and it makes chasing normal failures
burdensome. This commit removes the debug logging, which can be
reenabled again if needed.
2018-09-11 19:11:30 -04:00
Ryan Ernst 3a64e6d121
Test: Remove specifying zip distribution in qa tests (#28805)
Applying the rest test gradle plugin already uses the zip distribution
by default, so specifying it explicitly is not necessary. These are
leftovers from before zip was the default for rest tests.
2018-02-23 13:45:38 -08:00
Michael Basnight e0bea70070
Generalize BWC logic (#28505)
Generalizing BWC building so that there is less code to modify for a release. This ensures we do not
need to think about what major or minor version is in the gradle code. It follows the general rules of the
elastic release structure. For more information on the rules, see the VersionCollection's javadoc.

This also removes the additional bwc snapshots that will never be released, such as 6.0.2, which were
being built and tested against every time we ran bwc tests.

Additionally, it creates 4 new projects that correspond to the different types of snapshots that may exist
for a given version. Its possible to now run those individual tasks to work out bwc logic whereas
previously it was impossible and the entire suite of bwc tests had to be run to work out any logic
changes in the build tools' bwc project. Please note that if the project does not make sense for the 
version that is current, that an error will be thrown from that individual project if an attempt is made to 
run it.

This should allow for automating the version bumps as well, since it removes all the hardcoded version
logic from the configs.
2018-02-09 14:55:10 -06:00
David Turner 89ba8996c6 Consolidate version numbering semantics (#27397)
Fixes to the build system, particularly around BWC testing, and to make future
version bumps less painful.
2017-11-23 20:21:53 +00:00
Boaz Leskes c342cdeab5 Setup debug logging for qa.full-cluster-restart 2017-10-07 23:37:09 +02:00
Boaz Leskes 2d409a912f full-cluster-restart tests: prevent shards from going inactive
FullClusterRestartIT.testRecovery relies on the translogs not being flushed
2017-10-05 10:08:10 +02:00
Ryan Ernst 42e8940a3d Build: Ensure build metadata is written (#26427)
This commit adds writing build metadata to the `check` command for each
bwc project. This ensures the files will be written if doing a general
`gradle check`, which is what CI intake jobs do. In later jobs like
bwcTest, the extra bwc-release-snapshot info is needed.

Note this commit also has a little cleanup of the output for the bwc
checkout, as it was plastering a git warning, instead of the real info
we care about (the refspec and commit that were used).
2017-08-30 07:26:33 -07:00
Martijn van Groningen bcaa413b0b
test: Port the remaining old indices search tests to full cluster restart qa module
Also tweaked the qa module's gradle file to actually run bwc tests against all index compat versions.

Relates to #24939
2017-06-19 12:27:24 +02:00
Ryan Ernst 106e373412 Build: Add master flag for disabling bwc tests (#25230)
This commit adds a gradle project, set inside the root build.gradle,
which controls all our bwc tests. This allows for seamless (ie no errant
CI failures) backporting of behavior.
2017-06-14 22:01:49 -07:00
Nik Everett 66007078d4 Build: Allow preserving shared dir (#24962)
This adds an option to `ClusterConfiguration` to preserve the
`shared` directory when starting up a new cluster and switches
the `qa:full-cluster-restart` tests to use it rather than
disable the clean shared task.

Relates to #24846
2017-06-04 20:40:25 -04:00
Nik Everett e072cc7770 Begin replacing static index tests with full restart tests (#24846)
These tests spin up two nodes of an older version of Elasticsearch,
create some stuff, shut down the nodes, start the current version,
and verify that the created stuff works.

You can run `gradle qa:full-cluster-restart:check` to run these
tests against the head of the previous branch of Elasticsearch
(5.x for master, 5.4 for 5.x, etc) or you can run
`gradle qa:full-cluster-restart:bwcTest` to run this test against
all "index compatible" versions, one after the other. For master
this is every released version in the 5.x.y version *and* the tip
of the 5.x branch.

I'd love to add more to these tests in the future but these
currently just cover the functionality of the `create_bwc_index.py`
script and start to cover the assertions in the
`OldIndexBackwardsCompatibilityIT` test.
2017-05-26 14:07:48 -04:00