Commit Graph

1445 Commits

Author SHA1 Message Date
Mark Vieira 214cbb28df
Fix for build runtime classpath instability (#45347)
(cherry picked from commit dee4ee2f0d4190ab54d0a4f0aa251d8c03e9db6d)
2019-08-08 12:41:17 -07:00
Mark Vieira 0ae103c40f
Avoid unnecessary eager creation of Gradle tasks (#45098) (#45310) 2019-08-08 10:50:09 -07:00
Alpar Torok b3076adae6 Fix reaper build failures on Windows (#45205)
We configure the service ID as the node's toString but this containes
characters that Windows doesn't like.
This PR fixes it by allowing only alphanumeric characters
2019-08-06 14:36:36 +03:00
Mark Vieira 87fb487299
Fix failed dependency resolution with external build-tools users (#45107) (#45219) 2019-08-05 15:52:46 -07:00
Jason Tedor 89b21102e5
Upgrade to JDK 12.0.2 (#45172)
This commit bumps the embedded JDK from 12.0.1 to 12.0.2.
2019-08-04 19:22:31 -04:00
Jason Tedor e708710f38
Simplify handling of git revision (#45175)
This commit simplifies the handling of git revision in the build. In
particular we remove pushing git revision through the generate build
info and print build info tasks as the git revision does not need to be
cached.
2019-08-04 11:09:11 -04:00
Jason Tedor 78aaf1b2d5
Lift build date to global build info (#45166)
This commit adds the build date to global build info mainly so that it
is accessible as an extension property.
2019-08-03 14:49:46 -04:00
Jason Tedor d78ecd9c09
Use the full hash in build info (#45163)
This commit switches to using the full hash to build into the JAR
manifest, which is used in node startup and the REST main action to
display the build hash.
2019-08-03 11:27:53 -04:00
Ryan Ernst 925bd4faa1
Use reaper process instead of shutdown hooks for testclusters (#44927)
Testclusters currently provides protection from clusters living past the
life of a build by adding a shutdown hook to java. While this works in
some cases, it does not cover all cases like where the daemon is killed
with SIGKILL.

To handle these other cases, this commit replaces the shutdown hooks with
a separate process (one per build) that manages reaping external services
if gradle dies.
2019-08-02 18:58:04 -07:00
Jason Tedor 5dc22e29c6
Add build hash to global build info (#45162)
This commit adds the commit hash to the global build info, and adds the
git revision as an extension. There are a couple motivations for this
change:
 - the current mechanism of getting the build hash does not work with
   git worktrees (because jgit does not understand them)
 - a follow-up will want to use the git revision when building the
   Docker images, so we want it available as an extension
 - it allows us to simplify our usage around the build hash as we no
   longer have to hack around silliness in the info-scm plugin

A follow-up will also stop using the short hash in the product build, so
that we use the full hash there. We already know that short hashes in
our codebase do collide, so we should move to the full hash to avoid
this problem.
2019-08-02 19:02:08 -04:00
Andrey Ershov 5a0bd696fc
Snapshot tool S3 cleanup 7.x backport (#44575)
Backport of #44551
2019-07-30 11:02:08 +02:00
Ignacio Vera 821f6f893b
Upgrade to Lucene 8.2.0 release (#44859) (#44892) 2019-07-26 08:14:59 +02:00
Mark Vieira aebfdf1477
Fix issue with Gradle daemons hanging indefinitely on shutdown (#44867) (#44878) 2019-07-25 15:33:51 -07:00
Ryan Ernst b3d8b39f29 Convert logging related gradle classes to java (#44771)
This commit converts the logging related classes (only used for vagrant)
to java from groovy.

relates #34459
2019-07-24 11:20:51 -07:00
Ryan Ernst 226a753e93
Restore setting up temp dir for windows service (#44541) (#44661)
In https://github.com/elastic/elasticsearch/pull/41913 setting up the
temp dir for ES was moved from the env script to individual cli scripts.
However, moving it to the windows service cli was missed. This commit
restores setting up the temp dir for the windows service control script.
2019-07-21 13:54:46 -07:00
Marios Trivyzas 8745d38dc8
Compile subset of build-tools classes with Java 8 target compatibility (#44560)
Backport of #43177 so that VersionProperties is Java 8 compatible and
can be used by https://github.com/elastic/elasticsearch-hadoop
to retrieve snapshot versions for Lucene.

(cherry picked from commit ec3ac9b62452f04ce44dea0a904a6e2b31dd8076)
2019-07-18 21:15:56 +03:00
Andrey Ershov ef6ddd15c6 Revert "Snapshot tool: S3 orphaned files
cleanup (#44551)"

This reverts commit 09edeeb3
2019-07-18 17:21:45 +02:00
Andrey Ershov 09edeeb38e Snapshot tool: S3 orphaned files cleanup (#44551)
A tool to work with snapshots.
Co-authored by @original-brownbear.
This commit adds snapshot tool and the single command cleanup, that
cleans up orphaned files for S3.
Snapshot tool lives in x-pack/snapshot-tool.

(cherry picked from commit fc4aed44dd975d83229561090f957a95cc76b287)
2019-07-18 16:38:00 +02:00
Alpar Torok 9f97319b3c Detect process third party audit being killed by OOM (#44269)
* Detect process third party audit being killed by OOM

It's very common for the third party audit to be killed by the OOM
killer when the system is running low on memory.
Since the forbidden APIs call is expected to fail, we were ignoring
these and incorrectly interpreting the partial output.
With this change we detect and provide a proper error message when this
happens.
2019-07-18 10:07:23 +03:00
Ignacio Vera eb348d2593
Upgrade to lucene-8.2.0-snapshot-6413aae226 (#44480) 2019-07-17 13:28:28 +02:00
david raistrick ae5a917efe Add clarification around TESTSETUP docs and error message (#43306) 2019-07-16 14:58:16 +02:00
William Brafford 673c63bb00 Restrict testCreateEmptyDirNoPermissions to Unix (#44282) (#44297)
The test EmptyDirTaskTests#testCreateEmptyDirNoPermissions may fail on
Windows. However, the test is only meaningful for Unix permissions
structures, so we should assume a Unix-family OS and skip the test on
Windows.

Fixes #44064
2019-07-16 08:48:12 +03:00
Ryan Ernst 7e06888bae
Convert testclusters to use distro download plugin (#44253) (#44362)
Test clusters currently has its own set of logic for dealing with
finding different versions of Elasticsearch, downloading them, and
extracting them. This commit converts testclusters to use the
DistributionDownloadPlugin.
2019-07-15 17:53:05 -07:00
Yogesh Gaikwad 91c342a888
fix and enable repository-hdfs secure tests (#44044) (#44199)
Due to recent changes are done for converting `repository-hdfs` to test
clusters (#41252), the `integTestSecure*` tasks did not depend on
`secureHdfsFixture` which when running would fail as the fixture
would not be available. This commit adds the dependency of the fixture
to the task.

The `secureHdfsFixture` is a `AntFixture` which is spawned a process.
Internally it waits for 30 seconds for the resources to be made available.
For my local machine, it took almost 45 seconds to be available so I have
added the wait time as an input to the `AntFixture` defaults to 30 seconds
 and set it to 60 seconds in case of secure hdfs fixture.

The integ test for secure hdfs was disabled for a long time and so
the changes done in #42090 to fix the tests are also done in this commit.
2019-07-12 12:44:01 +10:00
Mark Vieira 5698f4dc27
Pass tests.jvms system property to test tasks for maxParallelForks (#44237) 2019-07-11 12:41:32 -07:00
Mark Vieira 7a82106de6
Ignore test seed when flag is passed (#44234) 2019-07-11 12:30:21 -07:00
Nick Knize 374030a53f
Upgrade to lucene-8.2.0-snapshot-860e0be5378 (#44171) (#44184)
Upgrades lucene library to lucene-8.2.0-snapshot-860e0be5378
2019-07-11 09:17:22 -05:00
Alpar Torok 7ba18732f7 Run some REST tests against a cluster running in docker containers (#39515)
* Run REST tests against a cluster running on docker

Closes #38053
2019-07-11 15:28:33 +03:00
Alpar Torok 47ab2bda72 Improve how log is tailed in testclusters on failure (#40600)
* Improoce how log is tailed in testclusters on failure

- only print last few lines
- print all errors and warnings
- compact repeating errors and warnings
2019-07-11 15:04:08 +03:00
Mark Vieira 7c2e4b2857
[Backport] Enable caching of rest tests which use integ-test distribution (#44181) 2019-07-10 15:42:28 -07:00
Alpar Torok bde5802ad6 Test fixtures improovements (#43956)
* Test fixtures improovements

Don't disable some of the precommit tasks on fixtures.
This no longer makes sense now that a project can both produce and use a
fixture.

In order for this to be possible, had to add an additional configuration
to make JarHell class accessible to the task even if it's not a
dependency of the project and fix some of the third party audit fallout
from  #43671 which wasn't detected at the time due to the issue being
fixed here.

Closes #43918
2019-07-10 21:21:06 +03:00
Alpar Torok 0c8294e633 Make sure the clean task doesn't break test fixtures (#43641)
Use a dedicated fixture dir.
2019-07-08 17:58:27 +03:00
Adrien Grand 680edbe3f1
Bump current version to 7.4. (#43927) 2019-07-03 20:32:04 +02:00
Mark Vieira b5d47e5643
Upgrade to Gradle 5.5 (#43788) (#43832) 2019-07-01 11:54:58 -07:00
Alpar Torok b4303290d9 Testclusters: improove timeout handling (#43440) 2019-07-01 11:39:53 +03:00
Alpar Torok a38f509284 Testclusters: convert left-overs from checkPart1 (#43370)
* Testclusters: convert left-overs from checkPart1
2019-06-25 19:14:45 +03:00
Ryan Ernst 97cd417829 Distribution download plugin cleanup (#43461)
This commit addresses some post-review comments.

relates #43247
2019-06-24 09:34:53 -07:00
Alpar Torok b4f5882fbe Improove error on waiting for cluster to come up (#43416)
Always include all messages from exceptions to make the reason for
failure more visible.
2019-06-24 12:12:21 +03:00
Alpar Torok ea44da6069 Testclusters: conver remaining x-pack (#43335)
Convert x-pack tests
2019-06-24 12:07:42 +03:00
Alpar Torok 5a9c48369b TestClusters: Convert the security plugin (#43242)
* TestClusters: Convert the security plugin

This PR moves security tests to use TestClusters.
The TLS test required support in testclusters itself, so the correct
wait condition is configgured based on the cluster settings.

* PR review
2019-06-18 11:55:44 +03:00
Mark Vieira 74813360ab Fix inverted condition so we never cache rest integ tests 2019-06-18 07:33:19 +03:00
Ryan Ernst d0a4d23027 Add elasticsearch distribution plugin (#43247)
Several types of distributions are built and tested in elasticsearch,
ranging from the current version, to building or downloading snapshot or
released versions. Currently tests relying on these have to contain
logic deciding where and how to pull down these distributions.

This commit adds an distributiond download plugin for each project to
manage which versions and variants the project needs. It abstracts away
all need for knowing where a particular version comes from, like a local
distribution or bwc project, or pulling from the elastic download
service. This will be used in a followup PR by the testclusters and
vagrant tests.
2019-06-17 17:53:39 -07:00
Alpar Torok 023a126b57 Correctly configure testclsuters for fips (#43186)
Configuration was not being triggered for all projects with the way it
was previusly set up.
With this change we do it for each project directly.
2019-06-14 10:55:04 +03:00
Alpar Torok 4ba94a5051 Testclusters: convert ccr tests (#42313) 2019-06-13 19:19:36 +03:00
Alpar Torok 167e51335d Convert ILM tests to use testclusters (#43076)
Also improove the error message when bin scripts are not found
2019-06-13 12:24:48 +03:00
Vincent Boulaye 209a493b27 convert EmptyDirTask.groovy to .java (#34672) 2019-06-13 12:21:23 +03:00
Gabryel Monteiro 47d861c23a Converting DependencyLicensesTask and UpdateShasTask to java (#41921) 2019-06-13 12:15:51 +03:00
Rafiullah Hamedy a1b8d48f17 Convert PluginPropertiesExtension Groovy to Java (#39605) 2019-06-13 09:28:12 +03:00
Jason Tedor 0eeb080a72
Do not use runtime Java when starting BWC nodes (#43071)
When starting BWC nodes, it could be that runtime Java home is set. Yet,
runtime Java home can advance beyond what a BWC node might be compatible
with. For example, if runtime Java home is set to JDK 13 and we are
starting a 7.1.2 node, we do not have any guarantees that 7.1.2 is
compatible with JDK 13 (since we never did any work to make it so). This
will continue to be the case as JDK releases advance, but we still need
to test against BWC nodes. This commit stops applying runtime Java home
when starting a BWC node. Instead, we would use the bundled JDK.
2019-06-11 10:26:57 -04:00
James Rodewig 5913723788 [DOCS] Change `// TESTRESPONSE[_cat]` to `// TESTRESPONSE[non_json]` (#43006) 2019-06-10 09:53:05 -04:00