Commit Graph

91 Commits

Author SHA1 Message Date
Colin Goodheart-Smithe c93b39c65b
Adds version 7.4.1 2019-10-01 16:03:11 +01:00
Mark Vieira 45605cfd7a
Use VAULT_TOKEN environment variable if it exists (#45525)
(cherry picked from commit b57f2ff68049a4d86ab37399531616b7ad26cfd9)
2019-09-30 15:16:05 -07:00
Alpar Torok f4e32a5f5f
Disable the use of artifactory in CI (#47100)
(cherry picked from commit cbe8b3645f74da0a7e2794536d84a3c46905e49e)
2019-09-30 15:15:55 -07:00
Alpar Torok 86d216cc9c Update .ci/bwcVersions 2019-09-19 09:35:01 +03:00
Mark Vieira 972d3569c1
Disable local build cache in CI (#46505)
(cherry picked from commit 4dcc226e142057ec44b313edf883e1c5401ec610)
2019-09-10 08:33:00 -07:00
Paul Sanwald 758680c549
version bump to 6.8.4 (#46409) 2019-09-05 15:14:36 -04:00
Paul Sanwald 8bdbc7d9bf
Bump version from 7.4 to 7.5 (#46142) 2019-08-29 15:03:26 -04:00
Jason Tedor d9be906afb
Start testing against AdoptOpenJDK (#45666)
This commit adds AdoptOpenJDK to the testing matrix.
2019-08-29 08:56:21 -04:00
Jason Tedor e38289b94c
Enable testing against JDK 14 (#45178)
This commit enables testing against JDK 14.
2019-08-22 19:35:24 -04:00
Jake Landis 3af36008bd
update bwcVersions 2019-08-22 14:20:34 -05:00
Mark Vieira b5c3c9767c
Publish CI build scans to Gradle Enterprise (#45249)
(cherry picked from commit 957a232afc3ecd43326509043ab73cc6c2a26411)
2019-08-09 15:53:50 -07:00
Mark Vieira fca458f1c8
Use system properties for build cache configuration (#45295) 2019-08-07 13:14:54 -07:00
Alpar Torok bc60eebfc8 Add retries to vault access in init scripts 2019-08-06 15:39:36 +03:00
Alpar Torok 81ea08b8ca Update ci bwc versions 2019-08-06 11:45:28 +03:00
Alpar Torok f3570aa27b CI specific init script updates (#45016)
- Add a vault integration so that we don't need Jenkins to do that for
us
   - This will make it easier to enable for windows too
- Move everything to a single file so we can read other secrets in the
same way
2019-08-05 18:27:10 +03:00
Alpar Torok 3a199d044c Add a list of BWC versions for CI (#44418)
This PR adds a list of index compatible versions to the `.ci` directory
as well as a way to generate and verify it.

Unfortunetly there is no easy way in Jenkins to have the build generate then
consume this YML axis config.
I hate that this would need maintenance on versions bumps, but the
potential benefir here is reducing the bwc builds that can take more than
24 hours to less than 20 minutes.
This is possible because the CI setup would use a matrix job to run
something like:
```
./graldew 'v7.0.0#bwcTest'
```
For every index compatible version.
On top of that `--parallel` should be possible even without testclusters
due to the limited number of clusters being set up here.

The example command above runs in exactly 10 minutes on my laptop,
thus I'm proposing to accept this compromise while we work out the
infra to do this more dinamically.
2019-07-17 17:44:49 +03:00
Alpar Torok eb0c7196cb Disable Gradle running in parallel 2019-07-01 18:18:38 +03:00
Jason Tedor 0a99019259
Disable building on JDK 13 in CI
This commit disables building on JDK 13 in CI. The reason for this is
because Gradle is not yet ready to run on JDK 13. We could re-introduce
infrastructure to enable Gralde to run on a different JDK than the build
JDK, but rather than introducing such complexity we will instead wait
for Gradle to be ready to run on JDK 13.
2019-06-05 09:55:37 -04:00
Jason Tedor 78be3dde25
Enable testing against JDK 13 EA builds (#40829)
This commit adds JDK 13 to the CI rotation for testing. For now, we will
be testing against JDK 13 EA builds.
2019-06-04 20:54:24 -04:00
Alpar Torok 4dbf6c0df9 Make packer cache branches explicit (#41990)
Before this change we would recurse to cache bwc versions.
This proved to be problematic due to  the number of steps it was
generating taking too long.
Also this required tricky maintenance to break the recursion for old
branches we don't really care about.

With this change we now cache specific branches only.
2019-05-27 09:46:43 +03:00
Mark Vieira 200b7e1888
Gradle init script for enabling remote build cache 2019-05-23 21:00:08 -07:00
Alpar Torok 7841fa4814 Add corretto 11 and 8 to runtime javas (#41946)
We now have coretto on the ephemeral CI images so we can add it to our
regular java test matrix.
On backporting will also add  corretto8 to branches that support it.
2019-05-08 16:35:46 +03:00
Jason Tedor d7fd51a84e
Provide names for all artifact repositories (#41857)
This commit adds a name for each Maven and Ivy repository used in the
build.
2019-05-07 06:35:28 -04:00
Jason Tedor cbdb20faba
Add more Java 12 distributions to testing (#40482)
This commit adds the Oracle JDK 12 and the Zulu JDK 12 distributions to
testing.
2019-03-26 18:48:43 -04:00
Alpar Torok 2b34923541 Add Gradle init script to configure artifactoy (#40412)
The script will be use in CI to configure our own Artifactory instance
so CI does not depend on external network resources nor do we generate
excessive load on these resources.
In case the repo is not accessible Gradle should fall back to using the
public ones.
2019-03-26 12:23:55 +02:00
Jason Tedor 3212557da4
Require JDK 12 for compilation (#40231)
This commit switches from requiring JDK 11 for compilation to requiring
JDK 12 for compilation.
2019-03-22 08:33:23 -04:00
Alpar Torok 818d8951b8 Make pullFixture a task dependency of resolveAllDependencies (#38956)
* Make pullFixture a task dependency of resolveAllDependencies

With this change we will pull the docker test fixtures ( and thus cache
the images ) for older versions too as the `resolveAllDependencies` is
already being called on the bwc checkouts too.
2019-02-18 14:13:40 +02:00
Alpar Torok f3379940c6
Fix the packer cache script (#38023)
The script is used to create a cache on ephemeral CI workers.
Changes:
    - create and use a `pullFixture` task that always exists regardless
    of docker support
    - wire dependencies correctly so any pre fixture setup runs for pull
    as well
    - set up java env vars so bwc versions can build
2019-01-30 14:41:35 +02:00
Jason Tedor b6f59ffd71
Add JDK 12 to CI rotation (#36915)
Now that JDK 12 has entered rampdown phase one, it is time for us to add
JDK 12 to the CI rotation. This commit adds matrix parameters such that
we try to build with JDK 12, and we test running on JDK 12.
2018-12-20 20:47:56 -05:00
Jason Tedor c556b9ef19
Try running CI against Zulu (#36391)
This commit adds Zulu to the CI rotation. This commit only means that we
are going to try running CI against Zulu, it does not carry any
implications of support in the past, now, or in the future.
2018-12-11 10:48:14 -05:00
Alpar Torok 45db829039
Replace fixtures with docker-compose (#35651)
Creates a new plugin to manage docker-compose based test fixtures. 
Convert the smb-fixture as a first example.
2018-11-29 09:43:16 +02:00
Alpar Torok 9a319ec412
Remove ILM specific gradle extra args (#35296) 2018-11-07 14:22:00 +02:00
Colin Goodheart-Smithe 7c7605d3d2
Merge branch 'master' into index-lifecycle 2018-10-17 09:29:06 +01:00
Alpar Torok 2e5e4e1a69
Switch to parametric CI jobs (#34179)
Switch to parametric CI jobs. Given the changes in CI this also switches to running Gradle with the build java version only ( we used to also run it with run-time java version )
2018-10-17 07:57:30 +03:00
Lee Hinman 90c55f5e36 Merge remote-tracking branch 'origin/master' into index-lifecycle 2018-10-03 09:11:28 -06:00
Jason Tedor 86642d29e5
Require JDK 11 for compilation (#34103)
Now that JDK 11 is GA, we would switch our 6.x and master branches to
the JDK 11 compiler. This commit makes this change, as well as removes
JDK 10 from the CI configuration.
2018-10-03 08:59:00 -04:00
Alpar Torok 3afb8e8e66
Add Gradle attributes to CI jobs (#34178)
Maintain the additional arguments in the branch so CI doesn't have to care.
2018-10-02 17:11:01 +03:00
Jason Tedor 92a48fa6b8
Add script to cache dependencies (#33726)
With the introduction of immutable workers into our CI, we now have a
problem where we would have to download dependencies on every single
build. To this end our infrastructure team has introduced the
possibility to download dependencies one time and then bake these into
the base immutable worker image. For this, we introduce our version of
this special script which runs a task that downloads all dependencies of
all configurations. With this script, our infrastructure team will be
rebuilding these images on an at-least daily basis. This helps us avoid
having to download the dependencies for every single build.
2018-09-14 16:14:03 -04:00
Ioannis Kakavas d0fd23669a
Enable FIPS JVM in CI (#32330)
Now that #31666 and #31989 are merged we can run our tests in
fips JVM. This commits enables us to run tests on a Java 8
JVM using BouncyCastleFIPS as a security Provider.
2018-07-25 08:48:19 +03:00
Alpar Torok cf2295b408
Add JDK11 support and enable in CI (#31644)
* Upgrade bouncycastle

Required to fix
`bcprov-jdk15on-1.55.jar; invalid manifest format `
on jdk 11

* Downgrade bouncycastle to avoid invalid manifest

* Add checksum for new jars

* Update tika permissions for jdk 11

* Mute test failing on jdk 11

* Add JDK11 to CI

* Thread#stop(Throwable) was removed

http://mail.openjdk.java.net/pipermail/core-libs-dev/2018-June/053536.html

* Disable failing tests #31456

* Temprorarily disable doc tests

To see if there are other failures on JDK11

* Only blacklist specific doc tests

* Disable only failing tests in ingest attachment plugin

* Mute failing HDFS tests #31498

* Mute failing lang-painless tests #31500

* Fix backwards compatability builds

Fix JAVA version to 10 for ES 6.3

* Add 6.x to bwx -> java10

* Prefix out and err from buildBwcVersion for readability

```
> Task :distribution:bwc:next-bugfix-snapshot:buildBwcVersion
  [bwc] :buildSrc:compileJava
  [bwc] WARNING: An illegal reflective access operation has occurred
  [bwc] WARNING: Illegal reflective access by org.codehaus.groovy.reflection.CachedClass (file:/home/alpar/.gradle/wrapper/dists/gradle-4.5-all/cg9lyzfg3iwv6fa00os9gcgj4/gradle-4.5/lib/groovy-all-2.4.12.jar) to method java.lang.Object.finalize()
  [bwc] WARNING: Please consider reporting this to the maintainers of org.codehaus.groovy.reflection.CachedClass
  [bwc] WARNING: Use --illegal-access=warn to enable warnings of further illegal reflective access operations
  [bwc] WARNING: All illegal access operations will be denied in a future release
  [bwc] :buildSrc:compileGroovy
  [bwc] :buildSrc:writeVersionProperties
  [bwc] :buildSrc:processResources
  [bwc] :buildSrc:classes
  [bwc] :buildSrc:jar

```

* Also set RUNTIME_JAVA_HOME for bwcBuild

So that we can make sure it's not too new for the build to understand.

* Align bouncycastle dependency

* fix painles array tets

closes #31500

* Update jar checksums

* Keep 8/10 runtime/compile untill consensus builds on 11

* Only skip failing tests if running on Java 11

* Failures are dependent of compile java version not runtime

* Condition doc test exceptions on compiler java version as well

* Disable hdfs tests based on runtime java

* Set runtime java to minimum supported for bwc

* PR review

* Add comment with ticket for forbidden apis
2018-07-05 03:24:01 +00:00
David Leatherman 1172b3b31b
Java versions for ci (#29320)
* Add test matrix axis files for periodic java testing

* Add properties file defining java versions to use

* We have no openjdk8

* Remove openjdk

Oracle Java and OpenJDK basically only differ in license, so we don't
need to test both.
2018-04-02 14:24:25 -04:00