1725 Commits

Author SHA1 Message Date
Nick Knize
88bc268e29
[Rename] ESTestCase stragglers to OpenSearchTestCase (#3053)
A few places still referenced legacy ESTestCase name. This refactors those
instances to OpenSearchTestCase.

Signed-off-by: Nicholas Walter Knize <nknize@apache.org>
2022-04-23 16:50:55 -05:00
Marc Handalian
ca9151fa24
Add Github Workflow to build and publish lucene snapshots. (#2906)
This change introduces a github workflow so that we can build and push snapshots of lucene.
The RepositoriesSetupPlugin is also updated with a url from where these snapshots can be retrieved.

Signed-off-by: Marc Handalian <handalm@amazon.com>
2022-04-15 15:09:33 -05:00
Owais Kazi
3c5d997a76
Added a new line linter (#2875)
* Added linter to add new line

Signed-off-by: Owais Kazi <owaiskazi19@gmail.com>

* Fixed new lines

Signed-off-by: Owais Kazi <owaiskazi19@gmail.com>

* Ignore empty files

Signed-off-by: Owais Kazi <owaiskazi19@gmail.com>

* Updated DEVELOPER GUIDE

Signed-off-by: Owais Kazi <owaiskazi19@gmail.com>

* Renamed workflow file

Signed-off-by: Owais Kazi <owaiskazi19@gmail.com>

* Fixed failing tests

Signed-off-by: Owais Kazi <owaiskazi19@gmail.com>
2022-04-13 14:14:18 -04:00
Marc Handalian
21f9950806
Update ThirdPartyAuditTask to check for and list pointless exclusions. (#2760)
This change swaps the order of the task to first check for no pointless exclusions.
If not caught first these will be thrown inside of the bogusExcludesCount block that logs a
useless error message.

Signed-off-by: Marc Handalian <handalm@amazon.com>
2022-04-04 23:18:32 -05:00
Nick Knize
cb238aae61
Bump version to 3.0.0 (#2556)
Bump version to 3.0.0

Signed-off-by: Nicholas Walter Knize <nknize@apache.org>
2022-04-04 17:40:31 -05:00
Andriy Redko
223efe68e6
Update Gradle to 7.4.1 (#2078)
* Update Gradle to 7.4.1

Signed-off-by: Andriy Redko <andriy.redko@aiven.io>

* Address code review comments, added @PathSensitive(PathSensitivity.RELATIVE) where applicable

Signed-off-by: Andriy Redko <andriy.redko@aiven.io>
2022-03-29 12:23:08 -04:00
dependabot[bot]
8f4aec109d
Bump forbiddenapis in /buildSrc/src/testKit/thirdPartyAudit (#2611)
Bumps [forbiddenapis](https://github.com/policeman-tools/forbidden-apis) from 3.2 to 3.3.
- [Release notes](https://github.com/policeman-tools/forbidden-apis/releases)
- [Commits](https://github.com/policeman-tools/forbidden-apis/compare/3.2...3.3)

---
updated-dependencies:
- dependency-name: de.thetaphi:forbiddenapis
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>

Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2022-03-28 13:13:49 -07:00
Tianli Feng
19eadb46ba
Deprecate setting 'cluster.initial_master_nodes' and introduce the alternative setting 'cluster.initial_cluster_manager_nodes' (#2463)
* Deprecate setting cluster.initial_master_nodes, and add setting cluster.initial_cluster_manager_nodes

Signed-off-by: Tianli Feng <ftianli@amazon.com>
2022-03-18 14:55:49 -07:00
Nick Knize
05a5819243
[Upgrade] Lucene 9.1.0-snapshot-ea989fe8f30 (#2487)
* [Upgrade] Lucene 9.1.0-snapshot-ea989fe8f30

Upgrades from Lucene 9.0.0 to 9.1.0-snapshot-ea989fe8f30 in preparation for
9.1.0 GA.

Signed-off-by: Nicholas Walter Knize <nknize@apache.org>

* Add spanishplural token filter

Signed-off-by: Nicholas Walter Knize <nknize@apache.org>

* fix KNOWN_TOKENIZERS

Signed-off-by: Nicholas Walter Knize <nknize@apache.org>
2022-03-16 15:47:25 -04:00
Rishikesh Pasham
10b9986e12
Override Default Distribution Download Url with Custom Distribution Url when it is passed from Plugin (#2420)
* Override default Distribution Download URL with custom Distribution URL

Signed-off-by: Rishikesh1159 <rishireddy1159@gmail.com>

* Accidently made commit to main branch, this revives it.Override default Distribution Download URL with custom Distribution URL

Signed-off-by: Rishikesh1159 <rishireddy1159@gmail.com>

* Override Default DistributionDownloadUrl with customDistribution Url passed from Plugins

Signed-off-by: Rishikesh1159 <rishireddy1159@gmail.com>
2022-03-15 14:01:28 -07:00
Nick Knize
006c832c5f
[Upgrade] Lucene 9.0.0 release (#1109)
This commit upgrades the core codebase from Lucene 8.10.1 to
lucene 9.0.0. It includes all necessary refactoring of features and
API changes when upgrading to a new major Lucene release.

Signed-off-by: Nicholas Walter Knize <nknize@apache.org>
Co-authored-by: Andriy Redko <drreta@gmail.com>
2022-03-15 15:48:13 -05:00
Nick Knize
5a9a11416d
[Remove] TrimUnsafeCommit logic for legacy 6.x indexes (#2225)
* [Remove] TrimUnsafeCommit logic for legacy 6.x indexes

Multiple txlog commits was introduced in legacy 7.x. Legacy 6.x indexes could
therefore not have a safe commit. Since OpenSearch 2.0 is no longer compatible
with legacy 6.x indexes, the logic to trim these unsafe commits is safely
removed.

Signed-off-by: Nicholas Walter Knize <nknize@apache.org>

* fix assertion typo

Signed-off-by: Nicholas Walter Knize <nknize@apache.org>

* rebase and incorporate pr feedback

Signed-off-by: Nicholas Walter Knize <nknize@apache.org>
2022-03-09 10:50:05 -08:00
Daniel Doubrovkine (dB.)
044f53630a
Set target and source compatibility to 11, required by Lucene 9. (#2407)
* Set target and source compatibility to 11, required by Lucene 9.

Signed-off-by: dblock <dblock@dblock.org>

* Uncomment commented code in #2321 for killing child processes that uses JDK9+ ProcessInfo.

Signed-off-by: dblock <dblock@dblock.org>

* Set distribution checker target JDK compatibility to 11.

Signed-off-by: dblock <dblock@dblock.org>

* Supress processing warnings.

Signed-off-by: dblock <dblock@dblock.org>
2022-03-08 14:48:51 -05:00
Andriy Redko
ae52008463
Fixing the --release flag usage for javac (#2343) (#2352)
* Fixing the --release flag usage for javac (#2343)

* Fixing the --release flag usage for javac

Signed-off-by: Andriy Redko <andriy.redko@aiven.io>

* Fixing the --html5 flag usage for javadoc

Signed-off-by: Andriy Redko <andriy.redko@aiven.io>

* Fix java-version-checker source/target compatibility settings (#2354)

Signed-off-by: Andriy Redko <andriy.redko@aiven.io>
2022-03-04 17:44:52 -05:00
Daniel Doubrovkine (dB.)
ae14259a2c
Restore Java 8 compatibility for build tools. (#2300) (#2321)
* Restore Java 8 compatibility for build tools.

Signed-off-by: dblock <dblock@dblock.org>

* Make source code compatible with Java 8.

Signed-off-by: dblock <dblock@dblock.org>
2022-03-03 14:34:53 -06:00
dependabot[bot]
cb57b9202b
Bump log4j-core in /buildSrc/src/testKit/thirdPartyAudit/sample_jars (#2281)
Bumps log4j-core from 2.17.1 to 2.17.2.

---
updated-dependencies:
- dependency-name: org.apache.logging.log4j:log4j-core
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>

Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2022-03-03 14:23:47 -05:00
Rishikesh Pasham
494c7bc436
Revert "Override Default Distribution Download Url with Custom Distribution Url When User Passes a Url" (#2256)
* Override default Distribution Download URL with custom Distribution URL

Signed-off-by: Rishikesh1159 <rishireddy1159@gmail.com>

* Accidently made commit to main branch, this revives it.Override default Distribution Download URL with custom Distribution URL

Signed-off-by: Rishikesh1159 <rishireddy1159@gmail.com>

* Revert Override Default Distribution Download Url with Custom Distribution Url When User Passes a Url

Signed-off-by: Rishikesh1159 <rishireddy1159@gmail.com>
2022-02-24 19:39:43 -08:00
dependabot[bot]
788ba99915
Bump forbiddenapis from 2.7 to 3.2 in /buildSrc/src/testKit/thirdPartyAudit (#2185)
* Bump forbiddenapis in /buildSrc/src/testKit/thirdPartyAudit

Bumps [forbiddenapis](https://github.com/policeman-tools/forbidden-apis) from 2.7 to 3.2.
- [Release notes](https://github.com/policeman-tools/forbidden-apis/releases)
- [Commits](https://github.com/policeman-tools/forbidden-apis/compare/2.7...3.2)

---
updated-dependencies:
- dependency-name: de.thetaphi:forbiddenapis
  dependency-type: direct:production
  update-type: version-update:semver-major
...

Signed-off-by: dependabot[bot] <support@github.com>

* Fixing failing check

Signed-off-by: Vacha Shah <vachshah@amazon.com>

Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: Vacha Shah <vachshah@amazon.com>
2022-02-24 11:27:44 -08:00
dependabot[bot]
5b3b98466a
Bump junit in /buildSrc/src/testKit/testingConventions (#2116)
Bumps [junit](https://github.com/junit-team/junit4) from 4.13.1 to 4.13.2.
- [Release notes](https://github.com/junit-team/junit4/releases)
- [Changelog](https://github.com/junit-team/junit4/blob/main/doc/ReleaseNotes4.13.1.md)
- [Commits](https://github.com/junit-team/junit4/compare/r4.13.1...r4.13.2)

---
updated-dependencies:
- dependency-name: junit:junit
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>

Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2022-02-18 15:28:13 -08:00
dependabot[bot]
be5b351a9b
Bump log4j-core in /buildSrc/src/testKit/thirdPartyAudit/sample_jars (#2131)
Bumps log4j-core from 2.11.1 to 2.17.1.

---
updated-dependencies:
- dependency-name: org.apache.logging.log4j:log4j-core
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>

Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2022-02-18 15:26:37 -08:00
Rishikesh Pasham
4a6f54bdeb
Override Default Distribution Download Url with Custom Distribution Url When User Passes a Url (#2086)
* Override Default Distribution Download Url with Custom Distribution Url When User Passes a Url

Signed-off-by: Rishikesh1159 <rishireddy1159@gmail.com>

* Adding test to check if correct IVY repos were added

Signed-off-by: Rishikesh1159 <rishireddy1159@gmail.com>

* Adding another test case when custom url is not passed and documenting usage of this in Developer Guide

Signed-off-by: Rishikesh1159 <rishireddy1159@gmail.com>

* Adding TOC and making changes in DEVELOPER_GUIDE

Signed-off-by: Rishikesh1159 <rishireddy1159@gmail.com>

* Making changes in DEVELOPER_GUIDE about Distribution Download plugin

Signed-off-by: Rishikesh1159 <rishireddy1159@gmail.com>
2022-02-18 14:17:07 -05:00
dependabot[bot]
9689a27b63
Bump com.diffplug.spotless from 5.6.1 to 6.2.0 (#1919)
* Bump com.diffplug.spotless from 5.6.1 to 6.2.0

Bumps com.diffplug.spotless from 5.6.1 to 6.2.0.

---
updated-dependencies:
- dependency-name: com.diffplug.spotless
  dependency-type: direct:production
  update-type: version-update:semver-major
...

Signed-off-by: dependabot[bot] <support@github.com>

* spotlessApply

Signed-off-by: Nicholas Walter Knize <nknize@apache.org>

Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: Nicholas Walter Knize <nknize@apache.org>
2022-02-16 22:08:40 -08:00
Martin Gaievski
309397520c
Adding support for JDK17 and removing JDK8 (#2025)
* Adding support for JDK17 and removing JDK8

Signed-off-by: Martin Gaievski <gaievski@amazon.com>

* Merge overlaping PR, bumping min java version to 11

Signed-off-by: Martin Gaievski <gaievski@amazon.com>

* Removing references to JDK8 from dev guide

Signed-off-by: Martin Gaievski <gaievski@amazon.com>
2022-02-02 20:59:10 -05:00
Leonidas Spyropoulos
6da253b8ff
build: introduce support for reproducible builds (#1995)
Reproducible builds is an initiative to create an independently-verifiable path from source to binary code [1]. This can be done by:
- Make all archive tasks in gradle reproducible by ignoring timestamp on files [2]
- Preserve the order in side the archives [2]
- Ensure GlobalBuildInfoPlugin.java use [SOURCE_DATE_EPOCH] when available

[SOURCE_DATE_EPOCH]: https://reproducible-builds.org/docs/source-date-epoch/
[1]: https://reproducible-builds.org/
[2]: https://docs.gradle.org/current/userguide/working_with_files.html#sec:reproducible_archives

Signed-off-by: Leonidas Spyropoulos <artafinde@gmail.com>
2022-02-02 09:13:54 -06:00
Sarat Vemulapalli
1f9517c4ca
Upgrading Shadow plugin to 7.1.2 (#2033)
Shadow plugin is used for publishing jars
and this upgrades Log4J dependency for build.

Signed-off-by: Sarat Vemulapalli <vemulapallisarat@gmail.com>
2022-02-02 08:51:23 -06:00
Andriy Redko
00416dc428
Update bundled JDK distribution to 17.0.2+8 (#2007)
Signed-off-by: Andriy Redko <andriy.redko@aiven.io>
2022-01-28 09:52:56 -08:00
ankitkala
56ae7fab63
Add hook to execute logic before Integ test task starts (#1969)
Signed-off-by: Ankit Kala <ankikala@amazon.com>
2022-01-25 14:47:45 +05:30
Rabi Panda
8fbd8cd998
Fix verifyVersions gradle task and cleanup bwcVersions (#1878)
Signed-off-by: Rabi Panda <adnapibar@gmail.com>
2022-01-11 09:19:51 -05:00
Andriy Redko
c192ecdfec
Attempt to fix :test:fixtures:s3-fixture:composeUp fails due to HTTP connection issue (#1866)
Signed-off-by: Andriy Redko <andriy.redko@aiven.io>
2022-01-10 17:34:52 -06:00
Ashish Agrawal
0b2eba568d
Update junit to 4.13.1 (#1837)
* Update junit to 4.13.1

Signed-off-by: Ashish Agrawal <ashisagr@amazon.com>

* update junit to 4.13.2

Signed-off-by: Ashish Agrawal <ashisagr@amazon.com>

* update SHA1 file

Signed-off-by: Ashish Agrawal <ashisagr@amazon.com>
2022-01-04 08:27:19 -05:00
Andriy Redko
65813b8320
RestIntegTestTask fails because of missed log4j-core dependency (#1815)
Signed-off-by: Andriy Redko <andriy.redko@aiven.io>
2021-12-28 14:51:25 -05:00
Andrew Ross
ca40ba9c64
Ignore file order in test assertion (#1755)
This unit test asserts that a SHA file for a groovy dependency gets
created. However, a SHA file for javaparser-core also gets created in
the same directory. For some reason, builds were failing on my machine
because `Files::list` was returning the javaparser-core file first. I
don't believe there are any ordering guarantees with that API, so I
relaxed the assertion to not depend on ordering.

Signed-off-by: Andrew Ross <andrross@amazon.com>
2021-12-20 13:58:05 -05:00
Nick Knize
b74d71fb74
Remove 6.0.* version constants (#1658)
This PR removes LegacyESVersion.V_6_0_* constants including all pre-release
versions and bug fixes.

Signed-off-by: Nicholas Walter Knize <nknize@apache.org>
2021-12-08 13:37:21 -06:00
Andriy Redko
8bf6fa6b4f
Support Gradle 7 (#1609)
Signed-off-by: Andriy Redko <andriy.redko@aiven.io>
2021-11-29 14:34:06 -05:00
Nick Knize
ac37cc99b2
Add staged version 1.1.1 (#1506)
Signed-off-by: Nicholas Walter Knize <nknize@apache.org>
2021-11-04 14:19:21 -05:00
Owais Kazi
a439371bc9
Cleanup for Checkstyle (#1370)
Signed-off-by: Owais Kazi <owaiskazi19@gmail.com>
2021-11-02 10:59:33 -07:00
Himanshu Setia
681e5548c1
Enabling spotless, disabling checkstyle check on plugins (#1488)
* Enabling spotless, disabling checkstyle on below modules

:plugins:mapper-annotated-text
:plugins:mapper-murmur3
:plugins:mapper-size
:plugins:repository-azure
:plugins:repository-gcs
:plugins:repository-hdfs
:plugins:repository-s3
:plugins:store-smb
:plugins:transport-nio
:qa:die-with-dignity

Signed-off-by: Himanshu Setia <setiah@amazon.com>

* Enabling spotless for more plugins

Signed-off-by: Himanshu Setia <setiah@amazon.com>

* Fixing error in merge conflict

Signed-off-by: Himanshu Setia <setiah@amazon.com>
2021-11-01 17:40:06 -07:00
kartg
2c1e89a1f9
Fixing support for a multi-node cluster via "gradle run" (#1455)
This change updates the logic in RunTask to use deterministic port values for all nodes rather than just the first node in a cluster. The local address of the first node is also used as the seed_hosts value to allow the cluster to form correctly. The seed_hosts setting is also allow-listed as an overrideable value.

Signed-off-by: Kartik Ganesh <85275476+kartg@users.noreply.github.com>
2021-11-01 09:18:21 -07:00
Andriy Redko
9c514d3734
Update bundled JDK distribution to 17.0.1+12 (#1476)
Signed-off-by: Andriy Redko <andriy.redko@aiven.io>
2021-11-01 10:34:04 -05:00
Sarat Vemulapalli
f6115ae160
Adding spotless support for subprojects under :test (#1464)
Signed-off-by: Sarat Vemulapalli <vemulapallisarat@gmail.com>
2021-10-29 13:20:39 -07:00
Owais Kazi
f4bdb94463
Run spotless and exclude checkstyle on rest-api-spec module (#1462)
Signed-off-by: Owais Kazi <owaiskazi19@gmail.com>
2021-10-29 12:11:24 -07:00
Owais Kazi
37ac3788a3
Run spotless and exclude checkstyle on modules module (#1442)
Signed-off-by: Owais Kazi <owaiskazi19@gmail.com>
2021-10-27 13:24:43 -07:00
Andriy Redko
f469d53cff
[BUG] SymbolicLinkPreservingUntarTransform fails on Windows (#1433)
Signed-off-by: Andriy Redko <andriy.redko@aiven.io>
2021-10-26 09:45:26 -05:00
Owais Kazi
8394f541bc
Run spotless and exclude checkstyle on libs module (#1428)
Signed-off-by: Owais Kazi <owaiskazi19@gmail.com>
2021-10-26 09:45:26 -05:00
Daniel Doubrovkine (dB.)
e601a68457
Fix windows build (mostly) (#1412)
* Updated developer guide with Windows specifics.

Signed-off-by: dblock <dblock@dblock.org>

* Correct windows task name.

Signed-off-by: dblock <dblock@dblock.org>

* Use Docker desktop installation on Windows.

Signed-off-by: dblock <dblock@dblock.org>

* Locate docker-compose on Windows.

Signed-off-by: dblock <dblock@dblock.org>

* Default docker-compose location.

Signed-off-by: dblock <dblock@dblock.org>
2021-10-22 07:55:41 -04:00
Owais Kazi
d02443a265
Run spotless and exclude checkstyle on plugins module (#1417)
Signed-off-by: Owais Kazi <owaiskazi19@gmail.com>
2021-10-21 20:49:03 -04:00
Owais Kazi
33e70a9886
Run spotless and exclude checkstyle on client module (#1392)
Signed-off-by: Owais Kazi <owaiskazi19@gmail.com>
2021-10-21 13:13:38 -07:00
Owais Kazi
996d33adb2
Run spotless and exclude checkstyle on server module (#1380)
Signed-off-by: Owais Kazi <owaiskazi19@gmail.com>
2021-10-19 08:32:54 -04:00
Andriy Redko
8ea3364bc5
Modernize and consolidate JDKs usage across all stages of the build. Update JDK-14 requirement, switch to JDK-17 instead (#1368)
* Modernize and consolidate JDKs usage across all stages of the build. Update JDK-14 requirement, switch to JDK-17 instead

Signed-off-by: Andriy Redko <andriy.redko@aiven.io>

* Updating phrasing based on review feedback

Signed-off-by: Andriy Redko <andriy.redko@aiven.io>

* Fixed runtime Java version usage, it has to be respected when RUNTIME_JAVA_HOME == JAVA_HOME

Signed-off-by: Andriy Redko <andriy.redko@aiven.io>

* Addressing review comments

Signed-off-by: Andriy Redko <andriy.redko@aiven.io>
2021-10-15 09:04:05 -04:00
Romain Tartière
ea0fe7bfae
Allow building on FreeBSD (#1091)
* Allow building on FreeBSD

With this set of change, we are able to successfuly run:

```
./gradlew publishToMavenLocal -Dbuild.snapshot=false
```

This step is used in the OpenSearch repository context when building
plugins in the current state of the CI.

While here, reorder OS conditions alphabetically.

Before building, the openjdk14 package was installed and the environment
was adjusted to use it:

```
sudo pkg install openjdk14
export JAVA_HOME=/usr/local/openjdk14/
export PATH=$JAVA_HOME/bin:$PATH
```

Signed-off-by: Romain Tartière <romain@blogreen.org>

* Unbreak CI with FreeBSD support

Signed-off-by: dblock <dblock@dblock.org>

Co-authored-by: dblock <dblock@dblock.org>
2021-10-14 14:42:28 -04:00