Commit Graph

55149 Commits

Author SHA1 Message Date
whitesource-for-github-com[bot] 762601fb68
Add .whitesource configuration file (#1525)
Co-authored-by: whitesource-for-github-com[bot] <50673670+whitesource-for-github-com[bot]@users.noreply.github.com>
2021-11-11 13:47:33 -08:00
Andriy Redko 0e6b0c5b4f
[repository-azure] Update to the latest Azure Storage SDK v12, remove privileged runnable wrapper in favor of access helper (#1521)
Signed-off-by: Andriy Redko <andriy.redko@aiven.io>
2021-11-11 12:57:34 -08:00
CEHENKLE 37233a9280
add codeowners file (#1530)
* initial commit of CODEOWNERS

Signed-off-by: CEHENKLE <henkle@amazon.com>

* put the @ sign in the right spot

Signed-off-by: CEHENKLE <henkle@amazon.com>

* updating wild card

Signed-off-by: CEHENKLE <henkle@amazon.com>

* removed extra comment

Signed-off-by: CEHENKLE <henkle@amazon.com>

* you know what?  I didn't like that other wildcard.  f++ would nnot wildcard again

Signed-off-by: CEHENKLE <henkle@amazon.com>
2021-11-11 11:47:47 -08:00
Megha Sai Kavikondala fb78d10f1a
Integration test that checks for settings upgrade (#1482)
* Made changes.

Signed-off-by: Megha Sai Kavikondala <kavmegha@amazon.com>

* Signed-off-by: Megha Sai Kavikondala <kavmegha@amazon.com>

Changes made by deleting the TestSettingsIT file and adding new lines in FullClusterRestartSettingsUpgradeIT.java
2021-11-09 10:57:38 -08:00
Andrew Ross 546ab21989
Use a non-default port for upgrade-cli unit tests (#1512)
I observed a [test failure][1] that I believe was caused by the fact
that an OpenSearch server happened to be running at localhost:9200 when
these unit tests were executed. The code under test has logic to try to
connect to localhost:9200 and then fall back to defaults if that port is
not open. The tests expect these defaults and will fail if different
data is returned. The change here is to use a non-default port to make
it very unlikely that a real instance will be running at the non-default
port. I don't know why an OpenSearch service happened to be running
during the linked test failure, but I think making these unit tests more
independent and isolated is helpful no matter the underlying cause in
this case.

[1]: https://fork-jenkins.searchservices.aws.dev/job/OpenSearch_CI/job/PR_Checks/job/Gradle_Check/975/artifact/gradle_check_975.log/*view*/

Signed-off-by: Andrew Ross <andrross@amazon.com>
2021-11-05 09:52:16 -04:00
Nick Knize eb5e454cc2
[bwc] reenable bwc testing after syncing staged branches (#1511)
Signed-off-by: Nicholas Walter Knize <nknize@apache.org>
2021-11-05 09:50:56 -04:00
Nick Knize 99a356bfc6
[main] Add staged version 1.3.0 for bwc (#1510)
Signed-off-by: Nicholas Walter Knize <nknize@apache.org>
2021-11-04 15:56:08 -05:00
Nick Knize ac37cc99b2
Add staged version 1.1.1 (#1506)
Signed-off-by: Nicholas Walter Knize <nknize@apache.org>
2021-11-04 14:19:21 -05:00
Nick Knize 4c42c142cb
[BWC] Diable BWC tests until branch versions are synced (#1508)
Signed-off-by: Nicholas Walter Knize <nknize@apache.org>
2021-11-04 12:03:41 -05:00
Ryan Bogan 286121f0dc
Added logic to allow {dot} files on startup (#1437)
* Added logic to allow {dot} files on startup

Signed-off-by: Ryan Bogan <rbogan@amazon.com>

* Ensures that only plugin directories are returned by findPluginDirs()

Signed-off-by: Ryan Bogan <rbogan@amazon.com>

* Prevents . files from being returned as plugins

Signed-off-by: Ryan Bogan <rbogan@amazon.com>
2021-11-04 10:01:55 -04:00
Xue Zhou f54cc382d5
remove codeQL warning about implicit narrowing conversion in compound assignment (#1403)
Signed-off-by: Xue Zhou <xuezhou@amazon.com>
2021-11-02 13:49:25 -07:00
Vacha af6ae752b4
Upgrading dependencies (#1491)
Signed-off-by: Vacha <vachshah@amazon.com>
2021-11-02 13:47:54 -07:00
Owais Kazi a439371bc9
Cleanup for Checkstyle (#1370)
Signed-off-by: Owais Kazi <owaiskazi19@gmail.com>
2021-11-02 10:59:33 -07:00
Himanshu Setia 681e5548c1
Enabling spotless, disabling checkstyle check on plugins (#1488)
* Enabling spotless, disabling checkstyle on below modules

:plugins:mapper-annotated-text
:plugins:mapper-murmur3
:plugins:mapper-size
:plugins:repository-azure
:plugins:repository-gcs
:plugins:repository-hdfs
:plugins:repository-s3
:plugins:store-smb
:plugins:transport-nio
:qa:die-with-dignity

Signed-off-by: Himanshu Setia <setiah@amazon.com>

* Enabling spotless for more plugins

Signed-off-by: Himanshu Setia <setiah@amazon.com>

* Fixing error in merge conflict

Signed-off-by: Himanshu Setia <setiah@amazon.com>
2021-11-01 17:40:06 -07:00
Vacha 389b7dfa18
Upgrading dependencies in hdfs plugin (#1466)
Signed-off-by: Vacha <vachshah@amazon.com>
2021-11-01 12:58:07 -07:00
kartg 2c1e89a1f9
Fixing support for a multi-node cluster via "gradle run" (#1455)
This change updates the logic in RunTask to use deterministic port values for all nodes rather than just the first node in a cluster. The local address of the first node is also used as the seed_hosts value to allow the cluster to form correctly. The seed_hosts setting is also allow-listed as an overrideable value.

Signed-off-by: Kartik Ganesh <85275476+kartg@users.noreply.github.com>
2021-11-01 09:18:21 -07:00
Andriy Redko 9c514d3734
Update bundled JDK distribution to 17.0.1+12 (#1476)
Signed-off-by: Andriy Redko <andriy.redko@aiven.io>
2021-11-01 10:34:04 -05:00
Rabi Panda d45f5dfcab
Make TranslogDeletionPolicy abstract for extension (#1456)
As part of the commit 2ebd0e04, we added a new method to the EnginePlugin to provide a 
custom TranslogDeletionPolicy. This commit makes minTranslogGenRequired method 
abstract in this class for implementation by child classes. The default implementation 
is provided by DefaultTranslogDeletionPolicy.

Signed-off-by: Rabi Panda <adnapibar@gmail.com>
2021-11-01 10:25:30 -05:00
Sarat Vemulapalli f6115ae160
Adding spotless support for subprojects under :test (#1464)
Signed-off-by: Sarat Vemulapalli <vemulapallisarat@gmail.com>
2021-10-29 13:20:39 -07:00
Owais Kazi f4bdb94463
Run spotless and exclude checkstyle on rest-api-spec module (#1462)
Signed-off-by: Owais Kazi <owaiskazi19@gmail.com>
2021-10-29 12:11:24 -07:00
Nick Knize 294ad0eb6f
[Upgrade] 1.2 BWC to Lucene 8.10.1 (#1460)
This commit upgrades the BWC 1.2 lucene dependency to version 8.10.1

Signed-off-by: Nicholas Walter Knize <nknize@apache.org>
2021-10-29 10:45:29 -07:00
Rabi Panda 2cb71a33b8
Remove deprecated settings and logic for translog pruning by retention lease. (#1416)
The settings and the corresponding logic for translog pruning by retention lease which were added as part of #1100 have been deprecated. This commit removes those deprecated code in favor of an extension point for providing a custom TranslogDeletionPolicy.

Signed-off-by: Rabi Panda <adnapibar@gmail.com>
2021-10-29 08:25:12 -07:00
CEHENKLE f08ee2c67c
Moving DCO to workflows (#1458)
Signed-off-by: CEHENKLE <henkle@amazon.com>
2021-10-28 17:11:28 -04:00
Nick Knize 53334b2ce4
Upgrade to Lucene 8.10.1 (#1440)
This commit upgrades to the latest release of lucene 8.10

Signed-off-by: Nicholas Walter Knize <nknize@apache.org>
2021-10-28 10:06:53 -05:00
Andrew Ross 12789f89a3
Close first engine instance before creating second (#1457)
When creating the second instance of an InternalEngine using the same
translog config of the default InternalEngine instance, the second
instance will attempt to delete all the existing translog files. I found
a deterministic test failure when running with the seed
`E3E6AAD95ABD299B`.

As opposed to creating a second engine instance with a different
translog location, just close the first one before creating the second.

Signed-off-by: Andrew Ross <andrross@amazon.com>
2021-10-28 01:11:05 -07:00
Owais Kazi 37ac3788a3
Run spotless and exclude checkstyle on modules module (#1442)
Signed-off-by: Owais Kazi <owaiskazi19@gmail.com>
2021-10-27 13:24:43 -07:00
Vacha 3f6e1df9eb
Fixing bwc test for repository-multi-version (#1441)
Signed-off-by: Vacha <vachshah@amazon.com>
2021-10-27 10:41:13 -04:00
Kyle J. Davis 39ddb774d8
changed work-in-progress language (#1275)
Signed-off-by: Kyle Davis <kyledvs@amazon.com>
2021-10-26 10:39:25 -07:00
Andriy Redko f469d53cff
[BUG] SymbolicLinkPreservingUntarTransform fails on Windows (#1433)
Signed-off-by: Andriy Redko <andriy.redko@aiven.io>
2021-10-26 09:45:26 -05:00
Owais Kazi 8394f541bc
Run spotless and exclude checkstyle on libs module (#1428)
Signed-off-by: Owais Kazi <owaiskazi19@gmail.com>
2021-10-26 09:45:26 -05:00
Andriy Redko a19c3736e5
Adjust CodeCache size to eliminate JVM warnings (and crashes) (#1426)
Signed-off-by: Andriy Redko <andriy.redko@aiven.io>
2021-10-26 09:45:26 -05:00
Daniel Doubrovkine (dB.) 2abd536f9b
Recommend Docker 3.6.0. (#1427)
Signed-off-by: dblock <dblock@dblock.org>
2021-10-26 09:45:26 -05:00
Rabi Panda c86d765e7c
Add extension point for custom TranslogDeletionPolicy in EnginePlugin. (#1404)
This commit adds a method that can be used to provide a custom TranslogDeletionPolicy
from within plugins that implement the EnginePlugin interface. This enables plugins to
provide a custom deletion policy with the current limitation that only one plugin can
override the policy. An exception will be thrown if more than one plugin overrides the
policy.

Signed-off-by: Nicholas Walter Knize <nknize@apache.org>
2021-10-26 09:45:10 -05:00
CEHENKLE 8f566123c1
adding in untriaged label to features (#1419)
Signed-off-by: CEHENKLE <henkle@amazon.com>
2021-10-22 12:42:03 -04:00
Daniel Doubrovkine (dB.) e601a68457
Fix windows build (mostly) (#1412)
* Updated developer guide with Windows specifics.

Signed-off-by: dblock <dblock@dblock.org>

* Correct windows task name.

Signed-off-by: dblock <dblock@dblock.org>

* Use Docker desktop installation on Windows.

Signed-off-by: dblock <dblock@dblock.org>

* Locate docker-compose on Windows.

Signed-off-by: dblock <dblock@dblock.org>

* Default docker-compose location.

Signed-off-by: dblock <dblock@dblock.org>
2021-10-22 07:55:41 -04:00
Andrew Ross 5fb270908a
Change comment to point to DEVELOPER_GUIDE.md (#1415)
The content being referred to by the comment is in DEVELOPER_GUIDE.md
and not CONTRIBUTING.md.

Signed-off-by: Andrew Ross <andrross@amazon.com>
2021-10-22 07:54:42 -04:00
Owais Kazi d02443a265
Run spotless and exclude checkstyle on plugins module (#1417)
Signed-off-by: Owais Kazi <owaiskazi19@gmail.com>
2021-10-21 20:49:03 -04:00
Vacha 8b4a7683d5
Upgrading mockito version to make it consistent across the repo (#1410)
Signed-off-by: Vacha <vachshah@amazon.com>
2021-10-21 19:40:43 -04:00
Owais Kazi 33e70a9886
Run spotless and exclude checkstyle on client module (#1392)
Signed-off-by: Owais Kazi <owaiskazi19@gmail.com>
2021-10-21 13:13:38 -07:00
Andriy Redko 119701f622
Removing Jenkinsfile (not used), replaced by opensearch-build/jenkins/opensearch/Jenkinsfile (#1408)
Signed-off-by: Andriy Redko <andriy.redko@aiven.io>
2021-10-21 14:45:51 -04:00
Andriy Redko acac3cc285
Fixing org.opensearch.repositories.azure.AzureBlobContainerRetriesTests and org.opensearch.action.admin.cluster.node.stats.NodeStatsTests (#1390)
Signed-off-by: Andriy Redko <andriy.redko@aiven.io>
2021-10-20 15:27:18 -04:00
Saurabh Singh 284968bb85
Update node attribute check to version update (1.2) check for shard indexing pressure serialization. (#1395)
This commit adds an explicit version check for shard indexing pressure 
serialization. This is required to not mandate test have the cluster service 
initialized while asserting node attributes.

Signed-off-by: Saurabh Singh <sisurab@amazon.com>
Co-authored-by: Saurabh Singh <sisurab@amazon.com>
2021-10-20 12:18:46 -05:00
Xue Zhou 574e42c31b
Remove old ES libraries used in reindex due to CVEs (#1359)
This commit removes old ES libraries version 090 and 176 due to CVE

Signed-off-by: Xue Zhou <xuezhou@amazon.com>
2021-10-20 11:51:45 -05:00
Nick Knize ecac8d3c38
Add EngineConfig extensions to EnginePlugin (#1387)
This commit adds an extension point to EngineConfig through EnginePlugin using
a new EngineConfigFactory mechanism. EnginePlugin provides interface methods to
override configurations in EngineConfig. The EngineConfigFactory produces a new
instance of the EngineConfig using these overrides. Defaults are used absent
overridden configurations.

This serves as a mechanism to override Engine configurations (e.g., CodecService,
TranslogConfig) enabling Plugins to have higher fidelity for changing Engine
behavior without having to override the entire Engine (which is only permitted for
a single plugin).

Signed-off-by: Nicholas Walter Knize <nknize@apache.org>
2021-10-19 23:04:28 -05:00
Sergey Nuyanzin 2da858ccb0
[typos] typos in DEVELOPER_GUIDE.md (#1381)
Signed-off-by: snuyanzin <snuyanzin@gmail.com>
2021-10-19 08:45:49 -04:00
Owais Kazi 996d33adb2
Run spotless and exclude checkstyle on server module (#1380)
Signed-off-by: Owais Kazi <owaiskazi19@gmail.com>
2021-10-19 08:32:54 -04:00
Andriy Redko 05dc4bf4b3
Fixing post merge 3rd party audit issues (#1384)
Signed-off-by: Andriy Redko <andriy.redko@aiven.io>
2021-10-18 22:06:50 -04:00
Andriy Redko 9612fe80b5
[repository-azure] plugin should use Azure Storage SDK v12 for Java (#1302)
Signed-off-by: Andriy Redko <andriy.redko@aiven.io>
2021-10-18 19:48:32 -04:00
Vacha c7f5c90a5f
Upgrading netty version to 4.1.69.Final (#1363)
Signed-off-by: Vacha <vachshah@amazon.com>
2021-10-18 17:30:19 -04:00
Andriy Redko 8ea3364bc5
Modernize and consolidate JDKs usage across all stages of the build. Update JDK-14 requirement, switch to JDK-17 instead (#1368)
* Modernize and consolidate JDKs usage across all stages of the build. Update JDK-14 requirement, switch to JDK-17 instead

Signed-off-by: Andriy Redko <andriy.redko@aiven.io>

* Updating phrasing based on review feedback

Signed-off-by: Andriy Redko <andriy.redko@aiven.io>

* Fixed runtime Java version usage, it has to be respected when RUNTIME_JAVA_HOME == JAVA_HOME

Signed-off-by: Andriy Redko <andriy.redko@aiven.io>

* Addressing review comments

Signed-off-by: Andriy Redko <andriy.redko@aiven.io>
2021-10-15 09:04:05 -04:00