First pass to remove types from CreateIndexRequest and CreateIndexRequestBuilder
mapping method. This method is overloaded several times so the most widely used
methods in the RequestBuilder are refactored from mapping to setMapping to avoid
confusion, conflicts, and to be consistent with other method names (e.g.,
setSettings, setCause, setAlias).
Signed-off-by: Nicholas Walter Knize <nknize@apache.org>
* [Upgrade] Lucene 9.1.0-snapshot-ea989fe8f30
Upgrades from Lucene 9.0.0 to 9.1.0-snapshot-ea989fe8f30 in preparation for
9.1.0 GA.
Signed-off-by: Nicholas Walter Knize <nknize@apache.org>
* Add spanishplural token filter
Signed-off-by: Nicholas Walter Knize <nknize@apache.org>
* fix KNOWN_TOKENIZERS
Signed-off-by: Nicholas Walter Knize <nknize@apache.org>
Unmutes IndexPrimaryRelocationIT.testPrimaryRelocationWhileIndexing which was
fixed by LuceneChangesSnapshot using accurate ops history.
Signed-off-by: Nicholas Walter Knize <nknize@apache.org>
Adds default implementation for getEngineFactory in EnginePlugin. The
default just returns Optional.empty(), allowing plugin developers to
implement this plugin without implementing this method.
Signed-off-by: John Mazanec <jmazane@amazon.com>
* Override default Distribution Download URL with custom Distribution URL
Signed-off-by: Rishikesh1159 <rishireddy1159@gmail.com>
* Accidently made commit to main branch, this revives it.Override default Distribution Download URL with custom Distribution URL
Signed-off-by: Rishikesh1159 <rishireddy1159@gmail.com>
* Override Default DistributionDownloadUrl with customDistribution Url passed from Plugins
Signed-off-by: Rishikesh1159 <rishireddy1159@gmail.com>
This commit upgrades the core codebase from Lucene 8.10.1 to
lucene 9.0.0. It includes all necessary refactoring of features and
API changes when upgrading to a new major Lucene release.
Signed-off-by: Nicholas Walter Knize <nknize@apache.org>
Co-authored-by: Andriy Redko <drreta@gmail.com>
Improves the LuceneChangesSnapshot to get an accurate count of recovery
operations using sort by sequence number optimization.
Signed-off-by: Nicholas Walter Knize <nknize@apache.org>
Removes the obsolete types based .addMapping method from CreateIndexRequest and
CreateIndexRequestBuilder. Tests are refactored to use the new source only
setMapping method.
Signed-off-by: Nicholas Walter Knize <nknize@apache.org>
Removes types from the TaskResults internal index along with the getMappings
method from IndexMetadata. This is needed to further remove types from
CreateIndexRequest.
Signed-off-by: Nicholas Walter Knize <nknize@apache.org>
* [Remove] Type from TermsLookUp
Signed-off-by: Suraj Singh <surajrider@gmail.com>
* Fix unit test failure
Signed-off-by: Suraj Singh <surajrider@gmail.com>
* Discrepancy in result from _validate/query API and actual query validity
Signed-off-by: Andriy Redko <andriy.redko@aiven.io>
* Moved the validate() check later into the flow to allow range validation to trigger first
Signed-off-by: Andriy Redko <andriy.redko@aiven.io>
Removes types from UID class along with cleaning up all obsolete MapperService
dependendices. This includes removing UID from the Translog Delete operation
which is no longer needed due to type dependency removal.
Signed-off-by: Nicholas Walter Knize <nknize@apache.org>
Removes multi-type support from IndexTemplateMetadata so that instead of holding
a map of multiple types to mappings, it only returns a single mapping for a
single type. Also removes type from documentMapper() method to avoid any
accidental NullPointerExceptions in the internal mapping retrieval.
Signed-off-by: Nicholas Walter Knize <nknize@apache.org>
This inner class is no longer required because its functionality has been moved to the generic GatedCloseable class.
Signed-off-by: Kartik Ganesh <gkart@amazon.com>
* Reducing duplication in plugins around ref-counted releasable classes
Both AmazonEc2Reference and AmazonS3Reference duplicate the same logic - a subclass of AbstractRefCounted that also implements Releasable. This change centralizes this paradigm into a AbstractRefCountedReleasable class and supports both clients via generics. It also updates all fetching implementations to use the get() method instead of client()
Signed-off-by: Kartik Ganesh <gkart@amazon.com>
* Introduce Reference classes for the Closeable and AutoCloseable interfaces
These classes allow you to wrap a reference instance with an onClose runnable that is executed when close() is invoked. Two separate classes are needed because the close() signatures for the two interfaces are different. This change takes the first step to have implementing classes extend from these generic superclasses, before attempting to remove the subclasses entirely. The get() method is also replaced throughout the code base.
Note that there is also a separate Releasable interface that has a similar access pattern, but is implemented separately. This is used in AbstractRefCountedReleasable introduced in a prior commit
Signed-off-by: Kartik Ganesh <gkart@amazon.com>
* More improvements and refactoring
* Functionality around one-way gating is now moved to a dedicated class - OneWayGate. This replaces duplicate functionality throughout the code.
* The two *Reference classes have been renamed to Gated* since that better represents their functionality
* The AbstractRefCountedReleasable has been improved to no longer be abstract by accepting the shutdown hook. This removes the need for the inner class in ReleasableBytesReference, and further simplifies the plugin subclasses (these could probably be removed entirely).
* Finally, unit tests have been added for some classes
Signed-off-by: Kartik Ganesh <gkart@amazon.com>
* Added tests for GatedCloseable
Also updated the license information in GatedAutoCloseableTests
Signed-off-by: Kartik Ganesh <gkart@amazon.com>
* Fixing license information in new files
Signed-off-by: Kartik Ganesh <gkart@amazon.com>
* Added unit tests for RefCountedReleasable
Signed-off-by: Kartik Ganesh <gkart@amazon.com>
* [Remove] TrimUnsafeCommit logic for legacy 6.x indexes
Multiple txlog commits was introduced in legacy 7.x. Legacy 6.x indexes could
therefore not have a safe commit. Since OpenSearch 2.0 is no longer compatible
with legacy 6.x indexes, the logic to trim these unsafe commits is safely
removed.
Signed-off-by: Nicholas Walter Knize <nknize@apache.org>
* fix assertion typo
Signed-off-by: Nicholas Walter Knize <nknize@apache.org>
* rebase and incorporate pr feedback
Signed-off-by: Nicholas Walter Knize <nknize@apache.org>
* MapperService has to be passed in as null for EnginePlugins CodecService constructor
Signed-off-by: Andriy Redko <andriy.redko@aiven.io>
* Addressing code review comments
Signed-off-by: Andriy Redko <andriy.redko@aiven.io>
* Delayed CodecService instantiation up to the shard initialization
Signed-off-by: Andriy Redko <andriy.redko@aiven.io>
* Added logger (associated with shard) to CodecServiceConfig
Signed-off-by: Andriy Redko <andriy.redko@aiven.io>
* Refactored the EngineConfigFactory / IndexShard instantiation of the CodecService
Signed-off-by: Andriy Redko <andriy.redko@aiven.io>
Removes include_type_name from the high level reset client along with relevant
deprecated methods in IndicesClient. All tests are updated to remove the
parameter from the rest requests along with various toXContent methods that are
no longer required.
Signed-off-by: Nicholas Walter Knize <nknize@apache.org>
* Add setting reindex.remote.allowlist, and deprecate setting reindex.remote.whitelist
Signed-off-by: Tianli Feng <ftianli@amazon.com>
* Add unit test for renaming the setting reindex.remote.allowlist
Signed-off-by: Tianli Feng <ftianli@amazon.com>
* Remove system.out.println()
Signed-off-by: Tianli Feng <ftianli@amazon.com>
* Adjust format by spotlessApply task
Signed-off-by: Tianli Feng <ftianli@amazon.com>
* Replace REMOTE_CLUSTER_WHITELIST with REMOTE_CLUSTER_ALLOWLIST
Signed-off-by: Tianli Feng <ftianli@amazon.com>
* Add a unit test to test final setting value when both settings have got a value
Signed-off-by: Tianli Feng <ftianli@amazon.com>
* Rename the unit test class name
Signed-off-by: Tianli Feng <ftianli@amazon.com>
* Remove the Access modifiers public from the constant REMOTE_CLUSTER_WHITELIST
Signed-off-by: Tianli Feng <ftianli@amazon.com>
* Initialize ReindexPlugin without using the @Before method
Signed-off-by: Tianli Feng <ftianli@amazon.com>
* Rename 'unwhitelisted' to 'unallowlisted' in a yml file used for REST api testing.
Signed-off-by: Tianli Feng <ftianli@amazon.com>
* Replace the exclusionary word whitelist with allowlist, and blacklist with denylist, in code commet and internal variable/method/class/package name.
Signed-off-by: Andreas <apre@gmx.at>
* Fixing the --release flag usage for javac (#2343)
* Fixing the --release flag usage for javac
Signed-off-by: Andriy Redko <andriy.redko@aiven.io>
* Fixing the --html5 flag usage for javadoc
Signed-off-by: Andriy Redko <andriy.redko@aiven.io>
* Fix java-version-checker source/target compatibility settings (#2354)
Signed-off-by: Andriy Redko <andriy.redko@aiven.io>
This test is slated for removal as it only applies to ancient indexes (Legacy
6.x). Muting test so bwc tests are consistent and no longer angry.
Signed-off-by: Nicholas Walter Knize <nknize@apache.org>
* Add '_name' field support to score functions and provide it back in explanation response
Signed-off-by: Andriy Redko <andriy.redko@aiven.io>
* Address code review comments
Signed-off-by: Andriy Redko <andriy.redko@aiven.io>
Nick Knize
John Mazanec
Lukáš Vlček
Andriy Redko
Rishikesh Pasham
Kyle J. Davis
Suraj Singh
dependabot[bot]
Kartik
Kartik
Daniel Doubrovkine (dB.)
Tianli Feng
Subhobrata Dey
Owais Kazi
aponb
Vacha Shah