* [Refactor] MapperService to QueryShardContext in valueFetcher
QueryShardContext encapsulates MapperService along with other mechanisms useful
for analyzers, highlighters, and readers. This commit refactors the valueFetcher
to use the QueryShardContext as a parameter instead of direct access to the
MapperService. This is an API change. Behavior of the method is unaffected.
Signed-off-by: Nicholas Walter Knize <nknize@apache.org>
* change unnecessary Optional usage
Signed-off-by: Nicholas Walter Knize <nknize@apache.org>
* spotlessApply
Signed-off-by: Nicholas Walter Knize <nknize@apache.org>
* Add regexp interval source
Add a regexp interval source provider so people can use regular
expressions inside of intervals queries.
Signed-off-by: Matt Weber <matt@mattweber.org>
* Fixes
- register regexp interval in SearchModule
- use fully-qualified name for lucene RegExp
- get rid of unnecessary variable
Signed-off-by: Matt Weber <matt@mattweber.org>
Lucene 9 changes from BigEndian to LittleEndian in DataInput and DataOutput.
The use of Lucene's Data IO classes are refactored in this commit to use
StreamInput which maintains the same method calls while preserving the byte
order of the data.
Signed-off-by: Nicholas Walter Knize <nknize@apache.org>
Lucene 9 removed CodecReader#ramBytesUsed and all file formats that no longer
consume large amounts of memory. As a result RAM estimation for segments is no
longer possible and is removed by this commit. backwards compatibility is retained
for the tranport layer.
Signed-off-by: Nicholas Walter Knize <nknize@apache.org>
* Adding support for JDK17 and removing JDK8
Signed-off-by: Martin Gaievski <gaievski@amazon.com>
* Merge overlaping PR, bumping min java version to 11
Signed-off-by: Martin Gaievski <gaievski@amazon.com>
* Removing references to JDK8 from dev guide
Signed-off-by: Martin Gaievski <gaievski@amazon.com>
Fix composite aggregation failed test cases introduce by missing_order parameter by using MissingOrder to decide null values's order in InternalBucket.
Signed-off-by: Peng Huo <penghuo@gmail.com>
Remove deprecated max_merge_at_once_explicit parameter since it has been
removed in lucene 9 and deprecated in opensearch 1.3
Signed-off-by: Nicholas Walter Knize <nknize@apache.org>
Lucene 9 removes the ability to define an explicit version on an analysis
component so remove the the version parameter and deprecation warning
prohibiting users from explicitly setting versioned analyzers.
Signed-off-by: Nicholas Walter Knize <nknize@apache.org>
* Added timeout to ensureGreen() for testClusterGreenAfterPartialRelocation
Signed-off-by: Owais Kazi <owaiskazi19@gmail.com>
* Fixed spotless imports
Signed-off-by: Owais Kazi <owaiskazi19@gmail.com>
Remove the deprecated sync flush API which was replaced by sequence
number and retention lease mechanisms and no longer used in 2.0.0.
Signed-off-by: Nicholas Walter Knize <nknize@apache.org>
Lucene 9 removes the ability to define an explicit version on an analysis
component. The version parameter is deprecated at parse time and a warning is
issued
to the user through the deprecation logger.
Signed-off-by: Nicholas Walter Knize <nknize@apache.org>
max_merge_at_once_explicit is removed in lucene 9 so the index setting is
deprecated for removal in the next major release.
Signed-off-by: Nicholas Walter Knize <nknize@apache.org>
Adds support for _first and _last parameters for missing bucket ordering in composite aggregation.
By default, if order is asc, missing_bucket at first, if order is desc, missing_bucket at last. If
missing_order is "_first" or "_last", regardless order, missing_bucket is at first or last respectively.
Signed-off-by: Peng Huo <penghuo@gmail.com>
Logically delay the request size calculation until it's absolutely required by the
indexing pressure framework.
Signed-off-by: Saurabh Singh <sisurab@amazon.com>
Add support for setting the max expansions on a wildcard interval.
The default value is still 128 and the max value is bounded by
`BooleanQuery.getMaxClauseCount()`.
Signed-off-by: Matt Weber <matt@mattweber.org>
Soft Deletes have been enabled by default since Legacy version 7.0 and made
mandatory in Version 2.0.0. This commit refactors the InternalEngine to always
use soft-deletes. It is a follow on to making soft deletes mandatory in 2.0.0.
Signed-off-by: Nicholas Walter Knize <nknize@apache.org>
* [BUG] Wait for outstanding requests to complete in LastSuccessfulSettingsUpdate test
Signed-off-by: Suraj Singh <surajrider@gmail.com>
* [BUG] Wait for outstanding requests to complete in LastSuccessfulSettingsUpdate test
Signed-off-by: Suraj Singh <surajrider@gmail.com>
When some node or set of nodes is excluded (based on some cluster setting)
BalancedShardsAllocator iterates over them in breadth first order picking 1 shard from
each node and repeating the process until all shards are balanced. Since shards from
each node are picked randomly it's possible the p and r of shard1 is relocated first
leaving behind both p and r of shard2. If the excluded nodes were to go down the
cluster becomes red.
This commit introduces a new setting "cluster.routing.allocation.move.primary_first"
that prioritizes the p of both shard1 and shard2 first so the cluster does not become
red if the excluded nodes were to go down before relocating other shards. Note that
with this setting enabled performance of this change is a direct function of number
of indices, shards, replicas, and nodes. The larger the indices, replicas, and
distribution scale, the slower the allocation becomes. This should be used with care.
Signed-off-by: Ankit Jain <jain.ankitk@gmail.com>
* Update protobuf-java to 3.19.3
Signed-off-by: Tianli Feng <ftl94@live.com>
* Exclude some API usage violations in the package com.google.protobuf for thirdPartyAudit task to pass
Signed-off-by: Tianli Feng <ftl94@live.com>
* [BWC] Ensure 2.x compatibility with Legacy 7.10.x
This commit fixes TransportHandshaker to send a spoofed Legacy 7.10.2 mincompat
version to ensure OpenSearch 2.x nodes can join a Legacy 7.10.x cluster for
rolling upgrade support. Without this change 7.10.x and OpenSearch 2.x mixed
cluster bwc tests would fail.
Signed-off-by: Nicholas Walter Knize <nknize@apache.org>
* fix AbstractSimpleTransportTestCase failures
Signed-off-by: Nicholas Walter Knize <nknize@apache.org>
* change v3 to local static
Signed-off-by: Nicholas Walter Knize <nknize@apache.org>
* make V_3_0_0 package private and reuse in test
Signed-off-by: Nicholas Walter Knize <nknize@apache.org>
Kunal Kotwani
Nick Knize
Ankit Jain
Matt Weber
Vacha
Andriy Redko
Martin Gaievski
Lukáš Vlček
Leonidas Spyropoulos
Sarat Vemulapalli
Peng Huo
Owais Kazi
Sergey Nuyanzin
Rishikesh Pasham
Saurabh Singh
Dave Lago
ankitkala
Venkata Jyothsna Donapati
Suraj Singh
Tianli Feng
Poojita Raj
dependabot[bot]