This commit refactors the ElasticsearchStatusException in the server module to
OpenSearchStatusException. References and usages throughout the rest of the
codebase are fully refactored.
Signed-off-by: Nicholas Knize <nknize@amazon.com>
This commit refactors ElasticsearchSecurityException class in the server module
to OpenSearchSecurityException. References and usages throughout the rest of the
codebase are fully refactored.
Signed-off-by: Nicholas Knize <nknize@amazon.com>
This commit refactors ElasticsearchGenerationException class in the server
module to OpenSearchGenerationException. References and usages throughout the
rest of the codebase are fully refactored.
Signed-off-by: Nicholas Knize <nknize@amazon.com>
This commit refactors all instances of elasticsearch in server/src/test/java/org/apache to opensearch.
Signed-off-by: Abbas Hussain <abbas_10690@yahoo.com>
This commit refactors the ElasticsearchCorruptionException in the server module
to OpenSearchCorruptionException. References and usages throughtout the rest of
the codebase are fully refactored.
Signed-off-by: Nicholas Knize <nknize@amazon.com>
This commit refactors the ElasticsearchClient class located in the server module to
OpenSearchClient. References and usages throughout the rest of the codebase are
fully refactored.
Signed-off-by: Nicholas Knize <nknize@amazon.com>
This commit refactors the ElasticsearchException class located in the server module
to OpenSearchException. References and usages throughout the rest of the
codebase are fully refactored.
Signed-off-by: Nicholas Knize <nknize@amazon.com>
This commit refactors all instances of elasticsearch in
server/src/main/java/org/apache to opensearch.
Signed-off-by: Nicholas Knize <nknize@amazon.com>
This reverts commit c50e8c83a2476ca4ca2f1dd05fa5a608bc0e9ef6
which went should have merged to the rename branch instead of
the main branch.
Signed-off-by: Peter Nied <petern@amazon.com>
This commit refactors all instances of elasticsearch in
server/src/main/java/org/apache to opensearch.
Signed-off-by: Nicholas Knize <nknize@amazon.com>
Signed-off-by: Peter Nied <petern@amazon.com>
This commit changes the building, packaging, and testing framework to only support OSS on different distributions.
Next steps:
completely remove -oss flag dependencies in package and build tests
move 6.x bwc testing to be an explicit option
remove any references to elastic.co download site (or replace with downloads from the OSS website)
Co-authored-by: Himanshu Setia <setiah@amazon.com>
Co-authored-by: Rabi Panda <pandarab@amazon.com>
Co-authored-by: Himanshu Setia <58999915+setiah@users.noreply.github.com>
Co-authored-by: Sarat Vemulapalli <vemsarat@amazon.com>
Signed-off-by: Peter Nied <petern@amazon.com>
ReplicationOperation can notify the listener twice if the primary shard
is demoted after it has completed the primary operation.
Closes#68049
Signed-off-by: Peter Nied <petern@amazon.com>
This commit fixes DeleteDataStreamRequestTests.testDeleteSnapshottingDataStream unit test failure by passing SnapshotsInProgress.Entry.SUCCESS in the createEntrymethod.
Signed-off-by: Peter Nied <petern@amazon.com>
This commit cleans up the following:
* Remove unused imports
* Remove ILM settings in hlrc testCluster formation
* Comment out security users settings in ElasticsearchNode creation for build-tools tests
Signed-off-by: Peter Nied <petern@amazon.com>
This fixes the constructor for IndexNameExpressionResolver to pass in
Settings.EMPTY to a ThreadContext used by the resolver.
Signed-off-by: Peter Nied <petern@amazon.com>
Drops an assertion that in 1911 Paris rolled its clocks back past
midnight. This assertion is so far in the past that it isn't
consistently included in the JDK's tzdb. When we upgraded to 15.0.1 it
fell out of my tzdb locally. It doesn't seem to have happened to CI,
oddly, but it just doesn't seem worth keeping.
Closes#67930
We have seen a case where the memory of IndexingPressure was
re-adjusted twice. With this commit, we will log that error with a
stack trace so that we can figure out the source of the issue.
Our test for early termination would break if we made many small
segments because none of them would be large enough to trigger the early
termination. This makes sure have only a single segment for these test,
making sure we terminate early.
Closes#62769
This test failed on WindowsFS. We failed to remove the corrupted file if
it's being opened (for a short window by ListShardStore action) and the
pending delete files were clear when we restarted that node.
This commit fixes the issue by shutting down the node before removing
the corrupted file to avoid any access to that file.
Closes#66893
We started passing down the root document's _source when processing
nested hits, to avoid reloading and reparsing the root source for each hit.
Unfortunately the approach did not work when there are multiple layers of
`inner_hits`. In this case, the second-layer inner hit received its immediate
parent's source instead of the root source. This parent source is filtered to
just contain the parts corresponding to the nested document, but the source
parsing logic is designed to always operate on the top-level root source. This
caused failures when loading the second-layer inner hits.
This PR makes sure to always pass the root document's _source when processing
inner hits, even if there are multiple layers.
Previously we treated attribute filtering for _tier-prefixed attributes a pass-through, meaning
that they were essentially always treated as matching in DiscoveryNodeFilters.match, however, for
exclude settings, this meant that the node was considered to match the node if a _tier* filter was
specified.
This commit prunes these attributes from the DiscoveryNodeFilters when considering the filters for
FilterAllocationDecider so that they are only considered in DataTierAllocationDecider.
Resolves#66679
This PR fixes two bugs that can arise when _source is disabled and we fetch nested documents:
* Fix exception when highlighting `inner_hits` with disabled _source.
* Fix exception in nested `top_hits` with disabled _source.
* Add more tests for highlighting `inner_hits`.
For async searches (EQL included) the client's request headers were
erroneously stored in the .tasks index. This might expose the requesting
client's HTTP Authorization header. This PR fixes that by employing the
usual approach to store only the security-internal headers, which carry
the authentication result, instead of the original Authorization header,
which is commonly utilized to redo authentication for scheduled tasks.
This PR removes outdated overrides in some tests that prevent them from testing
older index versions. Also removes an old comment + logic from
AggregatorFactoriesTests.
This commit fixes the cat tasks api parameter specification and the
handler so that the parameters are consumed during request preparation.
Closes#59493
Backport of #66272
This commit ensures that the after key is parsed with the doc value formatter.
This is needed for unsigned longs that uses shifted longs internally.
Closes#65685
In the assertion mentioned in the new comment we first get the `endingSnapshots`
and then check that we don't have a listener that isn't referred to by it so we need
to remove from the listers map before removing from `endingSnapshots` to avoid rare, random
assertion tripping here with concurrent repository operations.
If creating the latest translog file and retrieving a translog stats
happen within the same millisecond, then the earliestLastModifiedAge
will be zero.
Closes#66092
This PR fixes a regression where fvh fragments could be loaded from the wrong
document _source.
Some `FragmentsBuilder` implementations contain a `SourceLookup` to load from
_source. The lookup should be positioned to load from the current hit document.
However, since `FragmentsBuilder` are cached and shared across hits, the lookup
is never updated to load from the new documents. This means we accidentally
load _source from a different document.
The regression was introduced in #60179, which started storing `SourceLookup`
on `FragmentsBuilder`.
Fixes#65533.
This commit adjusts the behavior when calculating the diff between two
`AbstractScopedSettings` objects, so that the default values of settings
whose default values depend on the values of other settings are
correctly calculated. Previously, when calculating the diff, the default
value of a depended setting would be calculated based on the default
value of the setting(s) it depends on, rather than the current value of
those settings.
In #61906 we added the possibility for the master node to fetch
the size of a shard snapshot before allocating the shard to a
data node with enough disk space to host it. When merging
this change we agreed that any failure during size fetching
should not prevent the shard to be allocated.
Sadly it does not work as expected: the service only triggers
reroutes when fetching the size succeed but never when it
fails. It means that a shard might stay unassigned until
another cluster state update triggers a new allocation
(as in #64372). More sadly, the test I wrote was wrong as
it explicitly triggered a reroute.
This commit changes the InternalSnapshotsInfoService
so that it also triggers a reroute when fetching the snapshot
shard size failed, ensuring that the allocation can move
forward by using an UNAVAILABLE_EXPECTED_SHARD_SIZE
shard size. This unknown shard size is kept around in the
snapshot info service until no corresponding unassigned
shards need the information.
Backport of #65436
This change simplifies logic and allow more legit cases in Metadata.Builder.validateDataStreams.
It will only show conflict on names that are in form of .ds-<data stream name>-<[0-9]+> and will allow any names like .ds-<data stream name>-something-else-<[0-9]+>.
This fixes problem with rollover when you have 2 data streams with names like a and a-b - currently if a-b has generation greater than a you won't be able to rollover a anymore.
Co-authored-by: Elastic Machine <elasticmachine@users.noreply.github.com>
TransportService doesn't respond to the pending requests of proxy
connections when the underlying connections get disconnected because
proxy connections do not override the getCacheKey method. Some CCS
requests would never be completed because of this bug.
These strings are quite long individually and will be repeated
potentially up to the number of snapshots in the repository times.
Since these make up more than half of the size of the repository metadata
and are likely the same for all snapshots the savings from deduplicating them
can make up for more than half the size of `RepositoryData` easily in most real-world
cases.