This commit rebases the versioning to OpenSearch 1.0.0
Co-authored-by: Rabi Panda <adnapibar@gmail.com>
Signed-off-by: Nicholas Walter Knize <nknize@apache.org>
This commit adds the SPDX Apache-2.0 license header along with an additional
copyright header for all modifications.
Signed-off-by: Nicholas Walter Knize <nknize@apache.org>
This commit renames several files that contain the name elasticsearch and replace that with opensearch.
Signed-off-by: Rabi Panda <adnapibar@gmail.com>
This commit fixes the currently broken gradle build resulted from the renaming work. It reverts a few dependencies and comments out the `opensearch_distibutions` task which is currently failing for some builds. We will address these separately in the future once we have a working build.
Signed-off-by: Rabi Panda <adnapibar@gmail.com>
This commit refactors the remaining o.e.index and o.e.test packages in the
test/fixtures module. References throughout the codebase are also refactored.
Signed-off-by: Nicholas Walter Knize <nknize@apache.org>
Refactor the code in the `libs/x-content` module and any references to those in the entire code base. The refactoring is done as part of the renaming to OpenSearch work.
Signed-off-by: Rabi Panda <adnapibar@gmail.com>
This commit refactors the remaining classes from o.e to the o.opensearch
namespace. All references throughout the codebase have been refactored.
Signed-off-by: Nicholas Walter Knize <nknize@apache.org>
Refactor the code in the `libs/core` module and any references to those in the entire code base. The refactoring is done as part of the renaming to OpenSearch work.
Signed-off-by: Rabi Panda <adnapibar@gmail.com>
This commit refactors o.e.common.settings package to the
o.opensearch.common.setttings namespace. All references throughout the codebase
are refactored.
Signed-off-by: Nicholas Walter Knize <nknize@apache.org>
* Rename org.elasticsearch.gateway to org.opensearch.gateway
Signed-off-by: Harold Wang <harowang@amazon.com>
* Rename org.elasticsearch.http to org.opensearch.http
Signed-off-by: Harold Wang <harowang@amazon.com>
* Renames org.elasticsearch.plugins to org.opensearch.plugins
Signed-off-by: Harold Wang <harowang@amazon.com>
Refactor the remaining classes in the `org.elasticsearch.search` package in the server module,
- Rename `org.elasticserach.search.aggregations` to `org.opensearch.search.aggregations`
- Rename instances of `org.elasticsearch.search` `org.opensearch.search`
Signed-off-by: Rabi Panda <adnapibar@gmail.com>
This commit refactors the o.e.common.util package to the
o.opensearch.common.util namespace. All references throughout the codebase have
been refactored.
Signed-off-by: Nicholas Walter Knize <nknize@apache.org>
This commit refactors the o.e.common.xcontent package to
o.opensearch.common.xcontent. All references throughout the codebase have been
refactored.
Signed-off-by: Nicholas Walter Knize <nknize@apache.org>
Rename `org.elasticserach.search.aggregations.bucket` to `org.opensearch.search.aggregations.bucket` in package names and references.
Signed-off-by: Rabi Panda <adnapibar@gmail.com>
This commit refactors the following packages:
* o.e.common.logging
* o.e.common.lucene
to the o.opensearch.common parent package. References throughout the codebase
have also been refactored.
Signed-off-by: Nicholas Walter Knize <nknize@apache.org>
This commit refactors the o.e.action.search package to o.opensearch.action
namespace. All references throughout the code are also refactored.
Signed-off-by: Nicholas Walter Knize <nknize@apache.org>
This commit refactors the following:
* o.e.action.delete
* o.e.action.explain
* o.e.action.fieldcaps
* o.e.action.get
* o.e.action.index
* o.e.action.main
* o.e.action.resync
* o.e.action.update
to the o.opensearch namespace. All references throughout the codebase are also
refactored.
Signed-off-by: Nicholas Walter Knize <nknize@apache.org>
Rename `org.elasticserach.search` to `org.opensearch.search` in package names and references for top level classes in the search package.
Signed-off-by: Rabi Panda <adnapibar@gmail.com>
Rename `org.elasticserach.search.aggregations.support` to `org.opensearch.search.aggregations.support` in package names and references.
Signed-off-by: Rabi Panda <adnapibar@gmail.com>
Rename `org.elasticserach.search.aggregations.metrics` to `org.opensearch.search.aggregations.metrics` in package names and references.
Signed-off-by: Rabi Panda <adnapibar@gmail.com>
* [Rename] o.e.common subpackages round 1
This commit refactors the following subpackages of o.e.common:
* o.e.common.joda
* o.e.common.lease
* o.e.common.metrics
* o.e.common.network
* o.e.common.path
* o.e.common.recycling
* o.e.common.regex
* o.e.common.rounding
* o.e.common.text
* o.e.common.time
* o.e.common.transport
to the o.opensearch namespace. All references throughout the codebase have been
refactored.
Signed-off-by: Nicholas Knize <nknize@amazon.com>
* fix imports 1
Signed-off-by: Nicholas Knize <nknize@amazon.com>
This commit refactors the following packages:
* o.e.common.geo
* o.e.common.hash
* o.e.common.io
into the o.opensearch.common namespace. All references throughout the codebase
have been refactored.
Signed-off-by: Nicholas Knize <nknize@amazon.com>
This commit refactors the following:
* o.e.common.cache
* o.e.common.collect
* o.e.common.component
* o.e.common.compress
* o.e.common.document
to the o.opensearch namespace. All references throughout the codebase are also
refactored
Signed-off-by: Nicholas Knize <nknize@amazon.com>
As part of this commit we refactor the following in the o.e.search package:
- rename `org.elasticsearch.search.fetch` to `org.opensearch.search.fetch`
- rename `org.elasticsearch.search.internal` to `org.opensearch.search.internal`
- rename `org.elasticsearch.search.profile` to `org.opensearch.search.profile`
- rename `org.elasticsearch.search.query` to `org.opensearch.search.query`
- rename `org.elasticsearch.search.suggest` to `org.opensearch.search.suggest`
- rename other instances of Elasticsearch to OpenSearch in these packages.
Signed-off-by: Rabi Panda <adnapibar@gmail.com>
This commit refactors the following packages:
* o.e.common.blobstore
* o.e.common.breaker
* o.e.common.bytes
to the o.opensearch.common namespace. All references throughout the codebase
have been refactored.
Signed-off-by: Nicholas Knize <nknize@amazon.com>
This commit refactors classes under o.e.common to o.opensearch.common. All
references throughout the codebase have also been refactored.
Signed-off-by: Nicholas Knize <nknize@amazon.com>
Refactoring:
- rename `org.elasticsearch.search.builder` to `org.opensearch.search.builder`
- rename `org.elasticsearch.search.collapse` to `org.opensearch.search.collapse`
- rename `org.elasticsearch.search.dfs` to `org.opensearch.search.dfs`
- rename `org.elasticsearch.search.lookup` to `org.opensearch.search.lookup`
- rename `org.elasticsearch.search.lookup` to `org.opensearch.search.lookup`
- rename `org.elasticsearch.search.rescore` to `org.opensearch.search.rescore`
- rename `org.elasticsearch.search.searchafter` to `org.opensearch.search.searchafter`
- rename `org.elasticsearch.search.slice` to `org.opensearch.search.slice`
- rename `org.elasticsearch.search.sort` to `org.opensearch.search.sort`
Signed-off-by: Rabi Panda <adnapibar@gmail.com>
This commit refactors o.e.Version to o.opensearch.Version. This is retained in a
single commit to serve as a reference for re-versioning the opensearch codebase
from legacy 7.10 to 1.0.
Signed-off-by: Nicholas Knize <nknize@amazon.com>
This commit refactors all OpenSearch classes in the root server package to
o.opensearch. All references throughout the codebase are also refactored.
Signed-off-by: Nicholas Knize <nknize@amazon.com>
This commit refactors the o.e.cli and o.e.client packages from elasticsearch to
o.opensearch.cli and o.opensearch.client packages in the server module,
respectively.
Signed-off-by: Nicholas Knize <nknize@amazon.com>
This commit refactors the following subpackages:
* o.e.cluster.health
* o.e.cluster.metadata
* o.e.cluster.node
to o.opensearch.cluster.*. All other references throughout the codebase are
updated.
Signed-off-by: Nicholas Knize <nknize@amazon.com>
Refactor the package`org.elasticsearch.script` in server module to rename it to`org.opensearch.script`.
Signed-off-by: Rabi Panda <adnapibar@gmail.com>
This commit refactors the classes in o.e.action.support to
o.opensearch.action.support. The remaining directories will be refactored in a
separate commit.
Signed-off-by: Nicholas Knize <nknize@amazon.com>
This commit refactors the ElasticsearchDirectoryReader class located in the
server module to OpenSearchDirectoryReader. References and usages, along with
method names, throughout the rest of the codebase are fully refactored.
Signed-off-by: Nicholas Knize <nknize@amazon.com>
This commit refactors the ElasticsearchException class located in the server module
to OpenSearchException. References and usages throughout the rest of the
codebase are fully refactored.
Signed-off-by: Nicholas Knize <nknize@amazon.com>
This PR removes outdated overrides in some tests that prevent them from testing
older index versions. Also removes an old comment + logic from
AggregatorFactoriesTests.
Now that we're consistently using `cat_match` to filter which shards we
run on we can get this confusing case:
1. You have a search with, say, a range and a sub-agg.
2. That search has a query that `can_match` can recognize will match no
docs. On *any* shard.
3. So we dutifully run it on a single shard so it can produce the
"empty" aggs.
4. The shard we pick happens to not have the target of the range mapped.
5. This kicks in the special range aggregator that doesn't collect any
documents.
6. Before this commit, that range aggregator *also* never produced any
sub-aggs.
So, without this change, it was quite possible for a search that
happened to match no documents to "throw away" the sub-aggs of a range
and a few other aggs.
We've had this problem for a long, long time but it is more confusing
now because `can_match` is really kicking in and causing us to see cases
where it looks like you are targeting a lot of shards but you really are
only targeting a couple. It used to be that to get the "no sub-aggs"
behavior you had to explicitly target only shards that didn't map the
target field of the `range` agg. And, like, in that case it isn't too
bad because you targeted a sort of degenerate shard. But now that
`can_match` is doing its thing you can end up with the confusing steps
above. It took me several hours to track down what what happening I know
how the individual pieces of all of this works. It took four hours to
figure out how they fit together in this case....
Anyway! This replaces all the aggregator implementations that throw out
the sub-aggregators with ones that keep them. I think this'll be less
confusing in the future.
Closes#64142
In #57892 I broke *some* sub-aggregations inside of the `parent` and
`child` aggregator, specifically any sub-aggregations that do work in
the `postCollect` phase. This fixes it by delaying the post collect
phase of aggs under `parent` and `child` until `beforeBuildingBuckets`
because, well, we haven't done *any* collection until after that phase.
This PR adds factory methods for the most common implementations:
* `SourceValueFetcher.identity` to pass through the source value untouched.
* `SourceValueFetcher.toString` to simply convert the source value to a string.
When constructing a value fetcher, the 'parsesArrayValue' flag must match
`FieldMapper#parsesArrayValue`. However there is nothing in code or tests to
help enforce this.
This PR reworks the value fetcher constructors so that `parsesArrayValue` is
'false' by default. Just as for `FieldMapper#parsesArrayValue`, field types must
explicitly set it to true and ensure the behavior is covered by tests.
Follow-up to #62974.
For runtime fields, we will want to do all search-time interaction with
a field definition via a MappedFieldType, rather than a FieldMapper, to
avoid interfering with the logic of document parsing. Currently, fetching
values for runtime scripts and for building top hits responses need to
call a method on FieldMapper. This commit moves this method to
MappedFieldType, incidentally simplifying the current call sites and freeing
us up to implement runtime fields as pure MappedFieldType objects.
Most of our field types have the same implementation for their `existsQuery` method which relies on doc_values if present, otherwise it queries norms if available or uses a term query against the _field_names meta field. This standard implementation is repeated in many different mappers.
There are field types that only query doc_values, because they always have them, and field types that always query _field_names, because they never have norms nor doc_values. We could apply the same standard logic to all of these field types as `MappedFieldType` has the knowledge about what data structures are available.
This commit introduces a standard implementation that does the right thing depending on the data structure that is available. With that only field types that require a different behaviour need to override the existsQuery method.
At the same time, this no longer forces subclasses to override `existsQuery`, which could be forgotten when needed. To address this we introduced a new test method in `MapperTestCase` that verifies the `existsQuery` being generated and its consistency with the available data structures.
This implements the `fields` API in `_search` for runtime fields using
doc values. Most of that implementation is stolen from the
`docvalue_fields` fetch sub-phase, just moved into the same API that the
`fields` API uses. At this point the `docvalue_fields` fetch phase looks
like a special case of the `fields` API.
While I was at it I moved the "which doc values sub-implementation
should I use for fetching?" question from a bunch of `instanceof`s to a
method on `LeafFieldData` so we can be much more flexible with what is
returned and we're not forced to extend certain classes just to make the
fetch phase happy.
Relates to #59332
This PR adds support for the 'fields' option in the following places:
* Anytime `inner_hits` is used, for both fetching nested/ child docs and field collapsing
* The `top_hits` aggregation
Addresses #61949.
This commit introduces a new API that manages point-in-times in x-pack
basic. Elasticsearch pit (point in time) is a lightweight view into the
state of the data as it existed when initiated. A search request by
default executes against the most recent point in time. In some cases,
it is preferred to perform multiple search requests using the same point
in time. For example, if refreshes happen between search_after requests,
then the results of those requests might not be consistent as changes
happening between searches are only visible to the more recent point in
time.
A point in time must be opened before being used in search requests. The
`keep_alive` parameter tells Elasticsearch how long it should keep a
point in time around.
```
POST /my_index/_pit?keep_alive=1m
```
The response from the above request includes a `id`, which should be
passed to the `id` of the `pit` parameter of search requests.
```
POST /_search
{
"query": {
"match" : {
"title" : "elasticsearch"
}
},
"pit": {
"id": "46ToAwMDaWR4BXV1aWQxAgZub2RlXzEAAAAAAAAAAAEBYQNpZHkFdXVpZDIrBm5vZGVfMwAAAAAAAAAAKgFjA2lkeQV1dWlkMioGbm9kZV8yAAAAAAAAAAAMAWICBXV1aWQyAAAFdXVpZDEAAQltYXRjaF9hbGw_gAAAAA==",
"keep_alive": "1m"
}
}
```
Point-in-times are automatically closed when the `keep_alive` is
elapsed. However, keeping point-in-times has a cost; hence,
point-in-times should be closed as soon as they are no longer used in
search requests.
```
DELETE /_pit
{
"id" : "46ToAwMDaWR4BXV1aWQxAgZub2RlXzEAAAAAAAAAAAEBYQNpZHkFdXVpZDIrBm5vZGVfMwAAAAAAAAAAKgFjA2lkeQV1dWlkMioGbm9kZV8yAAAAAAAAAAAMAWIBBXV1aWQyAAA="
}
```
#### Notable works in this change:
- Move the search state to the coordinating node: #52741
- Allow searches with a specific reader context: #53989
- Add the ability to acquire readers in IndexShard: #54966
Relates #46523
Relates #26472
Co-authored-by: Jim Ferenczi <jimczi@apache.org>
Nick Knize
Rabi Panda
Nick Knize
Harold Wang
Nick Knize
Rabi Panda
Julie Tibshirani
Nik Everett
Alan Woodward
Luca Cavanna
Nhat Nguyen