Commit Graph

2704 Commits

Author SHA1 Message Date
Nick Knize 923ea001f5 [Rename] o.e.action.support classes (#253)
This commit refactors the classes in o.e.action.support to
o.opensearch.action.support. The remaining directories will be refactored in a
separate commit.

Signed-off-by: Nicholas Knize <nknize@amazon.com>
2021-03-21 20:56:34 -05:00
Rabi Panda 991b3650b6 [Rename] refactor server/snapshots package. (#251)
Refactor `server/snapshots` to rename the package names from `org.elasticsearch.snapshots` to `org.opensearch.snapshots` as part of the rename to OpenSearch work.

Signed-off-by: Rabi Panda <adnapibar@gmail.com>
2021-03-21 20:56:34 -05:00
Nick Knize e21c436cea [Rename] o.e.action.bulk (#248)
This commit refactors all classes in o.e.action.bulk to o.opensearch.action.bulk
all references throughout the rest of the codebase are updated.

Signed-off-by: Nicholas Knize <nknize@amazon.com>
2021-03-21 20:56:34 -05:00
Rabi Panda 3eee5183d1 [Rename] server/rest (#229)
This commit refactors the `server/rest` package as part of the Elasticsearch to OpenSearch renaming.

Signed-off-by: Rabi Panda <adnapibar@gmail.com>
2021-03-21 20:56:34 -05:00
Nick Knize 8aa818e93e [Rename] refactor o.e.action.admin.cluster (#207)
This commit refactors all classes in o.e.action.admin.cluster to 
org.opensearch.action.admin.cluster. References are updated 
throughout the codebase.

Signed-off-by: Nicholas Knize <nknize@amazon.com>
2021-03-21 20:56:34 -05:00
Rabi Panda 223ce87447 [Rename] client/sniffer (#243)
Refactor client/sniffer module as part of the Elasticsearch to OpenSearch renaming effort.

Signed-off-by: Rabi Panda <adnapibar@gmail.com>
2021-03-21 20:56:34 -05:00
Nick Knize 1203aa7302 [Rename] refactor o.e.action classes (#203)
This commit refactors top level classes in o.e.action to o.opensearch.action.
References throughout the rest of the codebase have been updated.

Signed-off-by: Nicholas Knize <nknize@amazon.com>
2021-03-21 20:56:34 -05:00
Nick Knize 0c81a5cf65 [Rename] refactor o.e.action.admin.indices (#209)
This commit refactors o.e.action.admin.indices package to
o.opensearch.action.admin.indices. References through out the codebase have been
updated to reflect the new package location.

Signed-off-by: Nicholas Knize <nknize@amazon.com>
2021-03-21 20:56:34 -05:00
Nick Knize 2aa9906c42 [Rename] ElasticsearchParseException class in server module (#169)
This commit refactors ElasticsearchParseException class in the server module to
OpenSearchParseException. References and usages throughout the rest of the
codebase are fully refactored.

Signed-off-by: Nicholas Knize <nknize@amazon.com>
2021-03-21 20:56:34 -05:00
Nick Knize e60906fc11 [Rename] ElasticsearchDirectoryReader class in server module (#176)
This commit refactors the ElasticsearchDirectoryReader class located in the
server module to OpenSearchDirectoryReader. References and usages, along with
method names, throughout the rest of the codebase are fully refactored.

Signed-off-by: Nicholas Knize <nknize@amazon.com>
2021-03-21 20:56:34 -05:00
Nick Knize bdc3158020 [Rename] ElasticsearchNodeCommand class in server module (#175)
This commit refactors the ElasticsearchNodeCommand class located in the server
module to OpenSearchNodeCommand. References and usages throughout the rest of
the codebase are fully refactored.

Signed-off-by: Nicholas Knize <nknize@amazon.com>
2021-03-21 20:56:34 -05:00
Nick Knize ccceb381db [Rename] ElasticsearchException class in server module (#165)
This commit refactors the ElasticsearchException class located in the server module
to OpenSearchException. References and usages throughout the rest of the
codebase are fully refactored.

Signed-off-by: Nicholas Knize <nknize@amazon.com>
2021-03-21 20:56:34 -05:00
Nick Knize 83e87f7e54 Remove any non oss from build, package, and distribution (#102)
This commit changes the building, packaging, and testing framework to only support OSS on different distributions.

Next steps:

completely remove -oss flag dependencies in package and build tests
move 6.x bwc testing to be an explicit option
remove any references to elastic.co download site (or replace with downloads from the OSS website)

Co-authored-by: Himanshu Setia <setiah@amazon.com>
Co-authored-by: Rabi Panda <pandarab@amazon.com>
Co-authored-by: Himanshu Setia <58999915+setiah@users.noreply.github.com>
Co-authored-by: Sarat Vemulapalli <vemsarat@amazon.com>
Signed-off-by: Peter Nied <petern@amazon.com>
2021-03-13 10:36:14 -06:00
Nick Knize b360d4fb61 [TEST] Fix Feature Flags in Test Framework and SortTemplates yaml failure (#82)
This commit adds parse logic to correctly parse feature flags in the test framework. It also fixes a test failure in cat.templates/Sort Templates yaml test.

Signed-off-by: Peter Nied <petern@amazon.com>
2021-03-13 10:36:13 -06:00
Tianli Feng fc3922aee4 Remove unused imports after x-pack feature flag removed from yaml test (#81)
Signed-off-by: Peter Nied <petern@amazon.com>
2021-03-13 10:36:13 -06:00
Nick Knize 755367f1c9 [PURIFY] Remove x-pack feature flag from yaml test (#68)
This commit removes the xpack no_xpack feature flag from the yaml test suite.

Signed-off-by: Peter Nied <petern@amazon.com>
2021-03-13 10:36:13 -06:00
Nicholas Knize f7d66c6718 Remove unused imports in RemoteClustersIT and InternalTestCluster
This commit removes unused imports that are causing checkStyle failures.

Signed-off-by: Peter Nied <petern@amazon.com>
2021-03-13 10:36:11 -06:00
Nick Knize 3f168ac85c [PURIFY] Revert "Move data stream transport and rest action to xpack (#59593)" (#28)
This commit reverts commit 2a89e13e43. Relicensing data streams from OSS to XPack.

Signed-off-by: Peter Nied <petern@amazon.com>
2021-03-13 10:36:09 -06:00
Nick Knize 82300bbe3d [PURIFY] remove all trace of x-pack monitoring (#22)
This commit removes all trace of Elastic licensed monitoring

Signed-off-by: Peter Nied <petern@amazon.com>
2021-03-13 10:36:09 -06:00
Nick Knize 70a07216bb [PURIFY] remove all trace of x-pack watcher (#21)
This commit removes all trace of Elastic licensed watcher

Signed-off-by: Peter Nied <petern@amazon.com>
2021-03-13 10:36:08 -06:00
Nick Knize 4aee8e56db [PURIFY] remove all trace of x-pack index lifecycle management (#20)
This commit removes all trace of Elastic licensed ILM.

Signed-off-by: Peter Nied <petern@amazon.com>
2021-03-13 10:36:08 -06:00
Nick Knize 3769b2c6a4 [PURIFY] remove all trace of x-pack rollups (#17)
This commit removes all trace of Elastic licensed rollups

Signed-off-by: Peter Nied <petern@amazon.com>
2021-03-13 10:36:08 -06:00
Nick Knize fb7ff93c42 [PURIFY] remove all trace of x-pack security (#16)
This commit removes all trace of the security high level rest client and other reference to x-pack security

Co-authored-by: Rabi Panda <rabipanda@icloud.com>
Signed-off-by: Peter Nied <petern@amazon.com>
2021-03-13 10:36:08 -06:00
Nick Knize 0dd4f9f281 [PURIFY] remove all trace of x-pack ccr (#11)
This committ removes all trace of Elastic licensed CCR.

Signed-off-by: Peter Nied <petern@amazon.com>
2021-03-13 10:36:07 -06:00
Nick Knize 479dd4b42a [PURIFY] remove all trace of x-pack async-search (#10)
This commit removes all trace of Elastic licensed asyc-search

Signed-off-by: Peter Nied <petern@amazon.com>
2021-03-13 10:36:07 -06:00
David Roberts 24a2bc3f62
Extend the selective muting of memory tests on Debian 8 (#67455)
The selective muting implemented for autoscaling in #67159
is extended to the ML tests that also fail when machine
memory is reported as 0.

Most of the logic to determine when memory will not be
accurately reported is now in a utility method in the
base class.

Relates #66885
Backport of #67422
2021-01-13 16:20:39 +00:00
Nhat Nguyen 6ba5ae2388 Retry synced-flush on conflict in tests (#66968)
Closes #66631
2021-01-10 22:35:06 -05:00
Nhat Nguyen 26d67c1662 Ensure notify when proxy connections disconnect (#65697)
TransportService doesn't respond to the pending requests of proxy
connections when the underlying connections get disconnected because
proxy connections do not override the getCacheKey method. Some CCS
requests would never be completed because of this bug.
2020-12-03 14:53:17 -05:00
Alan Woodward fb84b6710d
Restore use of default search and search_quote analyzers (#65491) (#65562)
In the refactoring of TextFieldMapper, we lost the ability to define
a default search or search_quote analyzer in index settings. This
commit restores that ability, and adds some more comprehensive
testing.

Fixes #65434
2020-11-26 18:34:59 +00:00
Jim Ferenczi 359b89a19b
Fix cacheability of custom LongValuesSource in TermsSetQueryBuilder (#65367) (#65389)
This change fixes the equals and hashCode methods of the custom FieldValuesSource
that is used internally to extract the value from a doc value field.
Using the field data instance to check equality prevented the query to be cached in
previous versions. Switching to the field name should make the query eligible for
caching again.
2020-11-23 22:21:01 +01:00
Nik Everett 56605e4d9a Fixup reduceRandom tests (#65263)
In aa1ea96b8698aa12bed1c4e8d704882a2a639791 I made all
`testReduceRandom` tests for aggs mimick production more precisely.
More precisely, they pick the correct "lead" result when performing
partial reduction. This is great, but, sadly, some tests assumed that we
always reduced against the "first" aggregator. This fixes those tests.

Closes #65163
2020-11-20 13:10:34 -05:00
Nik Everett 956ae93fdb Better mimick real reduction in test
This changes has `InternalAggregationTestCase` performs partial
reductions to better mimick `InternalAggregations#reduce`. In
particular, this fixes a bug when we'd attempt a partial reduction of
aggregations that were entirely unmapped but use a mapped aggregation as
the "lead" of the reduction. This was possible in the test but isn't
possible in production. The reduction "lead" is unmapped if all agg
results being reduced are unmapped.

Closes #65118
2020-11-20 13:10:34 -05:00
Sylvain Wallez b2475f9ccf
Fix parsing RareTerms aggregation response in RestHighLevelClient (#65144)
Backport of #64454

- Add LongRareTerms and StringRareTerms to the DefaultNamedXContents,
  ensure that the response of RareTerms aggregation can be parsed
  correctly.
- Add testSearchWithRareTermsAgg method to test the response of
  RareTerms aggregation can be parsed correctly.
- Add some test code to ensure the AggregationsTests can execute
  successfully.

Co-authored-by: bellengao <gbl_long@163.com>
2020-11-17 17:43:51 +01:00
Alan Woodward 0e2a9b4ac7 Fix sparse vector test 2020-11-12 20:10:07 +00:00
Alan Woodward caf143f4a5
Unused boost parameter should not throw mapping exception (#64999) (#65014)
We were correctly dealing with boosts that had an effect, but mappers
that had a silently accepted but ignored boost parameter were throwing
an error instead of continuing to ignore the boost but emitting a
warning.

Fixes #64982
2020-11-12 19:28:32 +00:00
Tim Brooks f96dccd1ec
Propogate rejected execution during bulk actions (#64886)
Currently a rejected execution exception can be swallowed when async
actions return during transport bulk actions. This includes scenarios
where we went async to perform ingest pipelines or index creation. This
commit resolves the issue by propagating a rejected exception.
2020-11-10 12:16:40 -07:00
Armin Braun 6de9198017
Increase Timeout for Waiting on Tasks in REST Tests (#64707) (#64718)
10s is pretty tight here considering that some x-pack related tests
might have a running create-index tasks in the backgroud after a test
that require multiple CS updates and can randomly take a few seconds on
slow IO etc.

closes #64580
2020-11-06 17:39:24 +01:00
Armin Braun 51e9d6f227
Revert Serializing Outbound Transport Messages on IO Threads (#64632) (#64654)
Serializing outbound transport message on the IO loop was introduced in https://github.com/elastic/elasticsearch/pull/56961. Unfortunately it turns out that this is incompatible with assumptions made by CCR code here: f22ddf822e/x-pack/plugin/ccr/src/main/java/org/elasticsearch/xpack/ccr/action/repositories/GetCcrRestoreFileChunkAction.java (L60-L61) and that are not easy to work around on short notice.

Raising reverting this move (as a temporary solution, it's still a valuable change long-term) as a blocker therefore as this seriously affects the stability of the initial phase of the CCR following by causing corrupted bytes to be send to the follower.
2020-11-05 16:29:12 +01:00
Jim Ferenczi 3e2fa09666 Fix merging of terms aggregation with compound order (#64469)
This change fixes a bug introduced in #61779 that uses a compound order to
compare buckets when merging. The bug is triggered when the compound order
uses a primary sort ordered by key (asc or desc).
This commit ensures that we always extract the primary sort when comparing keys
during merging.
The PR is marked as no-issue since the bug has not been released in any official version.
2020-11-05 12:05:19 +01:00
Jason Tedor 827dd39a12
Filter node.roles setting in transport client (#64276)
This commit filters out the node.roles setting from the transport
client, since the transport client does not take on these roles.
2020-10-28 16:24:14 -04:00
Jason Tedor 4b4c88496a
Fix checkstyle violation in ESTestCase
This commit fixes a line-length violation in ESTestCase.
2020-10-28 10:45:53 -04:00
Jason Tedor 5d42c2b06e
Deprecate the no-jdk distributions (#64275)
This commit adds logging to indicate that the no-jdk distributions are
deprecated and will be removed in a future release.
2020-10-28 10:35:23 -04:00
Armin Braun e0f73c96f7
Fix testStartCloneWithSuccessfulShardSnapshotPendingFinalization (#63966) (#64000)
We have to wait for no more operations here not for `1`. This mostly worked
because the test thread would add the listener quickly enough so that it sees the
state where either the snapshot or clone but not both have already finished
but randomly the test thread would be slow and time out on a state without snaphots in it.
2020-10-21 15:33:12 +02:00
Nik Everett 8d30766a7d
Fix scripted metric BWC serialization (backport of #63821) (#63897)
We had and an error when serializing fully reduced scripted metrics.
Small typo and sever lack of tests..... Anyway, this fixed the one
character typo and adds a bunch more tests.
2020-10-20 13:15:26 -04:00
Nhat Nguyen 9015b50e1b
Check docs limit before indexing on primary (#63273)
Today indexing to a shard with 2147483519 documents will fail that
shard. We should check the number of documents and reject the write
requests instead.

Closes #51136
2020-10-13 17:39:08 -04:00
Armin Braun f70391c6cc
Fix Broken Snapshot State Machine in Corner Case (#63534) (#63608)
This fixes a gap in testing and a bug that can occur in various forms:
When we would start a snapshot or clone related to a shard that was done
snapshotting/cloning but its overall operation was not yet finalized
at the time of starting the operation, we would base the operation off of
the wrong generation. This would not cause a corrupted repo, but would
cause the operation to be `PARTIAL`.
This commit fixes the state machine to take into account the correct generation
in this case.

Closes #63498
2020-10-13 16:05:34 +02:00
Julie Tibshirani 9e52513c7b
Add support for missing value fetchers. (#63585)
This PR implements value fetching for the following field types:
* `text` phrase and prefix subfields
* `search_as_you_type`, plus its subfields
* `token_count`, which is implemented by fetching doc values

Supporting these types helps ensure that retrieving all fields through
`"fields": ["*"]` doesn't fail because of unsupported value fetchers.
2020-10-12 17:34:21 -07:00
Przemyslaw Gomulka eadd69e1e4
Deprecate week_year in favour of weekyear date format backport(63307) (#63308)
week_year is misleading as the formatter only has a weekyear. A field
corresponding to 'Y'. 'weekyear' should be used instead

relates #60707
backports https://github.com/elastic/elasticsearch/pull/63307
2020-10-07 09:16:27 +02:00
Dawid Weiss dbcbdcc029
Set context class loader for plugin initialization (#63185)
Plugins are loaded in isolated child class loaders of the root class loader. However, some libraries depend on the context class loader being set. This commit sets the context class loader for the duration of calling each plugins constructor.

relates #52320

Co-authored-by: Ryan Ernst <ryan@iernst.net>
2020-10-06 18:00:21 -07:00
Gordon Brown 5c8b0662df
Deprecate REST access to System Indices (#63274) (Original #60945)
This PR adds deprecation warnings when accessing System Indices via the REST layer. At this time, these warnings are only enabled for Snapshot builds by default, to allow projects external to Elasticsearch additional time to adjust their access patterns.

Deprecation warnings will be triggered by all REST requests which access registered System Indices, except for purpose-specific APIs which access System Indices as an implementation detail a few specific APIs which will continue to allow access to system indices by default:

- `GET _cluster/health`
- `GET {index}/_recovery`
- `GET _cluster/allocation/explain`
- `GET _cluster/state`
- `POST _cluster/reroute`
- `GET {index}/_stats`
- `GET {index}/_segments`
- `GET {index}/_shard_stores`
- `GET _cat/[indices,aliases,health,recovery,shards,segments]`

Deprecation warnings for accessing system indices take the form:
```
this request accesses system indices: [.some_system_index], but in a future major version, direct access to system indices will be prevented by default
```
2020-10-06 13:41:40 -06:00