This commit adds Delete Repository, the associated docs and tests for
the high level REST API client. It also cleans up a seemingly innocuous
line in the RestDeleteRepositoryAction and some naming in SnapshotIT.
Relates #27205
* master:
Reduce CLI scripts to one-liners (#30759)
SQL: Preserve scoring in bool queries (#30730)
QA: Switch rolling upgrade to 3 nodes (#30728)
[TEST] Enable DEBUG logging on testAutoQueueSizingWithMax
[ML] Don't install empty ML metadata on startup (#30751)
Add assertion on removing copy_settings (#30748)
bump lucene version for 6_3_0
[DOCS] Mark painless execute api as experimental (#30710)
disable annotation processor for docs (#30610)
Add more script contexts (#30721)
Fix default shards count in create index docs (#30747)
Mute testCorruptFileThenSnapshotAndRestore
This commit reduces the Linux CLI scripts to one-liners by moving all of
the redundant logic to an elasticsearch-cli script. This commit is only
the Linux side, a follow-up will do this for Windows too.
Make all bool constructs use match/should (that is a query context) as
that is controlled and changed to a filter context by ES automatically
based on the sort order (_doc, field vs _sort) and trackScores.
Fix#29685
Switches the rolling upgrade tests from upgrading two nodes to upgrading
three nodes which is much more realistic and much better able to find
unexpected bugs. It upgrades the nodes one at a time and runs tests
between each upgrade. As such this now has four test runs:
1. Old
2. One third upgraded
3. Two thirds upgraded
4. Upgraded
It sets system properties so the tests can figure out which stage they
are in. It reuses the same yml tests for the "one third" and "two
thirds" cases because they are *almost* the same case.
This rewrites the yml-based indexing tests to be Java based because the
yml-based tests can't handle different expected values for the counts.
And the indexing tests need that when they are run twice.
This change is to support rolling upgrade from a pre-6.3 default
distribution (i.e. without X-Pack) to a 6.3+ default distribution
(i.e. with X-Pack).
The ML metadata is no longer eagerly added to the cluster state
as soon as the master node has X-Pack available. Instead, it
is added when the first ML job is created.
As a result all methods that get the ML metadata need to be able
to handle the situation where there is no ML metadata in the
current cluster state. They do this by behaving as though an
empty ML metadata was present. This logic is encapsulated by
always asking for the current ML metadata using a static method
on the MlMetadata class.
Relates #30731
The copy_settings parameter will be removed in Elasticsearch 8.0.0. This
commit adds an assertion message that to clean up this code when master
is bumped to 8.0.0.
* disable annotation processor for docs
Could not find evidence that the log4j annotation processor is used.
The compiler flag enables the Gradle 5.0 behavior
Closes#30476
* Disable annotation processors for all tests
* remove redundant `-proc:none` already handled by required plugins
* Revert unintentional changes
Added dedicated script contexts for:
* script function score
* script sorting
* terms_set query
Scripts for these contexts will either have a specific return value or
use scoring and therefor in the future will need their own scripting classes.
Relates to #30511
* master:
Scripting: Remove getDate methods from ScriptDocValues (#30690)
Upgrade to Lucene-7.4.0-snapshot-59f2b7aec2 (#30726)
[Docs] Fix single page :docs:check invocation (#30725)
Docs: Add uptasticsearch to list of clients (#30738)
[DOCS] Removes out-dated x-pack/docs/en/index.asciidoc
[DOCS] Removes redundant index.asciidoc files (#30707)
[TEST] Reduce forecast overflow to disk test memory limit (#30727)
Plugins: Remove meta plugins (#30670)
[DOCS] Moves X-Pack configurationg pages in table of contents (#30702)
TEST: Add engine log to testCorruptFileThenSnapshotAndRestore
[ML][TEST] Fix bucket count assertion in ModelPlotsIT (#30717)
[ML][TEST] Make AutodetectMemoryLimitIT less fragile (#30716)
Default copy settings to true and deprecate on the REST layer (#30598)
[Build] Add test admin when starting gradle run with trial license and
This implementation lazily (on 1st forecast request) checks for available diskspace and creates a subfolder for storing data outside of Lucene indexes, but as part of the ES data paths.
Tests: Fail if test watches could not be triggered (#30392)
[ML] add version information in case of crash of native ML process (#30674)
Make TransportClusterStateAction abide to our style (#30697)
Change required version for Get Settings transport API changes to 6.4.0 (#30706)
[DOCS] Fixes edit URLs for stack overview (#30583)
Silence sleep based watcher test
[TEST] Adjust version skips for movavg/movfn tests
[DOCS] Replace X-Pack terms with attributes
[ML] Clean left behind model state docs (#30659)
Correct typos
filters agg docs duplicated 'bucket' word removal (#30677)
top_hits doc example description update (#30676)
[Docs] Replace InetSocketTransportAddress with TransportAdress (#30673)
[TEST] Account for increase in ML C++ memory usage (#30675)
User proper write-once semantics for GCS repository (#30438)
Remove bogus file accidentally added
Add detailed assert message to IndexAuditUpgradeIT (#30669)
Adjust fast forward for token expiration test (#30668)
Improve explanation in rescore (#30629)
Deprecate `nGram` and `edgeNGram` names for ngram filters (#30209)
Watcher: Fix watch history template for dynamic slack attachments (#30172)
Fix _cluster/state to always return cluster_uuid (#30656)
[Tests] Add debug information to CorruptedFileIT
# Conflicts:
# test/framework/src/main/java/org/elasticsearch/indices/analysis/AnalysisFactoryTestCase.java
The getDate() and getDates() existed prior to 5.x on long fields in
scripting. In 5.x, a new Date type for ScriptDocValues was added. The
getDate() and getDates() methods were left on long fields and added to date
fields to ease the transition. This commit removes those methods for
7.0.
The docs/README shows how to run the :docs:check goal on a subset of pages,
however using the current example fails. Removing the masking character before
the first wildcard fixes the problem.
By default ML native processes are only allowed to use
30% of RAM, so the previous 2GB setting prevented the
test passing on VMs with only 4GB RAM. This change
reduces the limit to 1200MB, which means it can now
pass on VMs with 4GB RAM.
Meta plugins existed only for a short time, in order to enable breaking
up x-pack into multiple plugins. However, now that x-pack is no longer
installed as a plugin, the need for them has disappeared. This commit
removes the meta plugins infrastructure.
As the first record is random, there's a chance it will
be aligned on a bucket start. Thus we need to check the
bucket count is in [23, 24].
Closes#30715
These tests aim to check the set model memory limit is
respected. Additionally, it was asserting counts of
partition, by, over fields in an attempt to check that
the used memory is spent meaningfully. However, this
made the tests fragile, as changes in the ml-cpp could
lead to CI failures.
This commit removes those assertions. We are working on
adding tests in ml-cpp that will compensate.
diskspace and creates a subfolder for storing data outside of Lucene
indexes, but as part of the ES data paths.
Details:
- tmp storage is managed and does not allow allocation if disk space is
below a threshold (5GB at the moment)
- tmp storage is supposed to be managed by the native component but in
case this fails cleanup is provided:
- on job close
- on process crash
- after node crash, on restart
- available space is re-checked for every forecast call (the native
component has to check again before writing)
Note: The 1st path that has enough space is chosen on job open (job
close/reopen triggers a new search)
Watcher tests now always fail hard when watches that were
tried to be triggered in a test using the trigger() method,
but could not because they were not found on any of the
nodes in the cluster.
This change adds version information in case a native ML process crashes, the version is important for choosing the right symbol files when analyzing the crash. Adding the version combines all necessary information on one line.
relates elastic/ml-cpp#94
I still do not like == false. However, I am so use to reading it that
today I read this line of code and could not understand how it could
possibly be doing the right thing. It was only when I finally noticed
the ! that the code made sense. This commit changes this code to be in
our style of == false. I still do not like == false.
Get Settings API changes have now been backported to version 6.4, and
therefore the latest version must send and expect the extra fields when
communicating with 6.4+ code.
Relates #29229#30494
It is possible for state documents to be
left behind in the state index. This may be
because of bugs or uncontrollable scenarios.
In any case, those documents may take up quite
some disk space when they add up. This commit
adds a step in the expired data deletion that
is part of the daily maintenance service. The
new step searches for state documents that
do not belong to any of the current jobs and
deletes them.
Closes#30551
There's no need for an extra blobExists() call when writing a blob to the GCS service. GCS provides
an option (with stronger consistency guarantees) on the insert method that guarantees that the
blob that's uploaded does not already exist.
Relates to #19749
Adjust fast forward for token expiration test
Adjusts the maximum fast forward time for token expiration tests
to be 5 seconds before actual token expiration so that the test
won't fail even when upperlimit is randomly selected.
Resolves: #30062
Currently in a rescore request if window_size is smaller than
the top N documents returned (N=size), explanation of scores could be incorrect
for documents that were a part of topN and not part of rescoring.
This PR corrects this, but saving in RescoreContext docIDs of documents
for which rescoring was applied, and adding rescoring explanation
only for these docIDs.
Closes#28725
The camel case name `nGram` should be removed in favour of `ngram` and
similar for `edgeNGram` and `edge_ngram`. Before removal, we need to
deprecate the camel case names first. This change adds deprecation
warnings for indices with versions 6.4.0 and higher and logs deprecation
warnings.
Michael Basnight
Nhat Nguyen
Jason Tedor
Costin Leau
Nik Everett
Lee Hinman
David Roberts
Boaz Leskes
Martijn van Groningen
Alpar Torok
Adam Chalkley
Ryan Ernst
Christoph Büscher
William Dearden
lcawl
Dimitris Athanasiou
Hendrik Muhs
Alexander Reelsen
tomcallahan
Zachary Tong
Mayya Sharipova
Piotr Prądzyński
Yannick Welsch
Tim Vernum
Ioannis Kakavas