This commit adds a tombstone document into Lucene for every No-op.
With this change, Lucene index is expected to have a complete history
of operations like Translog. In fact, this guarantee is subjected to the
soft-deletes retention merge policy.
Relates #29530
* master:
Fix message content in users tool (#30293)
[DOCS] Fixes links to breaking changes
[DOCS] Adds new installation package details (#29590)
Revert "Build: Move gradle wrapper jar to a dot dir (#30146)"
The elasticsearch-users utility had various messages that were
outdated or incorrect. This commit updates the output from this
command to reflect current terminology and configuration.
The variadic constructor was only used in a few places and the
RepositoriesMetaData class is backed by a List anyway, so just using a
List will make it simpler to instantiate it.
Cause the CLI to ignore commands that are empty or consist only of
newlines. This is a fairly standard thing for SQL CLIs to do.
It looks like:
```
sql> ;
sql>
|
| ;
sql> exit;
Bye!
```
I think I *could* have implemented this with a `CliCommand` that throws
out empty string but it felt simpler to bake it in to the `CliRepl`.
Closes#30000
xpack core contains a fork of `Cron` from quartz who's javadoc has a
`<table>` with non-html5 compatible stuff. This html5ifies the table and
switches the `:x-pack:plugin:core` project to building javadoc with
HTML5.
* Adds ClusterState to AsyncActionStep.performAction
This is needed so a new step can be created for the shrink action which
can select a node to allocate to based on the current routing rules and
the node attributes on teh discovery nodes.
x-pack/plugin/core/src/main/java/org/elasticsearch/xpack/core/indexlifec
ycle/AsyncActionStep.java
x-pack/plugin/core/src/main/java/org/elasticsearch/xpack/core/indexlifec
ycle/DeleteStep.java
x-pack/plugin/core/src/main/java/org/elasticsearch/xpack/core/indexlifec
ycle/ForceMergeStep.java
x-pack/plugin/core/src/main/java/org/elasticsearch/xpack/core/indexlifec
ycle/RolloverStep.java
x-pack/plugin/core/src/main/java/org/elasticsearch/xpack/core/indexlifec
ycle/ShrinkSetAliasStep.java
x-pack/plugin/core/src/main/java/org/elasticsearch/xpack/core/indexlifec
ycle/ShrinkStep.java
x-pack/plugin/core/src/main/java/org/elasticsearch/xpack/core/indexlifec
ycle/UpdateSettingsStep.java
x-pack/plugin/core/src/test/java/org/elasticsearch/xpack/core/indexlifec
ycle/DeleteStepTests.java
x-pack/plugin/core/src/test/java/org/elasticsearch/xpack/core/indexlifec
ycle/ForceMergeStepTests.java
x-pack/plugin/core/src/test/java/org/elasticsearch/xpack/core/indexlifec
ycle/RolloverStepTests.java
x-pack/plugin/core/src/test/java/org/elasticsearch/xpack/core/indexlifec
ycle/ShrinkSetAliasStepTests.java
x-pack/plugin/core/src/test/java/org/elasticsearch/xpack/core/indexlifec
ycle/ShrinkStepTests.java
x-pack/plugin/core/src/test/java/org/elasticsearch/xpack/core/indexlifec
ycle/UpdateSettingsStepTests.java
x-pack/plugin/index-lifecycle/src/main/java/org/elasticsearch/xpack/inde
xlifecycle/IndexLifecycleRunner.java
x-pack/plugin/index-lifecycle/src/main/java/org/elasticsearch/xpack/inde
xlifecycle/IndexLifecycleService.java
x-pack/plugin/index-lifecycle/src/test/java/org/elasticsearch/xpack/inde
xlifecycle/IndexLifecycleRunnerTests.java
* Adds single node allocation to shrink
This change adds two new steps as the first steps of the shrink action:
1. A `SetSingleNodeAllocateStep` which:
1. Determines which of the active nodes match the existing index
allocation rules
2. Randomly (using Randomness so its deterministic for testing)
picks one of the matching nodes
3. Updates the index settings to add a require allocation rule for
the node that was picked (using the
`index.routing.allocation.require._name` setting)
2. An `AllocationRoutedStep` which ensures that at least one copy of
each shard is allocated according to the new allocation rules
Note that this change also modifies the `AllocationRoutedStep` to add a
boolean field which determines whether the allocation is complete when
at least one copy of each shard matches the allocation rulees or if it
needs to wait for all shard copies to be allocated according to the
rules.
Lastly, a `randomStepKey()` method is added to `AbstractStepTestCase`
for convenience.
x-pack/plugin/core/src/main/java/org/elasticsearch/xpack/core/indexlifec
ycle/AllocateAction.java
x-pack/plugin/core/src/main/java/org/elasticsearch/xpack/core/indexlifec
ycle/AllocationRoutedStep.java
x-pack/plugin/core/src/main/java/org/elasticsearch/xpack/core/indexlifec
ycle/SetSingleNodeAllocateStep.java
x-pack/plugin/core/src/main/java/org/elasticsearch/xpack/core/indexlifec
ycle/ShrinkAction.java
x-pack/plugin/core/src/main/java/org/elasticsearch/xpack/core/indexlifec
ycle/ShrinkSetAliasStep.java
x-pack/plugin/core/src/test/java/org/elasticsearch/action/admin/indices/
settings/put/UpdateSettingsTestHelper.java
x-pack/plugin/core/src/test/java/org/elasticsearch/xpack/core/indexlifec
ycle/AbstractStepTestCase.java
x-pack/plugin/core/src/test/java/org/elasticsearch/xpack/core/indexlifec
ycle/AllocationRoutedStepTests.java
x-pack/plugin/core/src/test/java/org/elasticsearch/xpack/core/indexlifec
ycle/SetSingleNodeAllocateStepTests.java
x-pack/plugin/core/src/test/java/org/elasticsearch/xpack/core/indexlifec
ycle/ShrinkActionTests.java
* Fixes AllocationRoutedStep when `waitForAllShardCopies=false`
This change fixes `AllocationRoutedStep` so that when
`waitForAllShardCopies=false` we wait for any shard copy of each shard
to be allocated according to the allocation rules rather than
specifically the primary of each shard.
x-pack/plugin/core/src/main/java/org/elasticsearch/xpack/core/indexlifec
ycle/AllocationRoutedStep.java
x-pack/plugin/core/src/test/java/org/elasticsearch/xpack/core/indexlifec
ycle/AllocationRoutedStepTests.java
* Corrects Licence headers and typo
x-pack/plugin/core/src/main/java/org/elasticsearch/xpack/core/indexlifec
ycle/SetSingleNodeAllocateStep.java
x-pack/plugin/core/src/test/java/org/elasticsearch/xpack/core/indexlifec
ycle/SetSingleNodeAllocateStepTests.java
This commit refactors the DataStreamDiagnostics class
achieving the following advantages:
- simpler code; by encapsulating the moving bucket histogram
into its own class
- better performance; by using an array to store the buckets
instead of a map
- explicit handling of gap buckets; in preparation of fixing #30080
Starting with the refactoring in https://github.com/elastic/elasticsearch/pull/22778 (released in 5.3) we may fail to properly replicate operation when a mapping update on master fails. If a bulk
operations needs a mapping update half way, it will send a request to the master before continuing
to index the operations. If that request times out or isn't acked (i.e., even one node in the cluster
didn't process it within 30s), we end up throwing the exception and aborting the entire bulk. This is
a problem because all operations that were processed so far are not replicated any more to the
replicas. Although these operations were never "acked" to the user (we threw an error) it cause the
local checkpoint on the replicas to lag (on 6.x) and the primary and replica to diverge.
This PR does a couple of things:
1) Most importantly, treat *any* mapping update failure as a document level failure, meaning only
the relevant indexing operation will fail.
2) Removes the mapping update callbacks from `IndexShard.applyIndexOperationOnPrimary` and
similar methods for simpler execution. We don't use exceptions any more when a mapping
update was successful.
I think we need to do more work here (the fact that a single slow node can prevent those mappings
updates from being acked and thus fail operations is bad), but I want to keep this as small as I can
(it is already too big).
The overall NOTICE file for the ML X-Pack module should
include the notices from the 3rd party C++ components as
well as the 3rd party Java components.
Currently, the only way to get the REST response for the `/_cluster/state`
call to return the `cluster_uuid` is to request the `metadata` metrics,
which is one of the most expensive response structures. However, external
monitoring agents will likely want the `cluster_uuid` to correlate the
response with other API responses whether or not they want cluster
metadata.
We had a number of awaitsFix links that weren't updated after the xpack
merge.
Where possible I changed the links to the new locations, but in some
circumstances the original ticket was closed (suggesting the awaitsfix
should be removed) or was otherwise unclear the status.
* master: (24 commits)
Watcher: Ensure mail message ids are unique per watch action (#30112)
REST: Remove GET support for clear cache indices (#29525)
SQL: Correct error message (#30138)
Require acknowledgement to start_trial license (#30135)
Fix a bug in FieldCapabilitiesRequest#equals and hashCode. (#30181)
SQL: Add BinaryMathProcessor to named writeables list (#30127)
Tests: Use buildDir as base for generated-resources (#30191)
Fix SliceBuilderTests#testRandom failures
Build: Fix deb version to use tilde with prerelease versions (#29000)
Fix edge cases in CompositeKeyExtractorTests (#30175)
Document time unit limitations for date histograms (#30177)
Add support for field capabilities to the high-level REST client. (#29664)
Remove licenses missed by the migration (#30128)
[DOCS] Updates docker installation package details (#30110)
Fix TermsSetQueryBuilder.doEquals() method (#29629)
[Monitoring] Remove unhelpful Monitoring tests (#30144)
[Test] Fix RenameProcessorTests.testRenameExistingFieldNullValue() (#29655)
add copyright/scope configuration for intellij to Contributing Guide (#29688)
[test] include oss tar in packaging tests (#30155)
TEST: Update settings should go through cluster state (#29682)
...
Email message IDs are supposed to be unique. In order to guarantee this,
we need to take the action id of a watch action into account as well,
not just the watch id from the watch execution context. This prevents
that two actions from the same watch execution end up with the same
message id.
This is related to #30134. It modifies the start_trial action to require
an acknowledgement parameter in the rest request to actually start the
trial license. There are backwards compatibility issues as prior ES
versions did not support this parameter. To handle this, it is assumed
that a request coming from a node prior to 6.3 is acknowledged. And
attempts to write a non-acknowledged request to a prior to 6.3 node will
throw an exception.
Additionally this PR adds messages about the trial license the user is
generating.
Currently the test picks random java.util.TimeZone ids in some places.
Internally we still need to convert back to joda DateTimeZone by id
occassionally (e.g. when serializing to pre 6.3 versions). There are
some deprecated "SystemV/*" time zones that Jodas DateTimeZone refuses
to convert. This change excludes those rare cases from the set of
allowed random time zones. It would be quiet odd for them to appear in
practice.
Closes#30156
A few of the old style license got kept around because their comment
string did not start with a space. This caused the license check to not
see it as a license and skip it. This commit cleans it up.
The purpose of this change is to reflect what is the future way of configuring aliases in
the Rollover action for ILM. With this change, users will be able to edit all desired rollover-related
information in the index's template, so that the index-lifecycle policy does not need to be dependent on it.
* es/master:
Watcher: Fold two smoke test projects into smoke-test-watcher (#30137)
In the field capabilities API, deprecate support for providing fields in the request body. (#30157)
Set JAVA_HOME before forking setup commands (#29647)
Remove animal sniffer from low-level REST client (#29646)
Cleanup .gitignore (#30145)
Do not add noop from local translog to translog again (#29637)
Build: Assert jar LICENSE and NOTICE files match
Correct transport compression algorithm in docs (#29645)
[Test] Fix docs check for DEB package in packaging tests (#30126)
Painless: Docs Clean Up (#29592)
Fixes Eclipse build for sql jdbc project (#30114)
Remove reference to `not_analyzed`.
[Docs] Add community analysis plugin (#29612)
Adds tasks that check that the all jars that we build have LICENSE.txt
and NOTICE.txt files and that the files are correct. Sets check to
depend on these task.
This is mostly there for extra parnoia because we automatically
configure all Jar tasks to include the LICENSE.txt and NOTICE.txt
files anyway. But it is quite possible to add configuration to those
tasks that would override either file.
This causes check to depend on several more things than it used to.
Take, for example, javadoc:
check depends on the new verifyJavadocJarNotice which depends on
extractJavadocJar which depends on javadocJar which depends on
javadoc, this check now depends on javadoc.
The bundled configuration isn't recognised by eclipse so these
dependencies are missed when it imports the `x-pack:plugin:sql:jdbc`
project. This change makes these dependencies compile dependencies if
the build is running for Eclipse.
Tests need to wait for changes to the job's established memory usage to
propagate and an over enthusiastic optimisation meant jobs were updated
from stale state causing recent change to be lost.
This commit fixes the classpath for the SQL CLI tool on Windows. As the
x-pack bin folder was collapsed into the distribution bin folder, the
location of the classpath here needed to no longer contain the old
plugins directory.
This commit adds the distribution type to the startup scripts so that we
can discern from log output and the main response the type of the
distribution (deb/rpm/tar/zip).
With the move of X-Pack to a module, the classpath for the scripts needs
to be adjusted. This was done on Unix, but not for Windows. This commit
addresses Windows.
