Necessary changes so that the licensing functionality can be
used in a JVM in FIPS 140 approved mode.
* Uses adequate salt length in encryption
* Changes key derivation to PBKDF2WithHmacSHA512 from a custom
approach with SHA512 and manual key stretching
* Removes redundant manual padding
Other relevant changes:
* Uses the SAH512 hash instead of the encrypted key bytes as the
key fingerprint to be included in the license specification
* Removes the explicit verification check of the encryption key
as this is implicitly checked in signature verification.
This commit removes the http.enabled setting. While all real nodes (started with bin/elasticsearch) will always have an http binding, there are many tests that rely on the quickness of not actually needing to bind to 2 ports. For this case, the MockHttpTransport.TestPlugin provides a dummy http transport implementation which is used by default in ESIntegTestCase.
closes#12792
The variadic constructor was only used in a few places and the
RepositoriesMetaData class is backed by a List anyway, so just using a
List will make it simpler to instantiate it.
xpack core contains a fork of `Cron` from quartz who's javadoc has a
`<table>` with non-html5 compatible stuff. This html5ifies the table and
switches the `:x-pack:plugin:core` project to building javadoc with
HTML5.
* Adds ClusterState to AsyncActionStep.performAction
This is needed so a new step can be created for the shrink action which
can select a node to allocate to based on the current routing rules and
the node attributes on teh discovery nodes.
x-pack/plugin/core/src/main/java/org/elasticsearch/xpack/core/indexlifec
ycle/AsyncActionStep.java
x-pack/plugin/core/src/main/java/org/elasticsearch/xpack/core/indexlifec
ycle/DeleteStep.java
x-pack/plugin/core/src/main/java/org/elasticsearch/xpack/core/indexlifec
ycle/ForceMergeStep.java
x-pack/plugin/core/src/main/java/org/elasticsearch/xpack/core/indexlifec
ycle/RolloverStep.java
x-pack/plugin/core/src/main/java/org/elasticsearch/xpack/core/indexlifec
ycle/ShrinkSetAliasStep.java
x-pack/plugin/core/src/main/java/org/elasticsearch/xpack/core/indexlifec
ycle/ShrinkStep.java
x-pack/plugin/core/src/main/java/org/elasticsearch/xpack/core/indexlifec
ycle/UpdateSettingsStep.java
x-pack/plugin/core/src/test/java/org/elasticsearch/xpack/core/indexlifec
ycle/DeleteStepTests.java
x-pack/plugin/core/src/test/java/org/elasticsearch/xpack/core/indexlifec
ycle/ForceMergeStepTests.java
x-pack/plugin/core/src/test/java/org/elasticsearch/xpack/core/indexlifec
ycle/RolloverStepTests.java
x-pack/plugin/core/src/test/java/org/elasticsearch/xpack/core/indexlifec
ycle/ShrinkSetAliasStepTests.java
x-pack/plugin/core/src/test/java/org/elasticsearch/xpack/core/indexlifec
ycle/ShrinkStepTests.java
x-pack/plugin/core/src/test/java/org/elasticsearch/xpack/core/indexlifec
ycle/UpdateSettingsStepTests.java
x-pack/plugin/index-lifecycle/src/main/java/org/elasticsearch/xpack/inde
xlifecycle/IndexLifecycleRunner.java
x-pack/plugin/index-lifecycle/src/main/java/org/elasticsearch/xpack/inde
xlifecycle/IndexLifecycleService.java
x-pack/plugin/index-lifecycle/src/test/java/org/elasticsearch/xpack/inde
xlifecycle/IndexLifecycleRunnerTests.java
* Adds single node allocation to shrink
This change adds two new steps as the first steps of the shrink action:
1. A `SetSingleNodeAllocateStep` which:
1. Determines which of the active nodes match the existing index
allocation rules
2. Randomly (using Randomness so its deterministic for testing)
picks one of the matching nodes
3. Updates the index settings to add a require allocation rule for
the node that was picked (using the
`index.routing.allocation.require._name` setting)
2. An `AllocationRoutedStep` which ensures that at least one copy of
each shard is allocated according to the new allocation rules
Note that this change also modifies the `AllocationRoutedStep` to add a
boolean field which determines whether the allocation is complete when
at least one copy of each shard matches the allocation rulees or if it
needs to wait for all shard copies to be allocated according to the
rules.
Lastly, a `randomStepKey()` method is added to `AbstractStepTestCase`
for convenience.
x-pack/plugin/core/src/main/java/org/elasticsearch/xpack/core/indexlifec
ycle/AllocateAction.java
x-pack/plugin/core/src/main/java/org/elasticsearch/xpack/core/indexlifec
ycle/AllocationRoutedStep.java
x-pack/plugin/core/src/main/java/org/elasticsearch/xpack/core/indexlifec
ycle/SetSingleNodeAllocateStep.java
x-pack/plugin/core/src/main/java/org/elasticsearch/xpack/core/indexlifec
ycle/ShrinkAction.java
x-pack/plugin/core/src/main/java/org/elasticsearch/xpack/core/indexlifec
ycle/ShrinkSetAliasStep.java
x-pack/plugin/core/src/test/java/org/elasticsearch/action/admin/indices/
settings/put/UpdateSettingsTestHelper.java
x-pack/plugin/core/src/test/java/org/elasticsearch/xpack/core/indexlifec
ycle/AbstractStepTestCase.java
x-pack/plugin/core/src/test/java/org/elasticsearch/xpack/core/indexlifec
ycle/AllocationRoutedStepTests.java
x-pack/plugin/core/src/test/java/org/elasticsearch/xpack/core/indexlifec
ycle/SetSingleNodeAllocateStepTests.java
x-pack/plugin/core/src/test/java/org/elasticsearch/xpack/core/indexlifec
ycle/ShrinkActionTests.java
* Fixes AllocationRoutedStep when `waitForAllShardCopies=false`
This change fixes `AllocationRoutedStep` so that when
`waitForAllShardCopies=false` we wait for any shard copy of each shard
to be allocated according to the allocation rules rather than
specifically the primary of each shard.
x-pack/plugin/core/src/main/java/org/elasticsearch/xpack/core/indexlifec
ycle/AllocationRoutedStep.java
x-pack/plugin/core/src/test/java/org/elasticsearch/xpack/core/indexlifec
ycle/AllocationRoutedStepTests.java
* Corrects Licence headers and typo
x-pack/plugin/core/src/main/java/org/elasticsearch/xpack/core/indexlifec
ycle/SetSingleNodeAllocateStep.java
x-pack/plugin/core/src/test/java/org/elasticsearch/xpack/core/indexlifec
ycle/SetSingleNodeAllocateStepTests.java
* master: (24 commits)
Watcher: Ensure mail message ids are unique per watch action (#30112)
REST: Remove GET support for clear cache indices (#29525)
SQL: Correct error message (#30138)
Require acknowledgement to start_trial license (#30135)
Fix a bug in FieldCapabilitiesRequest#equals and hashCode. (#30181)
SQL: Add BinaryMathProcessor to named writeables list (#30127)
Tests: Use buildDir as base for generated-resources (#30191)
Fix SliceBuilderTests#testRandom failures
Build: Fix deb version to use tilde with prerelease versions (#29000)
Fix edge cases in CompositeKeyExtractorTests (#30175)
Document time unit limitations for date histograms (#30177)
Add support for field capabilities to the high-level REST client. (#29664)
Remove licenses missed by the migration (#30128)
[DOCS] Updates docker installation package details (#30110)
Fix TermsSetQueryBuilder.doEquals() method (#29629)
[Monitoring] Remove unhelpful Monitoring tests (#30144)
[Test] Fix RenameProcessorTests.testRenameExistingFieldNullValue() (#29655)
add copyright/scope configuration for intellij to Contributing Guide (#29688)
[test] include oss tar in packaging tests (#30155)
TEST: Update settings should go through cluster state (#29682)
...
This is related to #30134. It modifies the start_trial action to require
an acknowledgement parameter in the rest request to actually start the
trial license. There are backwards compatibility issues as prior ES
versions did not support this parameter. To handle this, it is assumed
that a request coming from a node prior to 6.3 is acknowledged. And
attempts to write a non-acknowledged request to a prior to 6.3 node will
throw an exception.
Additionally this PR adds messages about the trial license the user is
generating.
The purpose of this change is to reflect what is the future way of configuring aliases in
the Rollover action for ILM. With this change, users will be able to edit all desired rollover-related
information in the index's template, so that the index-lifecycle policy does not need to be dependent on it.
With the move of X-Pack to a module, the classpath for the scripts needs
to be adjusted. This was done on Unix, but not for Windows. This commit
addresses Windows.
This commit makes x-pack a module and adds it to the default
distrubtion. It also creates distributions for zip, tar, deb and rpm
which contain only oss code.
Previously, phase X's `after` step had `X` as its associated phase. This causes confusion because
we have only entered phase `X` once the `after` step is complete. Therefore, this refactor
pushes the after's phase to be associated with the previous phase. This first phase is an exception.
The first phase's `after` step is associated with the first phase (not some non-existent prior phase).
- Renames EnoughShardsWaitStep to ShrunkShardsAllocatedStep
- Changes ShrunkShardsAllocatedStep to check the shards of the shrunken index rather than the current one
- shrink index prefix is now passed into the steps of the shrink aciton
- Related Test Changes
Adds some missing tests including checking the hashcode and equals methods of `DeleteStep`, `StepKey`, and `TerminalPolicyStep` as well as adding a test for `DeleteAction.toSteps()`
- added ShrinkStep/Tests
- AsyncActionStep now passes in IndexMetaData instead of Index
- Delete usage of ClusterStateActionStep
- with ClusterStateActionStep gone, InitializePolicyContextStep
is the only other ClusterState-nonWait step
- Migrate setting-updates to UpdateSettingsStep
Also renames EnoughShardsWaitStep to ReplicasAllocatedStep, removes it from the allocate action and adds a check that th number of replicas in the cluster state is correct to it.
Various classes had some code that was not used and is not going to be needed so this change cleans up those classes so we don’t have dead code hanging around
The force-merge is an a TODO state due to the
unresolved issue around best_compression.
- updated ReadOnlyStep with tests
- implemented an update to the ForceMergeAction
- added UpdateBestCompressionSettingsStep
- added tests for SegmentCountStep
Looks like we need to split out the tests of core classes to core
and index-lifecycle ones stay in index-lifecycle.
I believe I got everything, although I may have missed at least one thing
checked status with
$ ./gradlew :x-pack-elasticsearch:plugin:index-lifecycle:check -Dtests.seed=39838421912001B4
$ ./gradlew :x-pack-elasticsearch:plugin:core:check -Dtests.seed=39838421912001B4
other things done in this PR:
- removal of a few unused variables/thrown exceptions/imports
- fix TimeseriesLifecycleTypeTests
- an all null AllocateAction was created
- fix AllocateActionTests
- woops. -Dtests.seed=39838421912001B4 resulted in two `null`s and an emptyMap.
this resulted in a test failure.
Specifically this change makes it optional to:
* Specify `includes`, `excludes` and `requires`maps in the allocate action as long as at least one fo the options is specified and is not an empty map
* Specify an `after` parameter on a phase. If no `after` value is specified `TimeValue.ZERO` is used and the phase will be moved to as soon as the previous phase reports `ACTIONS COMPLETED`. `after` is always non-null when we are serialising the Phase.
* Specify a `type` for a LifecyclePolicy. If no `type` is specified `TimeSeriesLifecycleType.INSTANCE` is used since this is currently the only production `type`. `type` is always non-null when we are serialising the LifecyclePolicy.
This PR adds a new setting called `index.lifecycle.date` that
the ShrinkAction will be responsible for populating in the newly created index.
This way, we can continue to know when we should be executing the next phase
relative to the original index creation date, and not that of the shrunken index.
This does the following in sequential service polls
1. sets the index to read-only and runs shrink with a modified `index.lifecycle.name` setting set to `null`.
2. checks to see if shrink is complete, if it is...
b. set target index's `index.lifecycle.*` settings to the original index's values.
3. if not complete, just wait till next iteration
4. if operating on shrunken index, delete old index and add it as an alias to shrunken index
* Adds Allocate lifcycle action
* Addresses review comments
Still need to make a change in core for the FilterAllocationDecider to make the execute logic simpler
* Addresses more review comments
* Adds randomMap method to AllocateActionTests
* Addresses further review comments
* Improves handling of exceptions in Index Lifecycle
This change improves a few different aspects:
* If an exception occurs executing the lifecycle of one index it is caught, logged and other indexes are still processed
* If the lifecycle policy specified in the settings does not exist an error is logged
* Fixes the exception when the delete action is run which occurs because Phase attempts to update the phase and action settings for the deleted index. A `LifecycleAction.indexSurvives()` method is introduced which defaults to `true` but can be overridden to indicate whether the index survives following completion of the action.
* Adds test
This change moves the Action classes and referenced data model classes to the new :x-pack-elasticsearch:plugin:core project in preparation for splitting the x-pack features into their own gradle modules.
Note that the TransportAction classes had to be promoted to their own class file (rather than being inner classes to their Action) so they can remain in the plugin project (and will late be move to the `index-lifecycle` project when its created.
Ioannis Kakavas
Ryan Ernst
Colin Goodheart-Smithe
Nhat Nguyen
Michael Basnight
Nik Everett
Tal Levy
Tim Brooks
Martijn van Groningen
Jason Tedor