Previously the call to register a listener for settings updates was in
each individual service, rather than in the notification service
itself. This change ensures that each child of the notification service
gets registered with the settings update consumer.
ingest: Introduction of a bytes processor
This processor allows for human readable byte values (e.g. 1kb) to be converted to value in bytes (e.g. 1024). Internally this processor re-uses "ByteSizeValue.parseBytesSizeValue" which supports conversions up to Long.MAX_VALUE and the following units: "b", "kb", "mb", "gb", "tb", pb".
This change also introduces a generic return type for the AbstractStringProcessor to allow for code reuse while supporting a String -> T conversion. (String -> Long in this case).
The Rectangle constructor validates bounds before coerce has a chance
to normalize coordinates so it cannot be used as intermittent storage.
This commit removes the Rectangle as an intermittent storage for the
bounding box coordinates.
Fixes#31718
Significantly improve the example snippets in the documentation.
The examples are part of the test suite and checked nightly.
To help readability, the existing dataset was extended (test_emp renamed
to emp plus library).
Improve output of JDBC tests to be consistent with the CLI
Add lenient flag to JDBC asserts to allow type widening (a long is
equivalent to a integer as long as the value is the same).
AWS supports the creation and use of credentials that are only valid for a
fixed period of time. These credentials comprise three parts: the usual access
key and secret key, together with a session token. This commit adds support for
these three-part credentials to the EC2 discovery plugin and the S3 repository
plugin.
Note that session tokens are only valid for a limited period of time and yet
there is no mechanism for refreshing or rotating them when they expire without
restarting Elasticsearch. Nonetheless, this feature is already useful for
nodes that need only run for a few days, such as for training, testing or
evaluation. #29135 tracks the work towards allowing these credentials to be
refreshed at runtime.
Resolves#16428
This PR does the server side work for adding the Get Index API to the REST
high-level-client, namely moving resolving default settings to the
transport action. A follow up would be the client side changes.
* Default resolveFromHash to Hasher.NOOP
This changes the default behavior when resolving the hashing
algorithm from unrecognised hash strings, which was introduced in
#31234
A hash string that doesn't start with an algorithm identifier can
either be a malformed/corrupted hash or a plaintext password when
Hasher.NOOP is used(against warnings).
Do not make assumptions about which of the two is true for such
strings and default to Hasher.NOOP. Hash verification will subsequently
fail for malformed hashes.
Finally, do not log the potentially malformed hash as this can very
well be a plaintext password.
Resolves#31697
Reverts 58cf95a06f1defd31b16c831708ca32a5b445f98
So far the in-flight request circuit breaker has only accounted for the
on-the-wire representation of a request. However, we convert the raw
request into XContent internally which increases the overhead.
Therefore, we increase the value of the corresponding setting
`network.breaker.inflight_requests.overhead` from one to two. While this
value is still rather conservative (we assume that the representation as
structured objects has no overhead compared to the byte[]), it is closer
to reality than the current value.
Relates #31613
`MemoryCircuitBreakerTests` conflates two test aspects: It tests
individual circuit breakers as well as the circuit breaker hierarchy.
With this commit we split those two aspects into two test classes:
* Tests for individual circuit breakers stay in the current class
* Other tests are moved to `HierarchyCircuitBreakerServiceTests`
Adds a new parameter to the BlobContainer#write*Blob methods to specify whether the existing file
should be overridden or not. For some metadata files in the repository, we actually want to replace
the current file. This is currently implemented through an explicit blob delete and then a fresh write.
In case of using a cloud provider (S3, GCS, Azure), this results in 2 API requests instead of just 1.
This change will therefore allow us to achieve the same functionality using less API requests.
The xcontent parameters were not passed to the xcontent serialization
of the chain input for each chain. This could lead to wrongly stored
watches, which did not contain passwords but only their redacted counterparts, when an input inside of a chain input contained a password.
Proper cleanup of the docs snippet tests depends on detecting what is being tested (ML, Watcher, etc) this is deduced from the file path and so we must account for Windows and Unix path separators
testIncorrectPasswordHashingAlgorithm is based on the assumption
that the algorithm selected for the change password request is
different than the one selected for the NativeUsersStore.
pbkdf2_10000 is the same as pbkdf2 since 10000 is the default cost
factor for pbkdf2 and thus should not be used as an option for the
passwordHashingSettings.
Also make sure that the same algorithm is used for settings and
change password requests in other tests for consistency, even if
we expect to not reach the code where the algorithm is checked for
now.
Resolves#31696
Reverts 1c4f480794f2465c78e8e29645956f16971eeead
Some proxies require all requests to have paths starting with / since
there are no relative paths at the HTTP connection level. Elasticsearch
assumes paths are absolute. In order to run rest tests against a cluster
behind such a proxy, set the system property
tests.rest.client_path_prefix to /.
* remove explicit wrapper task
It's created by Gradle and triggers a deprecation warning
Simplify configuration
* Upgrade shadow plugin to get rid of Gradle deprecation
* Move compile configuration to base plugin
Solves Gradle deprecation warning from earlier Gradle versions
* Enable stable publishing in the Gradle build
* Replace usage of deprecated property
* bump Gradle version in build compare
It is useful to have a processor similar to
logstash-filter-fingerprint
in Elasticsearch. A processor that leverages a variety of hashing algorithms
to create cryptographically-secure one-way hashes of values in documents.
This processor introduces a pbkdf2hmac hashing scheme to fields in documents
for indexing
Before deleting a repository index generation file, BlobStoreRepository
checks for the existence of the file and then deletes it. We can save
a request here by using BlobContainer.deleteBlobIgnoringIfNotExists()
which ignores error when deleting a file that does not exist.
Since there is no way with S3 to know if a non versioned file existed
before being deleted, this pull request also changes S3BlobContainer so
that it now implements deleteBlobIgnoringIfNotExists(). It will now save
one more request (blobExist?) when appropriate. The tests and fixture
have been modified to conform the S3 API that always returns a 204/NO
CONTENT HTTP response on deletions.
Support multiple system store types
When falling back to using the system keystore and - most usually -
truststore, do not assume that it will be a JKS store, but deduct
its type from {@code KeyStore#getDefaultKeyStoreType}. This allows
the use of any store type the Security Provider supports by setting
the keystore.type java security property.
This commit removes some tests in the repository-s3 plugin that
have not been executed for 2+ years but have been maintained
for nothing. Most of the tests in AbstractAwsTestCase were
obsolete or superseded by fixture based integration tests.
This pull request merges the AzureStorageService interface and
the AzureStorageServiceImpl classes into one single
AzureStorageService class. It also removes some tests in the
repository-azure plugin that have not been executed for 2+ years.
As part of the changes in #31234,the password verification logic
determines the algorithm used for hashing the password from the
format of the stored password hash itself. Thus, it is generally
possible to validate a password even if it's associated stored hash
was not created with the same algorithm than the one currently set
in the settings.
At the same time, we introduced a check for incoming client change
password requests to make sure that the request's password is hashed
with the same algorithm that is configured to be used in the node
settings.
In the spirit of randomizing the algorithms used, the
{@code SecurityClient} used in the {@code NativeRealmIntegTests} and
{@code ReservedRealmIntegTests} would send all requests dealing with
user passwords by randomly selecting a hashing algorithm each time.
This meant that some change password requests were using a different
password hashing algorithm than the one used for the node and the
request would fail.
This commit changes this behavior in the two aforementioned Integ
tests to use the same password hashing algorithm for the node and the
clients, no matter what the request is.
Resolves#31670
Make password hashing algorithm/cost configurable for the
stored passwords of users for the realms that this applies
(native, reserved). Replaces predefined choice of bcrypt with
cost factor 10.
This also introduces PBKDF2 with configurable cost
(number of iterations) as an algorithm option for password hashing
both for storing passwords and for the user cache.
Password hash validation algorithm selection takes into
consideration the stored hash prefix and only a specific number
of algorithnm and cost factor options for brypt and pbkdf2 are
whitelisted and can be selected in the relevant setting.
