This is related to #27260. The elasticsearch-nio jar is supposed to be
a library opposed to a framework. Currently it internally logs certain
exceptions. This commit modifies it to not rely on logging. Instead
exception handlers are passed by the applications that use the jar.
This commit reduces the Linux CLI scripts to one-liners by moving all of
the redundant logic to an elasticsearch-cli script. This commit is only
the Linux side, a follow-up will do this for Windows too.
Make all bool constructs use match/should (that is a query context) as
that is controlled and changed to a filter context by ES automatically
based on the sort order (_doc, field vs _sort) and trackScores.
Fix#29685
This change is to support rolling upgrade from a pre-6.3 default
distribution (i.e. without X-Pack) to a 6.3+ default distribution
(i.e. with X-Pack).
The ML metadata is no longer eagerly added to the cluster state
as soon as the master node has X-Pack available. Instead, it
is added when the first ML job is created.
As a result all methods that get the ML metadata need to be able
to handle the situation where there is no ML metadata in the
current cluster state. They do this by behaving as though an
empty ML metadata was present. This logic is encapsulated by
always asking for the current ML metadata using a static method
on the MlMetadata class.
Relates #30731
By default ML native processes are only allowed to use
30% of RAM, so the previous 2GB setting prevented the
test passing on VMs with only 4GB RAM. This change
reduces the limit to 1200MB, which means it can now
pass on VMs with 4GB RAM.
Meta plugins existed only for a short time, in order to enable breaking
up x-pack into multiple plugins. However, now that x-pack is no longer
installed as a plugin, the need for them has disappeared. This commit
removes the meta plugins infrastructure.
As the first record is random, there's a chance it will
be aligned on a bucket start. Thus we need to check the
bucket count is in [23, 24].
Closes#30715
These tests aim to check the set model memory limit is
respected. Additionally, it was asserting counts of
partition, by, over fields in an attempt to check that
the used memory is spent meaningfully. However, this
made the tests fragile, as changes in the ml-cpp could
lead to CI failures.
This commit removes those assertions. We are working on
adding tests in ml-cpp that will compensate.
diskspace and creates a subfolder for storing data outside of Lucene
indexes, but as part of the ES data paths.
Details:
- tmp storage is managed and does not allow allocation if disk space is
below a threshold (5GB at the moment)
- tmp storage is supposed to be managed by the native component but in
case this fails cleanup is provided:
- on job close
- on process crash
- after node crash, on restart
- available space is re-checked for every forecast call (the native
component has to check again before writing)
Note: The 1st path that has enough space is chosen on job open (job
close/reopen triggers a new search)
Watcher tests now always fail hard when watches that were
tried to be triggered in a test using the trigger() method,
but could not because they were not found on any of the
nodes in the cluster.
This change adds version information in case a native ML process crashes, the version is important for choosing the right symbol files when analyzing the crash. Adding the version combines all necessary information on one line.
relates elastic/ml-cpp#94
It is possible for state documents to be
left behind in the state index. This may be
because of bugs or uncontrollable scenarios.
In any case, those documents may take up quite
some disk space when they add up. This commit
adds a step in the expired data deletion that
is part of the daily maintenance service. The
new step searches for state documents that
do not belong to any of the current jobs and
deletes them.
Closes#30551
Adjust fast forward for token expiration test
Adjusts the maximum fast forward time for token expiration tests
to be 5 seconds before actual token expiration so that the test
won't fail even when upperlimit is randomly selected.
Resolves: #30062
The part of the history template responsible for slack attachments had a
dynamic mapping configured which could lead to problems, when a string
value looking like a date was configured in the value field of an
attachment.
This commit fixes the template by setting this field always to text.
This also requires a change in the template numbering to be sure this
will be applied properly when starting watcher.
This commit removes xpack from being a meta-plugin-as-a-module.
It also fixes a couple tests which were missing task dependencies, which
failed once the gradle execution order changed.
Removes dependency for server's version from the JDBC driver code. This
should allow us to dramatically reduce driver's size by removing the
server dependency from the driver.
Relates #29856
This commit increases the logging level around search to aid in
debugging failures in LicensingTests#testSecurityActionsByLicenseType
where we are seeing all shards failed error while trying to search the
security index.
See #30301
This change adds a `listTasks` method to the high level java
ClusterClient which allows listing running tasks through the
task management API.
Related to #27205
* Refactors ClientHelper to combine header logic
This change removes all the `*ClientHelper` classes which were
repeating logic between plugins and instead adds
`ClientHelper.executeWithHeaders()` and
`ClientHelper.executeWithHeadersAsync()` methods to centralise the
logic for executing requests with stored security headers.
* Removes Watcher headers constant
When the encrpytion of sensitive date is enabled, test that a
scheduled watch is executed as expected and produces the correct value
from a secret in the basic auth header.
Make SSLContext reloadable
This commit replaces all customKeyManagers and TrustManagers
(ReloadableKeyManager,ReloadableTrustManager,
EmptyKeyManager, EmptyTrustManager) with instances of
X509ExtendedKeyManager and X509ExtendedTrustManager.
This change was triggered by the effort to allow Elasticsearch to
run in a FIPS-140 environment. In JVMs running in FIPS approved
mode, only SunJSSE TrustManagers and KeyManagers can be used.
Reloadability is now ensured by a volatile instance of SSLContext
in SSLContectHolder.
SSLConfigurationReloaderTests use the reloadable SSLContext to
initialize HTTP Clients and Servers and use these for testing the
key material and trust relations.
This commit is related to #28898. It adds an nio driven http server
transport. Currently it only supports basic http features. Cors,
pipeling, and read timeouts will need to be added in future PRs.
This commit exposes the master version to the REST test context. This
will be needed in a follow-up where the master version will be used to
determine whether or not a certain warning header is expected.
Due to the way composite aggregation works, ordering in GROUP BY can be
applied only through grouped columns which now the analyzer verifier
enforces.
Fix 29900
This commit removes the SecurityLifecycleService, relegating its former
functions of listening for cluster state updates to SecurityIndexManager
and IndexAuditTrail.
This change adds a grok_pattern field to the GET categories API
output in ML. It's calculated using the regex and examples in the
categorization result, and applying a list of candidate Grok
patterns to the bits in between the tokens that are considered to
define the category.
This can currently be considered a prototype, as the Grok patterns
it produces are not optimal. However, enough people have said it
would be useful for it to be worthwhile exposing it as experimental
functionality for interested parties to try out.
This is fixing an issue that has come up in some builds. In some
scenarios I see an assertion failure that we are trying to move to
application mode when we are not in handshake mode. What I think is
happening is that we are in handshake mode and have received the
completed handshake message AND an application message. While reading in
handshake mode we switch to application mode. However, there is still
data to be consumed so we attempt to continue to read in handshake mode.
This leads to us attempting to move to application mode again throwing
an assertion.
This commit fixes this by immediatly exiting the handshake mode read
method if we are not longer in handshake mode. Additionally if we swap
modes during a read we attempt to read with the new mode to see if there
is data that needs to be handled.
