Today we require a pre-shared key to use the token service. Beside the
additional setup step it doesn't allow for key-rotation which is a major downside.
This change adds a TokenService private ClusterState.Custom that is used to distribute
the keys used to encrypt tokens. It also has the infrastructur to add automatic key
rotation which is not in use yet but included here to illustrate how it can work down
the road.
This is considered a prototype and requires additioanl integration testing. Yet, it's fully
BWC with a rolling / full cluster restart from a previous version (also from 5.6 to 6.x)
since if the password is set it will just use it instead of generating a new one.
Once we implement the automatic key rotation via the clusterstate we need to ensure that we are
fully upgraded before we do that.
Also note that the ClusterState.Custom is fully transient and will never be serialized to disk.
Original commit: elastic/x-pack-elasticsearch@1ae22f5d41
This change means that newly created jobs will get an explicit 1GB
model memory limit if no model memory limit is specified when creating
the job. Existing jobs that had a null model memory limit will carry
on using the default model memory limit defined in the C++ code.
Relates elastic/x-pack-elasticsearch#546
Original commit: elastic/x-pack-elasticsearch@a4e6b73c2b
We rely on command extensions in our scripts but we do not actually
guarantee that they are enabled (usually they are, by default, but they
can be disabled outside of our control). This commit ensures that they
are enabled.
Relates elastic/x-pack-elasticsearch#2307
Original commit: elastic/x-pack-elasticsearch@a5eec8ca7b
Today we require the `bootstrap.password` to be present in the keystore in order to
bootstrap xpack. With the addition of `keystore.seed` we have a randomly generated password
per node to do the bootstrapping. This will improve the initial user experience significantly
since the user doesn't need to create a keystore and add a password, they keystore is created
automatically unless already present and is always created with this random seed.
Relates to elastic/elasticsearch#26253
Original commit: elastic/x-pack-elasticsearch@5a984b4fd8
The old message of "Cannot auto close job" implied the problem was with
closing the job. This change makes it clearer that the problem is that
the datafeed could not be stopped and hence auto-close will not even be
attempted.
Original commit: elastic/x-pack-elasticsearch@065e9930ce
These members are default initialized on contruction and then set by the
init() method. It's possible that another thread accessing the object
after init() is called could still see the null/0 values, depending on how
the compiler optimizes the code.
Original commit: elastic/x-pack-elasticsearch@668121e274
Today we try to bootstrap the security index with the bootstrap password and recommend the user to change the password with the user tool. This is trappy for instance if you happen to configure multiple nodes with a different bootstrap passwords (which is possible) it's unclear which password made it too bootstrap. Yet, we tell in the logs but it can still be very confusing. In general it should be possible to bootstrap with the user tool from any node unless the user is already created in the native user store. This change uses the bootstrap.password from the local node and always authenticate against it until the user is bootstrapped even if the passwords are different on different nodes. This will also work for authenticating against the cluster for instance if a user deletes the .security index or if that index has not been upgraded.
Original commit: elastic/x-pack-elasticsearch@8cebecb287
When writing the users and users_roles files, we wrap a custom writer in
a print writer. There is a problem with this though: when print writer
closes it closes our underlying custom writer and the close
implementation for our custom writer is not trivial, it executes code
that can throw an I/O exception. When print writer invokes this close
and an I/O exception is thrown, it swallows that exception and sets the
status on the print writer to error. One would think that we could
simply check this status but alas print writer is broken here. The act
of checking the status causes print writer to try to flush the
underyling stream which is going to be completely undefined because the
underlying stream might or might not be closed. This might cause another
exception to be thrown, losing the original. Print writer screwed the
pooch here, there is no good reason to try to do any I/O after the
underlying writer entered a failed state. To address this we remove the
use of print writer, we use our custom writer directly. This allows any
thrown exceptions to bubble up.
Relates elastic/x-pack-elasticsearch#2288
Original commit: elastic/x-pack-elasticsearch@11b8dd5641
When mappings are updated for an index are updated most settings are
merged, but not _meta. This change ensures that _meta is set when we
add per-job term mappings to our results index mappings. In order to
keep the logic for updating mappings after upgrade working, we now
have to put ALL the mappings for our results along with the latest _meta
section when updating per-job term mappings.
relates elastic/x-pack-elasticsearch#2265
Original commit: elastic/x-pack-elasticsearch@f58c11a13e
We close the secure settings in core before we pull bootstrap checks.
This means if a bootstrap check like the `TokenPassphraseBootstrapCheck`
accesses a secure setting that late it will fail due to an exception in
the `PKCS12KeyStore`. This change moves the bootstrap check creation
to the plugin constructor and adds a dummy setting to the integTest
that triggers the bootstrap checks.
Original commit: elastic/x-pack-elasticsearch@2b20865d1c
When the machine-learning-cpp repo is built locally, the zip file it
creates is preferred over that downloaded from s3 when creating the
overall x-pack-elasticsearch zip. However, prior to this change the
build would ALSO download an ml-cpp zip from s3, and just not use it.
Original commit: elastic/x-pack-elasticsearch@bd71637edd
PUT /_xpack/license with no content or content-type should fail with an appropriate error message rather than throwing NPE.
Original commit: elastic/x-pack-elasticsearch@f8c744d2a2
This change makes 2 improvements to the max_running_jobs setting:
1. Namespaces it by adding the xpack.ml. prefix
2. Renames "running" to "open", because the "running" terminology
is not used elsewhere
The old max_running_jobs setting is used as a fallback if the new
xpack.ml.max_open_jobs setting is not specified. max_running_jobs
is deprecated and (to ease backporting in the short term) will be
removed from 7.0 in a different PR closer to release of 7.0.
Relates elastic/x-pack-elasticsearch#2185
Original commit: elastic/x-pack-elasticsearch@18c539f9bb
These tests used to fail rarely, because during a watch execution
one of the watcher shards was relocated resulting in a second execution
of watch.
In order to prevent this, the tests do not need to actually create any
shards, which causes watcher potentially to be rebalanced.
This simplifies and speeds up the test as well.
relates elastic/x-pack-elasticsearch#1608
Original commit: elastic/x-pack-elasticsearch@1cfac1145d
This cleans up logging, when starting several elasticsearch instances,
as otherwise you cannot see, which node emits this log message.
Original commit: elastic/x-pack-elasticsearch@c8c2819d86
When a watch is executed, it sends an update request to the watch to
udpate its status.
This update request also updates the status.state field, which contains
information, if the watch is active. If the watch gets executed, and
during execution a watch gets disabled, then the current execution will
set the watch back to active.
This commit fixes the current behaviour and never changes the state of
a watch when updating the status after executing, allowing
activate/deactivate calls to work as expected, regardless if a watch
is being executed.
This will fix not only the current behaviour but also some flaky tests.
Original commit: elastic/x-pack-elasticsearch@ca69109ecb
It is really hard to debug some issues with watcher, when only the
e.getMessage() is returned as failure reasons instead of the whole
stack trace.
This commit gets rid of ExceptionsHelper.detailedMessage(e) and always
returns the whole exception.
This commit also extends the watch history to have all fields named
error be treated like an object to be sure they do not get
indexed. No matter where it's placed in the hierarchy
In addition a few Field interface classes were removed, that only contained parse fields.
relates elastic/x-pack-elasticsearch#1816
Original commit: elastic/x-pack-elasticsearch@b2ce680139
This commit adds the max_running_jobs setting from elasticsearch.yml
into a node attribute called ml.max_open_jobs. Previously there was
an assumption that max_running_jobs would be the same for all nodes in
the cluster. However, during a rolling cluster restart where the value
of the setting is being changed this clearly cannot be the case, and
would cause unexpected/unpredictable limits to be used during the period
when different nodes had different settings.
For backwards compatibility, if another node in the cluster has not added
its setting for max_running_jobs to the cluster state then the old
(flawed but better than nothing) approach is applied, i.e. assume the
remote node's setting for max_running_jobs is equal to that of the node
deciding the job allocation.
Relates elastic/x-pack-elasticsearch#2185
Original commit: elastic/x-pack-elasticsearch@1e62b89183
Validating job groups during parsing results into
the validation error being wrapped into a parse
exception. The UI then does not display the cause of the
error. Finally, it is conceptually not a parse error, so
it belongs outside the parsing phase.
Original commit: elastic/x-pack-elasticsearch@a03f002bdc
Only unit tests were broken. Production ML code was always terminating
bulk requests with newlines.
Original commit: elastic/x-pack-elasticsearch@96ed06fed3
If one of the old watcher templates does not exist when we try
to delete it, the upgrade should just continue.
Original commit: elastic/x-pack-elasticsearch@6a52bad329
This removes the `IndicesStatsCollector` and, instead, it reuses the superset version of the call from the `IndexStatsCollector`.
On clusters with a large number of indices, this should actually help a good amount in reducing wasted calls and memory allocation without any difference in the output.
Original commit: elastic/x-pack-elasticsearch@93b09878e4
This commit re-enables the OpenLDAP tests that were previously running against a one-off instance
in AWS but now run against a vagrant fixture. There were some IntegTests that would run against the
OpenLDAP instance randomly but with this change they no longer run against OpenLDAP. This is ok as
the functionality that is tested by these has coverage elsewhere.
relates elastic/x-pack-elasticsearch#1823
Original commit: elastic/x-pack-elasticsearch@ac9bc82297
record_count is no longer written to new results, but is still tolerated
for backwards compatibility. However, in the backwards compatibility case
the results index must already contain the required mapping. There's no
need to add this mapping to newly created results indices.
Original commit: elastic/x-pack-elasticsearch@e586f3ba96
Fix TemplateTransformMappingTests to work, even if date rolls over
during execution.
Reenable test in BootStrapTests, was forgotten.
Remove the SecurityF/MonitoringF/WatcherF classes, as there is a gradle
command to easily start elasticsearch with xpack
Remove HasherBenchmark, as it is not a test and relies on RandomContext
that is not available anymore (also I think a JMH benchmark would be
needed here).
Remove ManualPublicSmtpServersTester, was not usable anymore.
Remove OldWatcherIndicesBackwardsCompatibilityTests, now in dedicated
rolling upgrade tests.
Remove unused EvalCron class.
Original commit: elastic/x-pack-elasticsearch@100fa9e9b0
This commit enhances job/datafeed APIs that support acting
on multiple jobs/datafeeds at once so that they accept
expressions that may contain comma-separated lists or
wildcard patterns.
The APIs that are enhances are:
- get jobs API
- get job stats API
- close job API
- get datafeeds API
- get datafeed stats API
- stop datafeed API
relates elastic/x-pack-elasticsearch#1876
Original commit: elastic/x-pack-elasticsearch@45a1139d97
This change makes the setup password tool build the default URL from the settings provided by the
environment. This will ease the amount of work a user would have to do in order to run the tool as
http vs https will be selected automatically and the port/host will as well.
Original commit: elastic/x-pack-elasticsearch@79affe4a79
This commit moves the reading of the token passphrase to the creation of the bootstrap check to
avoid issues with the secure settings keystore already being closed and thus causing issues during
startup.
Original commit: elastic/x-pack-elasticsearch@bba1cc832d
* Don’t set detector field names to empty strings
* Check summary count field and categorisation field names are not empty strings
* Check a detector has a by field when using multivariate by fields
Original commit: elastic/x-pack-elasticsearch@461be12f9f
If the native controller dies or is killed then requests to open jobs
now immediately return with an error that says what the problem is.
The error that is logged also now clearly records the problem.
Previously open job requests would time out if the native controller
was not running, and logged errors were not easy to understand without
in-depth knowledge of the ML code.
relates elastic/x-pack-elasticsearch#2140
Original commit: elastic/x-pack-elasticsearch@fc7f074d4a
This commit adds detection of support for AES 256 bit ciphers and enables their use when the JVM
supports them. For OpenJDK, this is often the case without any changes but for the Oracle JVM, the
unlimited policy file needs to be installed. In order to simplify the work a user would need to do
we can detect this support and automatically enable the AES 256 bit versions of the ciphers we
already enable.
Original commit: elastic/x-pack-elasticsearch@5f23b18a1e
Replacing integration tests with rest tests and unit tests, thus removing integration tests that require start of a new cluster. Removing unused testing methods
Original commit: elastic/x-pack-elasticsearch@265966d80c
The deprecated handlers should have been removed earlier, but are now
going to to away finally.
Also the watcher restart action has been removed, mainly because users
should not blindly restart, but always make sure, that watcher is
stopped correctly before restarting. This had been removed from the
transport action previously.
Original commit: elastic/x-pack-elasticsearch@78a5ec3c05
This flaky tests was using sleep, latches and a custom script plugin,
causing issues with stopping/starting tests.
This removes the integration tests and replaces it with a unit test.
Also removed a couple of unused ctor/method parameters as cleanup.
relates elastic/x-pack-elasticsearch#1639
Original commit: elastic/x-pack-elasticsearch@2a42faf2db
As all the triggering is now done on the shards, people can use
bulk, *-by-query and regular index/delete operations on the watcher
index and do not need to use the dedicated APIs.
We can now remove the long existing rest operation hijacking code.
Original commit: elastic/x-pack-elasticsearch@08f4f4c3de
This commit fixes the handling of some exceptions when we attempt to create the security index and
alias. The issue here is provoked by a test that is currently muted with an AwaitsFix,
GroupMappingTests, which will be unmuted in another change.
Original commit: elastic/x-pack-elasticsearch@55f6b656cb
This commit proposes removing an unnecessary directory push/pop from the
X-Pack scripts. It is not clear exactly why these were added, the
original change was almost three years ago in
elastic/x-pack@ea9ba7cdd0 but
unfortunately the commit message does not elucidate the exact the
problem, nor is there an associated pull request. This change has
propogated into all of the X-Pack scripts yet still the reasons are
unclear. The little that we can glean from the commit message is that
there was a problem with the default paths if the script was executed
outside of the Elasticsearch home. It seems that such issues have been
addressed by the recent introduction of elasticsearch-env so maybe we
can simplify these scripts here?
Relates elastic/x-pack-elasticsearch#2125
Original commit: elastic/x-pack-elasticsearch@9548c47743
When invoking the elasticsearch-env.bat or x-pack-env.bat batch scripts
on Windows, if these scripts exits due to an error (e.g., Java can not
be found, or the wrong version of Java is found), then the script
exits. Sadly, on Windows, this does not also terminate the caller,
instead returning control. This means we have to explicitly exit so that
is what we do in this commit.
Relates elastic/x-pack-elasticsearch#2126
Original commit: elastic/x-pack-elasticsearch@18645db62c
This allows 6.0+ monitoring clusters to be used out of the box with CCS for extended read-only access.
Original commit: elastic/x-pack-elasticsearch@2b1e4ca4e4
The full cluster restart tests are currently geared towards the 5.6 -> 6.0 upgrade and have some
issues when the versions are changed to 6.x -> 7.0. One issue is a real code issue in that the
security code always expects the mappings to have the same version as the version of the node, but
we no longer update the mappings on the security index during a rolling upgrade. We know look at
the index format to determine if the index is up to date.
Original commit: elastic/x-pack-elasticsearch@14c1c72ff6
This commit removes the use of a now removed --path.conf command-line
flag from the retrieve roles and users tests.
Original commit: elastic/x-pack-elasticsearch@30d5f5f648
This commit fixes the users tool command tests which were broken because
of a guard added that es.path.conf is set. We do not want to set this
system property in tests so instead we override createEnv where the
problematic guard exists.
Original commit: elastic/x-pack-elasticsearch@78b757695b
This commit fixes the list X-Pack extension command tests which were
broken because of a guard added that es.path.conf is set. We do not want
to set this system property in tests so instead we override createEnv
where the problematic guard exists.
Original commit: elastic/x-pack-elasticsearch@b1bc4ddcb8
This commit fixes some failing CLI tests. The failure here is that a
guard against the system property es.path.conf was added yet these tests
were not adapted for this change. This commit implements this adapation
which overrides the createEnv method where the problematic guard is
invoked. We do this to avoid having to set es.path.conf in tests.
Original commit: elastic/x-pack-elasticsearch@20e1724823
This commit responds to an upstream change which removes the --path.conf
command-line flag and instead uses the replacement mechanism for setting
the configuration path via the system property es.path.conf.
Relates elastic/x-pack-elasticsearch#2113
Original commit: elastic/x-pack-elasticsearch@4fefbffecb
This commit moves the creation of the CryptoService to the createComponents method so that bootstrap
checks have been checked before the crypto service is instantiated. The cryptoservice was changed to
expect that the bootstrap check has passed before being instantiated in elastic/x-pack-elasticsearch#1831.
Original commit: elastic/x-pack-elasticsearch@cf11cf4782
This commit is fixing an issue with the build. Currently we have a
scenario where a call to build a transport client is not using the
security client. This modifies the settings source to ensure that the
security client is used.
Original commit: elastic/x-pack-elasticsearch@2d6ea1f4e4
This commit improves the error message in 6x if the security index has
not been upgraded, and warns the user that the native realm will not be
functional until the upgrade API is run.
Original commit: elastic/x-pack-elasticsearch@710b7634b4
This change will provide a exception with a message to any callers of the empty trust manager for
better visibility into issues.
Original commit: elastic/x-pack-elasticsearch@c8241aea98
Since change elastic/x-pack-elasticsearch@f796949 authentication is not allowed to respond with null, it must be AuthenticationResult.notHandled()
- Fixes 1 case where the native realm would respond null if the user was not found
- Fixes some edge cases in the LDAP realm.
Original commit: elastic/x-pack-elasticsearch@bc739a1d40
In logstash parlance there really is no pipeline.name, its pipeline.id
This also removes support for deprecated gauge types `text` and `boolean` we will be removing those soon in logstash
This also renames `counters` to `long_counters` to be more explicit and for consistency with the gauge type. Also, if we ever decide to add other types of counters this will be more clear
Original commit: elastic/x-pack-elasticsearch@8f44a94579
The user/role was added to simplify they setup surrounding Beats monitoring, but Beats monitoring has been delayed until the UI work is begun.
Original commit: elastic/x-pack-elasticsearch@1c0c85562e
When security initialization fails during startup today we throw an
Error. This triggers the uncaught exception handler immediately killing
the node. While the node is going to die either way, we should not be
triggering the killer and in general we should avoid using Throwables
that extend Error.
Relates elastic/x-pack-elasticsearch#2035
Original commit: elastic/x-pack-elasticsearch@546f7f9002
This commit removes a legacy check for an unsupported environment
variable. This environment variable has not been supported since 1.x so
it is safe to stop checking for the existence of this setting.
Relates elastic/x-pack-elasticsearch#2048
Original commit: elastic/x-pack-elasticsearch@023230fa9e
Today we explicitly export the HOSTNAME variable from scripts. This is
probably a relic from the days when the scripts were not run on bash but
instead assume a POSIX-compliant shell only where HOSTNAME is not
guaranteed to exist. Yet, bash guarantees that HOSTNAME is set so we do
not need to set it in scripts. This commit removes this legacy.
Relates elastic/x-pack-elasticsearch#2047
Original commit: elastic/x-pack-elasticsearch@7b833e061c
This commit fixes the croneval script to respect the CONF_DIR
environment variable used to locate the configuration directory.
Original commit: elastic/x-pack-elasticsearch@79974947f9
This commit fixes an issue with the usage of the environment variable
ES_HOME in the certgen script; the script was missing the use of $ to
obtain the value of the environment variable ES_HOME.
Relates elastic/x-pack-elasticsearch#2046
Original commit: elastic/x-pack-elasticsearch@63128db0eb
With the new template and mapping update mechanisms introduced as part
of the Upgrade API work, the NativeRealmMigrator is no longer needed or
used. This commit removes the NativeRealmMigrator code and the
associated tests for it.
Original commit: elastic/x-pack-elasticsearch@5d2d7a582c
This is related to elastic/x-pack-elasticsearch#1778. This commit adds a setting
(xpack.license.self_generated.type) which allows the user to specify
what type of license will be self-generated on node startup. The allowed
types are basic or trial.
Original commit: elastic/x-pack-elasticsearch@0a16a59e10
To achieve backwards compatibility the easiest way is
to restore the old watcher stats, which are supposed to run
on the master node only.
The distributed watcher stats have been moved under the statsdist
package and the action name has been changed as well.
This way there is no need to have a serialization BWC layer,
we can just call different actions.
Note: With the current approach developers still need to change
their java applications if they try to receive watcher stats,
as by default we are now using the distributed stats in the
watcher client.
Original commit: elastic/x-pack-elasticsearch@49b3a45452
Closing jobs can still use some or all of the threads that communicate
with the C++ process - the number of threads used will decrease as the
close progresses, but at the beginning of the closure all are still in
use. Therefore, to prevent the risk of EsRejectedExecution exceptions
for the autodetect communications threadpool, closing jobs need to be
considered when checking that enough threads exist to start a new
process. An explicit check produces a much more understandable error
message than an EsRejectedExecution exception.
relates elastic/x-pack-elasticsearch#1364
Original commit: elastic/x-pack-elasticsearch@845bfe0188
This is important for two reasons:
1. If a job hangs in the closing state for any reason there is now a
way to get rid of it
2. The force delete endpoint (as used by the UI) killed open jobs before
deleting them, but could not kill closing jobs, which created the
possibility that if a job was deleted from the UI while in the closing
state then the last few results could be indexed after the deletion
completed
relates elastic/x-pack-elasticsearch#1796
Original commit: elastic/x-pack-elasticsearch@1471106e06
Add an option to the ssl certificate generation tool (certgen) that generates PKCSelastic/x-pack-elasticsearch#12 (.p12) files in addition to the certificate (.crt) and key (.key) files.
A PKCSelastic/x-pack-elasticsearch#12 store is a container format for storing multiple crypto objects in a single file, which means we can put the cert and key into the same file.
These format is particularly useful for .NET environments, where .NET Core requires a single into file for PKI authentication.
Also adds documentation for all the command-line options in certgen.
Original commit: elastic/x-pack-elasticsearch@d10f88f12d
* [Monitoring] Add Cluster Alert for X-Pack License Expiration
* work on scripts round 1
* updates per feedback
* spaces
* fix NPE error in transform
* condition to allow updating metadata in the alerts index in every interval
* custom subject message
* update name of indexing actions
* ensure ctx.metadata is updated even if alert is not resolved
* fix omission of absoluteTime
* skip info-level alerts for trial-type license
* move break above `fromNow` declaration
* fix test
Original commit: elastic/x-pack-elasticsearch@f13718f5b5
the upgrade API is lacking some functionality in a special case,
where triggered_watches exists, but .watches does not. This
deletes the triggered watches index manually until we integrated
this properly in the upgrade API to fix the tests
Original commit: elastic/x-pack-elasticsearch@e9d1b0d35d
This commit adds the upgrade API functionality and script for security.
It also enables previously muted tests that would fail due to the lack
of security upgrade features in testing cluster restarts and old
security index backward compatibility.
Original commit: elastic/x-pack-elasticsearch@4abe9f1263
This change makes the restricted trust configuration delegate the list of files to monitor to the
trust configuration that it wraps so that all files that should be monitored for changes are
monitored for changes.
Relates elastic/x-pack-elasticsearch#1919
Original commit: elastic/x-pack-elasticsearch@227db92ac0
We were catching IndexNotFoundException, which was hiding the fact that delete index and update aliases APIs don't accept aliases anymore. Now that the exception changed this problem popped up. We now rather call get index providing .security as index name, then delete the concrete indices returned in the response.
Original commit: elastic/x-pack-elasticsearch@18f64f9a41
This change fixes some cases where calls to the LDAP library can result in a new connection being
created that were not wrapped in privileged connect calls. This would result in permission denied
errors when trying to make the connection.
Original commit: elastic/x-pack-elasticsearch@182c790dd4
This changes the SecurityTribeIT tests to use the same settings object when creating the settings
for the tribe node. Previously two different objects were being created and we would read regular
settings from one and secure settings from another. This causes problems since randomization means
that there may be settings added on the first call that do not get added on the second call. One
example is the randomization of when to only use a keystore or to use both a keystore and a
truststore. On the first call, we would add settings for both a keystore and a truststore but on
the second call only the keystore settings were added. This lead to failures as we would not be
able to open a password protected truststore since the password was never added to the secure
settings.
relates elastic/x-pack-elasticsearch#2005
Original commit: elastic/x-pack-elasticsearch@bbdb3ec662
This change clears the caches in the native realm and the composite roles store when there is a
a change in the health of the security index that necessitates this. When the security index goes
to a red state, the caches are left in tact as this allows for management operations to be
performed for a limited amount of time. When the index transitions out of the red state or exists
when it didn't exist before, the caches will be cleared so that we remove any stale values.
relates elastic/x-pack-elasticsearch#1789
Original commit: elastic/x-pack-elasticsearch@914959ea6b
This adds a bootstrap-check that makes it an error to configure a role mapping file that doesn't exist or cannot be parsed.
We are still lenient on dynamic reload because
(a) killing a running node is quite drastic
(b) file writes aren't atomic, so we might be picking up a file that is half way through being written (etc).
If you rely on the default role mapping filename, then it doesn't need to exist (because you might be using the role mapping API instead) but if it does exist it has to parse successfully
Original commit: elastic/x-pack-elasticsearch@5424dea4c4
The default for group_search.attribute is to search by DN, but explicitly setting that to dn
wouldn't work because the DN is returned in a special value in the result, and not in the attributes list.
This change detects when user_attribute is set to dn and treats it the same way as the default value.
Original commit: elastic/x-pack-elasticsearch@1933410a0b
- Changes the reloading test to always trust the "trusted" cert so that the health-ping works
- Adds some more logging in case we get new failures
Original commit: elastic/x-pack-elasticsearch@993bf9c721
This is related to elastic/x-pack-elasticsearch#1217. This commit requires that the elastic password
be bootstrapped for the user to be authenticated. As a result it removes
the special "setup" mode that allowed the user to be authenticated from
localhost.
Additionally, this commit updates the tests to work with this
functionality.
Original commit: elastic/x-pack-elasticsearch@d0d5d697a7
When a user asks for upgrade information for all indices and there are no indices in the cluster, upgrade assistance should just return an empty response indicating that no indices require upgrade or reindexing. This commit also reverts the temporary fix in WatchBackwardsCompatibilityIT tests that was added as a workaround for this issue.
Original commit: elastic/x-pack-elasticsearch@2ea9707867
Ensures that parent task is propagated to child operations to ensure that reindex operation can be cancelled if needed.
Original commit: elastic/x-pack-elasticsearch@fa40b5a951
This change fixes the creation of the bind DN string for active directory realms so that they do
not add the `@domain` suffix to the bind DN, when it is a actual DN value.
Original commit: elastic/x-pack-elasticsearch@bd04c07e16
This adds the Cloud detection mappings so that phone home can take advantage of their existence.
It also sets the system load fields to use `half_floats`.
Original commit: elastic/x-pack-elasticsearch@75f7992d38
This stops checking for older alerts now that we support emailing based on state changes. This only applies to 6.0 because the upgrade _can_ still happen usefully in 5.6 without the noise.
Original commit: elastic/x-pack-elasticsearch@9d73c64daa
The active directory user principal name format typically takes the form user@domain, which is what
the current implementation expects. However, active directory also allows the definition of other
suffixes that are not actual domains. A user can still authenticate using this user principal name
but the behavior of our realm would cause it to fail as it parsed the suffix as a domain and used it
as the search base for the user. Instead, we should use the default user search base and only look
for entries that have this exact user principal name. In a scenario where a realm is configured for
multiple domains in the same forest, the search base should be the base for the entire forest.
relates elastic/x-pack-elasticsearch#1744
Original commit: elastic/x-pack-elasticsearch@de00c4817e
Optional restrictions can be applied on top of an existing certificate trust scheme (PEM CAs, JKS TrustStore etc).
The restrictions reduce the set of certificate that would be otherwise trusted.
In this commit, the only supported restrictions are to filter by the certificate's SubjectAlternativeNames that are tagged as "other name" with an object-identifier of "cn"
Original commit: elastic/x-pack-elasticsearch@c6105a47df
This allows for messages to be returned, and distinguishes between 4 different results:
- I have authenticated the user
- I don't know how to authenticate that user. Try another realm.
- I tried to authenticate the user, but failed. Try another realm.
- I tried to authenticate the user, but failed. Fail the authentication attempt.
Original commit: elastic/x-pack-elasticsearch@f796949cfb
This commit adds support for a bind user when using the active directory realm. The addition of a
bind user also enables support for the user lookup mechanism, which is necessary to support the run
as functionality that we provide.
relates elastic/x-pack-elasticsearch#179
Original commit: elastic/x-pack-elasticsearch@40b07b3422
When there are data or master nodes in the cluster, that are older
than ES 6.0 alpha3, then watcher will only start on the master node.
Changed all transport actions to be master node actions, as there is
already a method to decide to run locally, which we can piggyback on.
Original commit: elastic/x-pack-elasticsearch@65cecb6d69
* [Monitoring] Update template version to 6000003
* update LAST_UPDATED_VERSION
* use the 6.0.0-beta1 version constant
* use the 6.0.0-beta1 version value (6000026)
* fix cluster uuid reference in terms filters
Original commit: elastic/x-pack-elasticsearch@40c822d396
Requests that execute a stored script will no longer be allowed to specify the lang of the script. This information is stored in the cluster state making only an id necessary to execute against. Putting a stored script will still require a lang.
Original commit: elastic/x-pack-elasticsearch@926a7b2d86
In case of mixed version in a cluster the update could potentially downgrade the mappings. This CL changes the check to avoid this situation.
Original commit: elastic/x-pack-elasticsearch@9ecd5df70e
This commit provides an explicit name for the log tail thread, otherwise
the thread shows up with a generic name like Thread-12 in stack
dumps. While the stack trace provides enough information to conclude
what this thread is doing, it is better to be more explicit about the
purpose of this thread so that that can be discerned directly from the
name of the thread.
Additionally, we mark this thread as a daemon thread. Since this thread
is created by the main thread during node initialization it defaults to
being a user thread. Since the JVM only exits when the only threads
running are not user threads, if this thread were to somehow block then
it could prevent the server JVM from exiting. As such, this thread
should be marked as a daemon thread.
Relates elastic/x-pack-elasticsearch#1969
Original commit: elastic/x-pack-elasticsearch@b3761c7d91
This commit adds a setting to allow changing the user search filter. Previously the filter was a
simple equality filter that mapped a given attribute to the value of the username. The default
behavior remains the same with this change but provides additional flexibility to users to who may
need more advanced LDAP searches. The user attribute setting has been deprecated due to the overlap
with the new filter setting.
relates elastic/x-pack-elasticsearch#1861
Original commit: elastic/x-pack-elasticsearch@e9d797e81c
When having a mixed cluster with 5.6 and 6.0 nodes, the template upgrade
service has a cluster state listener that deletes the old watches and
triggered_watches index template. However during that time the 5.6 node
WatcherIndexTemplateRegistry checks if the templates are missing and
adds them back. This results in a race, because the new .watches index
template does not get added by the WatcherIndexTemplateRegistry when the
6.0 node is node a master node.
This commit circumvents this issue, by only deleting the watches and
triggered watches template during the upgrade process.
Original commit: elastic/x-pack-elasticsearch@71380f460a
Following elastic/elasticsearch#23997 this was only working due to
the way we were suppressing certain errors during job deletion.
This PR makes the situations we want to ignore during job deletion
clearer and adheres to the intention of elastic/elasticsearch#23997
by only specifying concrete indices to the `indices` arguments of
deletion calls.
Original commit: elastic/x-pack-elasticsearch@2458c3db40
If the cluster is still making cluster state updates while disabling the license, cluster state updates might not go through,
triggering an assertion failure at the end of the test that checks if all cluster states have been applied.
relates elastic/x-pack-elasticsearch#1627
Original commit: elastic/x-pack-elasticsearch@e11863fd02
Adds a update procedure for ML index mappings in order to allow adding new fields. The ES version is stored in the "_meta" field of the ML mappings which then get applied to every index. When opening a job, this version is checked on the (shared) index and a mapping update is performed in case that the version is older than current.
Original commit: elastic/x-pack-elasticsearch@211608c7ad
* add index checks
The following checks apply to both 5.x as well as 6.0
index checks:
- coercion of boolean fields
- the _all meta field is now disabled by default
- the include_in_all mapping parameter is now disallowed
- unrecognized match_mapping_type options not silently ignored
- similarity settings
- store settings
- store throttling settings
- shadow replicas have been removed
Original commit: elastic/x-pack-elasticsearch@5583b21763
This is related to elastic/x-pack-elasticsearch#1217. This commit adds a ClusterStateListener at
node startup. Once the cluster and security index are ready, this
listener will attempt to set the elastic user's password with the
bootstrap password pulled from the keystore. If the password is not in
the keystore or the elastic password has already been set, nothing will
be done.
Original commit: elastic/x-pack-elasticsearch@7fc4943c45
As the TemplateUpgradeService requires permissions to add and
delete index template, we have to grant those to the _system user.
This commit adds such permissions plus an integration test.
Original commit: elastic/x-pack-elasticsearch@a76ca9c738
The monitoring watches are roughly executing the same queries even when
they run against different clusters. However the way they were created,
where the cluster name is replaced via search & replace instead of using
watch metadata implies, that every watch is different from a script
compilation cache perspective. On top of that every of those watches is
executed once a minute. So if a new node becomes master and you monitor
three clusters, this results in a fair share of compilations in the first minute. The
reason for the compilation is the fact, that the search input uses
mustache for being able to add dynamic parts into a search using
mustache.
Several of those watches also need to compile more than one search
request.
The maximum default value for script compilations is only 15 and thus at
least one watch will not be executed due to failing script compilations.
This commit changes the four watches, so that the search requests are
cacheable. This means, no matter how many clusters you monitor, there
will be only needed four compilations for the different watches and
that's it.
Relates elastic/support-dev-help#2090
Original commit: elastic/x-pack-elasticsearch@6c877421bb
This test creates watches in old versions of elasticsearch, upgrades them after upgrading cluster to the latest version and then tests that they were upgraded correctly.
Original commit: elastic/x-pack-elasticsearch@b9d45eb2a5
* [Monitoring] Email actions for Cluster Alerts
* fix quotations in email fields
* move email vars to transform, and rename for snake_case
* add state to email subject for cluster status alert
* remove types field in kibana_settings search
* simplify email action condition script
* uppercase the state for the email subject
* only append state to email subject if alert is new
* show state in email subject even when alert is resolved
Original commit: elastic/x-pack-elasticsearch@e6fdd8d620
This commit is related to elastic/x-pack-elasticsearch#1896. Currently setup mode means that the
password must be set post 6.0 for using x-pack. This interferes with
upgrade tests as setting the password fails without a properly
upgraded security index.
This commit loosens two aspects of the security.
1. The old default password will be accept in setup mode (requests
from localhost).
2. All request types can be submitted in setup mode.
Original commit: elastic/x-pack-elasticsearch@8a2a577038
This is step 2 of elastic/x-pack-elasticsearch#1604
This change stores `model_memory_limit` as a string with `mb` unit.
I considered using the `toString` method of `ByteSizeValue` but it
can lead to accuracy loss. Adding the fixed `mb` unit maintains
the accuracy, while making clear what unit the value is in.
Original commit: elastic/x-pack-elasticsearch@4dc48f0ce8
ML has two types of custom cluster state:
1. jobs
2. datafeeds
These need to be parsed from JSON in two situations:
1. Create/update of the job/datafeed
2. Restoring cluster state on startup
Previously we used exactly the same parser in both situations, but this
severely limits our ability to add new features. This is because the
parser was very strict. This was good when accepting create/update
requests from users, but when restoring cluster state from disk it meant
that we could not add new fields, as that would prevent reloading in
mixed version clusters.
This commit introduces a second parser that tolerates unknown fields for
each object that is stored in cluster state. Then we use this more
tolerant parser when parsing cluster state, but still use the strict
parser when parsing REST requests.
relates elastic/x-pack-elasticsearch#1732
Original commit: elastic/x-pack-elasticsearch@754e51d1ec
This is the x-pack side of https://github.com/elastic/elasticsearch/pull/24437
It changes two things, for the disable tests, it uses a valid endpoint instead
of a previously invalid endpoint that happened to return a 400 because the
endpoint was bad, regardless of if watcher was disabled.
The other change is to create the watches index by putting a watch using the
correct API, rather than manually creating the index. This is because
`RestHijackOperationAction` hijacks operations like this and stops accessing the
endpoint in a regular manner.
Original commit: elastic/x-pack-elasticsearch@3be78d9aea
Currently, the autodetect process has an `ignoreDowntime`
parameter which, when set to true, results to time being
skipped over to the end of the bucket of the first data
point received. After that, skipping time requires closing
and opening the job. With regard to datafeeds, this does not
work well with real-time requests which use the advance-time
API in order to ensure results are created for data gaps.
This commit improves this functionality by making it more
flexible and less ambiguous.
- flush API now supports skip_time parameter which
sends a control message to the autodetect process
telling it to skip time to a given value
- the flush API now also returns the last_finalized_bucket_end
time which allows clients to resume data searches correctly
- the datafeed start API issues a skip_time request when the
given start time is after the resume point. It then resumes
the search from the last_finalized_bucket_end time.
relates elastic/x-pack-elasticsearch#1913
Original commit: elastic/x-pack-elasticsearch@caa5fe8016
This fixes `testDeleteJobAfterMissingAliases` to not fail randomly.
The reason the test was failing is that at some point some aliases
are deleted and the cat-aliases API is called to verify they were
indeed deleted. This was checked by asserting an
index_not_found_exception was thrown by the cat-aliases request.
This was some times working as there were no other aliases. However,
that depends on whether other x-pack features had time to create their
infrastructure. For example, security creates an alias. When other
aliases had the time to be created, the cat-aliases request does not
fail and the test fails.
This commit simply changes the verification that the read/write
aliases were deleted by replacing the cat-aliases request with
two single get-alias requests.
Original commit: elastic/x-pack-elasticsearch@fe2c7b0cb4
The logging shows a wrong HTTP response status code from a previous
request. In addition the body now also gets logged, as debugging
is impossible otherwise.
Original commit: elastic/x-pack-elasticsearch@cc998cd587
This changes from collecting every index statistic to only what we actually want. This should help to reduce the performance impact of the lookup.
Original commit: elastic/x-pack-elasticsearch@80ae20f382
This is related to elastic/x-pack-elasticsearch#1217 and elastic/x-pack-elasticsearch#1896. Right now we are checking if an
incoming address is the loopback address or a special local addres. It
appears that we also need to check if that address is bound to a
network interface to be thorough in our localhost check.
This change mimicks how we check if localhost in `PatternRule`.
Original commit: elastic/x-pack-elasticsearch@a8947d6174
Depending on the random numbers fed to the analytics,
it is possible that the first planted anomaly ends up
in a different bucket due to the overlapping buckets feature.
Then that may result to a single interim bucket being available
due to overlapping buckets blocking the other interim bucket
from being considered.
I am removing the initial anomaly from the test as it is not useful
and it makes the test unstable.
relates elastic/x-pack-elasticsearch#1897
Original commit: elastic/x-pack-elasticsearch@aca7870708
This commit removes the system key from master and changes watcher to use a secure setting instead
for the encryption key.
Original commit: elastic/x-pack-elasticsearch@5ac95c60ef
This is related to elastic/x-pack-elasticsearch#1217. This PR removes the default password of
"changeme" from the reserved users.
This PR adds special behavior for authenticating the reserved users. No
ReservedRealm user can be authenticated until its password is set. The
one exception to this is the elastic user. The elastic user can be
authenticated with an empty password if the action is a rest request
originating from localhost. In this scenario where an elastic user is
authenticated with a default password, it will have metadata indicating
that it is in setup mode. An elastic user in setup mode is only
authorized to execute a change password request.
Original commit: elastic/x-pack-elasticsearch@e1e101a237
We made the mistake to generate way to many settings in xpack which makes
finding out the right string and where it's defined super difficult. If
we use constants we can just use commandline tools to find where the settings
are defined. This also removes 1.x and 2.x BWC from the enabled settings which should
be removed in 6.x
Original commit: elastic/x-pack-elasticsearch@ec25e6c40c
This is related to elastic/x-pack-elasticsearch#1217. This commit reads two environment variables on
startup: BOOTSTRAP_PWD and ELASTIC_CONTAINER. If BOOTSTRAP_PWD is
present, ELASTIC_CONTAINER must be set to true. Otherwise a new
bootstrap check will fail.
If ELASTIC_CONTAINER is set to true, the elastic user can be
authenticated with the BOOTSTRAP_PWD variable when its password
has not been explicitly set.
Original commit: elastic/x-pack-elasticsearch@78f53fd232
The logic of resetting acknowledgements is only executed, if the watch
wide condition is not met. However, if you dont specify a condition
(which makes it always true), but create a condition in your action
(this might make sense because it allows you to execute a transform and
then execute the condition), then after acking this action, it will
never get be unacked, because the watch wide condition is always met.
relates elastic/x-pack-elasticsearch#1857
Original commit: elastic/x-pack-elasticsearch@95aa402c27
The .security index used several different types to differentiate the
documents added to the index (users, reserved-users, roles, etc). Since
types are deprecated in 6.x, this commit changes the .security index
access layer to only use a single type and have all documents in the
index be of that single type. To differentiate documents that may have
the same id (e.g. the same user name and role name), the appropriate
type of the document is prepended to the id. For example, a user named
"jdoe" will now have the document id "user-jdoe".
This commit also ensures that any native realm security index operations
that lead to auto creation of the security index first go through the process
of creating the internal security index (.security-v6) and creating the alias
.security to point to the internal index.
Lastly, anytime the security index is accessed without having been
upgraded, an exception is thrown notifying the user to use the
upgrade API to upgrade the security index.
Original commit: elastic/x-pack-elasticsearch@cc0a474aed
Adds a new Upgrade API with the first action, index upgrade info, that returns that list of indices that require upgrade in the current cluster before the cluster can be upgraded to the next major version.
Relates to elastic/x-pack-elasticsearch#1214
Original commit: elastic/x-pack-elasticsearch@761e7d2128
* [Logstash] Change management license to Gold
Previously the license type for LS config management was `BASIC`. In order to use the security features in Standard/Gold, we had to bump Logstash as well to Gold license.
relates elastic/x-pack-elasticsearch#1841
Original commit: elastic/x-pack-elasticsearch@29194b2417
Adds REST endpoint and Transport Action for retrieving breaking-changes deprecations that exist in current version. This PR is just the framework for such an API, future checks will be added to the appropriate branches.
Original commit: elastic/x-pack-elasticsearch@990e3468e9
This commit adds new settings for the ssl keystore (not the ES keystore)
passphrase settings. New setting names are used, instead of trying to
support the existing names in both yml and the ES keystore, so that
there does not need to be complicated logic between the two. Note that
the old settings remain the only way to set the ssl passphrases for the
transport client, but the Settings object for transport clients are
created in memory by users, so they are already as "secure" as having a
loaded ES keystore. Also note that in the long term future (6.x
timeframe?) these settings should be deprecated and the keys/certs
themselves should be moved into the ES keystore, so there will be no
need for separate keystores/passphrases.
relates elastic/elasticsearch#22475
Original commit: elastic/x-pack-elasticsearch@be5275fa3d
* The TriggeredWatchStore now only has one method to put triggered
watches
* All code is async in TriggeredWatchStore, locking has been removed
* The dedicated WatchRecord.Fields interface has been removed
* TriggeredWatchTests integration test has been moved to a unit test
Original commit: elastic/x-pack-elasticsearch@bc4b5820fb
`index.mapper.single_type` will be removed in master. While there is still
one usage in the security template that we are working on, this change
will remove the remaining usage.
Original commit: elastic/x-pack-elasticsearch@6e7f63b9e0
This is just a workaround at the moment, but allows to use
mustache if you only provide the `url` part of a request,
instead of scheme, port, path, host, etc.
Original commit: elastic/x-pack-elasticsearch@3a4aa26665
Every cluster state update resulted in a log message, that watcher
pauses execution. This has been fixed to only log, if there was an
actual state switch from executing to pausing, but do nothing if
there are no local shards anyway.
This will reduce the logging noise in tests a lot.
Original commit: elastic/x-pack-elasticsearch@32ab86610c
* Give kill a chance to kill the process before closing input
* Remove variable that can be refactored out
Original commit: elastic/x-pack-elasticsearch@42f7a3cece
The graph API needs to be able to search in remote indices. Although it uses the Search API to perform the search and so doesn’t need to deal with remote indexes directly, the security feature needs to know it can be used with remote indexes so it knows to include remote indices in the list of indices accessible from the API for index level security
Original commit: elastic/x-pack-elasticsearch@e3cd84963e
This change removes all local security checks against remote cluster names.
Any user is allowed to attempt a cross-cluster search, and it is the responsibility of the remote cluster to authorise the search (or not).
This includes support for remote searches even if you have _no_ local search privileges.
Original commit: elastic/x-pack-elasticsearch@1620c3a8fa
Currently, aggregated datafeeds construct JSON from the aggregation
response by traversing all nested aggregations. In order to
achieve this, multiple leaf aggregations are not supported. Also,
scenarios it makes it impossible to effectively use pipeline
aggregations as it will not ignore the intermediate bucket
aggregations.
This commit refactors AggregationToJsonProcessor in order to
support the above scenarios. This is achieved by only converting
the fields of interest, that is the job analysis fields.
Original commit: elastic/x-pack-elasticsearch@8b575956ca
This changes the validation criteria we use for user and role
names in the file realm, native realm, and the
realm-agnostic code in x-pack security. The new criteria is:
A valid username's length must be at least 1 and no more than 1024
characters. It may not contain leading or trailing whitespace. All
characters in the name must be be alphanumeric (`a-z`, `A-Z`, `0-9`),
printable punctuation or symbols in the https://en.wikipedia.org/wiki/Basic_Latin_(Unicode_block)[Basic Latin (ASCII) block],
or the space character.
Original commit: elastic/x-pack-elasticsearch@f77640f269
Simon Willnauer
David Roberts
Jason Tedor
Colin Goodheart-Smithe
Alexander Reelsen
Yannick Welsch
Tim Vernum
Dimitris Athanasiou
Chris Earle
Tim Sullivan
Jay Modi
David Kyle
Tim Brooks
Luca Cavanna
Ryan Ernst
Lisa Cawley
Martijn van Groningen
Igor Motov
Michael Basnight
Tal Levy
jaymode
Ali Beyad
Andrew Cholakian
Jack Conradson
Adrien Grand
markharwood
Tim Vernum
Brandon Kobel
Hendrik Muhs
Nik Everett
Lee Hinman
Christoph Büscher
Suyog Rao
Daniel Mitterdorfer
Andy Bristol