The ci script checks out elasticsearch for x-plugins to use. However, it
always checks out the master branch. This change makes the script look
for the source branch of a PR (falling back to GIT_BRANCH which is provided by jenkins for non PR CI jobs) and checks if that branch exists in elasticsearch. If it does not, it falls back to the target branch for the PR.
Original commit: elastic/x-pack-elasticsearch@73146bb3b3
This commit migrates the actual authentication calls that are performed by internal realms
to be asynchronous so that we do not execute blocking calls on network threads. The realms
that used LDAP were especially bad as they can issue many different requests and/or open
multiple connections per authentication attempt.
The LdapRealm now uses the ThreadPool to fork a thread for authentication. This is necessary
because a LDAP bind operation is not allowed to be performed asynchronously and must be done
in a blocking manner. After the bind request has completed, all other requests can be done
asynchrnously. The LdapUtils class now provides asynchronous methods for searching and also
includes the handling of referral following as automated following is not supported by
UnboundID when using the asynchronous calls.
Original commit: elastic/x-pack-elasticsearch@dfb259e498
* [Watcher] HistoryStore should hide secrets
Some watcher actions like Jira stores the whole HTTP request in case of a failure. This is very helpful when something goes wrong, but it has to hide the password used for Basic authentication otherwise it will appear in the watch record in the .watcher-history index. In general, I would consider better to always hide secrets when storing something in the history index.
This commit removes the XContentParams "hide secrets" set at the JiraIssue so that it is now set at the HistoryStore level.
It also fixes test failures that can be reproduced with:
gradle :x-plugins:elasticsearch:test -Dtests.seed=BA730F93E726AECC -Dtests.class=org.elasticsearch.xpack.notification.jira.JiraIssueTests -Dtests.method="testToXContent" -Dtests.security.manager=true -Dtests.locale=es -Dtests.timezone=Asia/Choibalsan
Original commit: elastic/x-pack-elasticsearch@d686b3b3ad
The local exporter previously fired off asynchronous requests every time a cluster state was
observed that did not contain all of the required items for monitoring. This change adds a
flag so that monitoring can wait for the pending requests to complete before retrying. This
will reduce the number of duplicated log messages as well.
Additionally, the log message for adding modern aliases now contains the name of the indices.
Closeselastic/elasticsearch#3756
Original commit: elastic/x-pack-elasticsearch@727a0adfbe
The license test for the tribe node disables xpack features but these settings
are not passed to the tribe client nodes, so they need to manually be added.
Original commit: elastic/x-pack-elasticsearch@7273817c47
* Convert security rest filter to rest handler wrapper
This is the xpack side of elastic/elasticsearchelastic/elasticsearch#21905
Original commit: elastic/x-pack-elasticsearch@38bfa771b6
This commit removes some unnecessary code: collectors do not need to extend AbstractLifecycleComponent and do not need to be started/stopped. The extra
interface Collector is removed and AbstractCollector renamed to Collector.
Original commit: elastic/x-pack-elasticsearch@75893d66e3
Transform and condition scripts were only compiled on its initial creation, so
when a new watch is created or when the master node loads all the watches. However
changing a script (like a stored one) did not lead to any changes in the in memory
watch store and thus the old script was executed again.
We do however have a mechanism in Elasticsearch's ScriptService that already does some
caching, and should reuse that one.
Closeselastic/elasticsearch#4237
Original commit: elastic/x-pack-elasticsearch@477548e237
This commit adds basic tests that store a user and a role using the native API. The test checks
that the user and role can be used prior to starting the upgrade. The realm and roles caches are
also cleared to ensure the next authentication will require a read from the security index; this
ensures we are actually testing reads from the index.
Original commit: elastic/x-pack-elasticsearch@396862da94
This commit improves the output of the migrate tool in cases when there are errors parsing entries
in the roles or users files. This is done through the use of a logger that delegates its output to
the terminal.
Additionally, the `-c` option has been removed. This option was used to set the configuration directory
but this should be handled one way only and that is through the use of the `-Epath.conf` setting.
Closeselastic/elasticsearch#3757Closeselastic/elasticsearch#3758
Original commit: elastic/x-pack-elasticsearch@811e367766
Currently we have these logs for integration tests only.
This adds the following log at the start:
```
logger.info("[{}]: before test", getTestName());
```
and this is logged at the end, but before any clean up done in sub classes
```
logger.info("[{}]: after test", getTestName());
```
Original commit: elastic/x-pack-elasticsearch@2ca7296665
It may take a few clicks before the templates are added.
Waiting for started doesn't guarantee that the index templates have been added.
Original commit: elastic/x-pack-elasticsearch@ee94b740a8
This commit adds tests for DNS only hostname verification. This is a followup of elastic/elasticsearchelastic/elasticsearch#21828, which fixes issues with this type of hostname verification
due to some addresses losing the host information.
Original commit: elastic/x-pack-elasticsearch@8a63bb113d
When we encounter a composite request, we authorize at first without looking at the indices, to see whether the action can be executed at all. We then rely on the action to delegate to an inner action per sub-request, which will be authorized based on the indices it refers to. The first step works great for the simulate mode of search template, as it doesn't involve any index. The second step will make sure that when search template involves a search, it will be authorized as a normal search request would, based on the indices it reads from.
Note that the wildcard expansion happens now on the search side, it doesn't have to happen when executing the first authorization step, hence SearchTemplateRequest doesn't have to implement IndicesRequest, only SearchRequest has to (which it does already).
Closeselastic/elasticsearch#4171
Original commit: elastic/x-pack-elasticsearch@d586bd90cb
This commit applies the same fix merged in elastic/elasticsearch#4179 for XDocsClientYamlTestSuiteIT. It adds a waitForSecurityTemplate() method in order to wait for the security-index-template to be created by the SecurityTemplateService.
Original commit: elastic/x-pack-elasticsearch@1476f30e2d
Since there is no hard requirement for a strict POSIX shell
implementation, we can switch to /bin/bash.
Clean up left over break statements and consistently use $() for command
substitutions.
Original commit: elastic/x-pack-elasticsearch@0a33dfed56
This commit moves the iteration of realms for authentication and user lookup to
be done in an asynchronous fashion. The existing blocking methods have been deprecated
to allow custom realm implementors time to switch. All internal realms implement the
asynchronous methods.
This PR is another step toward the full migration to async authentication, but does not
complete the work. Additional work is needed for the LDAP realms, which make blocking
network calls. These blocking calls will be handled in a follow-up PR.
See elastic/elasticsearch#3790
Original commit: elastic/x-pack-elasticsearch@a65a9b2bb4
Remove duplicate definition of GRADLE_OPT_TEST and rename arguments as
discussed in
https://github.com/elastic/x-plugins/pull/4137#discussion_r89660620
Default to `check` if no argument is passed, but exit with error in an
unsupported parameter is provided.
Also ensure the pipefail option is only set when the executing shell
is *really* bash as opposed to POSIX sh which doesn't support it.
Original commit: elastic/x-pack-elasticsearch@64bd740295
If the HTTP response is an invalid one, it is still logged as success.
This commit changes the behaviour, that if the response status code is
set to -1 (which means it could not be interpreted), than an IOException
is thrown and thus the execution will be marked as a failure.
Closeselastic/elasticsearch#4152
Original commit: elastic/x-pack-elasticsearch@5736fbe3c0
When one of the 2 nodes in the old cluster is shut down, shards that were on that node will become unassigned and be marked to be
delay-allocated, i.e. either a node with shard data for that shard must be available or the allocation of the shards will be delayed for a minute.
In the mixed cluster the replica shard might not be allocated as the primary is already on the node with the newer version and replicas are not allowed
then to be allocated to a node of an older version of ES. Once both nodes are upgraded, the delay might still be in place, and can only be nullified if there
is shard data available on the node. If there never was a shard on that node though, it will take a minute and run into the timeout checking for green.
This commit ensures that all shards are fully-allocated before we do the rolling restart scenario
Original commit: elastic/x-pack-elasticsearch@a0d9b1b043
* Add pom info to all xplugins generated poms
This change is the xplugins side of elastic/elasticsearchelastic/elasticsearch#21757. We were
previously adding the apache license info to the xplugins poms (but the
zips still contained our commercial EULA). There is also a placeholder
here to add license info back into xpack artifact poms. I also created a
follow up issue to do better validation of these poms in the future
(with this change, without the corresponding change in ES, a broken pom
would be generated with two `developers` sections):
elastic/elasticsearchelastic/elasticsearch#21758.
* Add commercial license info to pom
Original commit: elastic/x-pack-elasticsearch@8f6941f82b
* Do not try bind to port range but use free ephemeral port
* Start a new email server in all tests, do not use static one
* Remove selection of username/password, as it was static anyway
* Remove Listener.Handle class, as it is not needed, when not running in static context
Original commit: elastic/x-pack-elasticsearch@8816cc25f6
OldMonitoringIndicesBackwardsCompatibilityTests fails because it waits for more shards stats to be collected but that can only work if new indices are created in the meanwhile.
Original commit: elastic/x-pack-elasticsearch@003c28cf93
If the primary shard of an index with (number_of_replicas > 0) ends up on a new node in a mixed cluster, the replica cannot be allocated to the old node as
the new node might have written segments that use a new postings format or codec that is not available on the older node.
As x-pack automatically creates indices with number_of_replicas > 0, for example monitoring-data-*, the test can only wait for yellow in a mixed cluster.
Original commit: elastic/x-pack-elasticsearch@945d9e3811
We prefer just passing a single cli argument that allows the script to
choose the corresponding gradle command from a preset list of
choices. For now, valid options are `packagingtests` or `tests`. The
script defaults to `tests` if no argument is supplied.
Original commit: elastic/x-pack-elasticsearch@f6315b1ad5
This commit updates the watch_history.json file so that it includes mappings for the new Jira action. It also update the JiraIssue format so that it now includes the name of the account used to create the Jira issue. It also update the REST tests to check that Jira action result are searchable and hide the user's password.
Original commit: elastic/x-pack-elasticsearch@75888f7748
Boaz Leskes
Ryan Ernst
Jay Modi
Tanguy Leroux
jaymode
Simon Willnauer
Alexander Reelsen
Simon Willnauer
Aaron Bull Schaefer
Martijn van Groningen
javanna
Jason Tedor
Adrien Grand
Luca Cavanna
Dimitrios Liappis
Yannick Welsch