Commit Graph

6176 Commits

Author SHA1 Message Date
Tim Brooks 1abc40c645 Rename elastic bootstrap password setting (elastic/x-pack-elasticsearch#2009)
This is related to elastic/x-pack-elasticsearch#1991.

Original commit: elastic/x-pack-elasticsearch@b265211e4a
2017-07-14 16:41:42 -05:00
jaymode 1a6e6411cf Test: use the same settings object to construct tribe settings
This changes the SecurityTribeIT tests to use the same settings object when creating the settings
for the tribe node. Previously two different objects were being created and we would read regular
settings from one and secure settings from another. This causes problems since randomization means
that there may be settings added on the first call that do not get added on the second call. One
example is the randomization of when to only use a keystore or to use both a keystore and a
truststore. On the first call, we would add settings for both a keystore and a truststore but on
the second call only the keystore settings were added. This lead to failures as we would not be
able to open a password protected truststore since the password was never added to the secure
settings.

relates elastic/x-pack-elasticsearch#2005

Original commit: elastic/x-pack-elasticsearch@bbdb3ec662
2017-07-14 14:12:32 -06:00
Drew Raines 49407c1069 [DOCS] Fix typo
Original commit: elastic/x-pack-elasticsearch@1aaaa88404
2017-07-14 14:31:53 -05:00
David Roberts 8365038584 [TEST] Mute failing test: SSLTrustRestrictionsTests testRestrictionsAreReloaded
See elastic/x-pack-elasticsearch#2007

Original commit: elastic/x-pack-elasticsearch@1deb4a7162
2017-07-14 16:30:47 +01:00
Jay Modi 6b4468ea5c Clear security caches on security index health changes (elastic/x-pack-elasticsearch#1957)
This change clears the caches in the native realm and the composite roles store when there is a
a change in the health of the security index that necessitates this. When the security index goes
to a red state, the caches are left in tact as this allows for management operations to be
performed for a limited amount of time. When the index transitions out of the red state or exists
when it didn't exist before, the caches will be cleared so that we remove any stale values.

relates elastic/x-pack-elasticsearch#1789

Original commit: elastic/x-pack-elasticsearch@914959ea6b
2017-07-14 09:28:28 -06:00
Tim Brooks 8ab167cccb Fix ReservedRealm test for failed authentication
Original commit: elastic/x-pack-elasticsearch@5759d9268c
2017-07-14 09:28:44 -05:00
Yannick Welsch dbbec0d37e Let primary own its replication group
Companion commit for elastic/x-pack-elasticsearch#25692

Original commit: elastic/x-pack-elasticsearch@ed93c56f07
2017-07-14 13:51:43 +02:00
David Roberts 81ec1a7ba5 [TEST] Mute failing test
See elastic/x-pack-elasticsearch#2003

Original commit: elastic/x-pack-elasticsearch@5f3611b1d3
2017-07-14 09:49:30 +01:00
Tim Vernum 1686add7ce The configured role-mapping file must be valid. (elastic/x-pack-elasticsearch#1940)
This adds a bootstrap-check that makes it an error to configure a role mapping file that doesn't exist or cannot be parsed.

We are still lenient on dynamic reload because
(a) killing a running node is quite drastic
(b) file writes aren't atomic, so we might be picking up a file that is half way through being written (etc).

If you rely on the default role mapping filename, then it doesn't need to exist (because you might be using the role mapping API instead) but if it does exist it has to parse successfully

Original commit: elastic/x-pack-elasticsearch@5424dea4c4
2017-07-14 15:04:26 +10:00
Tim Vernum d57e38fbed [LDAP] Support explicit "dn" attribute in group search (elastic/x-pack-elasticsearch#1995)
The default for group_search.attribute is to search by DN, but explicitly setting that to dn
wouldn't work because the DN is returned in a special value in the result, and not in the attributes list.

This change detects when user_attribute is set to dn and treats it the same way as the default value.

Original commit: elastic/x-pack-elasticsearch@1933410a0b
2017-07-14 13:12:28 +10:00
Tim Vernum 44a104cb26 [TESTS] Fix SSLTrustRestriction IntegrationTest
- Changes the reloading test to always trust the "trusted" cert so that the health-ping works
- Adds some more logging in case we get new failures

Original commit: elastic/x-pack-elasticsearch@993bf9c721
2017-07-14 13:10:29 +10:00
Tim Brooks 6d04eacdec Require elastic password be bootstrapped (elastic/x-pack-elasticsearch#1962)
This is related to elastic/x-pack-elasticsearch#1217. This commit requires that the elastic password
be bootstrapped for the user to be authenticated. As a result it removes
the special "setup" mode that allowed the user to be authenticated from
localhost.

Additionally, this commit updates the tests to work with this
functionality.

Original commit: elastic/x-pack-elasticsearch@d0d5d697a7
2017-07-13 19:59:50 -05:00
Igor Motov 4de6d9ebe5 Upgrade API: upgrade assistance shouldn't throw 404 on an empty cluster (elastic/x-pack-elasticsearch#1997)
When a user asks for upgrade information for all indices and there are no indices in the cluster, upgrade assistance should just return an empty response indicating that no indices require upgrade or reindexing. This commit also reverts the temporary fix in WatchBackwardsCompatibilityIT tests that was added as a workaround for this issue.

Original commit: elastic/x-pack-elasticsearch@2ea9707867
2017-07-13 17:01:50 -04:00
Igor Motov dd11fc3d0a Upgrade API: fix parent task propagation for upgrade (elastic/x-pack-elasticsearch#1986)
Ensures that parent task is propagated to child operations to ensure that reindex operation can be cancelled if needed.

Original commit: elastic/x-pack-elasticsearch@fa40b5a951
2017-07-13 16:25:38 -04:00
jaymode 20c06578f6 Do not add domain to the bind user when it is a DN
This change fixes the creation of the bind DN string for active directory realms so that they do
not add the `@domain` suffix to the bind DN, when it is a actual DN value.

Original commit: elastic/x-pack-elasticsearch@bd04c07e16
2017-07-13 12:15:08 -06:00
Chris Earle f4b9dff71a [Monitoring] Support new Kibana mappings for Detecting Cloud (elastic/x-pack-elasticsearch#1989)
This adds the Cloud detection mappings so that phone home can take advantage of their existence.

It also sets the system load fields to use `half_floats`.

Original commit: elastic/x-pack-elasticsearch@75f7992d38
2017-07-13 13:59:40 -04:00
Chris Earle 924ff446bf [Monitoring] Stop checking for old alerts in upgraded version (elastic/x-pack-elasticsearch#1992)
This stops checking for older alerts now that we support emailing based on state changes. This only applies to 6.0 because the upgrade _can_ still happen usefully in 5.6 without the noise.

Original commit: elastic/x-pack-elasticsearch@9d73c64daa
2017-07-13 13:24:32 -04:00
Jay Modi 6fdad6039f Allow the Active Directory UPN authenticator to work with suffixes (elastic/x-pack-elasticsearch#1958)
The active directory user principal name format typically takes the form user@domain, which is what
the current implementation expects. However, active directory also allows the definition of other
suffixes that are not actual domains. A user can still authenticate using this user principal name
but the behavior of our realm would cause it to fail as it parsed the suffix as a domain and used it
as the search base for the user. Instead, we should use the default user search base and only look
for entries that have this exact user principal name. In a scenario where a realm is configured for
multiple domains in the same forest, the search base should be the base for the entire forest.

relates elastic/x-pack-elasticsearch#1744

Original commit: elastic/x-pack-elasticsearch@de00c4817e
2017-07-13 10:08:22 -06:00
Tim Vernum c753ddf7f2 Enable TLS trust restrictions by subject alternative name (elastic/x-pack-elasticsearch#1919)
Optional restrictions can be applied on top of an existing certificate trust scheme (PEM CAs, JKS TrustStore etc).
The restrictions reduce the set of certificate that would be otherwise trusted.

In this commit, the only supported restrictions are to filter by the certificate's SubjectAlternativeNames that are tagged as "other name" with an object-identifier of "cn"

Original commit: elastic/x-pack-elasticsearch@c6105a47df
2017-07-13 23:45:00 +10:00
Alexander Reelsen eb118b365c Tests: Ignore 404 errors in watcher bwc tests
Original commit: elastic/x-pack-elasticsearch@8e583cf293
2017-07-13 13:41:51 +02:00
Colin Goodheart-Smithe 8aec1d4737 [DOCS] Remove reference to field stats in security limitations
Original commit: elastic/x-pack-elasticsearch@9ca673ea36
2017-07-13 12:00:16 +01:00
Colin Goodheart-Smithe 6abe6e5b54 Changes to support removal of Field stats API (elastic/x-pack-elasticsearch#1953)
* Changes to support removal of Field stats API

* iter

Original commit: elastic/x-pack-elasticsearch@bac97b8701
2017-07-13 11:56:43 +01:00
Alexander Reelsen 32bc0cd5d5 Tests: Add logging and error tracing to watcher bwc tests
Original commit: elastic/x-pack-elasticsearch@4a4b65bcb7
2017-07-13 11:08:48 +02:00
Luca Cavanna 56cfaf8cb9 Rename client artifacts (elastic/x-pack-elasticsearch#1985)
Adapt to upstream rename of rest and sniffer artifacts

Original commit: elastic/x-pack-elasticsearch@f43368b3fb
2017-07-13 09:44:53 +02:00
Ryan Ernst df41b8342d Convert put_template uses to put_script in rest tests
Original commit: elastic/x-pack-elasticsearch@5f5f52b955
2017-07-13 00:01:56 -07:00
Tim Vernum e4c8851a24 Convert Realm.authenticate to provide a richer result (elastic/x-pack-elasticsearch#1932)
This allows for messages to be returned, and distinguishes between 4 different results:
- I have authenticated the user
- I don't know how to authenticate that user. Try another realm.
- I tried to authenticate the user, but failed. Try another realm.
- I tried to authenticate the user, but failed. Fail the authentication attempt.

Original commit: elastic/x-pack-elasticsearch@f796949cfb
2017-07-13 14:24:08 +10:00
Tim Vernum a36121a725 [DOCS] [Security] Templates do not use bind_dn (elastic/x-pack-elasticsearch#1979)
Document that user_dn_template mode for LDAP authentication does not support bind_dn

Original commit: elastic/x-pack-elasticsearch@eef72615a8
2017-07-13 14:23:23 +10:00
Simon Willnauer 84ee21ed26 Followup for elastic/elasticsearch#25658 (elastic/x-pack-elasticsearch#1984)
This is the xpack side fo elastic/elasticsearch#25658 which is mainly refactorings
of a ctor and added tests.

Original commit: elastic/x-pack-elasticsearch@d8e2a2a057
2017-07-12 22:19:34 +02:00
Jay Modi e686d8a3bf Add active directory bind user and user lookup support (elastic/x-pack-elasticsearch#1956)
This commit adds support for a bind user when using the active directory realm. The addition of a
bind user also enables support for the user lookup mechanism, which is necessary to support the run
as functionality that we provide.

relates elastic/x-pack-elasticsearch#179

Original commit: elastic/x-pack-elasticsearch@40b07b3422
2017-07-12 14:01:39 -06:00
Brandon Kobel ef25568b2a Removing indices privileges from the reporting_user (elastic/x-pack-elasticsearch#1806)
* Removing indices privileges from the reporting_user

* Updating tests

* Adding comment

Original commit: elastic/x-pack-elasticsearch@ba7688b1dd
2017-07-12 15:56:35 -04:00
Alexander Reelsen e64cf23b13 Watcher: Start watcher on master node only with mixed versions (elastic/x-pack-elasticsearch#1983)
When there are data or master nodes in the cluster, that are older
than ES 6.0 alpha3, then watcher will only start on the master node.

Changed all transport actions to be master node actions, as there is
already a method to decide to run locally, which we can piggyback on.

Original commit: elastic/x-pack-elasticsearch@65cecb6d69
2017-07-12 20:58:47 +02:00
Tim Sullivan 51adbf0f51 [Monitoring] Update template version to 6.0.0 beta1 version (elastic/x-pack-elasticsearch#1948)
* [Monitoring] Update template version to 6000003

* update LAST_UPDATED_VERSION

* use the 6.0.0-beta1 version constant

* use the 6.0.0-beta1 version value (6000026)

* fix cluster uuid reference in terms filters

Original commit: elastic/x-pack-elasticsearch@40c822d396
2017-07-12 09:19:54 -07:00
David Roberts 176786de54 [ML] Improve mapping clash error message (elastic/x-pack-elasticsearch#1968)
relates elastic/x-pack-elasticsearch#1751

Original commit: elastic/x-pack-elasticsearch@01c221bf42
2017-07-12 15:58:37 +01:00
Jack Conradson 32dbfba0c2 Disallow lang to specified in requests where a stored script is used. (elastic/x-pack-elasticsearch#1949)
Requests that execute a stored script will no longer be allowed to specify the lang of the script. This information is stored in the cluster state making only an id necessary to execute against. Putting a stored script will still require a lang.

Original commit: elastic/x-pack-elasticsearch@926a7b2d86
2017-07-12 07:56:08 -07:00
Igor Motov 41f30b0ae4 Upgrade API: add common upgrade check method (elastic/x-pack-elasticsearch#1970)
Moves internal index format check to a central location.

Original commit: elastic/x-pack-elasticsearch@0869baac51
2017-07-12 08:46:59 -04:00
Hendrik Muhs f5cfb98ea2 do not update mapping if it is newer than the current version (elastic/x-pack-elasticsearch#1971)
In case of mixed version in a cluster the update could potentially downgrade the mappings. This CL changes the check to avoid this situation.

Original commit: elastic/x-pack-elasticsearch@9ecd5df70e
2017-07-12 09:19:06 +02:00
Chris Earle 753690bf5c [TEST][Monitoring] Use specific LAST_UPDATED constant to avoid issues when they're not the same (elastic/x-pack-elasticsearch#1976)
Original commit: elastic/x-pack-elasticsearch@a41bff564a
2017-07-11 20:47:36 -04:00
Chris Earle 44eda0962e [Monitoring] Use ctx.metadata.xpack not just ctx.metadata (elastic/x-pack-elasticsearch#1974)
This tweaks a recent fix so that it uses the right metadata variable.

Original commit: elastic/x-pack-elasticsearch@af9d9b0e41
2017-07-11 18:04:38 -04:00
Jason Tedor 5ed683538d Name the log tail thread and mark it as a daemon
This commit provides an explicit name for the log tail thread, otherwise
the thread shows up with a generic name like Thread-12 in stack
dumps. While the stack trace provides enough information to conclude
what this thread is doing, it is better to be more explicit about the
purpose of this thread so that that can be discerned directly from the
name of the thread.

Additionally, we mark this thread as a daemon thread. Since this thread
is created by the main thread during node initialization it defaults to
being a user thread. Since the JVM only exits when the only threads
running are not user threads, if this thread were to somehow block then
it could prevent the server JVM from exiting. As such, this thread
should be marked as a daemon thread.

Relates elastic/x-pack-elasticsearch#1969

Original commit: elastic/x-pack-elasticsearch@b3761c7d91
2017-07-11 16:05:07 -04:00
Tim Brooks c22f618a1a Modify password tool to pull password from keystore (elastic/x-pack-elasticsearch#1951)
This is related to elastic/x-pack-elasticsearch#1217. This commit modifies the password setup tool
to pull the password for the `elastic` user from the keystore.

Original commit: elastic/x-pack-elasticsearch@fbc71f393d
2017-07-11 12:50:53 -05:00
Igor Motov 9030c3ae77 Upgrade API: remove kibana upgraders (elastic/x-pack-elasticsearch#1963)
Also removes handling of extra parameters in the upgrade api

Original commit: elastic/x-pack-elasticsearch@9edb825da7
2017-07-11 11:29:31 -04:00
Jay Modi 03ed2bbbd0 Add setting for the LDAP user search filter and deprecate user attribute (elastic/x-pack-elasticsearch#1959)
This commit adds a setting to allow changing the user search filter. Previously the filter was a
simple equality filter that mapped a given attribute to the value of the username. The default
behavior remains the same with this change but provides additional flexibility to users to who may
need more advanced LDAP searches. The user attribute setting has been deprecated due to the overlap
with the new filter setting.

relates elastic/x-pack-elasticsearch#1861

Original commit: elastic/x-pack-elasticsearch@e9d797e81c
2017-07-11 09:27:24 -06:00
Alexander Reelsen 22b0560ccb Watcher: Ensure no template deletion race in mixed cluster environment (elastic/x-pack-elasticsearch#1964)
When having a mixed cluster with 5.6 and 6.0 nodes, the template upgrade
service has a cluster state listener that deletes the old watches and
triggered_watches index template. However during that time the 5.6 node
WatcherIndexTemplateRegistry checks if the templates are missing and
adds them back. This results in a race, because the new .watches index
template does not get added by the WatcherIndexTemplateRegistry when the
6.0 node is node a master node.

This commit circumvents this issue, by only deleting the watches and
triggered watches template during the upgrade process.

Original commit: elastic/x-pack-elasticsearch@71380f460a
2017-07-11 16:50:41 +02:00
David Roberts 5288791f41 [ML] Specify only concrete indices during deletion (elastic/x-pack-elasticsearch#1966)
Following elastic/elasticsearch#23997 this was only working due to
the way we were suppressing certain errors during job deletion.

This PR makes the situations we want to ignore during job deletion
clearer and adheres to the intention of elastic/elasticsearch#23997
by only specifying concrete indices to the `indices` arguments of
deletion calls.

Original commit: elastic/x-pack-elasticsearch@2458c3db40
2017-07-11 14:07:48 +01:00
Adrien Grand 2e7c725d03 Fix compile errors in src/test.
Original commit: elastic/x-pack-elasticsearch@2ab67bbbdc
2017-07-11 14:31:35 +02:00
Alexander Reelsen 82499b6e55 Fixing compilation issue after lucene snapshot update
Original commit: elastic/x-pack-elasticsearch@1d9e73ad9a
2017-07-11 14:27:21 +02:00
Clinton Gormley 81101b893a Added note to cross cluster search docs to specify minimum node version of 5.5
Original commit: elastic/x-pack-elasticsearch@98e440f1a4
2017-07-11 14:15:23 +02:00
Yannick Welsch 6b02197ca1 [TEST] Ensure stable cluster before disabling licensing in testEnableDisableBehaviour
If the cluster is still making cluster state updates while disabling the license, cluster state updates might not go through,
triggering an assertion failure at the end of the test that checks if all cluster states have been applied.

relates elastic/x-pack-elasticsearch#1627

Original commit: elastic/x-pack-elasticsearch@e11863fd02
2017-07-11 11:48:27 +02:00
Simon Willnauer 16f2de017d Trivial refactorings after elastic/elasticsearch#25627
Original commit: elastic/x-pack-elasticsearch@1cb9c2f959
2017-07-11 11:42:51 +02:00
Hendrik Muhs 261d65a9c4 [ML] add meta data field into mappings and update procedure (elastic/x-pack-elasticsearch#1925)
Adds a update procedure for ML index mappings in order to allow adding new fields. The ES version is stored in the "_meta" field of the ML mappings which then get applied to every index. When opening a job, this version is checked on the (shared) index and a mapping update is performed in case that the version is older than current.

Original commit: elastic/x-pack-elasticsearch@211608c7ad
2017-07-11 07:44:41 +02:00