Add beats_system user to security (elastic/x-pack-elasticsearch#4103)

This creates a new "beats_system" user and role with the same
privileges as the existing "logstash_system" user/role.

The "beat_system" user is also added as a managed user within
the "setup-passwords" command.

Users who upgrade from an earlier version of Elasticsearch/X-Pack
will need to manually set a password for the beats_system user via
the change password API (or Kibana UI)

Original commit: elastic/x-pack-elasticsearch@6087d3a18e

docs/en/commands/setup-passwords.asciidoc

@@ -3,7 +3,7 @@
 == setup-passwords
 
 The `setup-passwords` command sets the passwords for the built-in `elastic`,
-`kibana`, and `logstash_system` users.
+`kibana`, `logstash_system`, and `beats_system` users.
 
 [float]
 === Synopsis

docs/en/security/authentication.asciidoc

@@ -22,6 +22,7 @@ passwords have been set. The `elastic` user can be used to
 `elastic`:: A built-in _superuser_. See <<built-in-roles>>.
 `kibana`:: The user Kibana uses to connect and communicate with Elasticsearch.
 `logstash_system`:: The user Logstash uses when storing monitoring information in Elasticsearch.
+`beats_system`:: The user the Beats use when storing monitoring information in Elasticsearch.
 
 
 [float]
@@ -76,7 +77,7 @@ The +setup-passwords+ tool is the simplest method to set the built-in users'
 passwords for the first time. It uses the `elastic` user's bootstrap password to
 run user management API requests. For example, you can run the command in
 an "interactive" mode, which prompts you to enter new passwords for the
-`elastic`, `kibana`, and `logstash_system` users:
+`elastic`, `kibana`, `logstash_system`, and `beats_system` users:
 
 [source,shell]
 --------------------------------------------------
@@ -113,7 +114,7 @@ since at that point the bootstrap password is no longer required.
 
 [float]
 [[add-built-in-user-passwords]]
-==== Adding Built-in User Passwords To {kib} and Logstash
+==== Adding Built-in User Passwords To {kib}, Logstash, and Beats
 
 After the `kibana` user password is set, you need to update the {kib} server
 with the new password by setting `elasticsearch.password` in the `kibana.yml`
@@ -146,6 +147,24 @@ PUT _xpack/security/user/logstash_system/_enable
 ---------------------------------------------------------------------
 // CONSOLE
 
+The `beats_system` user is used internally within Beats when monitoring is
+enabled for Beats.
+
+To enable this feature in Beats, you need to update the configuration for each
+of your beats to reference the correct username and password. For example:
+
+[source,yaml]
+----------------------------------------------------------
+xpack.monitoring.elasticsearch.username: beats_system
+xpack.monitoring.elasticsearch.password: beatspassword
+----------------------------------------------------------
+
+If you have upgraded from an older version of {es}, then you may not have set a
+password for the `beats_system` user. If this is the case, then you should use
+the *Management > Users* page in {kib} or the
+{ref}/security-api-change-password.html[Change Password API] to set a password
+for this user.
+
 [float]
 [[disabling-default-password]]
 ==== Disabling Default Password Functionality

docs/en/security/authorization.asciidoc

@@ -99,6 +99,16 @@ change between releases.
 NOTE: This role does not provide access to the logstash indices and is not
 suitable for use within a Logstash pipeline.
 
+[[built-in-roles-beats-system]] `beats_system` ::
+Grants access necessary for the Beats system user to send system-level data
+(such as monitoring) to {es}. 
++
+NOTE: This role should not be assigned to users as the granted permissions may
+change between releases.
++
+NOTE: This role does not provide access to the beats indices and is not
+suitable for writing beats output to {es}.
+
 [[built-in-roles-ml-admin]] `machine_learning_admin`::
 Grants `manage_ml` cluster privileges and read access to the `.ml-*` indices.
 

docs/en/security/getting-started.asciidoc

@@ -18,7 +18,8 @@ To get started with {security}:
 
 .  Start {es} and {kib}.
 
-. Set the passwords of the built in `elastic`, `kibana`, and `logstash_system` users.
+. Set the passwords of the built in `elastic`, `kibana`, `logstash_system`, and
+`beats_system` users.
 In most cases, you can simply run the `bin/x-pack/setup-passwords` tool on one of the nodes in your cluster.
 Run that command with the same user that is running your {es} process.
 In "auto" mode this tool will randomly generate passwords and print them to the console.

plugin/core/src/main/java/org/elasticsearch/xpack/core/security/authc/esnative/ClientReservedRealm.java

@@ -18,6 +18,7 @@ public class ClientReservedRealm {
             case UsernamesField.ELASTIC_NAME:
             case UsernamesField.KIBANA_NAME:
             case UsernamesField.LOGSTASH_NAME:
+            case UsernamesField.BEATS_NAME:
                 return XPackSettings.RESERVED_REALM_ENABLED_SETTING.get(settings);
             default:
                 return AnonymousUser.isAnonymousUsername(username, settings);

plugin/core/src/main/java/org/elasticsearch/xpack/core/security/authz/store/ReservedRolesStore.java

@@ -83,6 +83,8 @@ public class ReservedRolesStore {
                         null, MetadataUtils.DEFAULT_RESERVED_METADATA))
                 .put("logstash_system", new RoleDescriptor("logstash_system", new String[] { "monitor", MonitoringBulkAction.NAME},
                         null, null, MetadataUtils.DEFAULT_RESERVED_METADATA))
+                .put(UsernamesField.BEATS_ROLE, new RoleDescriptor(UsernamesField.BEATS_ROLE,
+                        new String[] { "monitor", MonitoringBulkAction.NAME}, null, null, MetadataUtils.DEFAULT_RESERVED_METADATA))
                 .put("machine_learning_user", new RoleDescriptor("machine_learning_user", new String[] { "monitor_ml" },
                         new RoleDescriptor.IndicesPrivileges[] { RoleDescriptor.IndicesPrivileges.builder().indices(".ml-anomalies*",
                                 ".ml-notifications").privileges("view_index_metadata", "read").build() },

plugin/core/src/main/java/org/elasticsearch/xpack/core/security/user/BeatsSystemUser.java

@@ -0,0 +1,24 @@
+/*
+ * Copyright Elasticsearch B.V. and/or licensed to Elasticsearch B.V. under one
+ * or more contributor license agreements. Licensed under the Elastic License;
+ * you may not use this file except in compliance with the Elastic License.
+ */
+package org.elasticsearch.xpack.core.security.user;
+
+import org.elasticsearch.Version;
+import org.elasticsearch.xpack.core.security.support.MetadataUtils;
+
+/**
+ * Built in user for beats internals. Currently used for Beats monitoring.
+ */
+public class BeatsSystemUser extends User {
+
+    public static final String NAME = UsernamesField.BEATS_NAME;
+    public static final String ROLE_NAME = UsernamesField.BEATS_ROLE;
+    public static final Version DEFINED_SINCE = Version.V_6_3_0;
+    public static final BuiltinUserInfo USER_INFO = new BuiltinUserInfo(NAME, ROLE_NAME, DEFINED_SINCE);
+
+    public BeatsSystemUser(boolean enabled) {
+        super(NAME, new String[]{ ROLE_NAME }, null, null, MetadataUtils.DEFAULT_RESERVED_METADATA, enabled);
+    }
+}

plugin/core/src/main/java/org/elasticsearch/xpack/core/security/user/UsernamesField.java

@@ -18,6 +18,8 @@ public final class UsernamesField {
     public static final String XPACK_ROLE =  "_xpack";
     public static final String LOGSTASH_NAME = "logstash_system";
     public static final String LOGSTASH_ROLE = "logstash_system";
+    public static final String BEATS_NAME = "beats_system";
+    public static final String BEATS_ROLE = "beats_system";
 
     private UsernamesField() {}
 }

plugin/core/src/test/java/org/elasticsearch/xpack/core/security/authz/store/ReservedRolesStoreTests.java

@@ -84,6 +84,8 @@ import org.elasticsearch.xpack.core.security.authz.RoleDescriptor;
 import org.elasticsearch.xpack.core.security.authz.accesscontrol.IndicesAccessControl.IndexAccessControl;
 import org.elasticsearch.xpack.core.security.authz.permission.FieldPermissionsCache;
 import org.elasticsearch.xpack.core.security.authz.permission.Role;
+import org.elasticsearch.xpack.core.security.user.BeatsSystemUser;
+import org.elasticsearch.xpack.core.security.user.LogstashSystemUser;
 import org.elasticsearch.xpack.core.security.user.SystemUser;
 import org.elasticsearch.xpack.core.security.user.XPackUser;
 import org.elasticsearch.xpack.core.watcher.execution.TriggeredWatchStoreField;
@@ -130,6 +132,8 @@ public class ReservedRolesStoreTests extends ESTestCase {
         assertThat(ReservedRolesStore.isReserved("watcher_admin"), is(true));
         assertThat(ReservedRolesStore.isReserved("kibana_dashboard_only_user"), is(true));
         assertThat(ReservedRolesStore.isReserved(XPackUser.ROLE_NAME), is(true));
+        assertThat(ReservedRolesStore.isReserved(LogstashSystemUser.ROLE_NAME), is(true));
+        assertThat(ReservedRolesStore.isReserved(BeatsSystemUser.ROLE_NAME), is(true));
     }
 
     public void testIngestAdminRole() {
@@ -472,6 +476,28 @@ public class ReservedRolesStoreTests extends ESTestCase {
                 is(false));
     }
 
+    public void testBeatsSystemRole() {
+        RoleDescriptor roleDescriptor = new ReservedRolesStore().roleDescriptor(BeatsSystemUser.ROLE_NAME);
+        assertNotNull(roleDescriptor);
+        assertThat(roleDescriptor.getMetadata(), hasEntry("_reserved", true));
+
+        Role logstashSystemRole = Role.builder(roleDescriptor, null).build();
+        assertThat(logstashSystemRole.cluster().check(ClusterHealthAction.NAME), is(true));
+        assertThat(logstashSystemRole.cluster().check(ClusterStateAction.NAME), is(true));
+        assertThat(logstashSystemRole.cluster().check(ClusterStatsAction.NAME), is(true));
+        assertThat(logstashSystemRole.cluster().check(PutIndexTemplateAction.NAME), is(false));
+        assertThat(logstashSystemRole.cluster().check(ClusterRerouteAction.NAME), is(false));
+        assertThat(logstashSystemRole.cluster().check(ClusterUpdateSettingsAction.NAME), is(false));
+        assertThat(logstashSystemRole.cluster().check(MonitoringBulkAction.NAME), is(true));
+
+        assertThat(logstashSystemRole.runAs().check(randomAlphaOfLengthBetween(1, 30)), is(false));
+
+        assertThat(logstashSystemRole.indices().allowedIndicesMatcher(IndexAction.NAME).test("foo"), is(false));
+        assertThat(logstashSystemRole.indices().allowedIndicesMatcher(IndexAction.NAME).test(".reporting"), is(false));
+        assertThat(logstashSystemRole.indices().allowedIndicesMatcher("indices:foo").test(randomAlphaOfLengthBetween(8, 24)),
+                is(false));
+    }
+
     public void testMachineLearningAdminRole() {
         RoleDescriptor roleDescriptor = new ReservedRolesStore().roleDescriptor("machine_learning_admin");
         assertNotNull(roleDescriptor);

plugin/security/src/main/java/org/elasticsearch/xpack/security/authc/esnative/ReservedRealm.java

@@ -25,6 +25,7 @@ import org.elasticsearch.xpack.core.security.authc.support.Hasher;
 import org.elasticsearch.xpack.core.security.authc.support.UsernamePasswordToken;
 import org.elasticsearch.xpack.core.security.support.Exceptions;
 import org.elasticsearch.xpack.core.security.user.AnonymousUser;
+import org.elasticsearch.xpack.core.security.user.BeatsSystemUser;
 import org.elasticsearch.xpack.core.security.user.ElasticUser;
 import org.elasticsearch.xpack.core.security.user.KibanaUser;
 import org.elasticsearch.xpack.core.security.user.LogstashSystemUser;
@@ -143,6 +144,8 @@ public class ReservedRealm extends CachingUsernamePasswordRealm {
                 return new KibanaUser(userInfo.enabled);
             case LogstashSystemUser.NAME:
                 return new LogstashSystemUser(userInfo.enabled);
+            case BeatsSystemUser.NAME:
+                return new BeatsSystemUser(userInfo.enabled);
             default:
                 if (anonymousEnabled && anonymousUser.principal().equals(username)) {
                     return anonymousUser;
@@ -168,6 +171,9 @@ public class ReservedRealm extends CachingUsernamePasswordRealm {
                 userInfo = reservedUserInfos.get(LogstashSystemUser.NAME);
                 users.add(new LogstashSystemUser(userInfo == null || userInfo.enabled));
 
+                userInfo = reservedUserInfos.get(BeatsSystemUser.NAME);
+                users.add(new BeatsSystemUser(userInfo == null || userInfo.enabled));
+
                 if (anonymousEnabled) {
                     users.add(anonymousUser);
                 }
@@ -219,6 +225,8 @@ public class ReservedRealm extends CachingUsernamePasswordRealm {
         switch (username) {
             case LogstashSystemUser.NAME:
                 return LogstashSystemUser.DEFINED_SINCE;
+            case BeatsSystemUser.NAME:
+                return BeatsSystemUser.DEFINED_SINCE;
             default:
                 return Version.V_5_0_0;
         }

plugin/security/src/main/java/org/elasticsearch/xpack/security/authc/esnative/tool/SetupPasswordTool.java

@@ -27,6 +27,7 @@ import org.elasticsearch.common.xcontent.XContentBuilder;
 import org.elasticsearch.common.xcontent.json.JsonXContent;
 import org.elasticsearch.env.Environment;
 import org.elasticsearch.xpack.core.security.support.Validation;
+import org.elasticsearch.xpack.core.security.user.BeatsSystemUser;
 import org.elasticsearch.xpack.core.security.user.ElasticUser;
 import org.elasticsearch.xpack.core.security.user.KibanaUser;
 import org.elasticsearch.xpack.core.security.user.LogstashSystemUser;
@@ -48,6 +49,8 @@ import java.util.HashMap;
 import java.util.List;
 import java.util.Map;
 
+import static java.util.Arrays.asList;
+
 /**
  * A tool to set passwords of reserved users (elastic, kibana and
  * logstash_system). Can run in `interactive` or `auto` mode. In `auto` mode
@@ -59,7 +62,7 @@ import java.util.Map;
 public class SetupPasswordTool extends LoggingAwareMultiCommand {
 
     private static final char[] CHARS = ("ABCDEFGHIJKLMNOPQRSTUVWXYZabcdefghijklmnopqrstuvwxyz0123456789").toCharArray();
-    public static final List<String> USERS = Arrays.asList(ElasticUser.NAME, KibanaUser.NAME, LogstashSystemUser.NAME);
+    public static final List<String> USERS = asList(ElasticUser.NAME, KibanaUser.NAME, LogstashSystemUser.NAME, BeatsSystemUser.NAME);
 
     private final CheckedFunction<Environment, CommandLineHttpClient, Exception> clientFunction;
     private final CheckedFunction<Environment, KeyStoreWrapper, Exception> keyStoreFunction;
@@ -240,8 +243,8 @@ public class SetupPasswordTool extends LoggingAwareMultiCommand {
         }
 
         private void setParser() {
-            urlOption = parser.acceptsAll(Arrays.asList("u", "url"), "The url for the change password request.").withRequiredArg();
+            urlOption = parser.acceptsAll(asList("u", "url"), "The url for the change password request.").withRequiredArg();
-            noPromptOption = parser.acceptsAll(Arrays.asList("b", "batch"),
+            noPromptOption = parser.acceptsAll(asList("b", "batch"),
                     "If enabled, run the change password process without prompting the user.").withOptionalArg();
         }
 

plugin/security/src/test/java/org/elasticsearch/test/NativeRealmIntegTestCase.java

@@ -19,6 +19,7 @@ import org.elasticsearch.common.util.set.Sets;
 import org.elasticsearch.xpack.core.security.SecurityLifecycleServiceField;
 import org.elasticsearch.xpack.core.security.authc.support.UsernamePasswordToken;
 import org.elasticsearch.xpack.core.security.client.SecurityClient;
+import org.elasticsearch.xpack.core.security.user.BeatsSystemUser;
 import org.elasticsearch.xpack.core.security.user.ElasticUser;
 import org.elasticsearch.xpack.core.security.user.KibanaUser;
 import org.elasticsearch.xpack.core.security.user.LogstashSystemUser;
@@ -95,7 +96,7 @@ public abstract class NativeRealmIntegTestCase extends SecurityIntegTestCase {
             assertEquals(response.getStatusLine().getReasonPhrase(), 200, response.getStatusLine().getStatusCode());
         }
 
-        for (String username : Arrays.asList(KibanaUser.NAME, LogstashSystemUser.NAME)) {
+        for (String username : Arrays.asList(KibanaUser.NAME, LogstashSystemUser.NAME, BeatsSystemUser.NAME)) {
             String payload = "{\"password\": \"" + new String(reservedPassword.getChars()) + "\"}";
             HttpEntity entity = new NStringEntity(payload, ContentType.APPLICATION_JSON);
             BasicHeader authHeader = new BasicHeader(UsernamePasswordToken.BASIC_AUTH_HEADER,

plugin/security/src/test/java/org/elasticsearch/xpack/security/authc/esnative/NativeUsersStoreTests.java

@@ -27,6 +27,7 @@ import org.elasticsearch.test.ESTestCase;
 import org.elasticsearch.threadpool.ThreadPool;
 import org.elasticsearch.xpack.core.security.authc.AuthenticationResult;
 import org.elasticsearch.xpack.core.security.authc.support.Hasher;
+import org.elasticsearch.xpack.core.security.user.BeatsSystemUser;
 import org.elasticsearch.xpack.core.security.user.ElasticUser;
 import org.elasticsearch.xpack.core.security.user.KibanaUser;
 import org.elasticsearch.xpack.core.security.user.LogstashSystemUser;
@@ -87,7 +88,7 @@ public class NativeUsersStoreTests extends ESTestCase {
     public void testPasswordUpsertWhenSetEnabledOnReservedUser() throws Exception {
         final NativeUsersStore nativeUsersStore = startNativeUsersStore();
 
-        final String user = randomFrom(ElasticUser.NAME, KibanaUser.NAME, LogstashSystemUser.NAME);
+        final String user = randomFrom(ElasticUser.NAME, KibanaUser.NAME, LogstashSystemUser.NAME, BeatsSystemUser.NAME);
 
         final PlainActionFuture<Void> future = new PlainActionFuture<>();
         nativeUsersStore.setEnabled(user, true, WriteRequest.RefreshPolicy.IMMEDIATE, future);
@@ -105,7 +106,7 @@ public class NativeUsersStoreTests extends ESTestCase {
     public void testBlankPasswordInIndexImpliesDefaultPassword() throws Exception {
         final NativeUsersStore nativeUsersStore = startNativeUsersStore();
 
-        final String user = randomFrom(ElasticUser.NAME, KibanaUser.NAME, LogstashSystemUser.NAME);
+        final String user = randomFrom(ElasticUser.NAME, KibanaUser.NAME, LogstashSystemUser.NAME, BeatsSystemUser.NAME);
         final Map<String, Object> values = new HashMap<>();
         values.put(ENABLED_FIELD, Boolean.TRUE);
         values.put(PASSWORD_FIELD, BLANK_PASSWORD);

plugin/security/src/test/java/org/elasticsearch/xpack/security/authc/esnative/ReservedRealmIntegTests.java

@@ -11,6 +11,7 @@ import org.elasticsearch.common.settings.SecureString;
 import org.elasticsearch.test.NativeRealmIntegTestCase;
 import org.elasticsearch.xpack.core.security.action.user.ChangePasswordResponse;
 import org.elasticsearch.xpack.core.security.client.SecurityClient;
+import org.elasticsearch.xpack.core.security.user.BeatsSystemUser;
 import org.elasticsearch.xpack.core.security.user.ElasticUser;
 import org.elasticsearch.xpack.core.security.user.KibanaUser;
 import org.elasticsearch.xpack.core.security.user.LogstashSystemUser;
@@ -29,7 +30,7 @@ import static org.hamcrest.Matchers.notNullValue;
 public class ReservedRealmIntegTests extends NativeRealmIntegTestCase {
 
     public void testAuthenticate() {
-        for (String username : Arrays.asList(ElasticUser.NAME, KibanaUser.NAME, LogstashSystemUser.NAME)) {
+        for (String username : Arrays.asList(ElasticUser.NAME, KibanaUser.NAME, LogstashSystemUser.NAME, BeatsSystemUser.NAME)) {
             ClusterHealthResponse response = client()
                     .filterWithHeader(singletonMap("Authorization", basicAuthHeaderValue(username, getReservedPassword())))
                     .admin()
@@ -47,7 +48,7 @@ public class ReservedRealmIntegTests extends NativeRealmIntegTestCase {
      */
     public void testAuthenticateAfterEnablingUser() {
         final SecurityClient c = securityClient();
-        for (String username : Arrays.asList(ElasticUser.NAME, KibanaUser.NAME, LogstashSystemUser.NAME)) {
+        for (String username : Arrays.asList(ElasticUser.NAME, KibanaUser.NAME, LogstashSystemUser.NAME, BeatsSystemUser.NAME)) {
             c.prepareSetEnabled(username, true).get();
             ClusterHealthResponse response = client()
                     .filterWithHeader(singletonMap("Authorization", basicAuthHeaderValue(username, getReservedPassword())))
@@ -61,7 +62,7 @@ public class ReservedRealmIntegTests extends NativeRealmIntegTestCase {
     }
 
     public void testChangingPassword() {
-        String username = randomFrom(ElasticUser.NAME, KibanaUser.NAME, LogstashSystemUser.NAME);
+        String username = randomFrom(ElasticUser.NAME, KibanaUser.NAME, LogstashSystemUser.NAME, BeatsSystemUser.NAME);
         final char[] newPassword = "supersecretvalue".toCharArray();
 
         if (randomBoolean()) {

plugin/security/src/test/java/org/elasticsearch/xpack/security/authc/esnative/ReservedRealmTests.java

@@ -21,6 +21,7 @@ import org.elasticsearch.xpack.core.security.authc.esnative.ClientReservedRealm;
 import org.elasticsearch.xpack.core.security.authc.support.Hasher;
 import org.elasticsearch.xpack.core.security.authc.support.UsernamePasswordToken;
 import org.elasticsearch.xpack.core.security.user.AnonymousUser;
+import org.elasticsearch.xpack.core.security.user.BeatsSystemUser;
 import org.elasticsearch.xpack.core.security.user.ElasticUser;
 import org.elasticsearch.xpack.core.security.user.KibanaUser;
 import org.elasticsearch.xpack.core.security.user.LogstashSystemUser;
@@ -73,7 +74,8 @@ public class ReservedRealmTests extends ESTestCase {
     }
 
     public void testReservedUserEmptyPasswordAuthenticationFails() throws Throwable {
-        final String principal = randomFrom(UsernamesField.ELASTIC_NAME, UsernamesField.KIBANA_NAME, UsernamesField.LOGSTASH_NAME);
+        final String principal = randomFrom(UsernamesField.ELASTIC_NAME, UsernamesField.KIBANA_NAME, UsernamesField.LOGSTASH_NAME,
+                UsernamesField.BEATS_NAME);
 
         final ReservedRealm reservedRealm = new ReservedRealm(mock(Environment.class), Settings.EMPTY, usersStore,
                 new AnonymousUser(Settings.EMPTY), securityLifecycleService, new ThreadContext(Settings.EMPTY));
@@ -93,7 +95,7 @@ public class ReservedRealmTests extends ESTestCase {
         final ReservedRealm reservedRealm =
                 new ReservedRealm(mock(Environment.class), settings, usersStore,
                         new AnonymousUser(settings), securityLifecycleService, new ThreadContext(Settings.EMPTY));
-        final User expected = randomFrom(new ElasticUser(true), new KibanaUser(true), new LogstashSystemUser(true));
+        final User expected = randomReservedUser(true);
         final String principal = expected.principal();
 
         PlainActionFuture<AuthenticationResult> listener = new PlainActionFuture<>();
@@ -115,7 +117,7 @@ public class ReservedRealmTests extends ESTestCase {
     private void verifySuccessfulAuthentication(boolean enabled) throws Exception {
         final ReservedRealm reservedRealm = new ReservedRealm(mock(Environment.class), Settings.EMPTY, usersStore,
                 new AnonymousUser(Settings.EMPTY), securityLifecycleService, new ThreadContext(Settings.EMPTY));
-        final User expectedUser = randomFrom(new ElasticUser(enabled), new KibanaUser(enabled), new LogstashSystemUser(enabled));
+        final User expectedUser = randomReservedUser(enabled);
         final String principal = expectedUser.principal();
         final SecureString newPassword = new SecureString("foobar".toCharArray());
         when(securityLifecycleService.isSecurityIndexExisting()).thenReturn(true);
@@ -156,7 +158,7 @@ public class ReservedRealmTests extends ESTestCase {
         final ReservedRealm reservedRealm =
                 new ReservedRealm(mock(Environment.class), Settings.EMPTY, usersStore,
                         new AnonymousUser(Settings.EMPTY), securityLifecycleService, new ThreadContext(Settings.EMPTY));
-        final User expectedUser = randomFrom(new ElasticUser(true), new KibanaUser(true), new LogstashSystemUser(true));
+        final User expectedUser = randomReservedUser(true);
         final String principal = expectedUser.principal();
 
         PlainActionFuture<User> listener = new PlainActionFuture<>();
@@ -181,7 +183,7 @@ public class ReservedRealmTests extends ESTestCase {
         final ReservedRealm reservedRealm =
                 new ReservedRealm(mock(Environment.class), settings, usersStore, new AnonymousUser(settings),
                         securityLifecycleService, new ThreadContext(Settings.EMPTY));
-        final User expectedUser = randomFrom(new ElasticUser(true), new KibanaUser(true), new LogstashSystemUser(true));
+        final User expectedUser = randomReservedUser(true);
         final String principal = expectedUser.principal();
 
         PlainActionFuture<User> listener = new PlainActionFuture<>();
@@ -195,7 +197,7 @@ public class ReservedRealmTests extends ESTestCase {
         final ReservedRealm reservedRealm =
                 new ReservedRealm(mock(Environment.class), Settings.EMPTY, usersStore,
                         new AnonymousUser(Settings.EMPTY), securityLifecycleService, new ThreadContext(Settings.EMPTY));
-        final User expectedUser = randomFrom(new ElasticUser(true), new KibanaUser(true), new LogstashSystemUser(true));
+        final User expectedUser = randomReservedUser(true);
         final String principal = expectedUser.principal();
         when(securityLifecycleService.isSecurityIndexExisting()).thenReturn(true);
         final RuntimeException e = new RuntimeException("store threw");
@@ -221,7 +223,7 @@ public class ReservedRealmTests extends ESTestCase {
     }
 
     public void testIsReserved() {
-        final User expectedUser = randomFrom(new ElasticUser(true), new KibanaUser(true), new LogstashSystemUser(true));
+        final User expectedUser = randomReservedUser(true);
         final String principal = expectedUser.principal();
         assertThat(ClientReservedRealm.isReserved(principal, Settings.EMPTY), is(true));
 
@@ -231,7 +233,7 @@ public class ReservedRealmTests extends ESTestCase {
 
     public void testIsReservedDisabled() {
         Settings settings = Settings.builder().put(XPackSettings.RESERVED_REALM_ENABLED_SETTING.getKey(), false).build();
-        final User expectedUser = randomFrom(new ElasticUser(true), new KibanaUser(true), new LogstashSystemUser(true));
+        final User expectedUser = randomReservedUser(true);
         final String principal = expectedUser.principal();
         assertThat(ClientReservedRealm.isReserved(principal, settings), is(false));
 
@@ -244,8 +246,8 @@ public class ReservedRealmTests extends ESTestCase {
                 new AnonymousUser(Settings.EMPTY), securityLifecycleService, new ThreadContext(Settings.EMPTY));
         PlainActionFuture<Collection<User>> userFuture = new PlainActionFuture<>();
         reservedRealm.users(userFuture);
-        assertThat(userFuture.actionGet(), containsInAnyOrder(new ElasticUser(true), new KibanaUser(true),
+        assertThat(userFuture.actionGet(),
-                new LogstashSystemUser(true)));
+                containsInAnyOrder(new ElasticUser(true), new KibanaUser(true), new LogstashSystemUser(true), new BeatsSystemUser(true)));
     }
 
     public void testGetUsersDisabled() {
@@ -373,7 +375,7 @@ public class ReservedRealmTests extends ESTestCase {
                 new AnonymousUser(Settings.EMPTY), securityLifecycleService, new ThreadContext(Settings.EMPTY));
         PlainActionFuture<AuthenticationResult> listener = new PlainActionFuture<>();
 
-        final String principal = randomFrom(KibanaUser.NAME, LogstashSystemUser.NAME);
+        final String principal = randomFrom(KibanaUser.NAME, LogstashSystemUser.NAME, BeatsSystemUser.NAME);
         doAnswer((i) -> {
             ActionListener callback = (ActionListener) i.getArguments()[1];
             callback.onResponse(null);
@@ -395,12 +397,16 @@ public class ReservedRealmTests extends ESTestCase {
                 new AnonymousUser(Settings.EMPTY), securityLifecycleService, new ThreadContext(Settings.EMPTY));
         PlainActionFuture<AuthenticationResult> listener = new PlainActionFuture<>();
 
-        final String principal = randomFrom(KibanaUser.NAME, LogstashSystemUser.NAME);
+        final String principal = randomFrom(KibanaUser.NAME, LogstashSystemUser.NAME, BeatsSystemUser.NAME);
         reservedRealm.doAuthenticate(new UsernamePasswordToken(principal, mockSecureSettings.getString("bootstrap.password")), listener);
         final AuthenticationResult result = listener.get();
         assertThat(result.getStatus(), is(AuthenticationResult.Status.TERMINATE));
     }
 
+    private User randomReservedUser(boolean enabled) {
+        return randomFrom(new ElasticUser(enabled), new KibanaUser(enabled), new LogstashSystemUser(enabled), new BeatsSystemUser(enabled));
+    }
+
     /*
      * NativeUserStore#getAllReservedUserInfo is pkg private we can't mock it otherwise
      */
@@ -425,13 +431,20 @@ public class ReservedRealmTests extends ESTestCase {
                 assertThat(versionPredicate.test(Version.V_5_0_0), is(false));
                 assertThat(versionPredicate.test(Version.V_5_1_1), is(false));
                 assertThat(versionPredicate.test(Version.V_5_2_0), is(true));
+                assertThat(versionPredicate.test(Version.V_6_3_0), is(true));
+                break;
+            case BeatsSystemUser.NAME:
+                assertThat(versionPredicate.test(Version.V_5_6_9), is(false));
+                assertThat(versionPredicate.test(Version.V_6_2_3), is(false));
+                assertThat(versionPredicate.test(Version.V_6_3_0), is(true));
                 break;
             default:
                 assertThat(versionPredicate.test(Version.V_5_0_0), is(true));
                 assertThat(versionPredicate.test(Version.V_5_1_1), is(true));
                 assertThat(versionPredicate.test(Version.V_5_2_0), is(true));
+                assertThat(versionPredicate.test(Version.V_6_3_0), is(true));
                 break;
         }
-        assertThat(versionPredicate.test(Version.V_6_0_0_alpha1), is(true));
+        assertThat(versionPredicate.test(Version.V_7_0_0_alpha1), is(true));
     }
 }

qa/security-setup-password-tests/src/test/java/org/elasticsearch/xpack/security/authc/esnative/tool/SetupPasswordToolIT.java

@@ -97,7 +97,7 @@ public class SetupPasswordToolIT extends ESRestTestCase {
             }
         });
 
-        assertEquals(3, userPasswordMap.size());
+        assertEquals(4, userPasswordMap.size());
         userPasswordMap.entrySet().forEach(entry -> {
             final String basicHeader = "Basic " +
                     Base64.getEncoder().encodeToString((entry.getKey() + ":" + entry.getValue()).getBytes(StandardCharsets.UTF_8));